A multistage mixed-methods evaluation of the UKHSA testing response during the COVID-19 pandemic in England =========================================================================================================== * Reshania Naidoo * Billie Andersen-Waine * Prabin Dahal * Sophie Dickinson * Ben Lambert * Melinda C Mills * Catherine Molyneux * Emily Rowe * Sarah Pinto-Duschinsky * Kasia Stepniewska * Rima Shretta * Merryn Voysey * Marta Wanat * Gulsen Yenidogan * Lisa J White * the EY-Oxford Health Analytics Consortium ## ABSTRACT **Introduction** In 2020, the UK Health Security Agency (UKHSA) established a large-scale testing programme to rapidly identify individuals in England who were infected with SARS-CoV-2 and had COVID-19. This comprised part of the UK government’s COVID-19 response strategy, to protect those at risk of severe COVID-19 disease and death and to reduce the burden on the health system. To assess the success of this approach, UKHSA commissioned an independent evaluation of the activities delivered by the NHS testing programme in England. The primary purpose of this evaluation is to capture key learnings from the rollout of testing to different target populations via various testing services between October 2020 and March 2022 and to use these insights to formulate recommendations for future pandemic preparedness strategy. **Methods and analysis** The proposed study involves a stepwise mixed-methods approach, aligned with established methods for the evaluation of complex interventions in health, with retrospective and prospective components. A bottom-up approach will be taken, focusing on each of nine population-specific service settings. We will use a Theory of Change to understand the causal pathways and intended and unintended outcomes of each service, also exploring the effect of context on each individual service setting’s intended outcomes. Subsequently, the insights gained will be synthesised to identify process and outcome indicators to evaluate how the combined aims of the testing programme were achieved. A forward-looking, prospective component of this work will aim to inform testing strategy in preparation for future pandemics, through a participatory modelling simulation and policy analysis exercise. **Disclaimer** This is a provisional draft protocol that represents research in progress. This research was commissioned and funded by UKHSA, to be performed between August 2022 and March 2023. The scope and depth of testing services and channels covered by this research were pre-agreed with UKHSA and are limited to the availability and provision of data available at the time this protocol was written. **Ethics and dissemination** Findings arising from this evaluation will be used to inform lessons learnt and recommendations for UKHSA on appropriate pandemic preparedness testing programme designs; findings will also be disseminated in peer-reviewed journals and at academic conferences. **Strengths and limitations of the study** * Strengths of this mixed-methods evaluation protocol include the use of theory-based, complex evaluation approaches and an iterative and participatory approach with the stakeholder (UKHSA) to the evaluation process and prospective modelling. * Given the scale and complexity of the COVID-19 testing response in England, there is a scarcity of previous relevant research into this, either in England or appropriate international comparators, warranting the mixed-methods evaluation approach we are adopting. * This is the first national-scale evaluation of the testing response to COVID-19 in England to incorporate most service settings, a programme which formed an integral part of the UK pandemic response strategy. The approach proposed could be applied to the evaluation of pandemic responses in other contexts or to other types of interventions. * Whereas most complex interventions are accompanied by a prospective evaluation design initiated at the time of the intervention or earlier, this study predominantly comprises a retrospective evaluation and is therefore limited by the quality of existing research and the data available to the research team at the time of conducting the evaluation within the specified period allocated by UKHSA. ## INTRODUCTION In December 2019, a novel coronavirus was discovered in China and rapidly spread around the world. This virus was subsequently named severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and the disease it causes, novel coronavirus disease 2019 (COVID-19)1. On 11 March 2020, the World Health Organization (WHO) declared COVID-19 a pandemic and exhorted member states to “test, test, test”2. In response, the National Health Service (NHS) Test and Trace (T&T) Programme, England, part of the UK Health Security Agency (UKHSA), was established on 28 May 2020, with the purpose of enabling mass-scale testing and tracing systems to rapidly identify individuals with COVID-19 and their close contacts, thereby minimising the spread of the disease.3,4 The T&T programme had four main stated objectives: 1) to increase the speed and availability of testing, 2) to identify close contacts of positive cases and require them to isolate, 3) to contain local outbreaks via a coordinated response, and 4) to enable the government to learn more about the virus and explore ways to ease infection control measures as the science developed. The testing strategy and resultant policy has since been frequently adapted and revised in response to the changing epidemiological context of the pandemic.3 The overarching intended aim of the UK government’s COVID-19 response strategy was to deploy a coordinated and evolving programme of testing services to allow key sectors of society (such as health and social care; public; private; education; sport and leisure) to function smoothly and equitably during the pandemic while minimising transmission and harm.5 This strategy was deployed in the context of mitigating the impact of the pandemic and the measures to control it on other key sectors of society such as education, industry, hospitality, sport, and preventative and mental health.3 The overarching expectation was for the benefit and burden of this strategy to be shared equitably across the population.3 As part of this response, the testing programme, a component of NHS T&T, played an integral role through its various testing services, resulting in a complex balance of interconnected impacts on transmission, hospitalisations, mortality, societal productivity, and costs to the economy. To achieve its goal, the testing programme sought to work in partnership with local public health bodies, local authorities, the NHS, and commercial and academic providers; testing would then be rolled-out through a population-specific service delivery model across nine target populations6 (Table 1). The delivery of testing for each of these target populations was multi-modal, through combinations of in-person testing (e.g. public regional testing sites, mobile testing units), pharmacies, and home direct self-test kit deliveries. These were subject to ongoing revision by policymakers throughout the pandemic, dependent on factors such as changing epidemiological prevalence, emerging scientific evidence, and vaccination rollout.3 View this table: [Table 1.](http://medrxiv.org/content/early/2022/10/27/2022.10.27.22281604/T1) Table 1. The nine testing service target populations of the COVID-19 testing programme to be evaluated and the purpose of testing each population. The overall success and effectiveness of any national COVID-19 testing service is dependent on multiple contextual factors and the combined impacts of the various testing service settings. From a public health perspective, increased testing has been shown to result in reduced transmission of SARS-CoV-2 and associated hospitalisations and mortality.7-11 The economic impact of testing has been assessed via cost-effectiveness evaluations of testing strategies, including testing unit costs, operational deployment costs and, on a macro-economic level, quantifying the economic productivity gained from shortened isolation periods and savings to the taxpayer.12,13 Behavioural responses to testing strategies are dependent on public awareness and trust, with adherence to testing policy shown to be driven by perceptions of disease risk and socioeconomic factors.14 From an equity perspective, COVID-19 testing has been found to exacerbate existing health inequalities, with disparities in access to testing for individuals from black and ethnic minority backgrounds and those living in socially deprived areas.15 Tests for COVID-19 include those that detect the presence of the SARS-CoV-2 virus and those that detect the presence of antibodies to the virus.16 Tests for the virus, such as reverse-transcriptase polymerase chain reaction (RT-PCR) and loop-mediated isothermal amplification (LAMP), detect viral nucleic acid and are usually performed in a laboratory. Lateral flow tests (LFTs), which detect SARS-CoV-2 viral protein (antigen), are a quicker and easier approach to testing and can be used for self-testing, although they are less reliable than nucleic acid-based tests.17 UKHSA has appointed a team from Oxford University Innovation and Ernst and Young LLP to undertake an independent evaluation of the activities delivered by the national COVID-19 testing programme across nine service settings in England from October 2020 to March 2022 (hereafter “the evaluation”). The key purpose of this evaluation is to capture key learnings from the rollout of testing to the various target populations via the different testing services during this period and use these insights to formulate recommendations for future pandemic preparedness testing strategy. ### Evaluation aims and approach The testing programme can be considered to be a complex public health intervention due to the complexity of the intervention design and the context within which it was implemented.18 As such, the retrospective evaluation draws on a mixed-methods approach, utilising existing frameworks that have previously been applied to the evaluation of complex interventions, being broadly divided into process and cost-effectiveness evaluation components.19-22 Based on findings from the retrospective evaluation, the forward-looking prospective scenario modelling will aim to develop pandemic preparedness testing packages structured for low, medium, and high levels of preparedness. The project will consist of a phased process (Figure 1): ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/10/27/2022.10.27.22281604/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2022/10/27/2022.10.27.22281604/F1) Figure 1: Overview of the evaluation study design * A scoping phase, to develop a Theory of Change, evaluation aims, and research questions * A design phase, to agree evaluation approaches, methods, process and outcome indicators * A conduct phase, to collect, review and synthesise data * Evidence from the evaluation will then be used as the foundation for the prospective scenario modelling and to inform policy recommendations for future testing programmes Effective evaluation requires an understanding both of an intervention itself and how the intervention can achieve the expected outcomes. The “Theory of Change” (ToC) approach23,24 – a theory of how and why an initiative works – will therefore be used to map the causal pathways for each of the nine service settings shown in Table 1, as this approach lends itself to understanding complex interventions with multiple causal pathways.25 The ToC framework will be used to understand the causal pathways and intended and unintended outcomes of each service, also exploring the effect of context on each individual service setting’s intended outcomes. Subsequently, these separate insights will be used to define outcome and process indicators to determine if and how the combined aims of the testing programme were achieved. The ToC will be developed in a participatory manner with UKHSA stakeholders who, as the evaluation progresses, will be regularly consulted to discuss evolving causal assumptions and hypotheses. The evaluation will also consider the combined impacts of the nine population-specific service settings (Table 1) and whether/how they achieved the overall aims of the testing programme. Key research questions relating to each service, from October 2020 to March 2022, are detailed in Table 2. View this table: [Table 2:](http://medrxiv.org/content/early/2022/10/27/2022.10.27.22281604/T2) Table 2: Key research questions ## METHODS AND ANALYSIS ### Retrospective evaluation approach #### Overarching evaluation approach and design The Medical Research Council (MRC) framework for process evaluation details three key interrelated components as part of any complex public health process evaluation: implementation, mechanisms of impact, and context.22 To address these components, questions (1–3 in Table 2) for each testing service were developed. A comprehensive ToC for each of the nine service settings (Table 1) will be completed for the process evaluation, to inform outcome and process indicators that will be used to assess each service.26 The ToCs will be formulated utilising existing evidence provided by UKHSA, to detail the causal pathways for each service; the ToCs will also be subject to further refinement with UKHSA policy and operational stakeholders. A detailed, overarching Theory of Change for the testing service as a whole is illustrated in Figure 2. These ToCs will then be further consolidated into a narrative service overview summary for each of the nine service settings. Owing to the unpredictability of complex interventions such as this, sufficient flexibility will be allowed within the research design to allow emerging research questions to be addressed.22 ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/10/27/2022.10.27.22281604/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2022/10/27/2022.10.27.22281604/F2) Figure 2: Overarching Theory of Change for the COVID-19 testing programme in England (draft example -work in progress) Acknowledging the wide scope and breadth of the testing services rollout across England and limitations on the commissioned study period, preference will be given to identifying core research questions across targeted priority testing service settings (as identified in collaboration with the funder) and performing a “deep dive” into these for the evaluation, as opposed to covering multiple research questions across all settings. Insights from existing UK government-commissioned evidence in the form of previous COVID-19 service-specific testing evaluations and pilot testing programme data will be utilised where possible to avoid duplication of work. #### Process evaluation The process evaluation will address research questions 1–3 for each service: 1. How did the delivery and uptake of the service compare with what was planned over time and what factors affected this? 2. What were the barriers and facilitators to access, use, and delivery of the service? 3. What measurable impacts were there from the service according to its intended purpose? ### Process evaluation: qualitative research approach A rapid evidence review and synthesis of existing data within UKHSA and the public domain will be performed as part of the process evaluation; this will serve two purposes: * **To formulate a ToC and logic model framework****27** **for each of the nine testing services** A rapid review approach will be taken to synthesise existing evidence for each of the nine service settings, leveraging narrative reviews as a method for synthesising quantitative and qualitative data and feeding into the ToC.28,29 We will extract data using a predefined template, with the purpose of developing an understanding of the interventions used within each service, how they were implemented, how they were expected to work, intended outcomes, and the timeline of events and evolution of relevant policy. A gap analysis will also be conducted at this stage to determine whether any crucial data, research, or information is unavailable within the existing literature or data sources provided by UKHSA and whether subsequent adjustment of the research questions or descoping is required. This will be decided through a participatory consultation approach with key UKHSA stakeholders. * **To synthesise evidence on the behavioural barriers and facilitators for individuals undergoing COVID-19 testing in England** We will review existing studies to explore the process of implementing and delivering the COVID-19 testing service in England and any associated communication and engagement. We will provide a summary of identified studies, describing the setting in which testing took place, the types of tests used, participant characteristics, and the phase(s) of the pandemic. We will then perform a thematic synthesis using a behaviour change framework drawing on a relevant behaviour change framework to identify barriers and facilitators.30 ### Process evaluation: quantitative research approach #### Data collection Quantitative data will be obtained via UKHSA, from existing UKHSA repositories, the UK Office for National Statistics (ONS), NHS Digital, and other public sources of data where available. Datasets that relate generally to all testing services will be obtained for throughout the time course of the pandemic to March 2022; data will be aggregated at the local authority level and by age group. These data will also be used to inform the cost-effectiveness analyses. Where available, relevant individual-level data will also be utilised, such as those within the ISARIC (International Severe Acute Respiratory and Emerging Infection Consortium) database or from sources within NHS Digital. General datasets will include SARS-CoV-2 seroprevalence surveys; COVID-19 vaccination data; testing coverage data; and COVID-19 cases, hospitalisations, and deaths. Where available, these data will cover those who work in healthcare settings, essential employees, and university and school staff and students. Datasets specific to individual testing services will include cases and deaths in residents in care home settings, data from cases identified as having occurred through transmission between those attending a range of events, and hospital waiting times. Where possible, outcome data will be merged with publicly accessible datasets relating to population age-structures, income/GDP estimates, urban and rural populations, ethnicity, and indices of deprivation for each local authority; these merged datasets will then be analysed. #### Data analysis Data will be analysed with the aim of 1) providing summaries of outcome indicators of the implementation of each service identified in the ToC, to better understand the extent and reach of each testing service, and 2) providing estimates of the impact of each testing service, which will feed into the cost-effectiveness evaluation. For the impact assessment to feed into the process evaluation, the focus will be on a specified primary outcome for each testing service, as defined in the ToC. Outcomes and appropriate counterfactual comparators will be defined, contingent on obtaining access to available data sources; these definitions may be refined using an iterative process based on concurrent analysis of qualitative data. For each testing service, a multi-level regression-based approach or a method from the causal inference literature, such as the UK government Magenta book,19 will be undertaken together with analysis of data at the local authority level (or, if the data allow, at a finer spatial aggregation), accounting for potential confounding factors such as age, sex, and ethnicity profiles, as well as indicators of deprivation, population density, and relevant chronic illnesses. Where determined to be relevant, in collaboration with UKHSA stakeholders, predictors of engagement with health services, such as vaccination uptake or access to internet services will also be included. The ToCs will be populated in a collaborative manner, enabling the quantitative and qualitative research teams between them to define and align the process and outcome indicators for each service setting. Each team will conduct research concurrently for the various service settings and will communicate their findings at weekly meetings, allowing for emerging findings to be discussed and explored further if deemed necessary. Findings will also be synthesised across the testing services to inform programme-level insights to meet the programme-level research aims and inform the overarching ToC. ### Cost-effectiveness evaluation The cost-effectiveness evaluation will address research question 4: What was the cost to the government and the cost-effectiveness of the service? #### Data collection The cost-effectiveness analysis will take a provider perspective and analyse costs to the NHS/local authority. A literature review of publicly available costing data will be conducted using keyword searches of scientific databases as well as a grey literature search using Google Scholar. #### Data analysis Data will be analysed with the aim of 1) providing an estimate of the costs for each testing service using a cost-of-illness approach and 2) providing estimates of the value for money of each testing service, which will use the outputs from the statistical analysis, e.g. cost per quality-adjusted life-year (QALY) averted. #### Analysis and synthesis of data The analysis will use a thematic approach, which will involve the triangulation of findings from the literature reviews. ### Prospective scenario analysis The prospective evaluation will address research question 5: which aspects of the service might be beneficial for future programmes? #### Prospective scenario modelling Prospective scenario modelling will allow us to predict the impacts of a range of testing strategy packages in the context of various epidemiological challenges that may arise in the future. To ensure that the suite of recommendations that follow from this evaluation are relevant, feasible, applicable to likely future contexts, and grounded in robust data, the results from the retrospective evaluation will be used as inputs into a model designed to estimate the impact of possible testing packages. This will ensure that only those packages that will have the optimal impact are put forward as recommendations. We will use modified versions of existing age-structured, compartmental SEIRS (susceptible– exposed–infectious–recovered–susceptible) models31-34 (Figure 2) to estimate the trajectory of COVID-19 epidemiology based on a variety of scenarios. The models will be further modified to include details of key mass-testing interventions. They will then be used as simulation tools to explore future mass-testing strategy packages applied to a range of future pandemic threat use-case scenarios, having been populated with values estimated from data obtained during the retrospective analysis stage of the evaluation. The selection of future scenarios will be a participatory process, involving collaboration with UKHSA, to ensure realistic options are identified and simulated. The impact of each testing strategy package will then be determined in terms of reductions in transmission, hospital utilisation, mortality, and cost. Model outputs will also feed into a linked economic sub-model for costing and financial analysis. The prospective scenario modelling predictions will be used to inform recommendations to UKHSA on appropriate pandemic preparedness testing packages. ![Figure3](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/10/27/2022.10.27.22281604/F3.medium.gif) [Figure3](http://medrxiv.org/content/early/2022/10/27/2022.10.27.22281604/F3) Diagrammatic example of a typical baseline model structure to be used for prospective scenario modelling, representing an unmitigated spread scenario. Recommendations – implementation, policy interpretation, and preparedness packages The recommendations will address research question 6: for the programme overall, how can the above learnings be used to inform future pandemic preparedness testing strategy for England? A key objective of this evaluation is to develop pragmatic solutions for future pandemic preparedness, including how learnings from the COVID-19 experience should shape future testing policy and implementation. In any future pandemic, targeted testing strategies should be informed by public health intelligence, including disease epidemiology, health system capacity and utilisation, and public engagement, taking into account for example risk factors for disease and the needs of vulnerable populations, to understand priorities, concerns, and reasons for reduced testing uptake. Decisions about testing strategies should be primarily based on a situational assessment of the intensity of transmission and the capacity of the health system to respond, as well as a cost–benefit analysis of each strategy. Therefore, our recommendations will consider the interplay among the various strands of this evaluation and be used to inform proposals for pandemic preparedness testing packages. Elements such as expedited diagnostics development or improvements in public engagement will inform rollout timings, efficiencies, and levels of testing coverage for the prospective modelling scenarios and will come with different costs. Pandemic preparedness testing packages, structured for low, medium, and high levels of preparedness, will be made easily accessible and modifiable through a ready-to-use dashboard. This dashboard will demonstrate the potential impact of various testing strategies across different settings, allowing for adjustment of other parameters such as testing type. Recommendations for diagnostics will be made with a view to ultimately ensuring the research and development and regulatory pipeline is fit for purpose and ready to anticipate diagnostic requirements for diseases of epidemic potential. #### Data management plan All individuals on the research team will have Baseline Personnel Security Standard clearance. All “official-sensitive” data will be accessed through official, UKHSA-approved secure portals and UKHSA secure laptops. All data to be used in the proposed evaluation will be obtained from UKHSA, allied UK government bodies such as NHS Digital and ONS, and independent research organisations. ## Supporting information Ethics approval letter [[supplements/281604_file02.pdf]](pending:yes) ## Data Availability All data produced in the present work are contained in the manuscript ## Funding statement This study was funded by Secretary of State for Health and Social Care acting as part of the Crown through the UK Health Security Agency (UKHSA), reference number C80260/PRO5331. ## Ethics statements This study protocol has been granted ethical approval by the UKHSA Research Ethics and Governance Group Ref NR0347. All relevant ethics guidelines have been followed. Should the research approach or methods change following this version, the UKHSA Research Ethics and Governance Group will be notified and the appropriate ethical guidelines followed. ## Competing interests statement All authors have completed the ICMJE uniform disclosure form at [www.icmje.org/coi\_disclosure.pdf](http://www.icmje.org/coi_disclosure.pdf) and declare: all authors had financial support from the UK Health Security Agency (UKHSA) for the submitted work; EY LLP London has previously received payment for consultancy work and advisory on the NHS Test & Trace response from the UK Department of Health and Social Care, now known as the UK Health Security Agency, Prof. Melinda C Mills has served as consultancy for Independent Scientific Pandemic Insights Group on Behaviours (SPI-B) and the ongoing COVID-19 enquiry for the United Kingdom government to the UK government; there are no other relationships or activities that could appear to have influenced the submitted work. ## Acknowledgements Scientific writing assistance and editorial support was provided by Adam Bodley and Richard Lewis, according to Good Publication Practice guidelines. ## Appendix a: EY-Oxford Health Analytics Consortium Membership View this table: [Table3](http://medrxiv.org/content/early/2022/10/27/2022.10.27.22281604/T3) ## Footnotes * ** The EY-Oxford Health Analytics Consortium membership list is attached in appendix a * Received October 27, 2022. * Revision received October 27, 2022. * Accepted October 27, 2022. * © 2022, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. 1.Centers for Disease Control and Prevention. Basics of COVID-19 Atlanta2021 [Available from: [https://www.cdc.gov/coronavirus/2019-ncov/your-health/about-covid-19/basics-covid-19.html](https://www.cdc.gov/coronavirus/2019-ncov/your-health/about-covid-19/basics-covid-19.html) accessed 14 September 2022. 2. 2.World Health Organization. WHO Director-General’s opening remarks at the media briefing on COVID-19 - 16 March 2020: World Health Organization; 2020 [Available from: [https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-1916-march-2020](https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-1916-march-2020) accessed 14 September 2022. 3. 3.Department of Health and Social Care. Government launches NHS Test and Trace service London: HM Government; 2020 [Available from: [https://www.gov.uk/government/news/government-launches-nhs-test-and-trace-service](https://www.gov.uk/government/news/government-launches-nhs-test-and-trace-service) accessed 14 September 2022. 4. 4.SAGE. Thirty-second SAGE meeting on Covid-19, 1 May 2020, 2020. 5. 5.UK Cabinet Office. COVID-19 Response: Living with COVID-19 London: HM Government; 2022 [updated 6 May 2022. Available from: [https://www.gov.uk/government/publications/covid-19-response-living-with-covid-19/covid-19-response-living-with-covid-19](https://www.gov.uk/government/publications/covid-19-response-living-with-covid-19/covid-19-response-living-with-covid-19) accessed 14 September 2022. 6. 6.Department of Health and Social Care. Test and trace in England – progress update. London, 2021. 7. 7.Mercer TR, Salit M. Testing at scale during the COVID-19 pandemic. Nat Rev Gen 2021;22(7):415–26. doi: 10.1038/s41576-021-00360-w [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41576-021-00360-w&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33948037&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F10%2F27%2F2022.10.27.22281604.atom) 8. 8.Di Bari M, Balzi D, Carreras G, et al. Extensive Testing May Reduce COVID-19 Mortality: A Lesson From Northern Italy. Front Med (Lausanne) 2020;7:402. doi: 10.3389/fmed.2020.00402 [published Online First: 2020/08/09] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3389/fmed.2020.00402&link_type=DOI) 9. 9.Rannan-Eliya RP, Wijemunige N, Gunawardana J, et al. Increased Intensity Of PCR Testing Reduced COVID-19 Transmission Within Countries During The First Pandemic Wave. Health Aff (Millwood) 2021;40(1):70–81. doi: 10.1377/hlthaff.2020.01409 [published Online First: 2020/12/03] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1377/hlthaff.2020.01409&link_type=DOI) 10. 10.Liang LL, Tseng CH, Ho HJ, et al. Covid-19 mortality is negatively associated with test number and government effectiveness. Sci Rep 2020;10(1):12567. doi: 10.1038/s41598-020-68862-x [published Online First: 2020/07/28] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41598-020-68862-x&link_type=DOI) 11. 11.Liverpool University. Liverpool Covid-SMART Community Testing Pilot. Liverpool: Liverpool University, 2021. 12. 12.Du Z, Pandey A, Bai Y, et al. Comparative cost-effectiveness of SARS-CoV-2 testing strategies in the USA: a modelling study. Lancet Public Health 2021;6(3):e184–e91. doi: [https://doi.org/10.1016/S2468-2667(21)00002-5](https://doi.org/10.1016/S2468-2667(21)00002-5) 13. 13.Iacobucci G, Coombes R. Covid-19: Government plans to spend £100bn on expanding testing to 10 million a day. BMJ 2020;370:m3520. doi: 10.1136/bmj.m3520 [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE4OiIzNzAvc2VwMDlfMTcvbTM1MjAiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8xMC8yNy8yMDIyLjEwLjI3LjIyMjgxNjA0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 14. 14.Thorneloe RJ, Clarke EN, Arden MA. Adherence to behaviours associated with the test, trace, and isolate system: an analysis using the theoretical domains framework. BMC Public Health 2022;22(1):567. doi: 10.1186/s12889-022-12815-8 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12889-022-12815-8&link_type=DOI) 15. 15.Public Health England. Disparities in the risk and outcomes of COVID-19. London: Public Health England, 2020. 16. 16.Jarrom D, Elston L, Washington J, et al. Effectiveness of tests to detect the presence of SARS-CoV-2 virus, and antibodies to SARS-CoV-2, to inform COVID-19 diagnosis: a rapid systematic review. BMJ Evid Based Med 2022;27(1):33–45. doi: 10.1136/bmjebm-2020-111511 [published Online First: 2020/10/03] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiZWJtZWQiO3M6NToicmVzaWQiO3M6NzoiMjcvMS8zMyI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzEwLzI3LzIwMjIuMTAuMjcuMjIyODE2MDQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 17. 17.Leber W, Lammel O, Siebenhofer A, et al. Comparing the diagnostic accuracy of point-of-care lateral flow antigen testing for SARS-CoV-2 with RT-PCR in primary care (REAP-2). EClinicalMedicine 2021;38:101011. doi: 10.1016/j.eclinm.2021.101011 [published Online First: 2021/07/20] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.eclinm.2021.101011&link_type=DOI) 18. 18.Petticrew M. When are complex interventions ‘complex’? When are simple interventions ‘simple’? Eur J Public Health 2011;21(4):397–98. doi 10.1093/eurpub/ckr084 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/eurpub/ckr084&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21771736&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F10%2F27%2F2022.10.27.22281604.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000293025800001&link_type=ISI) 19. 19.HM Treasury. Magenta Book Central Government guidance on evaluation. London: HM Treasury, 2020. 20. 20.Skivington K, Matthews L, Simpson SA, et al. Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update. Health Technol Assess 2021;25(57):1–132. doi: 10.3310/hta25570 [published Online First: 2021/10/01] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3310/hta25590&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F10%2F27%2F2022.10.27.22281604.atom) 21. 21.Skivington K, Matthews L, Simpson SA, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ 2021;374:2061. doi: 10.1136/bmj.n2061 [published Online First: 2021/10/02] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1136/bmj.n2061&link_type=DOI) 22. 22.Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015;350:h1258. doi: 10.1136/bmj.h1258 [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNTAvbWFyMTlfNi9oMTI1OCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzEwLzI3LzIwMjIuMTAuMjcuMjIyODE2MDQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 23. 23.Aspen Institute Roundtable on Community Change. Community Change: Theories, Practice, and Evidence. Washington D.C.: Aspen Institute 2006. 24. 24.De Silva MJ, Breuer E, Lee L, et al. Theory of Change: a theory-driven approach to enhance the Medical Research Council’s framework for complex interventions. Trials 2014;15:267. doi: 10.1186/1745-6215-15-267 [published Online First: 2014/07/06] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1745-6215-15-267&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24996765&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F10%2F27%2F2022.10.27.22281604.atom) 25. 25.Blamey A, Mackenzie M. Theories of Change and Realistic Evaluation:Peas in a Pod or Apples and Oranges? Evaluation 2007;13(4):439–55. doi: 10.1177/1356389007082129 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1356389007082129&link_type=DOI) 26. 26.Funnell SC, Rogers PJ. Purposeful Program Theory: Effective Use of Theories of Change and Logic Models: Wiley 2011. 27. 27.Frechtling JA. Logic Modeling Methods in Program Evaluation: Wiley 2007. 28. 28.King VJ, Stevens A, Nussbaumer-Streit B, et al. Paper 2: Performing rapid reviews. Syst Rev 2022;11(1):151. doi: 10.1186/s13643-022-02011-5 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s13643-022-02011-5&link_type=DOI) 29. 29.Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 2005;10 Suppl 1:6–20. doi: 10.1258/1355819054308576 [published Online First: 2005/08/02] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1258/1355819054308576&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16053580&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F10%2F27%2F2022.10.27.22281604.atom) 30. 30.West R, Michie S. A brief introduction to the COM-B Model of behaviour and the PRIME Theory of motivation. Qeios 2020 doi: 10.32388/WW04E6.2 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.32388/WW04E6.2&link_type=DOI) 31. 31.Silal SP, Groome MJ, Govender N, et al. Leveraging epidemiology as a decision support tool during the COVID-19 epidemic in South Africa. S Afr Med J 2022;112(5b):361–65. doi: 10.7196/SAMJ.2022.v112i5b.16061 [published Online First: 2022/07/06] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7196/SAMJ.2022.v112i5b.16061&link_type=DOI) 32. 32.Silal SP, Pulliam JRC, Meyer-Rath G, et al. The National COVID-19 Epi Model (NCEM): Estimating cases, admissions and deaths in South Africa. medRxiv 2022:2022.09.05.22279174. doi: 10.1101/2022.09.05.22279174 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMi4wOS4wNS4yMjI3OTE3NHYxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMTAvMjcvMjAyMi4xMC4yNy4yMjI4MTYwNC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 33. 33.Aguas R, White L, Hupert N, et al. Modelling the COVID-19 pandemic in context: an international participatory approach. BMJ Glob Health 2020;5(12) doi: 10.1136/bmjgh-2020-003126 [published Online First: 2020/12/29] [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTI6IjUvMTIvZTAwMzEyNiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzEwLzI3LzIwMjIuMTAuMjcuMjIyODE2MDQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 34. 34.Águas R, Mahdi A, Shretta R, et al. Potential health and economic impacts of dexamethasone treatment for patients with COVID-19. Nat Commun 2021;12(1):915. doi: 10.1038/s41467-021-21134-2 [published Online First: 2021/02/12] [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41467-021-21134-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F10%2F27%2F2022.10.27.22281604.atom)