Public Health and Social Measures (PHSM) interventions to control COVID-19 An Overview of Systematic Reviews

Review question

The proposed overview of systematic reviews is intended to synthesize the literature addressing the following:

• The effectiveness of single and combined PHSMs to reduce transmission of COVID-19

• The unintended desirable and undesirable impacts of the single and combined PHSMs addressing the COVID-19 pandemic on health outcomes and non-health outcomes including social, educational, and economic outcomes.

Eligibility criteria

• Studies designs: We will include systematic reviews of primary studies of any design (experimental, observational and modelling studies) providing information on the effectiveness or unintended consequences of PHSMs in response to the COVID-19 pandemic. We will exclude abstracts, meetings proceedings, editorials and commentaries and?primary studies, protocols and literature reviews. We will also exclude reviews that did not include a risk of bias assessment of some kind as this is intended to help restrict the pool to reviews that were more likely to be well conducted and to contribute data that could be interpreted appropriately.

  For duplicate publications, we will select the most recent or complete version. Given the lack of consensus definition for a systematic review, Cochrane’s guidance for overviews of reviews recommends setting pre-established criteria for making decisions around inclusion. Thus, for this study, we define a systematic review as a publication where the following are provided: 1) an explicit search in at least two databases (at least one being electronic); 2) detailed description of the methods with explicit selection criteria; and 3) a summary of the included studies either in narrative or quantitative format (such as a meta-analysis). Both Cochrane and non-Cochrane systematic reviews will be considered eligible for inclusion, with or without meta-analysis, and regardless of language restriction and methodology of the included primary studies. Rapid reviews will be eligible for inclusion in this overview if they meet our pre-defined inclusion criteria noted above.

• Population: We will not restrict to any specific population as long as the study is in the context of the COVID-19 pandemic. We will exclude studies where we cannot isolate the data relevant to COVID-19.

• Interventions: The review will focus on PHSMs implemented in response to the COVID-19 pandemic. We will use the logic model developed by the WHO suggesting that PHSMs have two main objectives (14): to reduce the number of transmission-relevant contacts and to make contacts that take place safer. Based on a mapping of existing taxonomies, this would lead to 7 categories of PHSMs:

  • - Surveillance (e.g. screening, testing, contact tracing)

  • - Response (quarantine and isolation)

  • - Services (school measures incl closures), business measures including closures, use of immunity/testing/vaccination certificates

  • - Social interactions (e.g. physical distancing, stay at home orders, restrictions of mass gatherings)

  • - Physical environment (e.g. ventilation, cleaning of objects and surfaces)

  • - Individual protection (e.g. hand washing, respiratory etiquette, use of face masks)

  • - Movement (e.g. travel restrictions including quarantine and testing and bans, restriction of domestic mobility)

We will include both single and multi-component interventions involving the eligible PHSM.

We will exclude reviews that only described PHSM interventions without reporting on any of the outcomes of interest as these cannot contribute to answering the research questions related to effectiveness and impact. We will also exclude reviews of studies that did not assess a specific PHSM or combination of PHSMs but rather looked at the extent of PHSMs overall, using the Oxford Scale. Additionally, we will exclude reviews that looked at indirect associations (for example, focusing on the impact of distance learning and digital media (which are a result of COVID-19 pandemic rather than PHSM) as well as reviews s. focusing on disinfectant interventions in clinical settings (versus public spaces)

Outcome of interest

We will include systematic reviews reporting on any of the below outcomes (categorized according to the conceptual framework developed by the University of Munich and WHO¹, derived from a review and mapping of existing PHSM taxonomies and literature (14):

Search methods for identification of reviews

The primary search source will be the Epistemonikos’ COVID-19 Living Overview of Evidence (LOVE) repository, a living repository that has been validated for use as a single source of COVID-19 evidence (15-17). The search will have no language restrictions. For the purpose of this overview of reviews, the search strategy will include specific keywords to represent the concept of ‘Public Health and Social Measures’ including the seven categories of PHSMs listed under ‘interventions’ (see eligibility criteria). The final list of terms was reviewed by a PHSM expert, and the final Boolean search strategy was reviewed by methods experts. The search strategy is available in Appendix 1.

Additionally, to detect potentially relevant systematic reviews in the COVID-10 LOVE repository, we will use a validated automatic classifier (a machine learning classifier for the records with an abstract) and a heuristic classifier for the records without an abstract (18). Finally, we will screen the reference lists of included reviews.

Methodological assessment of included reviews

Two researchers will assess independently and in duplicate the quality of the included reviews using the AMSTAR tool 2 which enables more detailed assessment of systematic reviews that include randomized or non-randomized studies of healthcare interventions, or both (Shea et al., 2020). We will rate adherence to each item as follows: yes, partial yes, no, or not applicable (e.g. when a meta-analysis was not conducted). The overall confidence in the results of the review will be rated as “critically low”, “low”, “moderate” or “high”, according to the AMSTAR 2 guidance based on seven critical domains.

We will consider reviews that scored “low” or “very low” as having major limitations. We will also use the assessment of methodological limitation to choose between overlapping reviews.

Currently there is no guidance by the GRADE working group addressing the overview of reviews. We have proposed to abstract the GRADE certainty of evidence assessment by the review, if reported. If we identify at least one SR that is of moderate or high quality, we will choose the one with the highest quality and apply GRADE to its findings.

As indicated earlier, we will exclude all systematic reviews that don’t include a risk of bias assessment of some kind.

Data extraction

Two researchers will extract data in duplicate and independently using standardized data abstraction form. They will resolve disagreements through consensus or with the help of a third reviewer if consensus could not be achieved. They will abstract data on the following variables from the included studies (to the extent that they have been provided by the included reviews):

• Type of synthesis document (e.g. standard versus rapid review)

• Basic information about systematic reviews (e.g. title; authors; year of publication; date last assessed as up-to-date; number of included studies; objectives; specific research question(s) (if explicitly provided by the authors), eligibility criteria, and study designs).

• Information on primary studies included in the systematic reviews: the range of primary study designs that the included reviews considered and identified in the review

• Context (countries/region, setting (e.g. healthcare settings, education facilities, etc.), phase of outbreak (if reported a priori in the methodology section), and population of interest at review level

• Interventions and comparators at review level (these will subsequently be categorized using the WHO logic model described earlier)

• Reported outcomes (as specified in Methods section of the systematic reviews).

• Key findings (including narratively reported study-level data and/or meta-analyzed data)

• Risk of bias assessment of included studies, if reported

• GRADE certainty of evidence assessment, if reported

• Whether risk of bias and GRADE certainty assessments were taken into account in interpreting the findings of the review

• Additional information (e.g. overview author’s comments, conclusion, systematic review limitations, reported disclosures of interest, funding source).

We will pilot-test data abstraction form prior to data abstraction.

Data synthesis

We will report results in tabular formats and narratively. We will summarize outcome data as they are reported in the included systematic reviews, for both narratively reported study-level data, as well as meta-analyzed data.

For each outcome category (as specified by the WHO logic model), we will assess whether the intervention had a “desirable effect, little or no effect, an uncertain effect (very low□certainty evidence), no included studies, an undesirable effect, not reported (i.e. not specified as a type of outcome that was considered by the review authors), or not relevant (i.e. no plausible mechanism by which the type of intervention could affect the type of outcome)”. We will present the result in an intervention□outcome matrix.

Managing overlapping systematic reviews

We will not exclude overlapping systematic reviews because, according to Cochrane’s guidance, it may be appropriate to include all relevant reviews’ results if the purpose of the overview is to present and describe the current body of evidence on a topic (9).

Analysis of subgroups or subsets

None planned