WICID framework Version 1.0: Criteria and considerations to guide evidence-informed decision-making on non-pharmacological interventions targeting COVID-19
===========================================================================================================================================================

* Jan M Stratil
* Maike Voss
* Laura Arnold

## 1 Abstract

**Introduction** Decision-making on matters of public health and health policy requires the balancing of numerous, often conflicting factors. However, a broad societal discourse and a participatory decision-making process on the criteria underpinning the decision was often not feasible within the time constraints imposed on by the SARS-CoV-2 pandemic. While evidence-to-decision frameworks are not able or intended to replace stakeholder participation, they can serve as a tool to approach relevancy and comprehensiveness of the criteria considered.

**Objective** To develop a decision-making framework adapted to the challenges of decision-making on national and sub-national level implementation of non-pharmacological interventions (NPIs) measures to contain the global SARS-CoV-2 pandemic.

**Methods** We employed the “best-fit” framework synthesis technique and used the WHO-INTEGRATE framework Version 1.0 as a starting point. In a first phase adapted the framework through brainstorming exercises and application to exemplary case studies (e.g. school reopening). In a second phase we conducted a content analysis of comprehensive strategy documents intended to guide policymakers on the phasing out of applied lockdown measures in Germany. Based on factors and criteria identified in this process, we adapted previous framework versions into the WICID (*W*HO-*I*NTEGRATE *C*OV*ID-*19) framework Version 1.0.

**Results** Twelve comprehensive strategy documents were included in the content analysis. The revised WICID framework consists of eleven criteria, supported by 48 aspects, the metacriterion *quality of evidence* and embraces a complexity and systems-perspective. The criteria cover implications for the health of individuals and populations due to and beyond COVID-19, infringement on liberties and fundamental human rights, acceptability and equity considerations, societal, environmental, and economic implications, as well as resource and feasibility considerations.

**Discussion** In a third phase, the proposed framework will be expanded through a comprehensive document analysis focusing on key-stakeholder groups across the society. The WICID framework can be a tool to support comprehensive evidence-informed decision-making processes.

**What is already known?**Ad-hoc Decision-making on matters of public health and health policy such as non-pharmaceutical interventions to contain the global SARS-CoV-2 pandemic, requires decision-makers to balance numerous and often conflicting criteria. Insufficient consideration of relevant factors reduces acceptance and can limit the effectiveness of the intervention.

**What are the new findings?**Based on a content-analysis of comprehensive strategy documents, we newly developed WICID framework provides of 11+1 criteria informed by 47 aspects which are intended to support decision-makers in the balancing act of identifying and considering criteria of relevance.

**What do the new findings imply?**The usage of the WICID evidence-to-decision framework can support decision-makers and expert committees in making more balanced decision, even if not all voices of relevant stakeholders could be included in the process due to time constraints imposed by the rapid progress of the pandemic.

Keywords
*   decision-making
*   priority setting
*   health technology assessment
*   guidelines
*   policy making
*   public health
*   Covid-19

## 3 Introduction

The response to the SARS-CoV-2 pandemic highlights the challenges of inherent evidence informed of public health and health policy decision-making1-4. These include: decision-making under time constraints, under uncertainty due to limited evidence, balancing numerous tradeoffs, and the challenge of ensuring fair decision-making processes under such circumstances.

Due to exponential growth in the number of infections, the issue of timing is crucial in a pandemic. A delay on the implementation of public health interventions (e.g. physical-distancing regulations) by days can have grave consequences5. Therefore, to receive timely evidence-informed guidance, many governmental institutions set up expert committees to support public health and health policy decision makers6. One challenge these expert groups faced is the qualitatively poor scientific evidence, with often questionable transferability and applicability to the context of decision-making and the lack of reliable evidence7,8. Not only due to the novelty of the pathogen, but also as assessing the effects of public health and health policy which often do not arise directly from the intervention, but from the system reaction to the introduction of the intervention within in it9 (e.g. school closures cause can lead to parents reducing working hours, which can lead to shortages in staff in the health care sector, limiting its ability to provide medical services and care).

However, even if strong scientific evidence was available, this in itself is insufficient to make sound recommendations, as evidence-informed decision-making is a deeply value-laden and often politized process10-12. Decision-makers must balance numerous and often conflicting normative and technical factors to come to a promising and acceptable decision13-15. This leads to the question: which criteria should be considered and how these should be weighed against each other?

The extent to which decisions are considered acceptable and legitimate depends on *how* the decisions were made (procedural considerations). Key considerations include e.g. transparency, inclusion of relevant stakeholders, and an appropriate composition of the panel10,16-21. Such approaches can increase the acceptability and perceived legitimacy of a decision22-24 even if – given varying and sometimes contradictory interests – no consensus regarding the *right* selection and weighting of criteria can be achieved22. According to the Accountability for Reasonableness (A4R) framework20, a key condition is ‘relevance’: the decision or recommendation must rest on evidence, reasons, and principles that all fair-minded parties can agree to be relevant and must meet the diverse needs of affected stakeholders20.

Involvement of representatives of all relevant stakeholder groups to identifying reasons and principles for a given decision-making process is considered ideal10,16,25,26. However, this ideal is often difficult if not impossible to meet under the time constraints imposed by the rapid progression of the SARS-CoV-2 pandemic. While not intended nor able to replace stakeholder participation, Evidence-to-Decision (EtD) frameworks are a way to support this balancing act27,28. EtD frameworks, which tend to comprise criteria and procedural guidance, are intended to ensure that all relevant factors are considered and the underlying rationale is made transparent29. When developed and applied well, these frameworks can help identify and integrate the criteria of relevance, even if not all voices of all stakeholders could be heard.

One of these frameworks is the WHO-INTEGRATE framework version 1.030. It was developed in a research project commissioned by the World Health Organization (WHO) for complex public health and health system interventions31. Based on a conceptual and normative foundation30 primarily based on WHO norms and values derived and public health ethics frameworks32-39, it was developed based on a comprehensive literature review of real-world decision criteria40, an assessment of complexity features30 as well as qualitative research across four continents41. As with most EtDs, the WHO-INTEGRATE framework is generic framework. an requires adaption to the specific intervention and context.

## 4 Objective

The aim of this research project is to adapt the WHO-Integrate framework to support the development of recommendations and decision-making for lifting restrictions through non-pharmacological interventions (NPIs) to address the global SARS-CoV-2 pandemic. For this, the framework addresses the national and sub-national level and takes a plurality of viewpoints of affected stakeholders into account. The WICID framework (*W*HO-*I*NTEGRATE *C*OV*ID*-19 adaption) is intended to reflect decision-making challenges and opportunities on matters of public health in relation to COVID-19 by embracing a complex systems perspective. However, with the WICID framework, we aim to ensure that the tool is sufficiently generic to be applicable to a wide range of NPIs, contexts, and decision topics.

Although procedural criteria, norms, principles, and processual considerations are crucial for achieving fair processes, this research project focuses on the substantive decision-making criteria.

## 5 Methods

The development of the WICID framework was conducted in three phases, following an approach analogous to the “best fit” framework synthesis and using an adapted version of the WHO-INTEGRATE framework as a starting point42. An extended version of the methods used in this research project is provided as supplement 1.

In **phase I**, we adapted the WHO-INTEGRATE framework through brainstorming exercises and applying it to case studies, in order to develop an analytical, generic tool (a *priori* framework). This was done through discussion within the research team, (ii) assessment of real-world decision-making criteria derived from an comprehensive overview-of-reviews40, and (iii) conducting a brainstorming exercise guided by the application of the framework on two case studies (reopening high schools and reopening small businesses such as book shops). This preliminary *a priori* framework was then imported into the software MAXQDA20 (verbi, Berlin), with the criteria and sub-criteria of the a priori framework being translated into codes of the coding frame to be used in phase II.

**Phase II**, consisted of a content analysis of a purposive sample of comprehensive strategy papers on lifting the lockdown measures in Germany (e.g. reopening schools, increased testing measures), which were coded against the *a priori* framework developed in phase I following technique of "best fit" framework synthesis42-44. “Best fit” framework synthesis begins by creating a framework of *a priori* themes and coding data extracted from documents (in this case: the comprehensive strategy documents) against that thematic or conceptual framework. A new framework is created by performing a thematic analysis on any data that cannot be accommodated within the *a priori* framework42.

We assumed, that selecting comprehensive strategy documents by expert commissions or expert groups would provide a broad, multi-perspective set of recommendations (in contrast to e.g. scientific publications or statements by individual groups, which do not to reflect a broad range of relevant perspectives in conccluding). We defined these as documents (a) intended to provide a comprehensive strategy or strategic suggestions for phasing out the lockdown measures (rather than providing information or pointing out individual aspects), (b) not exclusively or primarily focused on mitigating the health related- consequences of the SARS-CoV-2 pandemic but also including other societal, economic, or health outcomes, (c) addressing various NPIs and their interplay, and (d) focusing on multiple considerations to be reflected in this process. Position papers of stakeholder groups reacting or positioning themselves to a document, measure, or event without providing comprehensive strategy guidance were excluded (n=8) but will be considered in upcoming **phase III**, where the current version of the WICID framework will be expanded (see 7.2). The eligibility criteria and the rationale for their selection are provided as a supplement in the expanded methods section.

The search was conducted through multiple pathways: two researchers (JMS, LA) independently conducted (1) grey literature searches in the search engine GoogleTM and (2) on the websites of major newspaper outlets in Germany, (3) one researcher (JMS) searched the websites of the 16 German federal states, the national government and selected national government ministries, (4) we submitted freedom of information requests to the federal states’ governments, the national government, as well as selected national government ministries, and (5) contacted a sample of experts involved in public health decision-making or expert groups to provide us with strategy documents. As the documents are considered grey literature and mostly written in in German, we did not conduct a literature search in scientific data bases for these types of documents at this point of time.

The coding process of the included strategy documents was conducted by two authors (JMS, LA). Following the coding of one sample strategy documents to assess the need for adaption of the coding frame, one author (JMS) used the Software MAXQDA 20 (VERBI GmbH, Berlin) to code all identified strategy papers followed by a critical review by a second author (LA). The researcher applied the level 1 codes (referring to the *criteria* in the framework) and level 2 codes (referring to the *aspects* in the framework) of the coding frame to passages in the strategy document making references to criteria, considerations, or values covered within the codes. When the content of such a passage was perceived as not adequately covered by the coding frame, new level 2 or level 1 codes were created. Unclear passages were assigned the code TBD code for later review. The coding frame is provided as supplement 2.

After coding all selected strategy documents, two authors (JMS, LA) critically reflected on content saturation and dimensions of the framework insufficiently covered within the strategy documents and it was concluded that content saturation was reached.

Based on the coded passages, the researchers adapted the a priori framework by assessing the need to newly create, adapt, merge, separate, or reword *criteria* (reflecting level 1 codes) and aspects (reflecting level 2 codes).

In a final step, two researchers (JMS, LA) went through each of the coded passages to critically reflect on whether the criteria, considerations, or values contained within these passages was adequately covered within the newly adapted phase II framework.

## 6 Results

### 6.1 Development of preliminary adaption of the framework and a coding frame (phase I)

Following the process outlined in the methods section, we developed in phase I a preliminary, adapted *(a priori*) framework, which consisted of 8 *criteria* comprising 36 *aspects* and the metacriterion quality of evidence (an criterion to be applied across the other substantive criteria). This framework is provided as supplement 2. A preliminary version of this phase I framework as well as the exemplary case studies are part of a strategy document developed by two of the authors, among other experts, for the federal ministry of interior and the federal chancellery45.

### 6.2 Documents identified and included for analysis for phase II

We identified 12 comprehensive strategy papers in total. Four strategy papers, developed by expert groups for federal states, the national government, or ministries of the national government and two strategy documents developed by the corona expert commission of the federal state government of North Rhine-Westphalia, were publicly available46. Two strategy documents were developed by an informal expert group for the German Federal Ministry of the Interior, one of which was publicly available through a press report the other was provided to us through the German Network Public Health COVID-19 *(Kompetenznetz Public Health on COVID-19*)45,47.

We identified eight additional comprehensive strategy documents, which were not directly commissioned by governmental institutions. Four of which were published by the National Academy of Sciences Leopoldina48-50, one by a political party in the federal city-state of Hamburg51, one by a research institute commissioned by the political foundation Hans-Böckler-Stiftung52, one by the Boston Consulting Group53, and one developed by researchers from diverse institutions under the coordination of two researchers with affiliation at the University of Wuerzburg and the IFO institut54.

### 6.3 The adapted WICID framework

The adapted WICID framework consists of 11 substantive decision-making criteria, containing 48 decision-making aspects, and the meta-criterion quality of evidence, to be applicable across all criteria and aspects (outer circle, Figure 1; table 1 and 2). Depending on the measure, the criteria and aspects are intended to be applied on and reflected for different population groups (center-most circle, figure 1). Depending on the measure and type of decision-making process, the decision-makers are intended to deliberate on the criteria and aspects taking one or multiple different perspectives (inner circle, figure 1). Analogous to the WHO-INTEGRATE framework, it aims to accommodate different features of complexity: depending on impact the measure is assumed to have on the system it is implemented in, direct (those caused by the intervention) and indirect (those resulting from the system reactively changing due to the intervention) effects should be taken into account, as well as local, regional, national and even global implications. At the same time both, the immediate and the short, medium- and long-term implications should be considered (figure 2). It is intended to guide the systematic reflection of the intervention in its context.

![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/08/13/2020.07.03.20145755/F1.medium.gif)

[Figure 1:](http://medrxiv.org/content/early/2020/08/13/2020.07.03.20145755/F1)

Figure 1: 
WICID Framework version 1.0

The color of the 11+1 criteria of the WICID refers so their grouping and relation to the criteria of the WHO-INTEGRATE framework they are derived from. The center-most circle describes population groups onto which the criteria and aspects should be applied to. The inner most circle describes the perspective the decision makers can take onto criteria and populations.

![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/08/13/2020.07.03.20145755/F2.medium.gif)

[Figure 2:](http://medrxiv.org/content/early/2020/08/13/2020.07.03.20145755/F2)

Figure 2: 
Complex system perspective on the implications caused by a measure being introduced into a system.

The intervention (grey circle on the left) is introduced as an “event” to a system. It directly effects other components of the system (non-grey) circles on the right) which again interact with other components of the system, causing a chain-reaction of the system reacting and adapting to the event; leading to the societal, economic, and health-related consequences of the intervention.

#### 6.3.1 Criteria and aspects of the WICID framework

The eleven criteria in the WICID framework consist of: three criteria focused on the balance of health benefits and harms: (I-III, in figure 1: light red), two criteria focus on the accordance with human rights principles and socio-cultural acceptability (IV-V, beige), one criterion focuses on equity, equality, and non-discrimination (VI, green), two criteria focus on the societal implications (VII-VIII, light yellow), and three criteria focus on feasibility (IX-X, blue) and health systems considerations (XI, light blue). Table 1 describes the 11+1 criteria in detail; with table 2 containing the 48 associated aspects in their concise formulation. Supplement 3 contains a more comprehensive version of the framework, providing more details and examples to guide users of the framework. The supplementary 4 contains exemplary passages from the coded strategy documents for criteria and aspects.

View this table:
[Table 1:](http://medrxiv.org/content/early/2020/08/13/2020.07.03.20145755/T1)

Table 1: 
Criteria of the WICID framework and description of what the criteria are intended to cover

View this table:
[Table 2:](http://medrxiv.org/content/early/2020/08/13/2020.07.03.20145755/T2)

Table 2: 
Concise version of the criteria and aspects of the WICID framework

#### 6.3.2 Considering criteria for different populations

Depending on the measure and decision-making context, criteria and aspects within the framework should be considered for the population as a whole, as well as for different population groups to assume relevant implications adequately for these groups. For example, an intervention such as regulation forcing people not to leave their houses can have adverse effects which disproportionate affect people affected by insecure housing circumstances or school closures disproportionately placing a burden on working mothers. The stakeholder groups to be considered will depend on the type of measure (e.g. closing schools vs. closing nursing homes to the public). Building on the WHO-INTEGRATE framework and the strategy documents, we suggest to consider the implications for (figure 2, center-most circle): (a) the general population, (b) those intended to benefit from the intervention (e.g. young school children in the case of school reopening), (c) those intended to implement the measures (e.g. teachers), and populations with a high risk profile (senior citizens with preexisting conditions) and (f) other affected stakeholder groups (e.g. employers). Within these population groups, further disaggregation based on relevant social characteristics with an emphasis on vulnerable and marginalized populations should be conducted (e.g. school children with a family with a low socio-economic status). These relevant characteristics are likely to vary depending on context, although socioeconomic status, age, and gender are likely to be important across context. The PROGRESS Plus framework can provide guidance on the identification of relevant characteristics56.

#### 6.3.3 Employing multiply perspectives: The population, the individual and the health system

Depending on the measure and decision-making process, decision-makers need to reflect on the different criteria from different perspectives to inform their deliberations (inner ring in figure 1). For example, the health implications of a measure on the SARS-CoV-2 related health risk (e.g. reopening of schools) can be approached from a population perspective (looking at the implications of the intervention for the population as a whole, which can take place on a local or community level, federal state level and/or national level; e.g. incidence rate of infections and associated mortality rates), a systems perspective (reflecting the intervention from the perspective of the health system, as well on a local, regional, and national level, e.g. taking the implications for the capacity of the health system and the availability of resources into account), or the individual perspective (taking the perspective of an individual affected by the population; e.g. the risk for individual teachers working in these schools).

#### 6.3.4 Taking a complex systems perspective

The implications of the NPIs can reach far. Therefore, the WICID framework embraces a complexity perspective2: the measures are regarded as “events in a system”57, with the (intended and unintended) effects resulting from the interaction of the measures’ components with each other and components of the larger system. Within this perspective, disentangling the effects *caused by* the measure itself from the interplay of context and measure can be challenging if not impossible to do9; posing challenges for transferability and generalizability of evidence.

Analogous to the ripple effects caused by dropping an object in a pool of water, the effects initiated by the introduction of the measure to the system can lead to a chain reaction that can be followed outwards incrementally (figure 2). The more profound the impact of the measure to the system, the further way the effects of the measure throughout the system can be observed. For example, the shutdown of a few companies in a region for a short time can lead to locally felt adverse economic consequences. However, a marginally more impactful event of closing the same companies for a marginally longer time and thereby exceeding an economic threshold can lead to the insolvency of these companies, causing – depending on the companies – a disruption of globalized production chains with economic consequences that can have regional, national, and even global effects. Therefore, depending on how profound the impact of the measure is assumed, decision makers need to consider whether the measure is likely to lead not only to immediate and local, but also regional, national, or global consequences over the short, medium, and long term. When reflecting on the measures, a focus should not only be on the direct effects along the intended causal pathway, but should also anticipate implications caused across several degrees of indirectness. These different dimensions are not necessarily related: for example, direct and indirect health-related, societal, and economic implication occurring from immediate to long term could arise exclusively on a local level (figure 2).

## 7 Discussion

### 7.1 Summary of findings

We adapted the WHO-INTEGRATE framework to decision-making processes on non-pharmaceutical interventions intended to suppress or mitigate the effects of the SARS-CoV 2 pandemic. We used brain storming exercises and content analysis of comprehensive strategy papers on the phasing out the implemented lockdown measures in Germany. The resulting WICID framework version 1.0 consists of 11+1 substantive decision-making criteria, containing 48 decision-making aspects. Depending on the needs of the decision-making processes, these are intended to be applied by policymakers for and with different affected stakeholder groups using a multi-perspective approach. In line with the underlying complexity perspective, rather than only focusing on direct, immediate, and local effects of the measure, the ripple effects caused by the introduction of the measure to a given system or policy field should be followed to adequately consider the implications a measure might have.

### 7.2 Advancing the WICID framework version 1.0 in phase III

The current version of the WICID framework version 1.0 will be expanded in a third phase. The limited diversity of expert groups established to inform policymakers on the handling of the SARS-CoV-2 pandemic has faced some criticism6. Our approach of adapting the WICID framework based on strategy documents therefore comes with the risk that relevant criteria were overlooked due to the limited selection of expert groups and the stakeholder groups (not) represented within them. The third phase of the research project aims to address this issue by including the perspectives of various stakeholder groups across the society and expanding the WICID framework version 1.0 with considerations not adequately covered previously.

In phase III we will conduct a content analysis of key-documents representing the opinions and perspectives of stakeholder representatives across the society (i.a. of affected populations, non-governmental organizations, private sector) and using the results to validate and – where needed – expand the framework version 1.0. Using a sample of NPIs with broad societal implication as a starting point (closure/reopening of schools, closure/reopening of businesses, and “shelter-in-place” regulations), we will include opinion pieces, position papers, or press statements aimed at informing political decision-making on these measures. A first set of stakeholder group clusters (e.g. social and welfare organizations) will be selected based on an initial brainstorming phase and stakeholder mapping58 and expanded in an iterative snowballing process. While it will be not feasible to cover all relevant organizations within a given cluster, we will analyze of a heterogeneous sample which will be expanded in an iterative process based on the assessment of saturation. While this approach is not able to capture all voices of affected stakeholders, it allows for a broad and representation of societal values in decision-making across the.

### 7.3 Short guidance on how to apply the WICID Framework

While some evidence-to-decision (EtD) frameworks are tailored to specific decision-making processes (e.g. on vaccination policies59) and provide a fixed set of “ready-to-use” decision-making criteria26, others are more generic and require some form of adaption. The WICID framework is intended to be adequately generic to be applicable across a broad range of NPIs and decision-making contexts. While the 11+1 criteria can be used as a “ready-to-use” evidence-to-decision framework, we believe the framework to be most useful as a guide to systematically reflect on NPIs as well as their interdependencies and adapting them based on the specific needs of the decision-making process, using the WICID framework as guidance.

First, (I) a comprehensive logic model60,61 or systems map of the measure and the context is intended to be implemented in should be created, in order to describe possible implications. Next, (II) the WICID framework should be used to expand on dimensions not adequately covered (e.g. by exploring the causal pathways from different perspectives, assessing the implications for different affected population groups, or using the criteria and aspects to assess its comprehensiveness). Informed by the logic model, (III) an identification of relevant stakeholders should be conducted, ideally in the form of a comprehensive or focused stakeholder mapping58. Next, (IVa) those involved in the decision-making process need to define criteria which are assumed to be of relevance for deliberating on the measure. This can be done e.g. by selecting individual aspects from within each criterion and adapting them to the context at hand. Using the example of school reopening, this could include the risk of outbreaks, health implications for teachers, for students, and for family members, the implications for the wellbeing of these groups, educational implications etc. Given the complexity of the decisions at hand, it is likely not all factors of relevance can be covered in depth. On the one hand, this reflects the reality of the decision-making process; on the other hand, the rationale for the selection should always be provided. (IVb) The assumed importance of the criteria should be rated (e.g. on a 1-5 scale from “less important” to “critical”). (V) Efforts should be made to receive feedback on the expanded logic model and the selected criteria from key stakeholder groups identified in the mapping. Repeated rounds of step I-IV are likely to produce the best results. Next (VI), efforts should be made to acquire appropriate sources of evidence to inform on the selected criteria (e.g. by commissioning research or inviting experts’ judgments). (VII) The retrieved evidence for each criterion should be summarized and presented alongside the assessment the quality of the evidence and transferability to the context at hand. The group of decision-makers are now asked (VIII) to engage in the deliberation to balance the criteria against each other, taking their weight, direction, quality, and transferability of the evidence into account. After all, the final judgment and the underlying rationale should be made transparent and public.

### 7.4 Relation to public health ethics framework

Various public health ethics frameworks providing guidance on principles and values to consider in public health and health policy decision-making have been published39,62-64, some of which are more generall33,34 37,65,66, while others focused on public health emergencies and pandemics67-71. Building on these foundations, institutions such as the German ethics council72 or the German Network Public Health COVID-1973,74 have outlined relevant values and principles for decision-making in the current public health crisis. These include the duty to provide care, health, non-discrimination, security, equity, individual liberty, privacy, proportionality, protection of the public from harm, reciprocity, and solidarity, among others73.

The WICID framework was developed to be in line with these documents; primarily due to the underlying WHO-INTEGRATE framework being developed with a foundation in WHO norms and values and key public health ethics frameworks30. The WICID framework aims to translate these principles and values into criteria applicable for real-world decision making processes in the pandemic (e.g. by translating the general moral considerations of producing benefits and avoiding, preventing, and removing harms33 into – among others – the criteria I,II,III,VII, and VIII on the different positive and negatives social, economic, or health-related effects a NPI might have for individuals and populations). Furthermore, the framework aims to place criteria derived from these values and principles alongside other factors of relevance for real- world decision-making often not covered in depth in public health ethics frameworks, such as considerations of feasibility or the wider implications for the (health) system. While some values and principles have a direct representation in the framework criteria (e.g. individual liberty, privacy, or proportionality) others are introduced on the level of the perspectives or the populations the criteria should be applied to (e.g. reciprocity being reflected in the consideration of *those intended to implement the interventions)*.

### 7.5 Need for fair and transparent processes

The WICID framework is in itself insufficient to achieve fair decision-making processes with results considered acceptable and legitimate20,21. It is important that not only the final decision, but also the underlying rationale, including the criteria and evidence, is made transparent. Other important values and principles underlying fair decision-making processes include accountability, inclusiveness, openness and transparency, reasonableness, and responsiveness73,74. While a comprehensive approach of stakeholder engagement is likely not possible within a pandemic, efforts to including the voices of affected stakeholders, for example through rapid reaction statements from stakeholder representation organizations, can not only improve acceptability and legitimacy but also lead to better outcomes. Especially, the needs of affected marginalized and vulnerable population groups without strong political capital (e.g. people affected by homelessness) are at risk of being overlooked. Furthermore, special attention needs to be placed to the composition of the stakeholder groups6. Documenting the process and the decisions made and providing – to some extend – access to these documents to the public can increase social acceptance for public health measures as well as lifting them. As the main focus within this research project lay on substantive criteria of WICID the framework and does *not* comprise procedural criteria, other procedural frameworks e.g. from the field of public health ethics can serve as guidance.

### 7.6 Strengths and Limitations

The WICID framework was developed by building on the WHO-INTEGRATE framework30 which was developed as a principles-based approach to ensure a solid, comprehensive normative foundation. It is also based on previous research such as the result of an overview of systematic reviews on public health and health system decision-making criteria40, and expanding this foundational framework through a broad set of comprehensive strategy documents informing decision-making processes in Germany. Application in other country contexts therefore need to be tested and the framework, if necessary, updated and revised.

The WICID framework was developed using strategy documents intended to inform the German government. While we believe the resulting WICID framework can prove useful and applicable to other regions within and outside Europe, the need for adapting to the respective decision-making contexts is necessary. Likely, the basis of the WHO-INTEGRATE framework, which was developed not only for the global level at the WHO, but also to be applicable on national and sub-national levels throughout the world, can cover factors not adequately captured in the German strategy documents.

The framework in its current version 1.0 was developed based primarily on comprehensive strategy documents developed by expert groups. The composition and the intention of the expert groups is likely to have influenced the criteria, consideration, values, and principles covered within them. While we believe possible shortcomings and blind spots are in part compensated using the WHO-INTEGRATE framework as a basis, there is a risk of relevant factors being missed. We aim to address this issue in phase III of this research project.

Despite multiple approaches to identify the comprehensive strategy documents, we acknowledge the possibility of having missed on individual statements. Furthermore, likely other relevant strategy documents exist, but were not disclosed by the governments or leaked through to the public and therefore are not captured in our analysis. An update of the searches will be conducted as part of phase III.

Another limitation is the distinction between comprehensive strategy documents and position papers by affected stakeholder groups, which was not always a clear cut. However, we believe this only to be a minor limitation, as all borderline documents were retained and will be included in the phase III of the project.

## 8 Conclusion

The WICID framework represents a comprehensive COVID-19 focused EtD framework intended to guide policy and public-health decision-makers on making decisions on NPIs. It is rooted in WHO norms and values, criteria and considerations used to inform decision-making, and a complex systems perspective. While adapted to COVID-19 related challenges, it is intended to be generic in a way to be applicable across a broad range of decisions-making processes, contexts, and on a diverse set of measures. It can be a useful tool for those involved in the difficult task of making decisions on NPIs by systematizing the decision-making process, making the underlying rationale more transparent and contributing to the relevance of the decision criteria.

## Data Availability

The included comprehensive strategy documents are - for the most part - publicly available. Those that are not, can be provided by the authors uppon request

## Author’s contribution

JMS and LA conceived the study. JMS, EAR, MV and LA engaged in the activities of phase I to develop the a priori framework. JMS and LA conducted the literature search for strategy documents, with support from MV. JMS and LA conducted the coding and analysis of the strategy documents and revised the a priori framework. JMS drafted the manuscript with support from LA and MV.

## Funding

No external funding source was used to fund the project.

## Competing interests

JMS is authors of the WHO-INTEGRATE framework. JMS and MV were part of an expert group which developed strategy documents intended to inform the Covid-19 crisis taskforce of the German government.

## Patient consent

Not required.

## Ethics approval

The research will be undertaken in accordance with the declaration of Helsinki in their respective current versions. As it is a document analysis, no review by an ethics committee was deemed necessary.

## 10 Annex

### Supplements 1 – Expanded methods description

The development of the WICID framework was conducted in three phases, following an approach analogous to the “best fit” framework synthesis and using an adapted version of the WHO-INTEGRATE framework as a starting point42. In phase I, we adapted the framework through brainstorming exercises and applying it to case studies, in order to develop an analytical, generic tool *(a priori* framework). Phase II, consisted of a content analysis of a purposive sample of comprehensive strategy papers on lifting the lockdown measures in Germany (e.g. reopening schools, increased testing measures), which were coded against the *a priori* framework and based on which the WICID framework version 1.0 was created. In a next step (phase III), which is yet to be completed, we will advance the WICID framework version 1.0, by integrating the perspectives of a diverse set of affected stakeholders across society. Phase III allows for an assessment of the comprehensiveness of the framework criteria and the integration of factors insufficiently covered in the expert-based strategy documents.

#### 10.1 Phase 1: Development of a preliminary adaption of the framework and of a coding frame

In phase I, we (i) discussed the WHO-INTEGRATE framework within the research team, (ii) assessed real-world decision-making criteria derived from an comprehensive overview-of-reviews40, and (iii) conducted a brainstorming exercise guided by the application of the WHO INTEGRATE Framework version 1.0 on two case studies (reopening high schools and reopening small businesses such as book shops).

This preliminary *a priori* framework was then imported into the software MAXQDA20 (verbi, Berlin), with the criteria and sub-criteria of the a priori framework being translated into codes of the coding frame to be used in phase II. In this process, we added two additional codes: (1) “evidence” – to cover considerations regarding information and evidence considered necessary of lacking in the decision-making process, and (2) the code “TBD” *(to be discussed)*, which was meant mark unclear passages for later review and discussion by the research team.

#### 10.2 Phase 2: Coding of strategy documents, best-fit framework synthesis, and development of the WICID framework

We used the technique of "best fit" framework synthesis42-44, which offers a method to build on an existing framework that is considered as relevant for the given circumstances, but potentially different in relevant areas (such as the WHO-INTEGRATE framework for decision-making on COVID-19 related decision-making). “Best fit” framework synthesis begins by creating a framework of *a priori* themes and coding data extracted from documents (in this case: the comprehensive strategy documents) against that thematic or conceptual framework (see 8.2.1). A new framework is created by performing a thematic analysis on any data that cannot be accommodated within the *a priori* framework. We used the adapted version of the WHO-INTEGRATE framework created in phase I, and translated this into the coding frame we used coded the strategy documents against (provided in as supplement 2)42.

##### 10.2.1 The strategy documents and the rationale behind their selection

We assumed, that selecting comprehensive strategy documents by expert commissions or expert groups would provide a broad, multi-perspective set of recommendations (in contrast to e.g. scientific publications or statements by individual groups, which do not claim nor intend to reflect multiple relevant perspectives in concluding).

We defined these as documents (a) intended to provide a comprehensive strategy or strategic suggestions for phasing out the lockdown measures (rather than providing information or pointing out individual aspects), (b) not exclusively or primarily focused on mitigating the health related-consequences of the SARS-CoV-2 pandemic but also including other societal, economic, or health outcomes, (c) addressing various NPIs and their interplay (e.g. not exclusively focusing on testing), and (d) focusing on multiple considerations to be reflected in this process (i.e. not exclusively focused on health impact). Position papers of stakeholder groups reacting or positioning themselves to a document, measure, or event without providing comprehensive strategy guidance were excluded (n=8) but will be considered in phase III.

While the main discourse on implementing lockdown measures in Germany was focused on suppressing the spread of the outbreak and on averting a collapse of the health care system, the debate on the controlled phasing out of the implemented lockdown measures was more nuanced: focusing on the challenge to balance the implications of the measures e.g. on health, society, or the economy. Therefore, we concentrated on strategy documents that focused on the latter.

To include strategy documents with impact on real world decision-making, we selected papers developed by expert groups or task forces commissioned by German policymakers on two levels (federal and national governments as well as ministries on these levels). We expanded the sample to comprehensive strategy documents on the exit from the lockdown measures developed by non-government affiliated expert groups intended to inform political decision-making, but not directly commissioned by governmental bodies.

In order to ensure that the strategy documents addressed a broad range of societal and economic implications beyond the health sector, we focused on strategy documents which – directly or indirectly – addressed one of the following NPIs: (a) closure and reopening of schools, (b) closure and reopening of businesses, and (c) “shelter-in-place” regulations. We excluded strategy papers that were not or only marginally concerned with these three NPIs. Strategy papers primarily concerned with hygiene measures or the testing capacity were excluded as well.

Table S1 in the annex displays the inclusion and exclusion criteria for phase II. All identified borderline cases were retained to be analyzed in phase III.

View this table:
[Table S1:](http://medrxiv.org/content/early/2020/08/13/2020.07.03.20145755/T3)

Table S1: 
Inclusion and exclusion criteria for the search of eligible documents for phase II

#### 10.2.2 Identification and selection of strategy papers

The search was conducted through multiple approaches channels:

*   Two researchers (JMS, LA) independently searched in the search engine Google™ with various versions of keyword combinations of the terms and synonyms of “strategy” or “expert commission” and “COVID-19” in German.

*   Two researchers (JMS, LA) independently searched the websites of major newspaper outlets in Germany (including: Die Zeit, Frankfurter Allgemeine Zeitung, Die Welt, Deutschlandfunk, Der Spiegel) outlets using the website’s search engine with similar keywords

*   One researcher (JMS) searched the websites of the 16 German federal states, the national government and selected national government ministries, focusing on the section of press releases.

*   We contacted a sample of experts involved in public health decision-making or expert groups to provide us with strategy documents; either directly or through the platform of the interdisciplinary *Kompetenznetz Public Health COVID-19 (Competence Network Public Health COVID-19, [www.public-health-covid19.de](http://www.public-health-covid19.de)*).

*   We posed freedom of information requests to the federal states’ governments, the national government, as well as selected national government ministries to provide us with strategy documents developed by expert groups, if available and publicly accessible.

As the documents are considered grey literature and mostly written in in German, we did not conduct a literature search in scientific data bases for these types of documents at this point of time.

#### 10.2.3 Coding of documents against the *a priori* framework

The coding process was conducted by two authors (JMS, LA; personal characteristics in line with the COREQ-checklist75 are reported in table S2). First, one sample strategy document was coded by the two authors to assess the need to adapt the preliminary coding frame and to develop a coding guidance document, outlining when a specific code should be used and which text passages to code. The coding frame (translated from German) is provided as a supplement (supplement 2). Based on this adapted frame, one author (JMS) used the Software MAXQDA 20 (VERBI GmbH, Berlin) to code all identified strategy papers. Afterwards, all coded documents were critically reviewed by a second author (LA) who highlighted conflicts (passages where this reviewer perceived a code was missing, suggested not to code, or to code differently). Conflicts were solved through discussion between the authors.

In the subsequent process, the researchers applied the level 1 codes (referring to the *criteria* in the framework) and level 2 codes (referring to the *aspects* in the framework) of the coding frame to passages in the strategy document making references to criteria, considerations, or values covered within the codes. Level 2 codes *(aspects)* are meant to describe considerations (e.g. factors, values, norms) contained within the level 1 codes *(criteria)* and support the user in the understanding, interpretation, and application of the criteria. When the content of such a passage was perceived as not adequately covered by the coding frame, new level 2 or level 1 codes were created. When the researchers identified passages containing references to criteria, considerations, or values of relevance which were assumed to be covered by a specific level 1 or level 2 code, but which seemed to expand on this or provide details or nuances (e.g. a passage on the implications of an measure risking the insolvency of small enterprises within the code *economic implications)*, the researchers took note of these passages for later review. Unclear passages were assigned the code TBD code for later review.

After coding all selected strategy documents, both authors critically reflected on content saturation and dimensions of the framework insufficiently covered within the strategy documents. As most criteria of the preliminary framework were adequately covered and due to an overlap of consideration across documents, a content saturation was reached.

View this table:
[Table S2:](http://medrxiv.org/content/early/2020/08/13/2020.07.03.20145755/T4)

Table S2: 
g characteristics of researchers involved in the content analysis

#### 10.2.4 Adapting the preliminary coding frame

Next, one researcher (JMS) conducted a thematic analysis of the passages assigned to the newly created codes as well as those passages noted down for expanding on or providing nuance within existing codes. A draft of an adapted phase II framework was created by reflecting on whether there was a need to adapt the *a priori* framework to cover the content in the coded passages. This included whether: (a) *criteria* (reflecting level 1 codes) should be created, (b) new aspects (reflecting level 2 codes) should be added to the framework, (c) new or preexisting *criteria* or *aspects* should be merged or separated, or (e) moved to another position within the framework, and (f) wording of the *criteria* or *aspects* needs to be adapted.

This newly developed draft of the adapted framework was critically reviewed by a second researcher (LA). Afterwards, the results of steps (a-f) were critically discussed within the research team (JMS, LA) to solve conflicts and revise the adapted phase II framework accordingly.

Afterwards, two researchers jointly reviewed all passages noted down for later review (code TBD) and assessed the need to add *criteria* or *aspects*, as well as the need for revision, rewording or repositioning within the framework.

In a final step, two researchers (JMS, LA) went through each of the coded passages to critically reflect on whether the criteria, considerations, or values contained within these passages was adequately covered within the newly adapted phase II framework.

### Supplement 2: *a priori* framework adapted in phase I and coding frame

View this table:
[Table S3:](http://medrxiv.org/content/early/2020/08/13/2020.07.03.20145755/T5)

Table S3: 
a priori framework adapted in phase I and coding frame

The right column presents the preliminary adapted framework of the WHO-INTEGRATE framework, which was developed in phase I. It was used as the a priori framework for the “best fit” framework synthesis of phase II and translated into a coding frame (left two columns)

### Supplement 3: WICID Framework – comprehensive Version 1.0

View this table:
[Table S4:](http://medrxiv.org/content/early/2020/08/13/2020.07.03.20145755/T6)

Table S4: 
Criteria and aspects within (comprehensive form).

### 10.3 Supplement 4: Table with exemplary passages for the criteria and aspects of the WICID framework

View this table:
[Table S5:](http://medrxiv.org/content/early/2020/08/13/2020.07.03.20145755/T7)

Table S5: 
Criteria and aspects of the WICID framework with exemplary codes from the trategy documents

## Acknowledgements

We would like to thank Prof. Eva Rehfuess for her comments and support in the development of the phase I framework. We furthermore would like to thank Prof. Rob Baltussen, Prof. Georg Marckmann, Karin Geffert, Dr. Lisa Pfadenhauer, Dr. Kerstin Sell, and Prof. Hajo Zeeb for taking the time to critically reviewing and commenting on the manuscript and framework, despite the time constraints the current pandemic imposes on them.

## Footnotes

*   We received friendly peer review feedback from five experts in the field. Based on this feedback, we made a number of changes to the wording of the manuscript and minor changes to the framework (e.g. merging aspects). We furthermore reduced the wordcount in order to meet the word limitations of the journal we intend to submit the manuscript to.

*   Received July 3, 2020.
*   Revision received August 12, 2020.
*   Accepted August 12, 2020.


*   © 2020, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/)

## 9 References

1.  1.Sanderson I. Intelligent Policy Making for a Complex World: Pragmatism, Evidence and Learning. Political Studies 2009;57(4):699–719. doi: 10.1111/j.1467-9248.2009.00791.x
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1467-9248.2009.00791.x&link_type=DOI) 

2.  2.Petticrew M, Knai C, Thomas J, et al. Implications of a complexity perspective for systematic reviews and guideline development in health decision making. BMJ Glob Health 2019;4(Suppl 1):e000899. doi: 10.1136/bmjgh-2018-000899 [published Online First: 2019/02/19]
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTc6IjQvU3VwcGxfMS9lMDAwODk5IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjAvMDgvMTMvMjAyMC4wNy4wMy4yMDE0NTc1NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

3.  3.Rutter H, Savona N, Glonti K, et al. The need for a complex systems model of evidence for public health. The Lancet 2017;390(10112):2602–04. doi: 10.1016/s0140-6736(17)31267-9
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/s0140-6736(17)31267-9&link_type=DOI) 

4.  4.Lancaster K, Rhodes T, Rosengarten M. Making evidence and policy in public health emergencies: lessons from COVID-19 for adaptive evidence-making and intervention. Evidence & Policy: A Journal of Research, Debate and Practice 2020 doi: 10.1332/174426420X15913559981103
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1332/174426420X15913559981103&link_type=DOI) 

5.  5.Pei S, Kandula S, Shaman J. Differential Effects of Intervention Timing on COVID-19 Spread in the United States. *medRxiv* 2020:2020.05.15.20103655. doi: 10.1101/2020.05.15.20103655
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4wNS4xNS4yMDEwMzY1NXYyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjAvMDgvMTMvMjAyMC4wNy4wMy4yMDE0NTc1NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

6.  6.Rajan D, Koch K, Rohrer K, et al. Governance of the Covid-19 response: a call for more inclusive and transparent decision-making. BMJ Global Health 2020;5(5):e002655. doi: 10.1136/bmjgh-2020-002655
    
    [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTE6IjUvNS9lMDAyNjU1IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjAvMDgvMTMvMjAyMC4wNy4wMy4yMDE0NTc1NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

7.  7.Ioannidis JP. A fiasco in the making? As the coronavirus pandemic takes hold, we are making decisions without reliable data. STAT 2020
    
    

8.  8.Ioannidis JPA. Coronavirus disease 2019: The harms of exaggerated information and non-evidence-based measures. European Journal of Clinical Investigation 2020;50(4):e13222. doi: 10.1111/eci.13222
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/eci.13222&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32191341&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

9.  9.Rhodes T, Lancaster K. Evidence-making interventions in health: A conceptual framing. Social Science & Medicine 2019;238:112488. doi: [https://doi.org/10.1016/j.socscimed.2019.112488](https://doi.org/10.1016/j.socscimed.2019.112488)
    
    

10. 10.Baltussen R, Jansen MP, Mikkelsen E, et al. Priority Setting for Universal Health Coverage: We Need Evidence-Informed Deliberative Processes, Not Just More Evidence on Cost-Effectiveness. International journal of health policy and management 2016;5(11):615-18. doi: 10.15171/ijhpm.2016.83
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15171/ijhpm.2016.83&link_type=DOI) 

11. 11.Baltussen R, Mikkelsen E, Tromp N, et al. Balancing efficiency, equity and feasibility of HIV treatment in South Africa – development of programmatic guidance. Cost Effectiveness and Resource Allocation 2013;11(1):26. doi: 10.1186/1478-7547-11-26
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1478-7547-11-26&link_type=DOI) 

12. 12.Kapiriri L, Lee N-M, Wallace LJ, et al. Beyond cost-effectiveness, morbidity and mortality: a comprehensive evaluation of priority setting for HIV programming in Uganda. BMC public health 2019;19(1):359. doi: 10.1186/s12889-019-6690-8
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12889-019-6690-8&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

13. 13.Guindo LA, Wagner M, Baltussen R, et al. From efficacy to equity: Literature review of decision criteria for resource allocation and healthcare decisionmaking. Cost Effectiveness and Resource Allocation 2012;10
    
    

14. 14.Youngkong S, Kapiriri L, Baltussen R. Setting priorities for health interventions in developing countries: A review of empirical studies. Tropical Medicine and International Health 2009;14:930–39.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1365-3156.2009.02311.x&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000268961300012&link_type=ISI) 

15. 15.UN UN. Transforming our world: the 2030 agenda for sustainable development. A/RES/70/1.. New York: UN 2015.
    
    

16. 16.Baltussen R, Jansen MPM, Bijlmakers L, et al. Value Assessment Frameworks for HTA Agencies: The Organization of Evidence-Informed Deliberative Processes. Value in Health 2017;20(2):256–60. doi: [https://doi.org/10.1016/uval.2016.11.019](https://doi.org/10.1016/uval.2016.11.019)
    
    

17. 17.Daniels N. Accountability for reasonableness. BMJ Global Health 2000;321(7272):1300–01. doi: 10.1136/bmj.321.7272.1300
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1136/bmj.321.7272.1300&link_type=DOI) 

18. 18.Daniels N, Porteny T, Urritia J. Expanded HTA: Enhancing Fairness and Legitimacy. International journal of health policy and management 2015;5(1):1–3. doi: 10.15171/ijhpm.2015.187 [published Online First: 2015/12/18]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15171/ijhpm.2015.187&link_type=DOI) 

19. 19.Singer PA, Martin DK, Giacomini M, et al. Priority setting for new technologies in medicine: qualitative case study. Bmj 2000;321(7272):1316–8. doi: 10.1136/bmj.321.7272.1316 [published Online First: 2000/11/25]
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMjEvNzI3Mi8xMzE2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjAvMDgvMTMvMjAyMC4wNy4wMy4yMDE0NTc1NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

20. 20.Daniels N, Sabin J. The Ethics Of Accountability In Managed Care Reform. Health Affairs 1998;17(5):50–64. doi: 10.1377/hlthaff.17.5.50
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjc6IjE3LzUvNTAiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMC8wOC8xMy8yMDIwLjA3LjAzLjIwMTQ1NzU1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

21. 21.Friedman A. Beyond accountability for reasonableness. Bioethics 2008;22(2):101–12. doi: 10.1111/j.1467-8519.2007.00605.x [published Online First: 2008/02/07]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1467-8519.2007.00605.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18251770&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000252805700005&link_type=ISI) 

22. 22.Daniels N. Accountability for reasonableness. BMJ 2000;7272(321):1300–01.
    
    

23. 23.Tyler TR. Psychological Perspectives on Legitimacy and Legitimation. Annual Review of Psychology 2005;57(1):375–400. doi: 10.1146/annurev.psych.57.102904.190038
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1146/annurev.psych.57.102904.190038&link_type=DOI) 

24. 24.Greenberg J, Folger R. Procedural Justice, Participation, and the Fair Process Effect in Groups and Organizations. In: Paulus P, ed. In Basic Group Processes. New York: Springer 1983: 235–56.
    
    

25. 25.Saltelli A, Giampietro M. What is wrong with evidence based policy, and how can it be improved? Futures 2017;91:62–71. doi: [https://doi.org/10.1016/Lfutures.2016.11.012](https://doi.org/10.1016/Lfutures.2016.11.012)
    
    

26. 26.WHO WHO. WHO Handbook for Guideline Development 2nd edition ed. Geneva, Switzerland: World Health Organization 2014.
    
    

27. 27.Ham C. Priority setting in health care: learning from international experience. Health policy (Amsterdam, Netherlands) 1997;42(1):49–66. [published Online First: 1997/09/04]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0168-8510(97)00054-7&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10173493&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1997XZ17000005&link_type=ISI) 

28. 28.Niessen LW, Bridges J, Lau BD, et al. ssessing the Impact of Economic Evidence on Policymakers in Health Care—A Systematic Review. Methods Research Report (Prepared by the Johns Hopkins University Evidence-based Practice Center under contract No. 290–2007-10061-I). AHRQ Publication No. 12(13)-EHC133-EF. Rockville: Agency for Healthcare Research and Quality 2012.
    
    

29. 29.Alonso-Coello P, Schünemann HJ, Moberg J, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ 2016;353:i2016.
    
    [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNTMvanVuMjhfOS9pMjAxNiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIwLzA4LzEzLzIwMjAuMDcuMDMuMjAxNDU3NTUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

30. 30.Rehfuess EA, Stratil JM, Scheel IB, et al. The WHO-INTEGRATE evidence to decision framework version 1.0: integrating WHO norms and values and a complexity perspective. BMJ Glob Health 2019;4(Suppl 1):e000844. doi: 10.1136/bmjgh-2018-000844 [published Online First: 2019/02/19]
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTc6IjQvU3VwcGxfMS9lMDAwODQ0IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjAvMDgvMTMvMjAyMC4wNy4wMy4yMDE0NTc1NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

31. 31.Norris SL, Rehfuess EA, Smith H, et al. Complex health interventions in complex systems: improving the process and methods for evidence-informed health decisions. BMJ Global Health 2019;4(Suppl 1):e000963. doi: 10.1136/bmjgh-2018-000963
    
    [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTc6IjQvU3VwcGxfMS9lMDAwOTYzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjAvMDgvMTMvMjAyMC4wNy4wMy4yMDE0NTc1NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

32. 32.Assasi N, Schwartz L, Tarride JE, et al. Methodological guidance documents for evaluation of ethical considerations in health technology assessment: a systematic review. Expert review of pharmacoeconomics & outcomes research 2014;14(2):203–20. doi: 10.1586/14737167.2014.894464 [published Online First: 2014/03/15]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1586/14737167.2014.894464&link_type=DOI) 

33. 33.Childress JF, Faden RR, Gaare RD, et al. Public health ethics: mapping the terrain. The Journal of law, medicine & ethics: a journal of the American Society of Law, Medicine & Ethics 2002;30(2):170–8. [published Online First: 2002/06/18]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1748-720x.2002.tb00384.x&link_type=DOI) 

34. 34.Marckmann G, Schmidt H, Sofaer N, et al. Putting public health ethics into practice: a systematic framework. Frontiers in public health 2015;3:23. doi: 10.3389/fpubh.2015.00023 [published Online First: 2015/02/24]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3389/fpubh.2015.00023&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

35. 35.Tannahill A. Beyond evidence--to ethics: a decision-making framework for health promotion, public health and health improvement. Health promotion international 2008;23(4):380–90. doi: 10.1093/heapro/dan032 [published Online First: 2008/10/31]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/heapro/dan032&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18971394&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000260979900010&link_type=ISI) 

36. 36.Coughlin SS. Ethical issues in epidemiologic research and public health practice. Emerging themes in epidemiology 2006;3:16. doi: 10.1186/1742-7622-3-16 [published Online First: 2006/10/05]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1742-7622-3-16&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17018147&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

37. 37.Krebs K, Unwin J. Public health: ethical issues. London: Nuffield Council on Bioethics, 2007.
    
    

38. 38.Liverani M, Hawkins B, Parkhurst JO. Political and institutional influences on the use of evidence in public health policy. A systematic review. PloS one 2013;8(10):e77404-e04. doi: 10.1371/journal.pone.0077404
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0077404&link_type=DOI) 

39. 39.ten Have M, de Beaufort ID, Mackenbach JP, et al. An overview of ethical frameworks in public health: can they be supportive in the evaluation of programs to prevent overweight? BMC public health 2010;10:638. doi: 10.1186/1471-2458-10-638 [published Online First: 2010/10/26]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1471-2458-10-638&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20969761&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

40. 40.Stratil J, Baltussen R, Scheel I, et al. Development of the WHO-INTEGRATE evidence-to-decision framework: An overview of systematic reviews of decision criteria for health decision-making. Cost Effectiveness and Resource Allocation 2020;8(18)
    
    

41. 41.Stratil J, Paudel D, Setty K, et al. The WHO-INTEGRATE framework as a tool for evidence-informed, deliberative decision-making processes. A qualitative study exploring the views of those developing and using WHO guidelines. International Journal for Health Policy and Management forthcoming
    
    

42. 42.Carroll C, Booth A, Leaviss J, et al. “Best fit” framework synthesis: refining the method. BMC medical research methodology 2013;13(1):37. doi: 10.1186/1471-2288-13-37
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1471-2288-13-37&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23497061&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

43. 43.Carroll C, Booth A, Cooper K. A worked example of “best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents. BMC medical research methodology 2011;11:29. doi: 10.1186/1471-2288-11-29 [published Online First: 2011/03/18]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1471-2288-11-29&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21410933&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

44. 44.Dixon-Woods M. Using framework-based synthesis for conducting reviews of qualitative studies. BMC medicine 2011;9:39. doi: 10.1186/1741-7015-9-39 [published Online First: 2011/04/16]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1741-7015-9-39&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21492447&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

45. 45.Bude H, von Donat C, Feldner D, et al. Kreativität – Verantwortung – Transformation. Grundlegende Voraussetzungen für die progressive Lockerung des COVID--19 Lockdown (17. April 2020) 2020
    
    

46. 46.Expertenrat Corona der Landesregierung Nordrhein-Westfalen, Di fabio U, Grünewald S, et al. Weg in eine verantwortungsvolle Normalität (11.04.2020). Expertenrat Corona der Landesregierung Nordrhein-Westfalen,: Düsseldorf 2020.
    
    

47. 47.Anonymous. COVID-19-Eindämmung: Übergang von Verlangsamung zu Viruskontrollphase. Politische Handlungsfähigkeit erlangen Erwartungshorizonte für Gesundheit, Gesellschaft und Wirtschaft schaffen (04.04.2020) 2020
    
    

48. 48.Nationale Akademie der Wissenschaften Leopoldina (Hrsg.). Erste Ad-hoc-Stellungnahme: Coronavirus-Pandemie in Deutschland: Herausforderungen und Interventionsmöglichkeiten (21. März 2020) Halle (Saale): Nationale Akademie der Wissenschaften Leopoldina 2020.
    
    

49. 49.Nationale Akademie der Wissenschaften Leopoldina (Hrsg.). Zweite Ad-hoc-Stellungnahme: Coronavirus-Pandemie in Deutschland: Coronavirus-Pandemie – Gesundheitsrelevante Maßnahmen (03. April 2020) Halle (Saale): Nationale Akademie der Wissenschaften Leopoldina 2020.
    
    

50. 50.Nationale Akademie der Wissenschaften Leopoldina (Hrsg.). Dritte Ad-hoc-Stellungnahme: Coronavirus-Pandemie in Deutschland: Herausforderungen und Interventionsmöglichkeiten (21. März 2020) Halle (Saale): Nationale Akademie der Wissenschaften Leopoldina 2020.
    
    

51. 51.Bündnis 90 Die Grünen Hamburg (Ed.). Autorenpapier unter Federführung von Katharina Fegebank. Corona-Pandemie – Voraussetzungen für eine Exit-Strategie in Hamburg. 09. April 2020. Hamburg: Bündnis 90 Die Grünen 2020.
    
    

52. 52.Institut für Makroökonomie und Konjunkturforschung IE, S D, Herzog-Stein A, et al. Schneller Ausstieg oder bedachte Lockerung? – Zur Zukunft von Kontaktbeschränkungen in der Covid-19-Krise. IMK Policy Brief Nr. 88 · April 2020. Düsseldorf: Institut für Makroökonomie und Konjunkturforschung (IMK) der Hans-Böckler-Stiftung 2020.
    
    

53. 53.Boston Consulting Group (Ed.), Türpitz A, Felden F, et al. BCG Perspektive zum Restart-Management in der Corona-Krise Munich: Boston Consulting Group (BCG) 2020.
    
    

54. 54.Abele-Brehm A, Dreier H, Fuest C, et al. Die Bekämpfung der Coronavirus-Pandemie tragfähig gestalten. Empfehlungen für eine flexible, risikoadaptierte Strategie. Koordination C Fuest und M Lohse. 2. Arpil 20202020.
    
    

55. 55.WHO WHO. Everybody’s business. Strengthening health systems to improve health outcomes : WHO’s framework for action. Geneva: WHO 2007.
    
    

56. 56.O’Neill J, Tabish H, Welch V, et al. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epidemiol 2014;67(1):56–64. doi: 10.1016/j.jclinepi.2013.08.005 [published Online First: 2013/11/06]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jclinepi.2013.08.005&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24189091&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

57. 57.Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. American journal of community psychology 2009;43(3–4):267-76. doi: 10.1007/s10464-009-9229-9 [published Online First: 2009/04/25]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s10464-009-9229-9&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19390961&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000265685500008&link_type=ISI) 

58. 58.Bernstein SL, Weiss J, Curry L. Visualizing implementation: contextual and organizational support mapping of stakeholders (COSMOS). Implementation Science Communications 2020;1(1):48. doi: 10.1186/s43058-020-00030-8
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s43058-020-00030-8&link_type=DOI) 

59. 59.Piso B, Wild C. Decision support in vaccination policies. Vaccine 2009;27(43):5923–8. doi: 10.1016/j.vaccine.2009.07.105 [published Online First: 2009/08/25]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2009.07.105&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19698809&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000271170500002&link_type=ISI) 

60. 60.Rehfuess EA, Booth A, Brereton L, et al. Towards a taxonomy of logic models in systematic reviews and health technology assessments: A priori, staged, and iterative approaches. Research synthesis methods 2018;9(1):13–24. doi: 10.1002/jrsm.1254 [published Online First: 2017/07/06]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/jrsm.1254&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=28677339&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

61. 61.Rohwer A, Pfadenhauer L, Burns J, et al. Series: Clinical Epidemiology in South Africa. Paper 3: Logic models help make sense of complexity in systematic reviews and health technology assessments. Journal of Clinical Epidemiology 2017;83:37–47. doi: 10.1016/j.jclinepi.2016.06.012
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jclinepi.2016.06.012&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

62. 62.Abbasi M, Majdzadeh R, Zali A, et al. The evolution of public health ethics frameworks: systematic review of moral values and norms in public health policy. Medicine, health care, and philosophy 2018;21(3):387–402. doi: 10.1007/s11019-017-9813-y [published Online First: 2017/11/11]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11019-017-9813-y&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

63. 63.Petrini C. Theoretical models and operational frameworks in public health ethics. International journal of environmental research and public health 2010;7(1):189–202. doi: 10.3390/ijerph7010189 [published Online First: 2010/03/03]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/ijerph7010189&link_type=DOI) 

64. 64.Lee LM. Public health ethics theory: review and path to convergence. The Journal of law, medicine & ethics : a journal of the American Society of Law, Medicine & Ethics 2012;40(1):85–98. doi: 10.1111/j.1748-720X.2012.00648.x [published Online First: 2012/03/31]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1748-720X.2012.00648.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22458465&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

65. 65.Baum N, Gollust S, Goold S, et al. Looking Ahead: Addressing Ethical Challenges in Public Health Practice. global health law, ethics, and policy 2007;4(35):657–67.
    
    

66. 66.Kass NE. An ethics framework for public health. American journal of public health 2001;91(11):1776–82. [published Online First: 2001/10/31]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2105/AJPH.91.11.1776&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11684600&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000171855400019&link_type=ISI) 

67. 67.Thompson AK, Faith K, Gibson JL, et al. Pandemic influenza preparedness: an ethical framework to guide decision-making. BMC medical ethics 2006;7:E12. doi: 10.1186/1472-6939-7-12
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1472-6939-7-12&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17144926&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

68. 68.Austin W. Ethics in a time of contagion: a relational perspective. The Canadian journal of nursing research = Revue canadienne de recherche en sciences infirmieres 2008;40(4):10–24. [published Online First: 2009/02/04]
    
    

69. 69.Gostin LO. Influenza pandemic preparedness: legal and ethical dimensions. The Hastings Center report 2004;34(5):10–1. [published Online First: 2004/11/24]
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15281721&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000222303400011&link_type=ISI) 

70. 70.Torda A. Ethical issues in pandemic planning. The Medical journal of Australia 2006;185(10 Suppl):S73–6. [published Online First: 2006/11/23]
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17115958&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000242773700013&link_type=ISI) 

71. 71.Schroder P, Brand H, Schroter M, et al. [Ethical discussion on criteria for policy makers in public health authorities for preventative measures against a pandemic caused by a novel influenza A virus]. Gesundheitswesen (Bundesverband der Arzte des Offentlichen Gesundheitsdienstes (Germany)) 2007;69(6):371–6. doi: 10.1055/s-2007-981675 [published Online First: 2007/07/21]
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1055/s-2007-981675&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17642042&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 

72. 72.Ethikrat. D. [Solidarity and responsibility in the Corona-Crisis. Ad Hoc recommendation]. Publication in German. Berlin: Deutscher Ethikrat 2020.
    
    

73. 73.Heilinger J, Thompson A, Wild V, et al. Public Health Ethics and Covid-19 – The ethical dimensions of public health decision-making during a pandemic German Network Public Health COVID-19 2020.
    
    

74. 74.Venkatapuram S, Heilinger J-C, Voss M, et al. Ethical principles for the Coronavirus Global Response: Ethical guidance for the development of fast and equitable access to safe, quality, effective and affordable diagnostics, therapeutics and vaccines against COVID-19. : German Network Public Health COVID-19 2020.
    
    

75. 75.Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care 2007;19(6):349–57. doi: 10.1093/intqhc/mzm042
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/intqhc/mzm042&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17872937&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F08%2F13%2F2020.07.03.20145755.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000251963800004&link_type=ISI)