Anticipating and assessing adverse and other unintended consequences of public health interventions: the (CONSEQUENT) framework

Jan M Stratil, Renke L Biallas, Ani Movsisyan, Kathryn Oliver, Eva A Rehfuess
doi: https://doi.org/10.1101/2023.02.03.23285408

1 Abstract

Despite the best intentions public health interventions (PHIs) can have adverse and other unintended consequences (AUCs). AUCs are rarely systematically examined when developing, evaluating or implementing PHIs. We used a structured, multi-pronged and evidence-based approach to develop a framework to support researchers and decision-makers in conceptualising and categorising AUCs of PHIs.

We employed the ‘best-fit’ framework synthesis approach. We designed the a-priori framework using elements of the WHO-INTEGRATE framework and the Behaviour Change Wheel. Next, we conducted a qualitative systematic review of theoretical and conceptual publications on the AUCs of PHIs in the databases Medline and Embase as well as through grey literature searches. Based on these findings, we iteratively revised and advanced the a-priori framework based on thematic analysis of the identified research. To validate and further refine the framework, we coded four systematic reviews on AUCs of distinct PHIs against it.

The CONSEQUENT framework includes two components: the first focuses on AUCs and serves to categorise them; the second component highlights the mechanisms through which AUCs may arise. The first component comprises eight domains of consequences – health, health system, human and fundamental rights, acceptability and adherence, equality and equity, social and institutional, economic and resource, and ecological.

The CONSEQUENT framework is intended to facilitate conceptualisation and categorising of AUCs of PHIs during their development, evaluation and implementation to support evidence-informed decision-making.

2 Research Manuscript section

2.1 Introduction

Promoting and improving the physical and mental health of populations is the central goal of public health (PH) interventions all over the globe. However, despite the best intentions, these interventions can have adverse effects, such as effects in the opposite direction of that intended or expected (paradoxical effects) or effects on unrelated outcomes (unintended externalities) (1). For example, providing pre-exposure prophylaxis against HIV may lead to an increase in risky sexual behaviour and in sexually transmitted infections other than HIV (2). The drilling of groundwater wells, which successfully reduced diarrheal disease mortality due to polluted surface water, has exposed an estimated 40 million Bangladeshis to harmful concentrations of arsenic contained in the groundwater (3). It has also been shown how obesity-focused PH interventions have led to an increase in stigmatisation and social exclusion of those living with obesity (4, 5).

To truly promote public health, it is essential not only to evaluate intended beneficial outcomes of PH interventions, but also to anticipate and assess their possible adverse and other unintended consequences (AUCs). In contrast to the well-established assessment of adverse drug reactions (ADRs), particular challenges arise in the assessment of AUCs of PH interventions: while ADRs primarily result directly from the drugs themselves and affect those taking the drugs, PH interventions often function as “events in systems”(6), where effects of the intervention arise as a result of the interaction between the intervention and the social, economic, or political context in which it is implemented (6-8). Individuals and populations not targeted by the intervention may even be those (most severely) affected by AUCs (9, 10). While ADRs are mostly health-related, PH interventions usually have social, economic, ecological, or political ramifications (e.g., large scale usage of DDT in malaria prevention leading to adverse effects on the ecosystem) (11-14). Furthermore, consideration of an unintended effect of an intervention as adverse, beneficial, or neutral is not always clear, as it depends on the perspective of the observer, as well as underlying sociocultural norms in different regions and populations; both perspectives and the sociocultural norms underlying them may change over time. For example, whether increased meat consumption is considered an adverse effect beyond (human) health is likely to depend on whether the evaluating person works in the meat industry or is an animal rights activist, whether the assessment takes place in Argentina or Nepal, and on the time of evaluation, such as in the 1980s compared to the 2020s.

Anticipating and understanding AUCs should be a priority for those deciding on or implementing PH interventions – as there are moral, ethical, political, and practical reasons for avoiding health and societal harms (15-17). However, these are often not thoroughly examined in public health research, practice, and policy, especially AUCs not directly related to health (18-20). While unintended consequences of social action have been discussed in the broader scientific literature (21-29), they constitute a largely neglected topic in empirical public health research (30, 31), except for specific areas, such as cancer screening (32) or illicit drug use (33).

In recent years, public health researchers have begun to identify and describe harms and to suggest typologies or classifications of harms (30, 31). However, these have primarily focused on health rather than broader societal consequences and/or have not been developed in a systematic manner (15, 34). Important questions remain on how to identify the unintended and potentially harmful effects of PH interventions (20), how best to evaluate them (19, 26), and how to incorporate the consideration of harms into the process of evidence-informed decision-making (14, 20, 35, 36). Being able to identify PH interventions and policies with substantive harmful effects and to subsequently revise, adapt or de-implement these interventions is essential for programme implementers, service providers, and policymakers.

The primary objective of the research project was to develop a framework which supports public health researchers, practitioners and decision-makers in conceptualising and categorising foreseeable AUCs of PH interventions (the consequences component of framework). The secondary objective was to map and conceptualise the mechanisms through which AUCs may arise (as a supplementary mechanisms component of framework).

In this context, we refer to conceptualising as the use of the framework as a tool to support stakeholders in systematically reflecting on (potential) AUCs of PH interventions when developing, evaluating, or implementing PH interventions. In the meantime, we refer to categorising as the use of the framework as a classification system of AUCs of PH interventions, for example when creating evidence (gap) maps.

2.2 Materials and Methods

2.2.1 Overview of framework development process

We developed the framework using a “best-fit” framework synthesis approach (38, 39), starting with an a priori framework based on limited evidence, followed by iterative revisions of the framework considering further evidence. We used key elements from the WHO-INTEGRATE framework (36) and the Behaviour Change Wheel (40) to create an a priori framework of AUCs and the possible mechanisms leading to these (38, 39). We then advanced and refined the framework based on theoretical and conceptual papers describing frameworks or systems of AUCs of PH interventions and/or their mechanisms, as well as empirical research on the AUCs of PH interventions implemented in policy and practice. These papers were identified using systematic searches in health databases and reference searches (Supplementary Files 1-3). Thematic analysis was used to identify new themes and topics and thereby to revise the framework. In the final step, the findings in systematic reviews of the AUCs of four specific PH interventions were coded against the empirically advanced framework components (41-44), which were conducted by or in cooperation with the members of the research team. This served to validate the framework using examples from practice. The framework revisions across all steps were guided by discussions within the study team. The entire framework development process is visualised in Figure 1. The terminology used is defined in Box 1. We used the Standards for Reporting Qualitative Research (SRQR) reporting guideline (37).

Figure 1.
Figure 1. Framework development process.

AUC: Adverse and other Unintended Consequences.

Box 1:

Terminology

A complex (public health) intervention implemented in a complex system is likely to have diverse effects that result directly from the intervention or indirectly from its interaction with the system. In this paper, we employ the term consequences to define a subset of these broad system effects, and namely those, which are assigned relevance within a given normative framework. Furthermore, we employ the term mechanisms to refer to the direct and indirect pathways or processes through which consequences (beneficial or adverse) arise.

Consequences may be intended or unintended, depending on whether they are the outcomes the intervention is supposed to produce (29) from the perspective of those conceptualising and implementing the intervention. Further to this, consequences can be anticipated, when they are predicted in advance by intervention developers or implementers, and unanticipated, when they occur unexpectedly (29). Anticipated consequences include those that are foreseeable, i.e., where through application of analytical frameworks and experiences, consequences can reasonably be predicted in their contours, even if not in their specifics (22). Unanticipated consequences include foreseeable consequences that were not anticipated, as well as consequences (mostly in complex systems) that may not be foreseeable (e.g., black swan events) (45) and even be truly random.

Whether consequences are regarded as adverse (or undesirable (21) or negative (29)), beneficial (or desirable (21) or positive (29)), or neutral, depends on the perspective, the underlying normative frame and the “zoom level” (i.e. how far the effect is followed in the pathway throughout the network of effects) (29). For example, the introduction of a sugar tax leading to a decrease in the consumption of sugar-sweetened beverages may be regarded as an (intended) beneficial effect by public health practitioners, whereas it may be regarded as an adverse effect by the sugar-sweetened beverage industry. Furthermore, from a societal perspective, the intervention may lead to a dismissal of workers, increase in unemployment rates, and a loss of a source of household income. To account for the perspective-dependency in the classification of unintended consequences as beneficial or adverse, we use the overarching term adverse and other unintended consequences (AUC) in this publication.

2.2.2 Development of the a priori framework

For the categorisation of consequences, we used the criteria and sub-criteria of the WHO-INTEGRATE framework version 1.0 (36, 46, 47). We chose this approach, as (i) it provides a reference frame that is firmly rooted in global health norms and values, as well as key public health ethics frameworks; (ii) it supports decision-making from a complexity perspective, viewing PH interventions as events in (complex) systems (6, 7, 48), and (iii) it considers outcomes of PH interventions beyond health, including social, ecological, and economic consequences.

For the categorisation of mechanisms, we used the Behaviour Change Wheel (40). We chose this as (i) it is the most widely used approach for examining behaviour change and (ii) it considers impacts at both individual and societal levels. We focused on the nine intervention functions in the Behaviour Change Wheel and derived a priori mechanisms based on these, including restriction, education, persuasion, incentivisation, coercion, training, enablement, modelling, and environmental restructuring.

Through brainstorming and discussions within the research team, these two frameworks were iteratively revised and advanced, resulting in the two components of the a priori framework (Supplementary Files 4 and 5).

2.2.3 Identification of eligible publications for “best fit” framework synthesis

To retrieve the publications of relevance to advance the a priori framework, we conducted comprehensive literature searches in Medline (Ovid), Embase (Ovid), and the Cochrane library for systematic reviews up until November 2020. The search strategy was developed by expanding the search strategy of the 2014 scoping review by Allen-Scott and colleagues (31) and by following a guidance document by the Cochrane Adverse Effects Methods Group (49-51). In brief, the search strategy combined terms related to unintended consequences with those related to public health. The search strategy for Embase (Ovid) is provided as an example in Supplementary File 1. Additionally, we conducted forward and backward citation searches of all included studies. We conducted these searches in Scopus, Google Scholar, and Microsoft Academic.

First, to incorporate existing concepts of AUCs of PH interventions, we examined theoretical or conceptual papers which categorised, explored, or explained AUCs in-depth, grounded in or alluding to empirical findings. These included papers (i) providing typologies or taxonomies of AUCs of PH interventions, such as those by Allen-Scott and colleagues (31) or Lorenc and Oliver (30), (ii) describing, discussing, or exploring mechanisms of how PH interventions may lead to unintended consequences, such as those by Allen-Scott and colleagues (31) and Bonell and colleagues (1), as well as (iii) offering guidance for identifying unintended consequences of PH interventions, such as those by Bonell and colleagues (1) and Mittelmark and colleagues (52).

Second, to incorporate empirical insights to date, we retrieved and assessed systematic reviews with the primary objective to assess AUCs of PH interventions. Reviews with a primary focus on the effectiveness of interventions (i.e., the intended beneficial effects of PH interventions) were excluded.

After removal of duplicate studies, the eligibility of studies was assessed independently by two researchers (JMS, RB). Disagreements were resolved by discussion, and where necessary, by consulting with the full research team.

When considering papers for inclusion, we adopted a broad understanding of PH interventions, including a range of measures of health promotion, disease prevention, health protection, and other interventions aimed at improving health, prolonging life, and improving the quality of life among populations64. We excluded papers focusing on medical prevention (e.g., medical treatment of risk factors, such as hypertension), as well as secondary (e.g., prostate or breast cancer screening programmes) or tertiary prevention. Regarding borderline cases, such as vaccinations, which can be regarded as a public health or as a medical prevention effort, we excluded those publications exclusively addressing the iatrogenic effects of vaccines as medical products (e.g., fever, immune reactions), but included those reporting on the AUCs of vaccination programmes beyond adverse drug reactions (53). The full list of inclusion and exclusion criteria is provided in Additional File 2.

2.2.4 Advancement of the a priori framework

As outlined above, we used the identified literature to revise the two components of the a priori framework. For this, we applied thematic analysis using a mix of inductive and deductive coding (38, 39). Specifically, the included papers were coded deductively against the categories and themes of the a priori framework, while the new themes not covered in the a priori framework were derived inductively (38, 39). The coding was done by two authors (JS, RB) using the software MAXQDA 20 (Verbi, Berlin). The thematic analysis and the framework revisions were implemented in an iterative manner (see Figure 1). The coding was conducted simultaneously for the consequences component and the mechanisms component of the framework.

First, the two components were revised and expanded based on the coding of the included theoretical and conceptual papers and the resulting new themes. The revisions were discussed in-depth within the research team, yielding conceptually advanced components. Next, the two components were further revised based on the coding of the systematic reviews of AUCs of PH interventions and discussions in the research team, yielding empirically advanced components.

2.2.5 Validation of the a priori framework

To validate the comprehensiveness of the framework, four systematic reviews on unintended consequences of different PH interventions were coded against the empirically advanced framework components. We chose four different PH interventions, i.e. setting-based drug prevention (44), prevention of SARS-CoV-2 transmission in schools (41), international travel-related control measures to control COVID-19 (42), and measures to reduce the consumption of sugar-sweetened beverages (43); the systematic reviews of the AUCs of these PH interventions had been conducted by or in cooperation with research team members. After a final review and discussion within the research team, the two-component framework was finalised as the adverse and other unintended CONSEQuences of pUblic hEalth iNTerventions (CONSEQUENT) framework.

2.3 Results

After the removal of duplicates, the literature searches identified 2998 records. The full texts of 150 records were screened for eligibility, and 15 records met the criteria for inclusion as theoretical or conceptual publications (1, 20, 30, 31, 52, 54-61). By screening the reference lists of the included records, as well as of the identified reviews, we included another 3 records (62-64). We also identified 15 systematic reviews (9, 33, 65-76) reporting on AUCs of different PH interventions through the database searches. No additional records yielding systematic reviews were identified through searches of the reference lists. Eventually, 18 unique records of theoretical or conceptual publications and 15 unique systematic reviews were included for thematic analysis and coding. The PRISMA flow-chart visualising this process is presented in supplementary file 3 (77).

The two-component CONSEQUENT framework is presented in Figure 2.

Figure 2:
Figure 2: CONSEQUENT framework comprising consequences and mechanisms.

AUC: adverse or other unintended consequences; M: mechanisms.

The consequences component of the CONSEQUENT framework comprises eight first-order domains: (i) health, (ii) health system, (iii) human and fundamental rights, (iv) acceptability and adherence, (v) equality and equity, (vi) social and institutional, (vii) economic and resource-related, and (viii) ecological. Each first-order domain also comprises several specific second-order domains. For example, the first-order domain health includes consequences for physical health and health behaviour as well as psychosocial health and wellbeing as second-order domains. Depending on the purpose and context of framework application, either the more generic first-order domains and/or the more granular second-order domains may be considered; second-order domains may also be adapted as needed (for example, differentiating the 1st order domain consequence health in COVID-19-related and non-COVID-19 related health consequence for the assessment of a PH interventions targeting SARS-CoV-2 pandemic). Descriptions of first- and second-order domains are provided in Table 1, and further examples—in Additional File 6.

View this table:
Table 1.

Consequences component of the CONSEQUENT framework: first-order domains, second-order domains, definition and description

The mechanisms component of the CONSEQUENT framework, which may be treated as a supplementary component, consists of eight mechanisms (Figure 2). AUCs may arise through: (i) bio-physiological mechanisms, (ii) (re-)action or behaviour change, (iii) perception, experience, and assessment, (iv) available opportunities for (re-)action, (v) environments and environmental exposures, (vi) social norms and practices, (vii) economic and market mechanisms, and (viii) the functioning of systems and system components. Each mechanism also includes a non-exhaustive list of more specific processes. For example, the mechanism of (re-)action or behaviour change includes sthe following processes: (ii.a) affecting behavioural practice(s), (ii.b) evasive, resistant, or counteractive (re-)action(s) or practices, (ii.c) supplementing practices or products, (ii.d) human error or misuse, (ii.e) triggering automated behaviours, and (iii.f) lack of action or (behaviour) change. In contrast to the second-order domains of consequences, these specific processes are not intended as standalone “sub-mechanisms”, but rather illustrate how the mechanisms may operate and are likely to vary for different PH interventions. Descriptions of the mechanisms and specific processes are presented in Table 2; further details and examples are provided in Additional File 7. The relationship between the final framework and the a priori and interim versions of the framework is depicted in Additional Files 4 and 5.

View this table:
Table 2.

Mechanisms component of the CONSEQUENT framework: description and specific process

2.4 Discussion

The CONSEQUENT framework represents a novel comprehensive system to conceptualise and categorise AUCs of PH interventions, as well as the potential mechanisms leading to these. The framework is rooted in global health norms and values as it was developed drawing on the WHO-INTEGRATE framework (36); it also reflects current best insights regarding behavioural science, given its roots in the Behaviour Change Wheel (40). Furthermore, it explicitly embraces a complexity perspective (48), and thus emphasises unintended consequences of PH interventions beyond the health of individuals and populations.

2.4.1 Conceptualisation of consequences and mechanisms in the CONSEQUENT framework

As shown in Figure 3, AUCs may arise through relatively simple or long and complex processes. AUCs may arise directly from the intervention (pathway A in Figure 3). For example, the taxation of sugar-sweetened beverages may lead to reduced revenue of vendors (the consequence) through a reduction in demand (the mechanism)). AUCs may also arise indirectly, when a mediator on the intended pathway leads to an unintended consequence (pathway B). For example, a public health campaign promoting physical activity may lead to an increase in road traffic injuries (consequence) due to uptake of cycling and increased exposure of cyclists to accident-prone environments (mechanism). Intended consequences may also lead to unintended consequences (pathway C). For example, skin cancer prevention programmes via a successful reduction of sun exposure (mechanism) may further lead to Vitamin D deficiencies and related health consequences (consequence) (30). Furthermore, AUCs can arise through one mechanism (pathway A) or through a combination of multiple mechanisms interacting with each other (pathway D).

Figure 3:
Figure 3: Depiction of the relations between interventions and unintended consequences.

AUC: adverse or other unintended consequences; M: mechanisms.

Finally, an unintended consequence may lead to additional “secondary” unintended consequences: an unintended consequence may lead to further unintended consequences in a chain (pathway E). For example, a public health media campaign promoting healthy eating patterns may interact with and reinforce social norms and attitudes regarding obesity and obese individuals more broadly, ultimately leading to an increase in weight-based discrimination and adversely affecting the mental health of obese individuals (e.g., depression). It may also lead to lower levels of physical activity among obese individuals due to behaviours that aim to avoid further stigmatization(4, 5).

The length and complexity of the causal pathways leading to AUCs depends on the perspective of the users conceptualising these: this entails the degree to which one “zooms in” on a particular pathway. Consider the example of conceptualising the unintended consequences of a public health nutrition guideline to reduce the consumption of cholesterol. This may lead to an increase in the consumption of trans fats in margarine (change in health behaviour) and because of pathophysiological mechanisms to a further increase of cardiovascular mortality. The pathway leading to an increase in cardiovascular mortality can be adequately depicted as a long-interlinked chain of bio-physiological processes in the human body. While this conceptualisation can be helpful from a biomedical perspective, a detailed understanding of the exact chain of bio-physiological mechanisms may not be useful for public health decision-makers developing or wanting to use the public health nutrition guideline. In line with a complexity perspective (48), we suggest that the degree to which the users “zoom in” or “zoom out” on the causal pathways and thereby the level of detail considered in theorising these pathways, should be guided by the principle of usefulness for the question at hand, rather than, by the principle of comprehensiveness.

2.4.2 Application of the framework

The framework was developed with two uses in mind:

The first intended use of the framework is to help public health researchers, practitioners and decision-makers conceptualise AUCs, that is, it can be used as a supporting tool to reflect on and anticipate AUCs of PH interventions when developing, evaluating, or implementing an intervention. In this application, the consequences listed in this framework are intended to guide deliberations on the potential AUCs of implementing the intervention in a given context, as well as on how different mechanisms can lead to these consequences. A comprehensive consideration of AUCs is important to appropriately judge the balance between benefits and harms of PH interventions, and anticipation of AUCs will inform their evaluation, as well as implementation of potential co-interventions or countermeasures. The CONSEQUENT framework is intended to organize these procedures and ensure that all relevant AUCs and mechanisms are considered. A simplified guidance on how to apply the framework in this conceptual manner and an illustration of this application is provided in Supplementary File 8.

The first intended use of the framework is to provide researchers with a classification system of unintended consequences of PH interventions and the mechanisms leading to them. Beyond classifying identified AUCs or mechanisms, such a classification system can reveal important gaps in the literature (41-44). For example, a preliminary version of the framework was used in a systematic review of PH interventions to prevent illicit drug use. The application of the framework showed that most publications examined in the review did not follow a structured approach for the assessment of AUCs or solely focused on health-related consequences. Furthermore, potential mechanisms were rarely described or explored. This indicated a gap in the literature on illicit drug use specifically related to the societal and ecological consequences of PH interventions for prevention (44).

2.4.3 Relationship with other frameworks of intervention harms

The proposed framework shares many features with other frameworks and classification systems of the harms of PH interventions (30, 31).. We describe these below.

Allen-Scott et al. (31) propose five underlying factors of AUCs of PH interventions, such as “ignoring root causes”, “limited and/or poor quality evidence” and “lack of community engagement”. These underlying factors deviate from what we refer to as mechanisms in the CONSEQUENT framework. We consider the underlying factors proposed by Allen-Scott et al. to operate on a more upstream level and rather align with what we refer to as root mechanisms. These are understood as mechanisms operating when planning or implementing a PH intervention. Based on the publications by Allen-Scott et al (31) others (21, 23, 25), we discuss a range of relevant root mechanisms, notably: (i) not taking context into account, (ii) insufficient buy-in and participation of relevant stakeholders, (iii) (not) acting based on poor quality or insufficient information, (iv) neglecting root causes and acting based on simple answers to complex problems, and (v) (mis-) allocating scarce resources. However, more work to explore these root mechanisms is needed.

While most categories of potential harms in the framework by Lorenc and Oliver (30) are also covered in our framework, the category of “opportunity cost” is not. We did not include it in the current framework, as it requires numerous assumptions about a counterfactual reality in which the intervention would not have been implemented. However, we consider this aspect as part of the root mechanisms (i.e., “through (mis-) allocating scarce resources”).

Unintended consequences regarding equity and equality have been addressed in several publications, such as in the framework by Glover et al. for identifying and mitigating the equity harms of COVID-19 policy interventions (30, 54, 73). While these are covered in the framework component of consequences, they are not explicitly mentioned in the framework component on mechanisms as a standalone mechanism. This decision was made, as equity and inequality can arise through different mechanisms in different populations (for example, an increase in health inequality (the consequence) can arise through different populations acting within the constraints of different degrees of freedom (i.e., opportunities)).

2.4.4 Strengths and limitations of the framework development process

A significant strength of the CONSEQUENT framework is the systematic, multi-pronged and iterative development of the framework. The framework has a strong and explicit normative foundation as it was modelled based on the WHO-INTEGRATE framework (36), and incorporates key insights from behavioural sciences (40). It was advanced using theoretical/conceptual, as well as empirical literature on AUCs of PH interventions derived from systematic literature searches; new insights were integrated using a mix of inductive and deductive approaches of qualitative inquiry. However, the project also has a few limitations. First, the literature searches regarding theoretical/conceptual papers and systematic reviews focusing on AUCs of PH interventions were likely not comprehensive. We conducted searches (primarily) in health-related databases; it is therefore likely that we missed insights on a broader range of consequences arising from PH interventions assessed and published by other disciplines (e.g., economics literature, environmental sciences literature). Second, the identified empirical literature itself is likely not comprehensive regarding all unintended consequences that may have occurred; for example, unintended ecological consequences were rarely addressed. Third, while we achieved content saturation in the coding process (i.e., themes were covered by multiple publications and those coded at a later stage did not provide new consequences or mechanisms), further publications may suggest additional consequences and mechanisms. For example, consideration of more publications on the AUCs of economic or market-based PH interventions derived from economic research may lead to additional insights. Finally, we focused on the literature of AUCs of PH interventions. In some cases, the distinction between economic or social policy measures and PH interventions was challenging. We aimed to overcome this issue through extensive discussions in the team and a clear definition of inclusion and exclusion criteria. For example, we are aware of the extensive literature on unintended consequences of social action from outside the field of public health. Therefore, expanding the framework based on this body of literature may provide additional insights. We therefore suggest that further validation should take place by applying it to a more diverse set of PH interventions. Based on this, a systematic collation of the experiences may lead to a further advancement of the CONSEQUENT framework, extending it into areas that are currently insufficiently covered and/or adding further granularity, such as for the second-order domains of consequences or for specific mechanisms.

3 Conclusion

The CONSEQUENT framework is a two-component framework to conceptualise and categorise the AUCs of PH interventions, reflecting on both outcomes (i.e., consequences), as well as the processes leading to these outcomes (i.e., mechanisms). The framework may help public health researchers, practitioners and decision-makers in conceptualising and anticipating AUCs when developing, evaluating, or implementing public health intervention. Furthermore, the framework can be used by researchers to categorise and classify AUCs of public health interventions, for example to reveal gaps in the literature on AUCs (41-44). Illustrations for the two forms of application of the CONSEQUENT Framework are provided (see Supplementary File 8) and will be further refined in future publications. Application and user-testing of the framework for practical utility may also inform further adaptations.

Data Availability

All data produced in the present work are contained in the manuscript as well as in the public domain.

4 Back Matter

4.1 Supplementary Materials

  • Supplementary File 1: Search strategy in Embase(Ovid)

  • Supplementary File 2: Eligibility criteria

  • Supplementary File 3: PRISMA-Flowchart

  • Supplementary File 4: Relationship between the a priori and the consequences component of the final CONSEQUENT framework

  • Supplementary File 5: Relationship between the a priori and the mechanism component of the final CONSEQUENT framework

  • Supplementary File 6: Consequences component of the final CONSEQUENT framework with exemplary quotes

  • Supplementary File 7: Mechanisms component of the final CONSEQUENT framework with exemplary quotes

  • Supplementary File 8: Application of CONSEQUENT

4.2 Funding

The authors received funding from the German Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung; BMBF) for this project. Grant number (001001EL2032)

4.4 Author contributions

The study was conceived by JMS and EAR in close collaboration with KO and with input and support from AM and Peter von Philipsborn (LMU Munich). The search strategy was developed by JMS in close collaboration with EAR with input from AM. Literature screening and selection was conducted by JMS and RB. A preliminary coding frame was conceived by JMS drawing on discussions among all members of the research team. Application of the preliminary framework to the documents included in the analysis and the development of the revised framework versions were conducted by JMS and RB. EAR, AM, and KO reviewed and provided in-depth feedback on intermediate versions of the framework, leading to further revisions by JMS in close collaboration with RB. JMS drafted the manuscript with RB contributing specific segments. Several versions of the manuscript were critically reviewed and revised by RB, EAR, AM, and KO.

4.5 Institutional Review Board Statement

All study procedures were reviewed and approved by the LMU Munich institutional ethics review board. No consent to participate in the study is required.

4.6 Informed consent for publication

Not applicable

4.7 Data availability statement

The underlying data (i.e., the publications used to develop the framework) are referenced in the article and are available in the public domain. The underlying scientific publications were published in international, scientific journals. Some, although not all, are available in open access journals.

4.8 Conflicts of interest

The authors declare that they have no financial or other economic competing interest regarding the content of this publication. JMS and EAR are authors of the WHO-INTEGRATE framework. KO is the author of a different, widely used framework for classifying unintended consequences of public health interventions.

4.3 Acknowledgements

We would like to thank Peter von Philipsborn for his support in finalising the application for the funding of this project. Without his contribution, this project would likely not have been possible.

Footnotes

  • Minor Changes in Wording. Primarily regarding the wording anticipating and assessing vs. conceptualization and classification.

4.9 References

  1. 1.↵
    Bonell C, Jamal F, Melendez-Torres GJ, Cummins S. ‘Dark logic’: theorising the harmful consequences of public health interventions. Journal of epidemiology and community health. 2015;69(1):95–8.
    OpenUrlAbstract/FREE Full Text
  2. 2.↵
    Molina JM, Charreau I, Spire B, Cotte L, Chas J, Capitant C, et al. Efficacy, safety, and effect on sexual behaviour of on-demand pre-exposure prophylaxis for HIV in men who have sex with men: an observational cohort study. The lancet HIV. 2017;4(9):e402–e10.
    OpenUrl
  3. 3.↵
    Alam MG, Allinson G, Stagnitti F, Tanaka A, Westbrooke M. Arsenic contamination in Bangladesh groundwater: a major environmental and social disaster. International journal of environmental health research. 2002;12(3):235–53.
    OpenUrlCrossRefPubMed
  4. 4.↵
    Sikorski C, Luppa M, Kaiser M, Glaesmer H, Schomerus G, Konig HH, et al. The stigma of obesity in the general public and its implications for public health - a systematic review. BMC Public Health. 2011;11:661.
    OpenUrlCrossRefPubMed
  5. 5.↵
    Ten Have M. Ethical aspects of obesity prevention. Best Pract Res Clin Gastroenterol. 2014;28(2):303–14.
    OpenUrl
  6. 6.↵
    Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. American journal of community psychology. 2009;43(3-4):267–76.
    OpenUrlCrossRefPubMedWeb of Science
  7. 7.↵
    Rutter H, Savona N, Glonti K, Bibby J, Cummins S, Finegood DT, et al. The need for a complex systems model of evidence for public health. The Lancet. 2017;390(10112):2602–4.
    OpenUrlCrossRef
  8. 8.↵
    Rhodes T, Lancaster K. Evidence-making interventions in health: A conceptual framing. Social Science & Medicine. 2019;238:112488.
    OpenUrl
  9. 9.↵
    Benjamin-Chung J, Abedin J, Berger D, Clark A, Jimenez V, Konagaya E, et al. Spillover effects on health outcomes in low- and middle-income countries: a systematic review. International journal of epidemiology. 2017;46(4):1251–76.
    OpenUrlPubMed
  10. 10.↵
    Fang D, Chen B, Hubacek K, Ni R, Chen L, Feng K, et al. Clean air for some: Unintended spillover effects of regional air pollution policies. Science advances. 2019;5(4).
  11. 11.↵
    Turcotte-Tremblay A-M, Gali Gali IA, Ridde V. The unintended consequences of COVID-19 mitigation measures matter: practical guidance for investigating them. BMC Medical Research Methodology. 2021;21(1):28.
    OpenUrl
  12. 12.
    Viner RM, Russell SJ, Croker H, Packer J, Ward J, Stansfield C, et al. School closure and management practices during coronavirus outbreaks including COVID-19: a rapid systematic review. The Lancet Child & Adolescent Health. 2020;4(5):397–404.
    OpenUrl
  13. 13.
    Bouwman H, van den Berg H, Kylin H. DDT and malaria prevention: addressing the paradox. Environ Health Perspect. 2011;119(6):744–7.
    OpenUrlCrossRefPubMedWeb of Science
  14. 14.↵
    Stratil JM, Voss M, Arnold L. WICID framework version 1.0: criteria and considerations to guide evidence-informed decision-making on non-pharmacological interventions targeting COVID-19. 2020;5(11):e003699.
    OpenUrl
  15. 15.↵
    Lorenc T, Oliver K. Adverse effects of public health interventions: a conceptual framework. J Epidemiol Community Health. 2014;68(3):288–90.
    OpenUrlAbstract/FREE Full Text
  16. 16.
    Childress JF, Faden RR, Gaare RD, Gostin LO, Kahn J, Bonnie RJ, et al. Public health ethics: mapping the terrain. The Journal of law, medicine & ethics : a journal of the American Society of Law, Medicine & Ethics. 2002;30(2):170–8.
    OpenUrlCrossRef
  17. 17.↵
    Bonell C, Jamal F, Melendez-Torres GJ, Cummins S. ‘Dark logic’: theorising the harmful consequences of public health interventions. Journal of Epidemiology and Community Health. 2015;69(1):95–8.
    OpenUrlAbstract/FREE Full Text
  18. 18.↵
    Oliver K. Can evidence-informed theory help us to avoid harmful public health polices? Soc Sci Med. forthcoming.
  19. 19.↵
    Oliver K, Lorenc T, Tinkler J. Evaluating unintended consequences: New insights into solving practical, ethical and political challenges of evaluation. Evaluation. 2019;26(1):61–75.
    OpenUrl
  20. 20.↵
    Oliver K, Lorenc T, Tinkler J, Bonell C. Understanding the unintended consequences of public health policies: the views of policymakers and evaluators. BMC public health. 2019;19(1):1057.
    OpenUrlPubMed
  21. 21.↵
    Merton RK. The Unanticipated Consequences of Purposive Social Action. American Sociological Review. 1936;1(6):894–904.
    OpenUrlCrossRefWeb of Science
  22. 22.↵
    Morell JA. Systematic iteration between model and methodology: A proposed approach to evaluating unintended consequences. Evaluation and program planning. 2018;68:243–52.
    OpenUrl
  23. 23.↵
    Morell JA. Why Are There Unintended Consequences of Program Action, and What Are the Implications for Doing Evaluation? American Journal of Evaluation. 2005;26(4):444–63.
    OpenUrlCrossRefWeb of Science
  24. 24.
    Morell JA. Evaluation in the face of uncertainty: Anticipating surprise and re-sponding to the inevitable. New York: Guildford; 2010.
  25. 25.↵
    Vernon R. Unintended Consequences. Political Theory. 1979;7(1):57–73.
    OpenUrlCrossRef
  26. 26.↵
    Jabeen S. Unintended outcomes evaluation approach: A plausible way to evaluate unintended outcomes of social development programmes. Evaluation and program planning. 2018;68:262–74.
    OpenUrl
  27. 27.
    Jabeen S. Do we really care about unintended outcomes? An analysis of evaluation theory and practice. Eval Program Plann. 2016;55:144–54.
    OpenUrl
  28. 28.
    Koch D-J, Schulpen L. Introduction to the special issue ‘unintended effects of international cooperation’. Evaluation and program planning. 2018;68:202–9.
    OpenUrlPubMed
  29. 29.↵
    Sherrill S. Identifying and measuring unintended outcomes. Evaluation and program planning. 1984;7(1):27–34.
    OpenUrlCrossRefPubMed
  30. 30.↵
    Lorenc T, Oliver K. Adverse effects of public health interventions: A conceptual framework. Journal of Epidemiology and Community Health. 2014;68(3):288–90.
    OpenUrlAbstract/FREE Full Text
  31. 31.↵
    Allen-Scott LK, Hatfield JM, McIntyre L. A scoping review of unintended harm associated with public health interventions: towards a typology and an understanding of underlying factors. International journal of public health. 2014;59(1):3–14.
    OpenUrl
  32. 32.↵
    Heleno B, Thomsen MF, Rodrigues DS, Jørgensen KJ, Brodersen J. Quantification of harms in cancer screening trials: literature review. BMJ : British Medical Journal. 2013;347:f5334.
    OpenUrl
  33. 33.↵
    Longshore D, Renter P, Derks J, Grapendaal M, Ebener P. Drug policies and harms: A conceptual framework. European Addiction Research. 1998;4(4):172–82.
    OpenUrlPubMed
  34. 34.↵
    Allen-Scott LK, Hatfield JM, McIntyre L. A scoping review of unintended harm associated with public health interventions: towards a typology and an understanding of underlying factors. Int J Public Health. 2014;59(1):3–14.
    OpenUrlCrossRefPubMed
  35. 35.↵
    Allen-Scott LK, Hatfield JM, McIntyre L, McLaren L. Operationalizing the ‘population health’ approach to permit consideration and minimization of unintended harms of public health interventions: a malaria control example. Critical Public Health. 2016;26(3):244–57.
    OpenUrl
  36. 36.↵
    Rehfuess E, Stratil J, Scheel I, Portela A, Norris S, Baltussen R. The WHO-INTEGRATE evidence to decision framework version 1.0: Integrating WHO norms and values and a complexity perspective BMJ Global Health. 2019(4):e000844.
  37. 37.↵
    O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for Reporting Qualitative Research: A Synthesis of Recommendations. Academic Medicine. 2014;89(9):1245–51.
    OpenUrlCrossRefPubMed
  38. 38.↵
    Carroll C, Booth A, Cooper K. A worked example of “best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents. BMC Med Res Methodol. 2011;11:29.
    OpenUrlCrossRefPubMed
  39. 39.↵
    Carroll C, Booth A, Leaviss J, Rick J. “Best fit” framework synthesis: refining the method. BMC Medical Research Methodology. 2013;13(1):37.
    OpenUrl
  40. 40.↵
    Michie S, van Stralen MM, West R. The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implementation Science. 2011;6(1):42.
    OpenUrl
  41. 41.↵
    Kratzer S, verboom B, Biallas R, Featherstone R, Klinger C, Movsisyan A, et al. Unintended Consequences of Measures Implemented in the School Setting to Contain the COVID-19 Pandemic: A Scoping Review Protocol. OSF. 2021.
  42. 42.↵
    Klinger C, Burns J, Movsisyan A, Biallas R, norris S, Rabe J, et al. Unintended health and societal consequences of international travel measures during the COVID-19 pandemic: a scoping review Journal of Travel Medicine. 2021.
  43. 43.↵
    Bulubas M, Rehfuess E, von Philipsborn P, Stratil J. Adverse and other unintended consequences of public health interventions to reduce sugar-sweetened beverage consumption: an overview of systematic reviews. forthcoming.
  44. 44.↵
    Biallas R, Stratil J, Rehfuess E. Adverse and other unintended consequences of setting-based interventions to prevent illicit drug use: A systematic review of reviews. forthcoming.
  45. 45.↵
    Taleb N. The Black Swan: The Impact of the Highly Improbable: Random House; 2007.
  46. 46.↵
    Stratil J, Baltussen R, Scheel I, Nacken A, Rehfuess E. Development of the WHO-INTEGRATE evidence-to-decision framework: An overview of systematic reviews of decision criteria for health decision-making. Cost Effectiveness and Resource Allocation. 020;8(18).
  47. 47.↵
    Stratil JM, Paudel D, Setty KE, Menezes de Rezende CE, Monroe AA, Osuret J, et al. Advancing the WHO-INTEGRATE Framework as a Tool for Evidence-Informed, Deliberative Decision-Making Processes: Exploring the Views of Developers and Users of WHO Guidelines. J International Journal of Health Policy and Management. 2020:-.
  48. 48.↵
    Petticrew M, Knai C, Thomas J, Rehfuess EA, Noyes J, Gerhardus A, et al. Implications of a complexity perspective for systematic reviews and guideline development in health decision making. BMJ Glob Health. 2019;4(Suppl 1):e000899.
    OpenUrlAbstract/FREE Full Text
  49. 49.↵
    Golder S, McIntosh HM, Loke Y. Identifying systematic reviews of the adverse effects of health care interventions. BMC Med Res Methodol. 2006;6:22.
    OpenUrlCrossRefPubMed
  50. 50.
    1. Higgins J,
    2. Green S, editors
    Loke Y, Price D, Herxheimer A, Cochrane Adverse Effects Methods Group. Chapter 14: Adverse effects. In: Higgins J, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 510 [updated March 2011]: The Cochrane Collaboration; 2011.
  51. 51.↵
    Loke YK, Price D, Herxheimer A. Systematic reviews of adverse effects: framework for a structured approach. BMC Med Res Methodol. 2007;7:32.
    OpenUrlCrossRefPubMed
  52. 52.↵
    Mittelmark MB. Unintended effects in settings-based health promotion. Scandinavian journal of public health. 2014;42(15 Supplement):17–24.
    OpenUrlCrossRefPubMed
  53. 53.↵
    Hansen BT. No evidence that HPV vaccination leads to sexual risk compensation. Human vaccines & immunotherapeutics. 2016;12(6):1451–3.
    OpenUrl
  54. 54.↵
    Glover RE, van Schalkwyk MCI, Akl EA, Kristjannson E, Lotfi T, Petkovic J, et al. A framework for identifying and mitigating the equity harms of COVID-19 policy interventions. Journal of Clinical Epidemiology. 2020;128:35–48.
    OpenUrlPubMed
  55. 55.
    Grummon AH, Hall MG, Block JP, Bleich SN, Rimm EB, Taillie LS, et al. Ethical Considerations for Food and Beverage Warnings. Physiology and Behavior. 2020;222:112930.
    OpenUrl
  56. 56.
    Guttman N, Salmon CT. Guilt, fear, stigma and knowledge gaps: Ethical issues in public health communication interventions. Bioethics. 2004;18(6):531–52.
    OpenUrlCrossRefPubMedWeb of Science
  57. 57.
    Levy DT, Cummings KM, Villanti AC, Niaura R, Abrams DB, Fong GT, et al. A framework for evaluating the public health impact of e-cigarettes and other vaporized nicotine products. Addiction (Abingdon, England). 2017;112(1):8–17.
    OpenUrlPubMed
  58. 58.
    Loss J, Nagel E. Problems and ethical challenges in public health communication. Bundesgesundheitsblatt, Gesundheitsforschung, Gesundheitsschutz. 2009;52(5):502–11.
    OpenUrlPubMed
  59. 59.
    Quinn P. Crisis Communication in Public Health Emergencies: The Limits of ‘Legal Control’ and the Risks for Harmful Outcomes in a Digital Age. Life sciences, society and policy. 2018;14(1):4.
    OpenUrl
  60. 60.
    Trudell B, Whatley MH. School sexual abuse prevention: Unintended consequences and dilemmas. Child Abuse and Neglect. 1988;12(1):103–13.
    OpenUrlCrossRefPubMedWeb of Science
  61. 61.
    Wilson N, Thomson G. Tobacco taxation and public health: Ethical problems, policy responses. Social Science and Medicine. 2005;61(3):649–59.
    OpenUrl
  62. 62.↵
    Cho H, Salmon CT. Unintended Effects of Health Communication Campaigns. Journal of Communication. 2007;57(2):293–317.
    OpenUrlCrossRefWeb of Science
  63. 63.
    Dishion TJ, McCord J, Poulin F. When interventions harm. Peer groups and problem behavior. The American psychologist. 1999;54(9):755–64.
    OpenUrl
  64. 64.↵
    MacLean L, Edwards N, Garrard M, Sims-Jones N, Clinton K, Ashley L. Obesity, stigma and public health planning. Health promotion international. 2009;24(1):88–93.
    OpenUrlCrossRefPubMedWeb of Science
  65. 65.
    Brown AW, Allison DB. Unintended consequences of obesity-targeted health policy. The virtual mentor : VM. 2013;15(4):339–46.
    OpenUrl
  66. 66.
    Burgess DJ, Fu SS, van Ryn M. Potential Unintended Consequences of Tobacco-Control Policies on Mothers Who Smoke. A Review of the Literature. American Journal of Preventive Medicine. 2009;37(2 SUPPL.):S151–S8.
    OpenUrl
  67. 67.
    Kuiper N, Goldston D, Godoy Garraza L, Walrath C, Gould M, McKeon R. Examining the Unanticipated Adverse Consequences of Youth Suicide Prevention Strategies: A Literature Review with Recommendations for Prevention Programs. Suicide & life-threatening behavior. 2019;49(4):952–65.
    OpenUrl
  68. 68.
    Werch CE, Owen DM. Iatrogenic effects of alcohol and drug prevention programs. Journal of Studies on Alcohol. 2002;63(5):581–90.
    OpenUrlPubMedWeb of Science
  69. 69.
    Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet (London, England). 2020;395(10227):912–20.
    OpenUrl
  70. 70.
    Carter FA, Bulik CM. Childhood obesity prevention programs: how do they affect eating pathology and other psychological measures? Psychosomatic medicine. 2008;70(3):363–71.
    OpenUrlAbstract/FREE Full Text
  71. 71.
    Evans-Polce RJ, Castaldelli-Maia JM, Schomerus G, Evans-Lacko SE. The downside of tobacco control? Smoking and self-stigma: A systematic review. Social science & medicine (1982). 2015;145:26–34.
    OpenUrl
  72. 72.
    Haignere CS, Gold R, McDanel HJ. Adolescent abstinence and condom use: are we sure we are really teaching what is safe? Health education & behavior : the official publication of the Society for Public Health Education. 1999;26(1):43–54.
    OpenUrl
  73. 73.↵
    Lorenc T, Petticrew M, Welch V, Tugwell P. What types of interventions generate inequalities? Evidence from systematic reviews. J Epidemiol Community Health. 2013;67(2):190–3.
    OpenUrlAbstract/FREE Full Text
  74. 74.
    Ruiter RAC, Abraham C, Kok G. Scary warnings and rational precautions: A review of the psychology of fear appeals. Psychology & Health. 2001;16(6):613–30.
    OpenUrl
  75. 75.
    Thomson K, Hillier-Brown F, Todd A, McNamara C, Huijts T, Bambra C. The effects of public health policies on health inequalities in high-income countries: an umbrella review. BMC Public Health. 2018;18(1):869.
    OpenUrlPubMed
  76. 76.
    Fong V, Iarocci G. Child and Family Outcomes Following Pandemics: A Systematic Review and Recommendations on COVID-19 Policies. Journal of pediatric psychology. 2020;45(10):1124–43.
    OpenUrl
  77. 77.↵
    Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. 2021;372:n71.
    OpenUrl
  78. 78.
    Peltzman S. The Effects of Automobile Safety Regulation. Journal of Political Economy. 1975;83(4):677–726.
    OpenUrlCrossRefWeb of Science
  79. 79.
    Weiss B, Caron A, Ball S, Tapp J, Johnson M, Weisz JR. Iatrogenic effects of group treatment for antisocial youths. Journal of consulting and clinical psychology. 2005;73(6):1036–44.
    OpenUrlCrossRefPubMedWeb of Science
  80. 80.
    Hedlund J. Risky business: safety regulations, risks compensation, and individual behavior. Injury prevention : journal of the International Society for Child and Adolescent Injury Prevention. 2000;6(2):82–90.
    OpenUrl
Back to top
Posted February 08, 2023.
Download PDF

Supplementary Material

Data/Code
Email
Anticipating and assessing adverse and other unintended consequences of public health interventions: the (CONSEQUENT) framework
Jan M Stratil, Renke L Biallas, Ani Movsisyan, Kathryn Oliver, Eva A Rehfuess
medRxiv 2023.02.03.23285408; doi: https://doi.org/10.1101/2023.02.03.23285408
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area