Optimization of an adult immunization program in Canada
=======================================================
* Nathan Locke
* Mike Paulden
* Shannon E. MacDonald
* Stephanie Montesanti
* Ashleigh Tuite
* Karsten Hempel
* Wade McDonald
* Ellen Rafferty
## Abstract
**Background** Provincial decisions to fund a new immunization program are generally made on a case-by-case basis, without systematic consideration of how the new immunization program may fit within the larger provincial immunization portfolio.
**Aim** The goal of this study was to develop evidence and tools to guide policy-makers in making fiscally and ethically responsible decisions on which adult immunization programs to include in their portfolio under various constrained budgetary scenarios.
**Methods** Using previously published infectious disease models, cost-utility data was estimated for adult pneumococcal, influenza, pertussis, and shingles immunization programs. This data was then inputted into a newly developed constrained optimization model to determine portfolios of immunization programs that maximize either population health or incremental net monetary benefit, subject to a budget constraint. Sensitivity analyses were conducted on model parameters such as vaccine costs, cost-effectiveness thresholds, and the budget constraint.
**Results** Optimized solutions changed dramatically based on the number of immunization programs included, total budget, what was optimized for (i.e., population health or incremental net monetary benefit), the cost-effectiveness threshold and the assumed vaccine prices. Maximal health gains and budget spending was achieved when optimizing based on population health. Reductions in health gains and budget spending were observed at a CAN$50,000 cost-effectiveness threshold, and at a CAN$30,000 threshold, the budget was significantly underutilized and health gains were noticeably reduced.
**Conclusion** If budgets for the adult immunization portfolio are fixed, then shifting to more expensive programs that offer large health benefits may be preferable. However, if budgets can be spread across various public health programs (i.e., childhood immunization, well-baby programs), it may make more sense to optimize based on cost-effectiveness. Constrained optimization tools could improve goals-based decision-making and allow for transparent and effective methods to make allocation decisions.
**Highlights**
* Optimized solutions changed dramatically based on the number of immunization programs available, total budget, and the cost-effectiveness threshold.
* If budgets for the adult immunization portfolio are fixed, then shifting to more expensive programs that offer large health benefits may be preferable.
* If budgets can be spread across various public health programs, it may make more sense to optimize based on cost-effectiveness.
* Constrained optimization tools could improve goals-based decision-making and allow for transparent and effective methods to make allocation decisions.
## I. Introduction
Healthcare systems across the globe are under increasing financial strain. Canada has one of the highest health care spending to gross domestic product (GDP) ratios (12.3%)(1) amongst Organisation for Economic Co-operation and Development countries, behind only the United States, Germany and the United Kingdom (UK). This ratio has been steadily increasing since 1975, when healthcare made up 7% of total GDP spending in Canada(2). By taking up a larger percentage of total spending, the healthcare system puts pressure on other sectors of the economy, making it essential to identify ways to increase the efficiency of spending in healthcare.
Evidence suggests that preventive and public health interventions are generally good value for the money; and cuts to public health budgets could potentially increase costs in acute care medicine and the wider economy(3). A study from the UK found that additional spending in public health produces more health per dollar spent than identical spending in treatment activities(4). However, public health typically constitutes a substantially smaller percentage of the overall healthcare budget (6.1% of total health spending in Canada)(1), leading to public health and immunization departments grappling with tough decisions about which programs to fund to optimize health and achieve the best use of scarce financial resources.
Across Canada, one of the main responsibilities of provincial/territorial public health departments is immunization. Since immunization programs make up a substantial proportion of public health budgets, it is important that policy-makers understand the benefits and costs of these programs, and make decisions about which large-scale immunization programs they should fund. While historically immunization programs have been great value for the money(5), new programs, especially adult immunization programs, can be very expensive, raising questions about their cost-effectiveness. With multiple new adult vaccines either recently licensed or coming down the pipeline, including vaccines for influenza(6), respiratory syncytial virus (RSV)(7), and Lyme disease(8), among others, public health decision-makers across the world will need to make budgetary decisions, including whether to expand their budgets, replace pre-existing adult immunization programs, or exclude any new vaccines.
Public health researchers and policy-makers need evidence-based research and methods for determining where the limited amount of immunization budget available to them is most efficiently spent. The objectives of this research were to: 1) conduct a case study on the optimal mix of adult immunization programs in a provincial public health system based on their incremental net monetary benefit (INMB) or health benefits (quality adjusted life years – QALYs); 2) develop an interactive optimization model that can be adapted by various jurisdictions. This analysis can provide insight into the trade-offs associated with budgetary changes to a portfolio of adult immunization programs, as well as inform decisions around where new adult vaccines may fit, both compared to other programs for that same disease, and within the larger immunization portfolio.
## II. Methods
In this analysis, we conducted a constrained optimization experiment to determine the optimal (i.e., most efficient) mix of adult immunization programs that maximized either health outcomes or INMB under specific budgetary constraints. We took the perspective of a regional public health program, with the province of Alberta, Canada as the case study. We focused on sustained large-scale immunization programs available in Canada that targeted either the entire adult population or a large subset of the population (e.g., specific age groups) on an ongoing or recurring (i.e., annual/seasonal) basis. We selected vaccine-preventable diseases, with specific immunization programs for adults that all or some Canadian provinces had publicly funded, including influenza, herpes zoster (shingles), pertussis in pregnant women and pneumococcal disease. We did not include vaccines that are currently funded nationally in Canada (e.g., COVID-19), or were not licensed at the time of the analysis.
This analysis involved four key steps: 1) identification of peer-reviewed infectious disease models and relevant immunization scenarios; 2) estimation of health outcomes for each immunization scenario using the identified infectious disease models; 3) application of healthcare costs and QALYs to the estimated health outcomes and calculation of the INMB; and 4) optimization of the adult immunization portfolio based on INMB or health benefit, and subject to a budget constraint.
We received ethical approval from the University of Alberta Health Research Ethics Board (Ethics ID: Pro00112164) and funding from the Canadian Institute for Health Research (CIHR) Catalyst Grant: Impacts of financial and organizational restructuring of public health (Grant Number: 435188). CIHR had no role in the design or conduct of the study.
### Identification of peer reviewed disease models & immunization scenarios
To estimate health outcomes for each disease, we first identified pre-existing infectious disease models that were suitable for the needs of this analysis. We conducted a rapid review of the literature to identify relevant infectious disease models. Inclusion criteria for the disease models included: 1) full text article available; 2) English language; 3) evaluated an immunization program; 4) Canadian setting; and 5) covered one of the four diseases under study (i.e., pertussis, shingles, pneumococcal, and influenza). We identified 64 articles that described potentially relevant infectious disease models. We then selected the four models (9-12) used in the analysis based on the following criteria: 1) model was available and adaptable; 2) Canada-specific (with preference given to those that were Alberta-focused); 3) included the vaccines of interest; 4) time horizon of the analysis; 5) year of study; 6) stratification of the results (e.g., age groups, sex, health risk factors); and 7) health outcomes captured.
Table 1 lists the immunization scenarios that we included in the analysis. These scenarios were selected based on the capabilities of the infectious disease models available for use in this analysis and expert opinion on which vaccines would be considered for inclusion in their adult immunization portfolio. Multiple scenarios are included to capture the effects on health outcomes that occur from changes in infectious disease models parameters such as coverage rate, the age group eligible for vaccination, or the type of vaccine used. A baseline scenario is included in each disease’s set of scenarios, where no adult immunization program is offered (except for influenza, where it is assumed those under 20 are vaccinated at the rates reported in Appendix Table A.1). The baseline scenarios allow us to compare the performance of the adult immunization programs individually and against each other. Table 2 provides information on the effectiveness and costs of the vaccines listed in Table 1.
View this table:
[Table 1](http://medrxiv.org/content/early/2024/06/11/2024.06.10.24308663/T1)
Table 1 Immunization Scenarios, by Disease and Vaccine Characteristics
View this table:
[Table 2.](http://medrxiv.org/content/early/2024/06/11/2024.06.10.24308663/T2)
Table 2. Vaccine Effectiveness and Costs
### Health Outcome Data
We used the four identified infectious disease models to estimate number of vaccines provided and health outcomes (e.g., infections, physician visits, hospitalizations, number of vaccinations) for each immunization scenario. Each model was run over a 10-year time horizon for consistency between scenarios. All models were adjusted to represent the Alberta population in terms of size and age-groups.
The model selected for influenza is an age-structured stochastic compartmental model with a 10-year time horizon(11). The outcomes we estimated using this model included annual vaccinations, influenza symptomatic infections, physician visits, emergency department visits, hospitalizations, ICU cases, and deaths. Estimates of these health outcomes are broken down by year and into three age groups: children (<20), adults (20-64), and seniors (65+). The pneumococcal model is a Markov Chain model developed by the National Advisory Committee on Immunization for the evaluation of newly licensed pneumococcal vaccines(10). The model outputs the number of cases of invasive pneumococcal disease (IPD), community acquired pneumonia (inpatient or outpatient severity separately), post-meningitis sequelae (mild or severe separately) and deaths. Our analysis looks at two age cohorts, where vaccination was provided at age 50 or 65. For pertussis, we used an agent-based model developed by Hempel et al. (2023)(12) that included maternal vaccination against pertussis. The model is a stochastic dynamic infectious disease model that estimates pertussis cases across 1-year age increments between 0 and 99.
We selected an agent-based model by Rafferty et al. (2018)(9) to estimate shingles-related health outcomes associated with immunization scenarios. The outcomes produced by this model include vaccinations, infections, cases of PHN, and hospitalizations.
### Calculating Total Costs, QALYs and INMB
Once we had produced health outcomes for each vaccine scenario, we then applied healthcare costs and QALY estimates to these outputs., We used costs and QALY estimates for influenza from Fisman et al. (2011)(14), for pneumococcal from PHAC (2023)(10), for pertussis from McGirr et al. (2019)(15) and for shingles from Rafferty et al. (2021)(16) and Friesen et al. (2017)(17). All costs were adjusted to 2022 Canadian Dollars and both costs and QALYs were discounted at a rate of 1.5% per year.
Using the cost and benefit outputs, we estimated INMB for each immunization scenario in comparison to baseline (no adult vaccination). We calculated INMB using both a CAN$30,000 and CAN$50,000 threshold (18, 19).
### Optimization
Our optimization analysis was conducted in accordance with good practice recommendations set out by the Professional Society for Health Economics and Outcomes Research (Crown et al., 2017; Crown et al., 2018). Two different optimization approaches were used in this analysis, varying only in how their objective function is defined. One approach is concerned with maximizing INMB, the other is concerned with maximizing population health based on QALYs (i.e., immunization scenarios are compared based on effectiveness rather than cost-effectiveness). Occurring under a budget constraint, our approach sought to maximize the total INMB or health benefits achieved in our selected combination of immunization program scenarios. Only one immunization scenario can be selected for each disease and scenarios must be fully funded, so our optimization approach is based on integer linear programming. The notation for the objective function and constraints are as follows in Equation Set A: ![Formula][1]</img> subject to: ![Formula][2]</img> ![Formula][3]</img>
The objective function (line 1) aims to maximize the sum of benefits from a limited combination of immunization scenarios, where *j* represents a given immunization program scenario, *x* represents the benefit of immunization program scenario *j*, and *c* represents our choice variable (either a 1 or 0) on whether a given scenario is included in the portfolio. The objective function is subject to four constraints: an integer constraint on the choice variable (line 3), a budget constraint (line 2), and a disease constraint which limits the number of immunization programs that can be selected per disease in the optimal solution to one (line 2). This final constraint on programs per disease is included as our immunization program scenarios are mutually exclusive within each disease category and we did not want to include two scenarios that only differ in their coverage rate or age (for pertussis, shingles, pneumococcal), or vaccinate a specific age group with two types of vaccines (for influenza, pneumococcal). Line 2 includes *a**ij*, where *a*1*j* represents the cost of vaccination program scenario *j*, and *a*2*j* to *a*5*j* which indicate what disease scenario *j* targets. The sum across each *i* must be less than or equal to its corresponding constraint value of *b**i* which is a vector of upper bound constraints with a length of five: with *b*1 representing the adult immunization budget and the remaining four constraints, which limit the number of immunization programs per disease, having a value of one.
In the main scenario analysis, we estimated both health-maximizing and INMB at a hypothetical CAN$500M adult immunization budget. We estimated these results for portfolios that contain one, two, three or four immunization program scenarios and at two cost-effectiveness thresholds: CAN$30,000/QALY and CAN$50,000/QALY. With the exception of pneumococcal vaccines, vaccine costs used in the main scenario represent a 50% reduction of published estimates, under an assumption that public pricing is lower than vaccine list prices from the United States, and so that all diseases could be considered in the cost-effectiveness optimization. Pneumococcal vaccine prices were obtained from Canadian sources and therefore did not include this 50% reduction. We then conducted sensitivity analyses around the total adult immunization budget, looking at lower (CAN$100M) and higher (CAN$1B) budgets, higher vaccine costs (i.e., costs consistent with published estimates), and a cost-effectiveness threshold of CAN$100,000/QALY. The selection of the lower budget threshold was based on an assumption that the actual 10-year budget is likely more than CAN$100M(20), whereas the upper bound budget threshold was set at CAN$1B because values beyond this amount did not yield new optimization solutions.
We solved the optimization problem in *R(21)* using the package *FLSSS(22)*. Its performance was validated by comparing its results against the solutions of Microsoft Excel’s Solver(23), which uses an appropriate branch and bound method (24-26). We then input the optimization model into R shiny, which allows users to change key parameters, including vaccine price, cost and quality of life loss associated with health events, the type of analysis (cost-effective vs. health maximizing solutions), the immunization program scenarios, the total budget and the cost-effectiveness threshold. The model is available at [https://eshiny.ihe.ca/cvop/](https://eshiny.ihe.ca/cvop/).
## III. Results
Overall, we were able to optimize the adult immunization portfolio using both the health maximizing and the INMB approach with thresholds of CAN$30,000 and CAN$50,000, assuming a budget of CAN$500 million. We estimated all results assuming a one, two, three or four immunization program portfolio. The results of this main scenario, including total health benefit (QALYs gained), percentage of the budget used, estimated incremental cost-effectiveness ratio (ICER) based on the portfolio selected, and immunization program(s) selected, at each portfolio size, are presented in Table 3.
View this table:
[Table 3](http://medrxiv.org/content/early/2024/06/11/2024.06.10.24308663/T3)
Table 3 Vaccine Portfolio Selections, Main Scenario, CAN$500M Budget
Our results found that the health-maximizing approach always achieves a greater health benefit to that of the INMB approach. As influenza immunization scenarios offer the largest net gains in health benefits, the health-maximizing approach selects the influenza scenario that provides the most QALYs possible while remaining under the CAN$500M budget for its one program portfolio. Table 4 provides an overview of the average health benefits and vaccination program costs within each disease category. At a cost-effectiveness threshold of CAN$50,000/QALY, the INMB optimization allocates its budget to a shingles program for the one program portfolio. Note, this optimization did not select the most cost-effective immunization scenario overall, which was the Tdap immunization program (50% coverage). While the pertussis programs are very cost-effective, the total health benefit that they offer is smaller than that of shingles programs.
View this table:
[Table 4](http://medrxiv.org/content/early/2024/06/11/2024.06.10.24308663/T4)
Table 4 Average Health Benefits and Vaccination Program Costs of Scenarios, by Disease
The only portfolio without a solution was the four program portfolio at a threshold of CAN$30,000 (see Table 3). We found no feasible solution for this portfolio because there were no influenza programs deemed cost-effective at that threshold. For this reason, the INMB optimization with a CAN$30,000/QALY threshold must instead select between immunization scenarios for pertussis, pneumococcal, or shingles. As seen in Table 4, shingles would generally offer the most potential health benefit out of these three diseases, yet the optimization begins by selecting a pertussis vaccination scenario instead. Unlike the INMB optimization with a CAN$50,000/QALY threshold, shingles scenarios do not have a sufficiently large enough health benefit over pertussis to be favoured at a cost-effectiveness threshold of CAN$30,000/QALY. To illustrate the trade-off between cost-effectiveness and net health gains, Figure 1 shows how the INMB of pertussis and shingles immunization scenarios change as the cost-effectiveness threshold increases. Shingles immunization scenarios generally achieve a positive INMB at just below CAN$30,000/QALY; however, it is only around the CAN$50,000/QALY cost-effectiveness threshold that one of the shingles scenarios can achieve a larger INMB than a pertussis scenario.
[Figure 1](http://medrxiv.org/content/early/2024/06/11/2024.06.10.24308663/F1)
Figure 1
Incremental Net Monetary Benefit of Pertussis (Gold) and Shingles Vaccination (Black) Scenarios, by Cost-Effectiveness Threshold
We also present the calculated ICER of each portfolio solution. The maximum ICER value is CAN$55,818/QALY, occurring under the one program portfolio for the health-maximizing approach. As the size of the health-maximizing portfolio increases, the ICER continuously decreases. The association between portfolio size and the ICER value is a byproduct of the dynamics between scale and cost-effectiveness of immunization scenarios between disease types. Influenza vaccination programs scenarios offer the largest health benefits, whereas pneumococcal vaccination programs offer the smallest amount. At the same time, pneumococcal programs are some of the most cost-effective scenarios and influenza programs are the least. Because of these dynamics, the health-maximizing approach is gradually adding more cost-effective disease categories as portfolio size increases. Moreover, it may need to reallocate budget to move from a one-program portfolio to a two-program one (99.47% of the budget is used for the one-program portfolio), so it must also consider a more cost-effective influenza program, which leads to the ICER value decreasing further.
In addition to our main scenario, we conducted additional sensitivity analyses which involved either using the full cost of each vaccine or a CAN$100,000/QALY cost-effectiveness threshold. Results of these sensitivity analyses are presented in Appendix A (Tables B.1 and B.2). Increasing the cost-effectiveness threshold to CAN$100,000/QALY allows additional influenza scenarios with an ICER of more than CAN$50,000/QALY to be considered in the optimization.
Lastly, we also conducted a sensitivity analysis around the total budgetary constraints, where the health-maximizing optimization and CAN$50,000/QALY INMB optimization for the main scenario were re-estimated for budget values ranging from CAN$5M to CAN$1B, in increments of CAN$5M. The results of these analyses are shown in Figures 2 and 3, which highlight how INMB and health benefit grow within each portfolio size as the adult immunization budget is increased. The cost-effectiveness approach reaches its maximum potential INMB for all portfolio sizes at a budget of roughly CAN$400M; the health-maximizing approach reaches its maximum health benefit for all portfolio sizes at a budget of roughly CAN$700M. For the full table of results at budgets of CAN$100M and CAN$1B see Appendix A (Table B.3 and Table B.4). Similar figures are available in Appendix B for the sensitivity analysis using the full cost of each vaccine, looking at solutions across a range of budget values under CAN$30,000/QALY (Figure B.5), CAN$50,000/QALY (Figure B.6), and CAN$100,000/QALY (Figure B.7) cost-effectiveness thresholds, as well as the health-maximizing approach (Figure B.8).
[Figure 2](http://medrxiv.org/content/early/2024/06/11/2024.06.10.24308663/F2)
Figure 2
Main Scenario, Incremental Net Monetary Benefit Optimization at CAN$50,000/QALY
[Figure 3](http://medrxiv.org/content/early/2024/06/11/2024.06.10.24308663/F3)
Figure 3
Main Scenario, Health-maximizing Optimization
## IV. Discussion
In this analysis, we optimized the adult immunization portfolio in Alberta based on INMB and health benefit. We found that the optimized solutions changed dramatically based on the number of immunization programs available, total budget, what we optimized for (i.e., health or INMB), the cost-effectiveness threshold and the assumed vaccine prices. When looking across the entire adult immunization program, we found that on average adult immunizations were not as good value for the money as what has previously been reported for childhood immunization programs (5), which are often found to be cost-saving (whereas none of the adult programs were found to be cost-saving). In comparison, at a CAN$30,000 cost-effectiveness threshold, even when assuming a 50% discount from the listed vaccine prices, there was no adult influenza program that was considered as part of the cost-effective optimization solution. Similarly, if we assumed no discount from vaccine list price at a threshold of CAN$30,000, only two programs were included in the final optimization model (Maternal Tdap [50% coverage] and Pneu-P-23 [at age 50]) using up only 5.67% of the CAN$500 million budget.
The chosen immunization programs, however, change substantially when optimizing health benefit instead of INMB at the CAN$30,000 threshold, with influenza vaccine programs having the highest health benefits, and therefore being the first program funded. This suggests that how budgets are distributed across public health and vaccines may have a big impact on which vaccine programs decision-makers selected for inclusion in the adult immunization portfolio. For instance, if budgets for the adult immunization portfolio are fixed, then the goal should be to maximize health benefits within that portfolio, with the potential to shift to more expensive programs that offer large health benefits. As observed in this analysis, a health maximizing approach will typically use up a higher percentage of the total budget, ensuring more health benefits for the population. However, if budgets can be spread across various programs, it may make more sense to optimize based on INMB, allowing budgets to be reallocated to more cost-effective programs within the public health or immunization portfolios (e.g., shifting more resources to improving coverage rates of childhood vaccines that are typically more cost-effective).
These findings demonstrate the complexity of decision-making around how to allocate scarce resources for immunizations and the need for methods and tools to allow these decisions to be made in real time with the best evidence available. By setting up an interactive optimization model, we allow for flexibility in decision-making, where we can incorporate new adult vaccines into the model as they become licensed or as immunization budgets change, providing an entire program perspective on where these new vaccines will fit within the larger adult immunization portfolio, and aiding in more efficient and transparent resource allocation decision-making.
While previous immunization studies have focused on constrained optimization of a specific vaccine program based on vaccines available (e.g., influenza, pneumococcal) or the efficient allocation of scarce vaccine resources (e.g., COVID-19 vaccines)(27-29), this is the first time to our knowledge of the approach being employed across an entire immunization portfolio. This study recognizes that a policy-decision based on one vaccine (e.g., to fund a new vaccine), can have big impacts on funding for other vaccines and health programs. By looking across the portfolio, the costs and benefits of multiple vaccination programs can be weighed simultaneously by policy-makers. As noted by Crown (2020)(30) and Standaert et al. (2020)(31), these methods are not being commonly applied within the healthcare system, but have the potential to support a range of programming decisions, including optimizing public health subject to a specific budget as we present here. Other areas of application include efficient delivery of supplies, health facility capacity management, staff scheduling, clinical decision-making, clinical trial design and optimal resource allocation, among others(32).
The analysis had a few limitations. First, since we estimated health outcomes associated with the different immunization programs using infectious disease models outside of the optimization model, we did not incorporate uncertainty around health outcomes in the optimization analysis. Future analyses could embed the infectious disease models directly in the optimization model. While this would allow for more complex analysis of uncertainty around the optimization experiment, this would substantially increase the run-time of the optimization model. Second, since the optimization results are based on the health outcomes estimated in the infectious disease models, they are subject to the limitations of each infectious disease model. While all the infectious disease models are Canadian-based and have undergone some level of peer review, they all include structural and methodological assumptions that may vary across the models and that should be taken into consideration when using the information produced by the optimization model. Third, there remains key uncertainties around vaccine price, which could substantially change the optimization results. In the main scenario we assumed a 50% reduction in listed vaccine prices; however, due to confidentiality surrounding government-negotiated vaccine prices, it is hard to judge whether this is a reasonable assumption. This highlights the importance of the interactive optimization model, where users can change input parameters, including vaccine price, and see how that impacts the optimization results. Fourth, we have only included vaccines that are publicly-funded by at least one province in Canada, and therefore vaccines funded through the federal government (e.g., COVID-19) and those coming down the pipeline at the time of our analysis (e.g., RSV vaccine for older adults) were not included in this initial analysis; however, these can be added to the future iterations of the analysis. Fifth, based on the population subgroups modelled in the infectious disease models, our ability to consider equity weighting for at-risk populations was limited. Through the interactive application, each immunization scenario can be assigned a specific cost-effectiveness threshold, which would allow weighting to be considered for at-risk groups were the necessary data to become available. Finally, Canada has no empirically derived cost-effectiveness threshold, which can substantially alter the vaccines selected as part of the cost-INMB optimization. Having an idea of the opportunity costs associated with resource allocation decisions would allow a more robust analysis of the tradeoffs associated with funding new vaccine programs.
This model could be expanded in several ways in the future. As noted above, the inclusion of vaccines in the pipeline for adults, as well as those currently funded through a different source, could help decision-makers who may need to make resource allocation decision about these vaccines in the near future. Moreover, more complex inclusion of uncertainty around health outcomes associated with each vaccination program could improve our understanding of optimization model unknowns. The methods used in this analysis could be included more broadly within the world of immunization, to evaluate the entire immunization portfolio, and provide decision makers a way to evaluate re-allocation of funds across a multitude of immunization programs. This more complete analysis would also have the added benefits of being able to effectively calculate a cost-effectiveness threshold for the immunization portfolio, which would also help with allocation decision-making. Future analyses could also add relevant constraints that were not included at this stage, such as vaccine administration constraints (e.g., the number of public health nurses or pharmacists available to vaccinate). Moreover, consideration could be given to how decision-making, and the factors considered, may change when adding a new vaccine to the portfolio versus optimizing an existing suite of immunization programs.
## V. Conclusion
This model demonstrates the informative power behind the constrained optimization approach when evaluating an entire portfolio of immunizations, rather than the current system of evaluating each new immunization program individually. As more adult vaccines come down the pipeline, it is essential decision-makers start to make allocation decisions within the larger context, especially as our analysis found adult vaccine programs may not be as good of value for the money, in comparison to childhood immunization programs or other public health interventions(4, 5).
Moreover, having optimization tools for a range of interventions available to decision-makers in real-time could improve goals-based decision-making, and improve effectiveness and transparency in making allocation decisions. This method requires decision-makers to think about the ultimate goals of their portfolios, along with any elements that may constrain their decision-making.
## Supporting information
Supplemental File [[supplements/308663_file03.pdf]](pending:yes)
## Data Availability
The optimization model is available on request, and there is a publicly available version available online.
[https://eshiny.ihe.ca/cvop/](https://eshiny.ihe.ca/cvop/).
## Conflict of Interests
The Authors declare that they have no conflicts of interest.
## Acknowledgements
We would like to acknowledge Danica Wolitski, Amy Elefson and staff at Alberta Health for the valuable expertise and feedback they shared with us. We would also like to thank Austin Nam for the use of the pneumococcal model he developed.
* Received June 10, 2024.
* Revision received June 10, 2024.
* Accepted June 11, 2024.
* © 2024, Posted by Cold Spring Harbor Laboratory
The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission.
## References
1. 1.Canadian Institute for Health Information. National health expenditure trends, 2023 — Snapshot. Accessed November 21, 2023. Available from: [https://www.cihi.ca/en/national-health-expenditure-trends-2023-snapshot](https://www.cihi.ca/en/national-health-expenditure-trends-2023-snapshot).
2. 2.Canadian Institute for Health Information. National health expenditure trends. Accessed November 21, 2023. Available from: [https://www.cihi.ca/en/national-health-expenditure-trends](https://www.cihi.ca/en/national-health-expenditure-trends).
3. 3.Masters R, Anwar E, Collins B, et al. Return on investment of public health interventions: a systematic review. J Epidemiol Community Health. 2017; 71: 827–34. DOI:10.1136/jech-2016-208141.
[Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiamVjaCI7czo1OiJyZXNpZCI7czo4OiI3MS84LzgyNyI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzA2LzExLzIwMjQuMDYuMTAuMjQzMDg2NjMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)
4. 4.Martin S, Lomas J, Claxton K. Is an ounce of prevention worth a pound of cure? A cross-sectional study of the impact of English public health grant on mortality and morbidity. BMJ Open. 2020; 10: e036411. DOI:10.1136/bmjopen-2019-036411.
[Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMzoiMTAvMTAvZTAzNjQxMSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzA2LzExLzIwMjQuMDYuMTAuMjQzMDg2NjMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)
5. 5.Zhou F, Shefer A, Wenger J, et al. Economic evaluation of the routine childhood immunization program in the United States, 2009. Pediatrics. 2014; 133: 577–85. DOI:10.1542/peds.2013-0698.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1542/peds.2013-0698&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=24590750&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F06%2F11%2F2024.06.10.24308663.atom)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000335343500003&link_type=ISI)
6. 6.University of Minnesota. The IVR Initiative (Influenza Vaccines Roadmap). Last update May 3, 2024. Accessed May 7, 2024. Available from: [https://ivr.cidrap.umn.edu/universal-influenza-vaccine-technology-landscape](https://ivr.cidrap.umn.edu/universal-influenza-vaccine-technology-landscape).
7. 7.Public Health Agency of Canada. Canadian Immunization Guide: Part 4. Immunizing Agents [Internet]. Ottawa (ON): Government of Canada. Last update: Jan 1, 2024. Accessed May 7, 2024. Available from: [https://www.canada.ca/en/public-health/services/publications/healthy-living/canadian-immunization-guide-part-4-active-vaccines/respiratory-syncytial-virus.html](https://www.canada.ca/en/public-health/services/publications/healthy-living/canadian-immunization-guide-part-4-active-vaccines/respiratory-syncytial-virus.html).
8. 8.Pfizer (2022). Pfizer and Valneva Initiate Phase 3 Study of Lyme Disease Vaccine Candidate VLA15 [press release]. Available from: [https://www.pfizer.com/news/press-release/press-release-detail/pfizer-and-valneva-initiate-phase-3-study-lyme-disease](https://www.pfizer.com/news/press-release/press-release-detail/pfizer-and-valneva-initiate-phase-3-study-lyme-disease).
9. 9.Rafferty E, McDonald W, Qian W, et al. Evaluation of the effect of chickenpox vaccination on shingles epidemiology using agent-based modeling. Peerj. 2018; 6: e5012. DOI:10.7717/peerj.5012.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7717/peerj.5012&link_type=DOI)
10. 10.National Advisory Committee on Immunization. Recommendations on the use of conjugate pneumococcal vaccine – 15 valent (PNEU-C-15) and 20 valent (PNEU-C-20) in adults: Economic evidence supplementary appendix [Internet]. Date published: 2023-02-24. Accessed November 21, 2023 Available from [https://www.canada.ca/en/public-health/services/immunization/national-advisory-committee-on-immunization-naci/public-health-level-recommendations-use-pneumococcal-vaccines-adults-including-use-15-valent-20-valent-conjugate-vaccines/economic-evidence-supplementary-appendix.html](https://www.canada.ca/en/public-health/services/immunization/national-advisory-committee-on-immunization-naci/public-health-level-recommendations-use-pneumococcal-vaccines-adults-including-use-15-valent-20-valent-conjugate-vaccines/economic-evidence-supplementary-appendix.html).
11. 11.Fisman DN, Tuite AR. Estimation of the health impact and cost-effectiveness of influenza vaccination with enhanced effectiveness in Canada. PLoS ONE. 2011; 6: e27420. DOI:10.1371/journal.pone.0027420.
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=22110645&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F06%2F11%2F2024.06.10.24308663.atom)
12. 12.Hempel K, McDonald W, Osgood ND, et al. Evaluation of the effectiveness of maternal immunization against pertussis in Alberta using agent-based modeling: A Canadian immunization research network study. Vaccine. 2023; 41: 2430–38. DOI:10.1016/j.vaccine.2022.12.071.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2022.12.071&link_type=DOI)
13. 13.Public Health Agency of Canada. Herpes zoster (shingles) vaccine: Canadian Immunization Guide [Internet]. Ottawa (ON): Government of Canada. Last update: 2023 Oct. Accessed November 21, 2023. Available from: https://www.canada.ca/en/public-health/services/publications/healthy-living/canadian-immunization-guide-part-4-active-vaccines/page-8-herpes-zoster-(shingles)-vaccine.html.
14. 14.Andradottir S, Chiu W, Goldsman D, et al. Reactive strategies for containing developing outbreaks of pandemic influenza. BMC Public Health. 2011; 11 Suppl 1: S1. DOI:10.1186/1471-2458-11-S1-S1.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1471-2458-11-S6-S1&link_type=DOI)
15. 15.McGirr A, Fisman DN, Tuite AR. The health and economic burden of pertussis in Canada: A microsimulation study. Vaccine. 2019; 37: 7240–47. DOI:10.1016/j.vaccine.2019.09.070.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2019.09.070&link_type=DOI)
16. 16.Rafferty ERS, McDonald W, Osgood ND, et al. What We Know Now: An Economic Evaluation of Chickenpox Vaccination and Dose Timing Using an Agent-Based Model. Value Health. 2021; 24: 50–60. DOI:10.1016/j.jval.2020.10.004.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jval.2020.10.004&link_type=DOI)
17. 17.Friesen KJ, Chateau D, Falk J, et al. Cost of shingles: population based burden of disease analysis of herpes zoster and postherpetic neuralgia. BMC Infect Dis. 2017; 17: 69. DOI:10.1186/s12879-017-2185-3.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12879-017-2185-3&link_type=DOI)
18. 18.Ochalek JM LJ, Claxton KP. Assessing health opportunity costs for the Canadian health care systems. 2018.
19. 19.Griffiths EA, Vadlamudi NK. CADTH’s $50,000 Cost-Effectiveness Threshold: Fact or Fiction? Value Health. 2016; 19: A488–A89. DOI:10.1016/j.jval.2016.09.821.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jval.2016.09.821&link_type=DOI)
20. 20.Government of Alberta (2023). New funding to stabilize primary health care [Press release]. Available from [https://www.alberta.ca/release.cfm?xID=895228C5726BF-03E2-A938-C9107415EA1FC1BA](https://www.alberta.ca/release.cfm?xID=895228C5726BF-03E2-A938-C9107415EA1FC1BA).
21. 21.Team R. R: A Language and Environment for Statistical Computing. RStudio 2022.07.2+576 ed: R Foundation for Statistical Computing. Available from [https://www.R-project.org/](https://www.R-project.org/).
22. 22.Liu CW. FLSSS: Mining Rigs for Problems in the Subset Sum Family. Available from [https://CRAN.R-project.org/package=FLSSS](https://CRAN.R-project.org/package=FLSSS).
23. 23.Microsoft Corporation. Microsoft Excel 2016.
24. 24.Frontline Systems Inc. Excel Solver – Integer Programming. 2024. Accessed November 21, 2023. Available from [https://www.solver.com/excel-solver-integer-programming](https://www.solver.com/excel-solver-integer-programming).
25. 25.Land AH, Doig AG. An Automatic Method of Solving Discrete Programming Problems. Econometrica. 1960; 28. DOI:10.2307/1910129.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2307/1910129&link_type=DOI)
[Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1960CAL7500001&link_type=ISI)
26. 26.Shih W. A Branch and Bound Method for the Multiconstraint Zero-One Knapsack Problem. The Journal of the Operational Research Society. 1979; 30. DOI:10.2307/3009639.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2307/3009639&link_type=DOI)
27. 27.Olbrecht J, Gomez JA, Van Bellinghen LA, et al. Constrained Optimization for Pneumococcal Vaccination in Brazil. Value Health Reg Issues. 2021; 26: 40–49. DOI:10.1016/j.vhri.2020.11.011.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vhri.2020.11.011&link_type=DOI)
28. 28.Buhat CAH, Lutero DSM, Olave YH, et al. Using Constrained Optimization for the Allocation of COVID-19 Vaccines in the Philippines. Appl Health Econ Health Policy. 2021; 19: 699–708. DOI:10.1007/s40258-021-00667-z.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s40258-021-00667-z&link_type=DOI)
29. 29.Warth J, Desforges JF. Determinants of intracellular pH in the erythrocyte. Br J Haematol. 1975; 29: 369–72. DOI:10.1111/j.1365-2141.1975.tb01833.x.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1365-2141.1975.tb01833.x&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=34&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F06%2F11%2F2024.06.10.24308663.atom)
30. 30.Crown WH. The Potential Role of Constrained Optimization Methods in Healthcare Decision Making. Appl Health Econ Health Policy. 2020; 18: 461–62. DOI:10.1007/s40258-020-00559-8.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s40258-020-00559-8&link_type=DOI)
31. 31.Standaert B, Van Vlaenderen I, Van Bellinghen LA, et al. Constrained Optimization for the Selection of Influenza Vaccines to Maximize the Population Benefit: A Demonstration Project. Appl Health Econ Health Policy. 2020; 18: 519–31. DOI:10.1007/s40258-019-00534-y.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s40258-019-00534-y&link_type=DOI)
32. 32.Crown W, Buyukkaramikli N, Thokala P, et al. Constrained Optimization Methods in Health Services Research-An Introduction: Report 1 of the ISPOR Optimization Methods Emerging Good Practices Task Force. Value Health. 2017; 20: 310–19. DOI:10.1016/j.jval.2017.01.013.
[CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jval.2017.01.013&link_type=DOI)
[PubMed](http://medrxiv.org/lookup/external-ref?access_num=28292475&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F06%2F11%2F2024.06.10.24308663.atom)
[1]: /embed/graphic-5.gif
[2]: /embed/graphic-6.gif
[3]: /embed/graphic-7.gif