Abstract
Background China has low seasonal influenza vaccination rates among children and older adults. This quasi-experimental pragmatic trial examined the effectiveness of a pay-it-forward intervention on influenza vaccine uptake among children and older adults compared to standard of care (user-paid vaccination) and free vaccination strategies in China. Pay-it-forward is a community-engaged social innovation in which people receive a free influenza vaccination from a local group and are then asked if they would like to donate financially and/or create a postcard message to promote vaccination among future individuals.
Methods We iteratively co-created the pay-it-forward intervention with the community through public engagement and a participatory hackathon. We implemented the three study arms in three clinics (rural, suburban, urban) in Guangdong Province, China. A total of 225 children aged between six months and eight years old and 225 older adults (60 years old or above) were recruited into the study. Regression methods were used to compare influenza vaccine uptake (administrative records) and vaccine confidence (self-report) between the three arms.
Results Among all participants, 55/150 (36.7%) in the standard of care arm received an influenza vaccine. 111/150 (74.0%) of people in the pay-it-forward arm received an influenza vaccine. 114/150 (76.0%) of people offered free vaccination received an influenza vaccination. Similar trends were observed among children and older adults. The pay-it-forward arm had significantly higher vaccine confidence when compared to the standard of care arm. In the pay-it-forward arm, 107/111 (96.4%) of participants donated money for subsequent vaccinations and 19 of 60 invited (31.7%) created postcard messages.
Conclusions Pay-it-forward was effective in improving influenza vaccine uptake and engagement among children and older adults. Our data have implications for pro-social interventions to enhance influenza uptake in the many countries where influenza vaccines are available for a fee.
Trial registration ChiCTR2000040048
Introduction
In mainland China, an average of 10 people die from influenza-related illnesses each hour.1 Influenza vaccination is the most effective way to prevent morbidity and mortality attributable to influenza.2 Influenza vaccine is increasingly important during COVID-19 because it might help reduce risks of acquiring SARS-CoV-2 that causes COVID-19.3,4 The Chinese Center for Diseases Control and Prevention (China CDC) guidelines recommend influenza vaccination for high-risk populations, including children aged below 5 years old and older adults (people older than 60 years old). However, influenza immunization policies widely vary,5 and most cities in China do not provide free influenza vaccines to high-risk individuals. A meta-analysis reported less than one-fifth of children and older adults in China received an influenza vaccine in the past year.6
There are several reasons for low influenza vaccination uptake in China.6,7 First, most people in China are unaware of influenza vaccination and many people are hesitant about vaccine safety and effectiveness.8 Second, there is minimal public engagement in vaccinations.9 Despite a strong rationale for public engagement, few programs engage the public regarding influenza vaccinations. Third, there are limited public resources to support influenza vaccination among high-risk populations. The influenza vaccine is largely not covered by mandatory health insurance schemes and, as a result, most people have to pay $8.5-23.5 out-of-pocket to be vaccinated.10 Innovative strategies are needed to improve influenza vaccine uptake.
Pay-it-forward is a community-engaged social innovation, which has an individual receive a free influenza vaccine and a hand-written postcard message co-created by previous participants informing them that someone else has paid for them to receive a free vaccine.11 After they receive vaccination, they are asked if they would like to support the vaccination of a subsequent person (supplementary figures: Fig 1). Our previous pay-it-forward studies focused on increasing testing for sexually transmitted infections among sexual minorities. The pay-it-forward arm had a chlamydia and gonorrhea dual test uptake of 56% compared to 18% in the standard of care arm, where participants had to pay out-of-pocket.12,13 In addition, over 90% of participants donated to the rolling finance pool, 12,13 and qualitative data showed that trust in health services improved among participants in the pay-it-forward arm.14
In this quasi-experimental pragmatic trial, we assessed the effectiveness of a pay-it-forward intervention to increase influenza vaccination uptake at three sites among children (aged between 6 months and 8 years) and older adults (aged 60 or above) in comparison to free vaccination and the current standard of care in China.
Methods
Study design and participants
Guangdong is a subtropical province in southern China with a population of over 120 million. In southern China, influenza is prevalent throughout the year.15 In this study, we selected three research sites where influenza vaccination was only available on a for-fee basis. These three study sites were: a rural site (Yangshan County, Qingyuan City), a suburban site (Zengcheng District, Guangzhou City), and an urban site (Tianhe District, Guangzhou, City). Study sites included community health centers (primary care facilities providing day-to-day healthcare in China) and vaccine centers. Clinics were selected because they had sufficient influenza vaccines in stock and health professionals (nurses, doctors) familiar with influenza vaccination.
This study consisted of three stages 1) co-creation of the intervention with stakeholders and engagement strategies during a three-day hackathon; 2) a feasibility pilot to inform the recruitment process and sample size calculations; and 3) a pay-it-forward quasi-experimental pragmatic trial to evaluate the effectiveness of the intervention.
Co-creation of intervention
Our team of three individuals joined a participatory hackathon from November 4-6 2019, to co-create the pay-it-forward intervention. Co-creation is an iterative, bidirectional partnership between researchers and the public to develop new ideas.16 Participants included potential end users, public health practitioners, health innovators, communication experts and vaccine experts. We mapped out the following elements of the study: 1) key stakeholders of the study; 2) potential user journeys; 3) engagement strategies; 4) behavioral mechanisms; and 5) donation strategies. Community engagement strategies used in this study included the following: inviting community members to design postcards (Supplementary figures: Fig 2); working in partnership with a local older adult to develop a video to explain pay-it-forward; inviting study participants to create hand-written postcard messages during recruitment for future participants (Supplementary figures: Fig 3); and engaging local community staff in implementing the quasi-experimental study, including having one-to-two community staff members at each study site to help adjust recruitment and communication efforts to the local dialect.
Pilot
Before the quasi-experimental study, we carried out a feasibility pilot at the rural study site from January to April 2020, which occurred during the social distancing period due to COVID-19 in China. The primary outcome of the study was influenza vaccination uptake. The purpose of the pilot was to finalize the pay-it-forward intervention process, assess feasibility, and estimate effect size to inform power calculations. This pilot demonstrated that, in the pay-it-forward arm, 90.9% (40/44) of participants received an influenza vaccine and 93% (37/40) of participants donated funds. Thirteen of 57 participants (22.3%) in the standard of care arm received a vaccine.
Sample size calculation
We stratified sample size calculations by age groups, given the differences between children and older adults. Based on our pilot data, we anticipated that the proportion of vaccine uptake in the standard-of-care arm was 30%, and the proportion of vaccine uptake in the pay-it-forward arm was 80%, a significance level of 0.025; therefore, a sample size 100 (50 in the control arm and 50 in the intervention) would give us 90% power to test the proportion difference with a margin of 10%. We increased the sample size by 50% to allow for secondary analyses, resulting in a sample size of 75 for each age group in each arm. In addition, we included a free vaccine arm with the same sample size as the other two arms. This free vaccination arm was included because it has important implications for policy and global relevance to countries that already provide free influenza vaccinations. In sum, we required the enrollment of 225 children and 225 older adults in order to have sufficient power.
Quasi-experimental pragmatic trial
This trial evaluated the pay-it-forward intervention arm against both the standard of care and a free vaccination program implemented in rural, suburban, and urban study sites. Each study site implemented all study arms and recruited participants were chronologically allocated (non-random) into the specified study arms because of practical considerations. Influenza vaccine services are usually available in China from September to April. Influenza vaccine availability is idiosyncratic at specific health facilities because of the periodical supply and procurement system in local settings. We allocated study arms to ensure a stable supply of vaccines (Supplementary figures: Fig 4: time-based recruitment). At each site, the standard of care arm was followed by the pay-it-forward arm. Despite discussions with health authorities and vaccine manufacturers, study sites encountered lapses in supply. The duration of time needed to recruit each study arm was related to the availability of vaccine and the number of people willing to participate.
The inclusion criteria for this study differed by age group and were determined according to China’s national influenza vaccine guidelines.17 Childhood eligibility criteria included the following: aged between six months and eight years old; no acute moderate or severe illnesses; eligible to receive an influenza vaccine based on clinical evaluation from a physician; has a legal guardian (e.g. a parent or grandparent) who lives in China and consents to participate in the study; and has not received an influenza vaccine in the past year. Older adult eligibility criteria included the following: ≥ 60 years old; no acute moderate or severe illness; eligible to receive an influenza vaccine based on clinical evaluation from a physician; capable of making informed decisions and consenting to participate in the study; and have not received an influenza vaccine in the past year. If multiple people in a family were eligible to join the study, we only allowed one person to join. All eligible children and older adults presenting to these sites were invited to participate by local medical staff involved in the study during the recruitment periods.
Ethical approval for this study was obtained from the institutional review boards at the London School of Hygiene & Tropical Medicine (approval number 19100) and the Zhuhai Center for Disease Control. Online consent was obtained from guardians of children and older adults.
Procedures
Among all potential participants visiting the selected clinics, health care workers assessed eligibility for the study based on inclusion criteria and introduced eligible participants to project staff.
Standard of care
Participants recruited in the standard of care arm were provided with a brief introduction to the influenza vaccine by project staff using a pamphlet about influenza and influenza vaccination (Supplementary figures: Fig 5). They were then asked if they were willing to pay out of pocket at the standard market price (US$8.5-23.5 depending on the market price of vaccines provided at the clinic) to receive an influenza vaccination. Those who agreed to pay were screened for vaccination eligibility, and those without any contraindications received the vaccine.
Pay-it-forward
Participants recruited in the pay-it-forward arm were provided with the same introductory pamphlet about influenza and influenza vaccination. Project staff then explained the pay-it-forward program, including its purpose, the opportunity to receive one dose of influenza vaccination for free, and the opportunity to donate money towards someone else’s vaccine dose and write postcard messages (Supplementary figures: Fig 6). Participants were told that the normal price to receive an influenza vaccine, including administration fees were RMB 56(US$ 8.5) for children and 153(US$ 23.5) for adults, and that previous participants had donated money to cover the costs and had also created handwritten postcards for them.
If the participants decided to receive vaccination, they were asked prior to receiving the vaccination whether they were willing to donate any amount of money into a pool of funds to support subsequent participants in receiving the same vaccine. They were assured that the donation was entirely voluntary, and any donation amount was acceptable and would not affect whether they received a vaccination or subsequent care. They were also invited to write anonymous postcard messages for future participants. A donation collection box was provided on-site for those who preferred to donate cash. A QR code using WeChat (a multifunctional social mobile app embedded with anonymous money transfer functions) was provided to those who chose to make online donations.
Donations were used to support the vaccination of subsequent participants and aggregated data on donation amounts were made publicly available on the website and WeChat newsletter of Social Entrepreneurship to Spur Health (a research hub in the UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases Social Innovation in Health Initiative). COVID-19 conditions at the rural site prevented participants from creating handwritten postcards during some periods of the trial.
Free vaccination
Participants in the free vaccination arm were invited to participate using the same introductory pamphlet and were provided with free influenza vaccination. They did not receive any community-created messages about the pay-it-forward program.
Participation in each arm was voluntary and anonymous. After introducing the intervention, all participants were asked to complete a short, self-administered online questionnaire to collect information about sociodemographic characteristics and attitudes towards influenza vaccines. Vaccine confidence in importance, safety, and effectiveness were measured using survey items adapted to assess influenza vaccine confidence in China.18,19 Those who had difficulty reading the questionnaire were assisted by the project and healthcare staff on-site. A small gift worth around RMB10 (US$1.5) was given to each participant after completing the questionnaire survey.
Data collection
Data collection was conducted from September 2020 to March 2021. The study collected the following information: administrative data recorded by research staff using a standard information tracking sheet including the number of invited and participating individuals, the number of participants who received the vaccine, the number of individuals who donated and the amount donated in the pay-it-forward arm, as well as survey data through a self-administered survey instrument (supplementary questionnaire). Administrative and survey data were linked using numerical IDs. We collected information about the number of participants in the pay-it-forward arm who donated and corresponding donation amount, and those who created a postcard text for subsequent people. Costs associated with each arm were collected for an economic evaluation.
Data analysis
Descriptive analyses were conducted to summarize each arm’s sociodemographic and behavioral characteristics, participation rate, and vaccination rate. We used a Chi-squared test to investigate differences in vaccination uptake between the standard of care, pay-it-forward, and free vaccination arms. We ran multivariable logistic regression to examine the association between vaccine uptake and interventions after adjusting for age, sex, study sites, education, occupation, income and marital status. We also summarized the participants’ donations in the pay-it-forward arm, and compared proportions of participants between rural, suburban and urban sites who contributed US$7.6 (close to a child vaccine cost) or more. All data were analyzed using SPSS Version 25 and SAS.
Cost Analysis
A decision tree was built to calculate and compare the costs and outcomes of the three influenza vaccination arms examined in the quasi-experimental study. We evaluated the costs of all three arms using a micro-costing approach and reported this in 2020 USD. The costs of implementing each strategy were estimated using invoices, onsite staff’s self-reporting the wages of healthcare workers, and estimated opportunity costs of community staff’s time (supplementary economic evaluation). The analysis was performed from the perspective of the healthcare provider, the Guangdong Department of Health. The time horizon considered was the duration of the seasonal influenza vaccination program. We reported the total economic and financial cost for each arm, the cost per person vaccinated, and the incremental cost-effectiveness ratios. Incremental economic costs were greater for PIF compared to free vaccination because of additional costs related to volunteer time in the PIF design, as well as recruitment and donation process associated with the start-up, and recurrent costs. However, incremental financial costs were greater for free vaccination compared to pay-it-forward because financial costs were obtained from subtracting donation contributions from the total economic cost.
Results
In total, 254 children’s caregivers and 262 older adults were approached at the three study sites (Fig 1). Forty-six people declined to participate, and 30 had recently received the influenza vaccine. In total, 450 participants were enrolled and completed the survey, including 150 in the pay-it-forward arm, 150 in the free vaccine arm, and 150 in the standard of care arm. All 450 responses were screened for completeness and included in the final statistical analyses.
Characteristics of caregivers and older adult participants were similar across the three arms (Table 1). Overall, 111 (74.0%) of 150 pay-it-forward participants, 114 (76.0%) of 150 participants offered free vaccination, and 55 (36.7%) of 150 participants in the standard of care arm received influenza vaccination (χ² test p<0.001; fig 2, supplementary tables: table 1). Among children, the pay-it-forward arm had an uptake rate of 88.0% (66/75) compared to 53.3% (40/75) in the standard of care arm. Among older adults, the pay-it-forward arm had an uptake rate of 60.0% (45/75) compared to 20.0% (15/75) in the standard of care arm. Differences in uptake between the pay-it-forward and standard of care arms remained statistically significant for both groups after adjusting for study site and educational level (Supplementary tables: table 1). Uptake in the pay-it-forward and free vaccination arms were not significantly different for both age groups.
Sample characteristics of recruited child caregivers and older adults in Guangdong Province, China, 2021 (N=450)
Multivariable logistic regression to compare vaccine uptake rates of three arms by age groups in Guangdong Province, China, 2020-2021 (N=450)
Influenza vaccine uptake rates by intervention arms and age group in Guangdong Province, China, 2020-2021 (N=450)
Table 3 suggests that people in the pay-it-forward arm were more likely to receive the vaccine compared to people in the standard of care arm (adjusted odd ratio (aOR)=7.9, 95%CI [3.2-19.7] among children; aOR=6.1, [2.8-13.2] among older adults). Free-of-charge vaccination participants had greater odds than those in the standard of care arms to receive a vaccination (aOR=5.5, [2.4-12.6] among children; aOR=9.1, [4.2-20.0] among older adults). People in the pay-it-forward arm had higher vaccine confidence in safety compared to people in the standard of care arm (aOR=2.25, [1.27-4.01], with greater odds than those in the free vaccination arm (aOR=1.55, [0.91-2.63]) (Table 3). Similar trends were observed in confidence in vaccine effectiveness and importance.
Association between study arm and vaccine confidence in influenza vaccine in Guangdong Province, China, 2020-2021 (N=450) Study arm Vaccine confidence - Safety Vaccine confidence - Importance Vaccine confidence – Effectiveness
Study participants contributed to the development of influenza vaccination materials in several important ways. 19/60 (31.7%) people created handwritten postcards for subsequent participants. Six postcard designs were subsequently used to explain the pay-it-forward system to potential participants. Most handwritten messages expressed general good wishes. In addition, we co-created a video and one local older adult contributed to this video design, implementation, and evaluation (supplementary video link).
Among 111 participants in the pay-it-forward arm who received the influenza vaccine, 107 (96.4%) donated money, with a total contribution of US$597.62. Donations covered 36.0% of vaccination costs in the pay-it-forward arm. The median donation was US$4.6. Only 30% of donors in the rural site contributed US$7.6 or more compared to 61.9% in the suburban and 40.0% in urban sites (supplementary Fig 7 and supplementary table 2).
The total financial cost of implementing an influenza vaccination intervention for children and older adults was US$2,725 for the standard of care arm, US$4,477 for the pay-it-forward arm, and US$4,665 for the free vaccination arm. The incremental cost for each treatment arm, the incremental number of people vaccinated, and the incremental cost-effectiveness ratios (ICERs) based on financial and economic costs are shown in Table 4 and in the supplementary economic evaluation file. Based on the ICER obtained from comparing pay-it-forward to the standard of care, the financial cost required per additional person vaccinated was US$31.29. The financial cost required per additional person vaccinated was US$62.67 in the free vaccination arm.
Economic evaluation of the influenza vaccine strategies
When economic costs are considered, the economic cost of implementing an influenza vaccination intervention for children and older adults was US$3,557 for standard of care, US$5,062 for pay-it-forward, US$4,665 for the free vaccination arm.
Discussion
Low influenza vaccination rates among high-risk populations are a major health problem in low and middle-income countries. Our quasi-experimental study assessed the effectiveness of a pay-it-forward influenza vaccination intervention to improve uptake and engagement. Our study contributes to the literature by determining the effectiveness of a social innovation using a quasi-experimental study, developing new methods for influenza vaccination public engagement, and identifying a new method to enhance influenza vaccine uptake. Our data suggest that the pay-it-forward strategy may increase influenza vaccine uptake among high-risk individuals compared to the current self-pay strategy for vaccination. This strategy substantially increased vaccine uptake compared to the standard of care, elicited financial contributions, improved vaccine confidence, and co-created participatory messages.
We found that children and older adults who took part in pay-it-forward had higher influenza vaccine uptake than they did if they needed to self-pay for vaccination. This finding is consistent with previous intervention studies using pay-it-forward to improve health services uptake.12,13 The effect of pay-it-forward might be related to the reduced costs associated with vaccination, enhanced public engagement, or both. The pay-it-forward arm had a similar vaccination rate to that of the free vaccine arm. This suggests that asking participants to make some financial contribution to support the vaccination of subsequent participants did not deter them from participating.
We also observed that, among those enrolled in the pay-it-forward arm, nearly all voluntarily donated to support another person in receiving an influenza vaccine, including those with a low annual income from a study site in a poor rural area. Compared to standard of care, the pay-it-forward arm had a higher financial cost, but increased the number of people vaccinated. The incremental financial cost per person vaccinated was lower than the median cost (US $50.78) per additional enrollee vaccinated from a systematic review published in 2018.20 Donations collected using a pay-it-forward system can support more individuals in receiving influenza vaccine services and can potentially reduce the financial burden for local governments. Pay-it-forward could also potentially transition from out-of-pocket payments to government-funded influenza vaccine programs.
Pay-it-forward has additional social benefits; it generated many messages aimed at driving influenza vaccine uptake. This is a rare example of public engagement in an influenza vaccination program.9 Public engagement is central to the success of public health programs; given that some engagement methods (in-person events) could facilitate influenza transmission,21 it is especially important to identify public engagement methods that are safe and effective. Engaging the community in vaccination services through cultivating kindness and reciprocity may also strengthen community solidarity, and increase confidence in vaccine services.14,22
The study has several limitations. First, although our study was implemented after COVID-19 lockdowns were lifted, all sites were heavily focused on COVID-19 prevention, COVID-19 vaccination, and related COVID-19 activities. This caused some delays in recruitment despite the availability of influenza vaccines. At the same time, this demonstrates the feasibility of pay-it-forward, even during an event as disruptive as COVID-19. Second, we examined people from only three sites. However, our sites all had a high influenza prevalence, included different settings (rural, suburban, urban), and reflected common pathways for vaccination in China. Third, our study did not capture granular data on implementation. Future effectiveness research to examine different pay-it-forward implementation strategies is needed to differentiate effective components and determine optimal pay-it-forward practices. Finally, the study was mainly implemented by our project staff with assistance from local health workers. It remains unclear how feasible it is to decentralize implementation and integrate pay-it-forward into existing vaccine services.
Our study has implications for research, implementation, and policy. From a research perspective, this study expands the limited literature on public engagement in influenza vaccine programs. It demonstrates how social innovation can engage key communities in the implementation process and build confidence in influenza vaccination. This might help address vaccine hesitancy and anti-vaccine movements. The success of the pay-it-forward initiative and different donation levels across three study sites shows the potential to mobilize financial resources between areas with different economic status (e.g., mobilize financial resources from economically better-off areas to subsidize essential preventive services for people in more impoverished areas). Randomized controlled trials and qualitative research are needed to better understand the implementation of this system and integrate this intervention within health systems.
Pay-it-forward may be particularly relevant in the large number of countries that charge fees for influenza vaccines, which contributes to disparities in influenza vaccine uptake. Developing pay-it-forward programs could help financially support expanded influenza vaccination programs in these settings while also generating community-engaged messages. Further pay-it-forward influenza vaccination research could help explore how to expand vaccine programs in a more equitable and pro-social manner.
Data Availability
All data produced in the present study are available upon reasonable request to the authors.
https://drive.google.com/file/d/1vXEVpVSi_x_8P8bva58qL_Wqh7u5bRRB/view?usp=sharing
Funding support
This work received financial support from Bill & Melinda Gates Foundation (OPP1217240), and the National Institute for Health Research (NIHR).
Acknowledgement
We appreciate support from Dr. David Sarley from Bill & Melinda Gates Foundation for his contribution and support to the early development of the intervention package. We are grateful for comments from Dr. Jing Li at West China School of Public Health Sichuan University to help improve clarity of the paper. We thank Xu Chen, Xiaolin Qiu, Yuxin Ni, Yuan Xiong, Ruoyu Zhu, Jie Fan, Yumeng Du from University of North Carolina at Chapel Hill Project-China, Kaiyi Han from Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine for their help with data collection, video production, data visualization and interpretation.