Bias in control selection associated with the use of rapid tests in influenza vaccine effectiveness studies

Eero Poukka, Caitriona Murphy, Loretta Mak, Samuel M. S. Cheng, Malik Peiris, Tim K. Tsang, Sheena G. Sullivan, Benjamin J. Cowling
doi: https://doi.org/10.1101/2024.11.16.24317422

Abstract

In test-negative design studies that use rapid tests to estimate influenza vaccine effectiveness (VE) a common concern is case/control misclassification due to imperfect test sensitivity and specificity. However, an imperfect test can also fail to exclude from the control group people that do not represent the source population, including people infected with other influenza types or other vaccine-preventable respiratory viruses for which vaccination status is correlated. We investigated these biases by comparing the effectiveness of seasonal 2023/24 influenza vaccination against influenza A and B based on PCR versus rapid test results, excluding controls who tested positive for SARS-CoV-2 or the other type of influenza. By PCR, VE against influenza A was 49% (95%CI 26–65%) after exclusion of PCR-confirmed influenza B and SARS-CoV-2 controls. Corresponding VE against influenza B was 65% (95%CI 35–81%). VE estimated by adjusting for COVID-19 vaccination status yielded similar estimates to the scenario that excluded SARS-CoV-2-positive controls. When case/control status and exclusions from test-negative controls were determined by rapid test, VE was reduced by 5–15 percentage points. Bias correction methods were able to reduce these discrepancies. When estimating VE from a test-negative study using rapid test results, methods to correct misclassification bias are recommended.

Background

The test-negative study design (TND) is a commonly used approach for evaluating vaccine effectiveness (VE) and has long been used to monitor influenza VE in Hong Kong (1,2) and elsewhere (3–6). In a TND, participants are enrolled based on clinical criteria and tested for the pathogen of interest. The participants are classified as test-positive cases or test-negative controls according to the diagnostic test. The key premise in any case control study, including the TND, is that the enrollment of participants to the control group is independent of the exposure (vaccination), and controls represent the source population of the cases (7).

Imperfect diagnostic accuracy can lead to several potential issues. Consider the example of estimating influenza VE against influenza A(H1N1). A diagnostic test with imperfect sensitivity for A(H1N1) will give false negative results for some patients with A(H1N1), erroneously placing them in the test-negative control group. Under realistic assumptions of test sensitivity, this typically causes minimal bias in VE (8). Imperfect specificity may also be an issue, resulting in false positives being classified as test-positive cases, and potentially causing substantial bias (8). Some methods to correct this bias have been developed (9–11), one of them published by Endo et al (9). This method used multiple overimputation to correct VE estimates and provided unbiased estimates in simulation scenarios.

However, imperfect diagnostic accuracy causes other complications. In the analysis of VE against A(H1N1) discussed here, patients with influenza A(H3N2) or B should be excluded from the analysis because they do not have the same risk of infection by A(H1N1). Imperfect test sensitivity could result in failure to exclude some of these patients. In addition, as Doll et al. noted (12), health-seeking behavior that drives a person to be vaccinated against influenza may also drive them to be vaccinated against COVID-19, resulting in correlated vaccination status against the two diseases. In a TND, this can lead to increased enrollment of SARS-CoV-2 positive control participants who have not received either the influenza or COVID-19 vaccination and therefore do not represent the source population (Figure 1). Either adjusting for COVID-19 vaccination status or excluding SARS-CoV-2-positive controls have been proposed (12), and shown (13) to address this issue, but imperfect test accuracy for SARS-CoV-2 would affect such exclusions (Figure 1B).

Figure 1.
Figure 1. Directed acyclic graph showing the selection of participants in TND, including (A) or excluding (B) SARS-CoV-2 cases.

HS = Health-seeking behaviour, Vi = Influenza vaccination, Vc = COVID-19 vaccination, AC = Age and chronic conditions, I = Infection due to influenza, C = Infection due to SARS-CoV-2, I* = Test detected influenza in a TND, C* = Test detected SARS-CoV-2, DAi = Diagnostic accuracy for influenza, DAc = Diagnostic accuracy for SARS-CoV-2, MARI = Medically attended acute respiratory infection, TND = Enrolment to a TND.

In a TND evaluating VE against influenza, detection of influenza (I*) among the study participants influences case-control status in two ways: 1) other seasonal influenza types should be excluded from the controls due to cross protection of influenza vaccination, and 2) misclassification of true-positive cases as false-negative controls (6). The detection of influenza is dependent on diagnostic accuracy (DAI) and characteristics of the influenza infection (I). Another source of bias is correlation between the influenza (Vi) and COVID-19 vaccination (Vc) by health-seeking behaviour (HS) (12) which alters the selection of SARS-CoV-2 cases by influenza vaccination status (Vc ➔ C ➔ MARI ➔ TND). Notably, the correlation between VI and VC might be influenced by age and chronic conditions (AC) and can be expected to be more pronounced among priority groups, such as those with chronic illnesses and older adults. The correlation can be addressed by adjusting for COVID-19 vaccination (Vc in Figure 1A) or excluding positive SARS-CoV-2 cases (C* in Figure 1B) (12), with the latter depending on detection of the SARS-CoV-2 cases.

Our objective was to examine the potential bias introduced through use of rapid tests for estimation of influenza VE against influenza A and B in a TND among outpatients. We compared VE estimates obtained when PCR or rapid test results were used and evaluated the magnitude of bias resulting from the use of rapid tests to classify cases and controls and to make exclusions. We also applied the bias correction method of Endo et al (9) to assess whether valid VE estimates could be recovered after correcting for outcome misclassification. Finally, we compared the adjustment of COVID-19 vaccination and exclusions of SARS-CoV-2 from the controls to address the correlation between influenza and COVID-19 vaccination.

Methods

Study setting

In Hong Kong influenza vaccination is recommended for individuals aged ≥6 months, and almost all influenza vaccines administered each year are inactivated influenza vaccines (14). Hong Kong is located in a subtropical climate and influenza epidemics can occur at any time of the year, with winter peaks occurring most years and spring or summer peaks in some years (15). During the COVID-19 pandemic there was no influenza circulation in Hong Kong between March 2020 and February 2023 (16,17), but influenza activity resumed in March 2023 (18). Influenza circulated in Hong Kong in a series of epidemics from October 2023 through to August 2024, including spread of A(H3N2) from October to February, and A(H1N1) from March 2024 onwards (19).

Study design and population

We conducted a TND study among outpatients aged at least 6 months of age and experiencing acute respiratory infection with at least two symptoms (fever ≥37.8°C, cough, sore throat, runny nose, headache, myalgia, and phlegm) starting within the preceding 72 hours. The study participants visited an outpatient clinic in Hong Kong between December 15, 2023, and August 13, 2024. Participants were interviewed by research staff using a standardized questionnaire and these responses were compared to medical records and vaccination certificates where possible. Swabs were collected for testing by PCR, and separate swabs were collected to conduct the rapid test “SARS-CoV-2 & Influenza A/B & RSV Antigen Kit” produced by Goldsite Diagnostics Inc (Shenzhen, China, Table S1) on-site. Since this rapid test was not widely available in the clinic before December 15, 2023, the study period began on this date. We excluded individuals that were not tested with this rapid test, had received seasonal influenza vaccination for 2023/24 within 13 days of the time of enrollment, or had incomplete background data (Figure S1).

Specimens and outcomes

Pooled nasal and throat swabs were collected in a viral transport medium by trained research staff and tested by PCR for SARS-CoV-2, influenza A and B. In addition, separate nasal swabs were collected for rapid test to detect SARS-CoV-2, influenza A and B. If a rapid test gave an invalid result (control line did not appear), the test was repeated once. The outcomes of interest were influenza A and B confirmed by PCR or rapid test.

Exposure

The exposure of interest was influenza vaccination for the 2023/24 season received at least 14 days prior to the time of enrollment. Those that had not received the seasonal 2023/24 influenza vaccination were classified as unexposed.

Statistical analysis

We estimated VE against influenza A and B by PCR or rapid test using logistic regression. VE was calculated from the odds ratio (OR) comparing the odds of vaccination among influenza-positive cases versus influenza-negative controls, adjusted for confounders (VE = 1 – ORadj x 100%). All models were adjusted for age, age-squared, sex, presence of diagnosed chronic medical conditions (cardiac, respiratory, hepatic, renal, hematological, immunological disease or diabetes ascertained by patient interview), receipt of seasonal influenza vaccination during the preceding season (2022/23) and calendar time (two-week bands).

To investigate the consequences of excluding influenza B and SARS-CoV-2 cases from the test-negative controls or alternatively addressing the correlation of influenza and COVID-19 vaccination by adjusting for COVID-19 vaccination status, we estimated VE against influenza A separately by PCR or rapid test using seven different analytic scenarios presented in Table 1.

View this table:
Table 1. Seven scenarios for estimating VE against influenza A.

When estimated against influenza A, scenarios 1 and 2 were considered the baseline analyses (the most valid estimates). For VE against influenza B the scenarios were similar, but the exclusions were instead based on influenza A positivity. As additional analyses we estimated the VE against influenza A separately among participants aged less than 18 years and participants aged 18 years or more. The age stratified analyses were not conducted against influenza B due to a small number of cases. We also estimated VE against influenza A during H3N2 (from December 15, 2023, until February 28, 2024) H1N1 (from March 1 until August 13, 2024) predominance periods to obtain estimates against specific subtype of influenza A.

To correct the VE estimates arising from imperfect diagnostic accuracy of rapid tests we used the multiple overimputation method introduced by Endo et al (9). The inputs for this method were the rapid test results, the predicted probability of a positive rapid test, the number of multiple overimputation iterations (chosen as 500) and estimates for sensitivity and specificity of the rapid test. The predicted probability of a positive rapid test for each study participant was estimated with the same logistic regression model used to estimate the OR between exposed and unexposed for different outcomes (i.e., the model used to estimate ORadj). For the sensitivity and specificity, we used estimates from the current study and in a separate analysis we used the values provided by the manufacturer (Table S1). The sensitivity and specificity of the rapid test were estimated from the study participants assuming PCR as gold standard (20). Separate sensitivity and specificity estimates were estimated by age groups (those aged less than 18 years and those aged 18 years or more) and these were similarly implemented to correct VE for the age group stratified analyses. The confidence intervals (CI) for sensitivity and specificity were estimated via the Clopper-Pearson intervals. The analyses were performed using R version 4.2.2 (R Foundation for Statistical Computing, Vienna, Austria).

Ethical approval

The study received approval from the Institutional Review Board of the University of Hong Kong. Written informed consent was obtained for each participant and parental consent was obtained for participants below 18 years of age.

Results

A total of 1,691 study participants were included in the study during the study period between December 2023 and August 2024 (Figure S1–S2). Among the participants 410 (24%) had PCR-confirmed influenza A and 178 (11%) had influenza B (Table 2). Among the influenza A cases, 184 (45%) were caused by influenza A(H1N1), 175 (43%) by A(H3N2), and 51 (12%) had influenza A which was not subtyped. Approximately half of the study participants were under 18 years of age (N=879, 52%) and 9% (N=154) had at least one chronic medical condition.

View this table:
Table 2. Characteristics of cases and controls.

For influenza A, 97 study participants received false negative rapid test results and five received false positive rapid test results in comparison to PCR (Table S2). The estimated sensitivity of the rapid test for influenza A was therefore 76.3% (95% CI 71.9–80.4%; Table 3) while the specificity was 99.6% (95% CI 99.1–99.9%). For influenza B, there were 36 false negatives and one false positive (Table S3), corresponding to a sensitivity of 79.8% (95% CI 73.1–85.4%; Table 3) and specificity of 99.9% (95% CI 99.6–100%). For SARS-CoV-2, there were 24 false negative and no false positive rapid test results. The estimated sensitivity for SARS-CoV-2 was 88.2% (95% CI 82.9–92.3%) and specificity was 100% (95% CI 99.8–100%) compared to PCR gold standard. Among participants aged less than 18 years the sensitivity for influenza A was higher compared to those aged 18 years or more (85.1%; 95% CI 79.0–89.9% vs 69.4%; 95% CI 63.0–75.3%). Similarly, the sensitivity for influenza B seemed higher among participants aged less than 18 years (84.3%; 95% 74.7–91.4% vs 75.8%; 95% 65.9–84.0%). For SARS-CoV-2 the sensitivity was comparable across the age groups (Table 3).

View this table:
Table 3. Sensitivity and specificity of Goldsite rapid test for influenza A, B and SARS-CoV-2 among study participants across age groups.

VE against influenza A was 49% (95% CI 26–65%, Table 4) when cases were confirmed by PCR and participants testing positive for influenza B or SARS-CoV-2 by PCR were excluded. VE was 47% (95% CI 24–63%) against influenza A by PCR when influenza B-positive controls were excluded by PCR and COVID-19 vaccination status was adjusted. With other scenarios the VE by PCR varied from 42% to 47% (see Table 4). When influenza A case status was confirmed by PCR and no control exclusions were made, VE was underestimated at 42% (95% CI 17–60%).

View this table:
Table 4. Vaccine effectiveness (VE) against influenza A based on PCR or rapid test with 95% confidence interval.

VE against influenza A confirmed by rapid test was 43% (95% CI 16–62%, Table 4) when participants testing positive for influenza B or SARS-CoV-2 by rapid test were excluded. Corresponding VE was 42% (95% CI 15–61%) when influenza B was excluded by rapid test and COVID-19 vaccination were adjusted. VE against influenza A by rapid test was 39% (95% CI 9– 59%) when no control group exclusions were made. Using both PCR results for the exclusion of influenza B and SARS-CoV-2 and applying the bias correction method increased VE estimates (Table 4).

VE against influenza B confirmed by PCR was 65% (95% CI 35–81%) after exclusion of PCR-confirmed influenza A and SARS-CoV-2 patients from the control group (Table 5). Corresponding VE was 63% (95% CI 34–80%) when influenza A were excluded by PCR and COVID-19 vaccination was adjusted. If these controls were not excluded VE was 58% (95% CI 25–77%). VE against influenza B by the rapid test was 49% (95% CI 6–73%) when patients with influenza A and SARS-CoV-2 by rapid test were excluded from the control group. Excluding PCR-confirmed influenza A and SARS-CoV-2 and applying bias correction increased VE estimates. However, the best estimates by rapid test for influenza B were still approximately 10 percentage points lower compared to baseline analyses despite applying both exclusions by PCR and bias correction.

View this table:
Table 5. Vaccine effectiveness (VE) against influenza B based on PCR or rapid test with 95% confidence interval.

In the stratified analyses, VE against influenza A by PCR was among participants aged less than 18 years 55% (95% CI 30–72%, Table S4) when participants testing positive for influenza B or SARS-CoV-2 by PCR were excluded. Corresponding VE estimate among participants aged 18 years or more was 38% (95% CI -21–69%, Table S5). Among participants aged less than 18 years the VE against influenza A by rapid test was 50% (95% CI 21–69%) with exclusions of participants testing positive for influenza B or SARS-CoV-2 by rapid test. However, among participants aged 18 years or more the corresponding VE was 26% (95% CI -52–65%). Overall, VE against influenza A by PCR or rapid test appeared to be lower among participants aged 18 years or more and the difference between VE by PCR or rapid test appeared greater indicating more profound bias in this age group. However, statistical power was limited in the age group stratified analysis. During H3N2 and H1N1 periods, VE against influenza A by PCR were 59% (95% CI 31–76%) and 33% (-14–61%, Tables S6–7) when participants testing positive for influenza B or SARS-CoV-2 by PCR were excluded. Corresponding VE against influenza A by rapid test were 53% (95% CI 19–72%) and 26% (95% CI -32–59%) with exclusions of participants testing positive for influenza B or SARS-CoV-2 by rapid test.

Overall, adjusting for COVID-19 vaccination and exclusion of SARS-CoV-2 to address correlation between influenza and COVID-19 vaccination provided comparable VE estimates for influenza A and B with only a 2 percentage point difference between estimates and compatible precision (Tables 4–5). Similar findings were observed for influenza A in the age and influenza A subtype stratified analysis (Tables S4–7).

Discussion

In this TND study among outpatients, the most valid VE estimates of seasonal 2023/24 vaccination for PCR-confirmed influenza A were approximately 50% while they were approximately 65% for PCR-confirmed influenza B from December 2023 to August 2024. When utilizing rapid test results, VE against influenza was approximately 5 and 15 percentage points lower for influenza A and B compared to the most valid VE estimates by PCR due to the bias caused by misclassification of cases and controls, and failure to exclude other types of influenza and SARS-CoV-2 cases from the controls. Both exclusion of influenza and SARS-CoV-2 by PCR (instead of rapid test) and the correction method by Endo et al (9) were able to mitigate the bias although estimates remained lower compared to VE against PCR-confirmed influenza. This bias was especially prominent for influenza B, with the best estimate being approximately 10 percentage points lower by rapid test compared to PCR, despite using the bias correction method and excluding PCR-confirmed influenza A and SARS-CoV-2.

Our VE estimates for PCR-confirmed influenza A were comparable to mid-season estimates among outpatient clinics in Canada during October 2023 – January 2024 (VE against PCR-confirmed influenza A 59%, 95% CI 48–68%) (21). Similarly, comparable VE estimates were reported in other studies conducted in 2023/24 among outpatients in the United Kingdom (22), the USA (3), China (23), and in a multinational European study (4). For influenza B, VE has ranged from 51% to 89% in different outpatient settings during 2023/24 seasons similar to our results (3,23). Gào et al. reported VE against laboratory-confirmed influenza virus infection by PCR or rapid test in a TND conducted in outpatient and emergency settings during the 2023/24 season and observed approximately a 15 to 20 percentage point drop in VE against influenza A or B when using rapid tests (24). Together with our results, these highlight the importance to address the bias related to use of rapid test results in a TND.

Notably, several recent TND studies evaluating VE against influenza or COVID-19 have used rapid tests results without bias correction (25–30). When estimating VE against respiratory viruses with rapid tests in a TND study two sources of bias should be considered: 1) misclassification of cases and controls, and 2) failure to exclude other types of influenza, SARS-CoV-2, and RSV cases (if RSV immunization has been introduced) from the test-negative control group. Currently no widely used method is available to correct these sources of bias and new correction methods would be useful in the future. The rapid test-based TND could have promising prospects and could be used to estimate VE against symptomatic disease in participatory cohorts limiting cost of a TND and complementing other studies estimating VE against medically-attended illness or hospitalization.

We also observed the diagnostic accuracy of the rapid test to be dependent on age group with higher sensitivity observed among those aged less than 18 years. Therefore, the magnitude of bias associated with use of rapid tests might be dependent on age and this was also indicated in the age group stratified analyses. Other factors that might influence diagnostic accuracy include viral load (31) and, for SARS-CoV-2, the patient’s COVID-19 vaccination status (32). There have also been some reports of reduced disease severity for influenza among vaccine recipients (33–37). Given that disease severity is correlated with viral load (38) and therefore the chance of a positive rapid test result, this could result in differential outcome misclassification by vaccination status, which in a case-control design can be difficult to correct (39). Future studies to confirm the importance of outcome misclassification should additionally consider these potential sources of bias.

We found that both methods to account for the correlation between influenza and COVID-19 vaccination – adjusting for COVID-19 vaccination or excluding SARS-CoV-2 positive controls – provided comparable VE estimates. Payne et al (13) similarly observed compatible estimates when estimating VE against COVID-19 with adjustment for influenza vaccination status or exclusion of influenza-positive controls, as did DeCuir et al. and Laniece et al (40,41). In our study, addressing the correlation increased VE estimates by around 5 percentage points, suggesting that the possible bias was relatively small. In Canada, for the same season the increase in VE observed for influenza A(H3N2) was just 1 percentage point (21). In the prior 2022/23 season, similar-magnitude increases of 1 to 6 percentage points were reported from Europe (42,43) and Canada (44). Slightly larger 5–10 percentage increase was observed in the 2021/22 season (45), and it is possible that the confounding caused by correlated vaccination status was stronger in the US setting where vaccination uptake for both COVID-19 and influenza vaccines was higher than in Hong Kong, Europe and Canada. This bias may diminish over time but could still be considerable in some study settings if the magnitude of the bias depends on factors such as seasonal activity of SARS-CoV-2 and influenza, vaccine protection, circulating variants and study population characteristics.

There are several potential limitations of our study. Residual confounding is possible, although we were able to control the most relevant confounders in our analysis (6,46). As another limitation, our sample was relatively small especially in the stratified analyses. Rapid tests were not able to distinguish influenza A(H3N2) and A(H1N1) and we therefore had to approximate VE by subtype using calendar time. Despite our attempts to verify the data collected, it is possible that the study might have included additional exposure and covariate misclassification, the influence of which were not explored. Moreover, we only had information of number of COVID-19 doses and the date of the last COVID-19 vaccination was missing for most of the participants and therefore we could not explore adjusting for time since the last COVID-19 vaccination as a confounder, which could have addressed the correlation between influenza and COVID-19 vaccination more adequately. Finally, we only estimated VE against medically-attended influenza among outpatients that sought care at a clinic, most of whom were young and did not have chronic conditions. The results might not be generalizable to other outcomes, such as influenza associated hospitalizations, or other populations, such as the elderly or individuals with conditions that increase the risk of severe influenza disease.

Conclusions

In this outpatient TND we estimated VE was approximately 50% against PCR-confirmed influenza A and 65% against influenza B. We found that VE estimated using rapid test results was approximately 5 to 15 percentage points lower than VE by PCR. The reduced sensitivity of current rapid tests compared to PCR is not only an issue for correct classification of cases and controls, but also an issue for making appropriate exclusions from the control group. New methods for controlling misclassification bias could help adapt participatory cohorts for monitoring of VE against influenza, COVID-19, and RSV.

Data Availability

All data produced in the present study are available upon reasonable request to the authors.

Author contributions

All authors meet the ICMJE criteria for authorship. EP and BJC conceived the study. Data were collected by LM and SMSC. Data were analyzed by EP and CM. MP and BJC supervised the work. EP wrote the first draft of the manuscript. All authors provided critical review and revision of the text and approved the final version.

Competing interests

EP is involved in an estate that holds shares of Astra Zeneca. BJC consults for AstraZeneca, Fosun Pharma, GSK, Haleon, Moderna, Novavax, Pfizer, Roche and Sanofi Pasteur. SGS reports honoraria from CSL Seqirus, Evo Health, Moderna, Novavax and Pfizer. The authors report no other potential conflicts of interest.

Acknowledgments

This project was supported by the Health and Medical Research Fund, Health Bureau, The Government of the Hong Kong Special Administrative Region (grant no. INF-HKU-3), the Theme-based Research Scheme (Project No. T11-712/19-N) of the Research Grants Council of the Hong Kong SAR Government, and the National Institute of General Medical Sciences (grant no. R01 GM139926). EP is supported by Finnish Medical Foundation. BJC is supported by an RGC Senior Research Fellowship from the University Grants Committee of Hong Kong (grant number: HKU SRFS2021-7S03).

References

  1. 1.↵
    Tam YH, Ng TWY, Chu DKW, et al. The effectiveness of influenza vaccination against medically-attended illnesses in Hong Kong across three years with different degrees of vaccine match, 2014–17. Vaccine. 2018;36(41). doi:10.1016/j.vaccine.2018.08.075
    OpenUrlCrossRef
  2. 2.↵
    Chua H, Kwan MYW, Chan ELY, et al. Influenza vaccine effectiveness against influenza-associated hospitalization in children in Hong Kong, 2010–2020. Vaccine. 2021;39(34):4842–4848. doi:10.1016/j.vaccine.2021.07.014
    OpenUrlCrossRefPubMed
  3. 3.↵
    Frutos AM, Price AM, Harker E, et al. Interim Estimates of 2023–24 Seasonal Influenza Vaccine Effectiveness — United States. MMWR Morb Mortal Wkly Rep. 2024;73(8):168-174. doi:10.15585/mmwr.mm7308a3
    OpenUrlCrossRefPubMed
  4. 4.↵
    Maurel M, Howard J, Kissling E, et al. Interim 2023/24 influenza A vaccine effectiveness: VEBIS European primary care and hospital multicentre studies, September 2023 to January 2024. Eurosurveillance. 2024;29(8). doi:10.2807/1560-7917.ES.2024.29.8.2400089
    OpenUrlCrossRef
  5. 5.
    Smolarchuk C, Ickert C, Zelyas N, Kwong JC, Buchan SA. Early influenza vaccine effectiveness estimates using routinely collected data, Alberta, Canada, 2023/24 season. Eurosurveillance. 2024;29(2). doi:10.2807/1560-7917.ES.2024.29.2.2300709
    OpenUrlCrossRefPubMed
  6. 6.↵
    Sullivan SG, Tchetgen Tchetgen EJ, Cowling BJ. Theoretical Basis of the Test-Negative Study Design for Assessment of Influenza Vaccine Effectiveness. Am J Epidemiol. 2016;184(5):345–353. doi:10.1093/aje/kww064
    OpenUrlCrossRefPubMed
  7. 7.↵
    Wacholder S, McLaughlin JK, Silverman DT, Mandel JS. Selection of Controls in Case-Control Studies. American Journal of Epidemiology. 1992;135(9):1019–1028. doi:10.1093/oxfordjournals.aje.a116396
    OpenUrlCrossRefPubMedWeb of Science
  8. 8.↵
    Jackson ML, Rothman KJ. Effects of imperfect test sensitivity and specificity on observational studies of influenza vaccine effectiveness. Vaccine. 2015;33(11):1313–1316. doi:10.1016/j.vaccine.2015.01.069
    OpenUrlCrossRefPubMed
  9. 9.↵
    Endo A, Funk S, Kucharski AJ. Bias correction methods for test-negative designs in the presence of misclassification. Epidemiol Infect. 2020;148:e216. doi:10.1017/S0950268820002058
    OpenUrlCrossRef
  10. 10.
    Habibzadeh F. Correction of vaccine effectiveness derived from test-negative case–control studies. BMC Med Res Methodol. 2023;23(1):137. doi:10.1186/s12874-023-01962-0
    OpenUrlCrossRefPubMed
  11. 11.↵
    Eusebi P, Speybroeck N, Hartnack S, Stærk-Østergaard J, Denwood MJ, Kostoulas P. Addressing misclassification bias in vaccine effectiveness studies with an application to Covid-19. BMC Med Res Methodol. 2023;23(1):55. doi:10.1186/s12874-023-01853-4
    OpenUrlCrossRefPubMed
  12. 12.↵
    Doll MK, Pettigrew SM, Ma J, Verma A. Effects of Confounding Bias in Coronavirus Disease 2019 (COVID-19) and Influenza Vaccine Effectiveness Test-Negative Designs Due to Correlated Influenza and COVID-19 Vaccination Behaviors. Clinical Infectious Diseases. 2022;75(1):e564–e571. doi:10.1093/cid/ciac234
    OpenUrlCrossRefPubMed
  13. 13.↵
    Payne AB, Ciesla AA, Rowley EAK, et al. Impact of accounting for correlation between COVID-19 and influenza vaccination in a COVID-19 vaccine effectiveness evaluation using a test-negative design. Vaccine. 2023;41(51):7581–7586. doi:10.1016/j.vaccine.2023.11.025
    OpenUrlCrossRefPubMed
  14. 14.↵
    Government of the Hong Kong Special Administrative Region. 2023/24 seasonal influenza vaccination programmes to start tomorrow. Accessed March 7, 2024. https://www.info.gov.hk/gia/general/202310/04/P2023100400218.htm
  15. 15.↵
    Wu P, Presanis AM, Bond HS, Lau EHY, Fang VJ, Cowling BJ. A joint analysis of influenza-associated hospitalizations and mortality in Hong Kong, 1998–2013. Sci Rep. Published online April 20, 2017. doi:10.1038/s41598-017-01021-x
    OpenUrlCrossRef
  16. 16.↵
    Cowling BJ, Ali ST, Ng TWY, et al. Impact assessment of non-pharmaceutical interventions against coronavirus disease 2019 and influenza in Hong Kong: an observational study. The Lancet Public Health. 2020;5(5):e279–e288. doi:10.1016/S2468-2667(20)30090-6
    OpenUrlCrossRefPubMed
  17. 17.↵
    Mak GCK, Lau SSY, Wong KKY, Lau AWL, Hung DLL. Low prevalence of seasonal influenza viruses in Hong Kong, 2022. Influenza Resp Viruses. 2023;17(3):e13123. doi:10.1111/irv.13123
    OpenUrlCrossRef
  18. 18.↵
    Xiong W, Cowling BJ, Tsang TK. Influenza Resurgence after Relaxation of Public Health and Social Measures, Hong Kong, 2023. Emerg Infect Dis. 2023;29(12):2556–2559. doi:10.3201/eid2912.230937
    OpenUrlCrossRefPubMed
  19. 19.↵
    Centre for Health Protection, Department of Health - COVID-19 & Flu Express. Accessed April 2, 2024. https://www.chp.gov.hk/en/resources/29/100148.html
  20. 20.↵
    Murphy C, Mak L, Cheng SMS, et al. Diagnostic performance of multiplex lateral flow tests in ambulatory patients with acute respiratory illness. Diagnostic Microbiology and Infectious Disease. 2024;110(1):116421. doi:10.1016/j.diagmicrobio.2024.116421
    OpenUrlCrossRefPubMed
  21. 21.↵
    Skowronski DM, Zhan Y, Kaweski SE, et al. 2023/24 mid-season influenza and Omicron XBB.1.5 vaccine effectiveness estimates from the Canadian Sentinel Practitioner Surveillance Network (SPSN). Eurosurveillance. 2024;29(7). doi:10.2807/1560-7917.ES.2024.29.7.2400076
    OpenUrlCrossRef
  22. 22.↵
    Whitaker H, Findlay B, Zitha J, et al. Interim 2023/2024 Season Influenza Vaccine Effectiveness in Primary and Secondary Care in the United Kingdom. Influenza Resp Viruses. 2024;18(5):e13284. doi:10.1111/irv.13284
    OpenUrlCrossRef
  23. 23.↵
    Mi J, Wang J, Chen L, et al. Real-world effectiveness of influenza vaccine against medical-attended influenza infection during 2023/24 season in Ili Kazakh Autonomous Prefecture, China: A test-negative, case-control study. Human Vaccines & Immunotherapeutics. 2024;20(1):2394255. doi:10.1080/21645515.2024.2394255
    OpenUrlCrossRefPubMed
  24. 24.↵
    Gào X, Sun Y, Shen P, et al. Population-Based Influenza Vaccine Effectiveness Against Laboratory-Confirmed Influenza Infection in Southern China, 2023–2024 Season. Open Forum Infectious Diseases. 2024;11(9):ofae456. doi:10.1093/ofid/ofae456
    OpenUrlCrossRefPubMed
  25. 25.↵
    Castelli JM, Rearte A, Olszevicki S, et al. Effectiveness of mRNA-1273, BNT162b2, and BBIBP-CorV vaccines against infection and mortality in children in Argentina, during predominance of delta and omicron covid-19 variants: test negative, case-control study. BMJ. Published online November 30, 2022:e073070. doi:10.1136/bmj-2022-073070
    OpenUrlAbstract/FREE Full Text
  26. 26.
    Cerqueira-Silva T, Shah SA, Robertson C, et al. Effectiveness of mRNA boosters after homologous primary series with BNT162b2 or ChAdOx1 against symptomatic infection and severe COVID-19 in Brazil and Scotland: A test-negative design case–control study. Suthar AB, ed. PLoS Med. 2023;20(1):e1004156. doi:10.1371/journal.pmed.1004156
    OpenUrlCrossRefPubMed
  27. 27.
    Halasa NB, Olson SM, Staat MA, et al. Maternal Vaccination and Risk of Hospitalization for Covid-19 among Infants. N Engl J Med. 2022;387(2):109–119. doi:10.1056/NEJMoa2204399
    OpenUrlCrossRefPubMed
  28. 28.
    Florentino PTV, Millington T, Cerqueira-Silva T, et al. Vaccine effectiveness of two-dose BNT162b2 against symptomatic and severe COVID-19 among adolescents in Brazil and Scotland over time: a test-negative case-control study. The Lancet Infectious Diseases. Published online August 2022:S1473309922004510. doi:10.1016/S1473-3099(22)00451-0
    OpenUrlCrossRefPubMed
  29. 29.
    Tenforde MW, Patel MM, Lewis NM, et al. Vaccine Effectiveness Against Influenza A(H3N2)–Associated Hospitalized Illness: United States, 2022. Clinical Infectious Diseases. 2023;76(6):1030–1037. doi:10.1093/cid/ciac869
    OpenUrlCrossRef
  30. 30.↵
    Shinjoh M, Sugaya N, Yamaguchi Y, et al. Effectiveness of Trivalent Inactivated Influenza Vaccine in Children Estimated by a Test-Negative Case-Control Design Study Based on Influenza Rapid Diagnostic Test Results. McVernon J, ed. PLoS ONE. 2015;10(8):e0136539. doi:10.1371/journal.pone.0136539
    OpenUrlCrossRefPubMed
  31. 31.↵
    Eyre DW, Futschik M, Tunkel S, et al. Performance of antigen lateral flow devices in the UK during the alpha, delta, and omicron waves of the SARS-CoV-2 pandemic: a diagnostic and observational study. The Lancet Infectious Diseases. 2023;23(8):922–932. doi:10.1016/S1473-3099(23)00129-9
    OpenUrlCrossRefPubMed
  32. 32.↵
    Meiners L, Horn J, Jones TC, et al. SARS-CoV-2 rapid antigen test sensitivity and viral load in newly symptomatic hospital employees in Berlin, Germany, December, 2020 to February, 2022: an observational study. The Lancet Microbe. 2024;5(6):e538–e546. doi:10.1016/S2666-5247(23)00412-3
    OpenUrlCrossRef
  33. 33.↵
    Chung JR, Kim SS, Flannery B, et al. Vaccine-associated attenuation of subjective severity among outpatients with influenza. Vaccine. 2022;40(32):4322–4327. doi:10.1016/j.vaccine.2022.06.019
    OpenUrlCrossRefPubMed
  34. 34.
    Thompson MG, Pierse N, Sue Huang Q, et al. Influenza vaccine effectiveness in preventing influenza-associated intensive care admissions and attenuating severe disease among adults in New Zealand 2012-2015. Vaccine. 2018;36(39):5916–5925. doi:10.1016/j.vaccine.2018.07.028
    OpenUrlCrossRefPubMed
  35. 35.
    VanWormer JJ, Sundaram ME, Meece JK, Belongia EA. A cross-sectional analysis of symptom severity in adults with influenza and other acute respiratory illness in the outpatient setting. BMC Infect Dis. 2014;14(1):231. doi:10.1186/1471-2334-14-231
    OpenUrlCrossRefPubMed
  36. 36.
    Arriola C, Garg S, Anderson EJ, et al. Influenza Vaccination Modifies Disease Severity Among Community-dwelling Adults Hospitalized With Influenza. Clinical Infectious Diseases. 2017;65(8):1289–1297. doi:10.1093/cid/cix468
    OpenUrlCrossRefPubMed
  37. 37.↵
    Castilla J, Godoy P, Domínguez Á, et al. Influenza Vaccine Effectiveness in Preventing Outpatient, Inpatient, and Severe Cases of Laboratory-Confirmed Influenza. Clinical Infectious Diseases. 2013;57(2):167–175. doi:10.1093/cid/cit194
    OpenUrlCrossRefPubMed
  38. 38.↵
    Launes C, Garcia-Garcia JJ, Jordan I, Selva L, Rello J, Muñoz-Almagro C. Viral load at diagnosis and influenza A H1N1 (2009) disease severity in children. Influenza Resp Viruses. 2012;6(6). doi:10.1111/j.1750-2659.2012.00383.x
    OpenUrlCrossRefPubMed
  39. 39.↵
    Greenland S. Basic Methods for Sensitivity Analysis of Biases. International Journal of Epidemiology. 1996;25(6):1107–1116. doi:10.1093/ije/25.6.1107
    OpenUrlCrossRefPubMedWeb of Science
  40. 40.↵
    DeCuir J, Payne AB, Self WH, et al. Interim Effectiveness of Updated 2023–2024 (Monovalent XBB.1.5) COVID-19 Vaccines Against COVID-19–Associated Emergency Department and Urgent Care Encounters and Hospitalization Among Immunocompetent Adults Aged ≥18 Years — VISION and IVY Networks, September 2023–January 2024. MMWR Morb Mortal Wkly Rep. 2024;73(8):180–188. doi:10.15585/mmwr.mm7308a5
    OpenUrlCrossRefPubMed
  41. 41.↵
    Lanièce Delaunay C, Martínez-Baz I, Sève N, et al. COVID-19 vaccine effectiveness against symptomatic infection with SARS-CoV-2 BA.1/BA.2 lineages among adults and adolescents in a multicentre primary care study, Europe, December 2021 to June 2022. Eurosurveillance. 2024;29(13). doi:10.2807/1560-7917.ES.2024.29.13.2300403
    OpenUrlCrossRefPubMed
  42. 42.↵
    Maurel M, Mazagatos C, Goerlitz L, et al. Exploring the effect of clinical case definitions on influenza vaccine effectiveness estimation at primary care level: Results from the end-of-season 2022–23 VEBIS multicentre study in Europe. Vaccine. 2024;42(16):3547–3554. doi:10.1016/j.vaccine.2024.04.060
    OpenUrlCrossRefPubMed
  43. 43.↵
    Domnich A, Orsi A, Ogliastro M, et al. Influenza vaccine effectiveness in preventing hospital encounters for laboratory-confirmed infection among Italian adults, 2022/23 season. Vaccine. 2023;41(33):4861–4866. doi:10.1016/j.vaccine.2023.06.072
    OpenUrlCrossRefPubMed
  44. 44.↵
    Skowronski DM, Chuang ES, Sabaiduc S, et al. Vaccine effectiveness estimates from an early-season influenza A(H3N2) epidemic, including unique genetic diversity with reassortment, Canada, 2022/23. Eurosurveillance. 2023;28(5). doi:10.2807/1560-7917.ES.2023.28.5.2300043
    OpenUrlCrossRef
  45. 45.↵
    Price AM, Flannery B, Talbot HK, et al. Influenza Vaccine Effectiveness Against Influenza A(H3N2)-Related Illness in the United States During the 2021–2022 Influenza Season. Clinical Infectious Diseases. 2023;76(8):1358–1363. doi:10.1093/cid/ciac941
    OpenUrlCrossRefPubMed
  46. 46.↵
    Sullivan SG, Feng S, Cowling BJ. Potential of the test-negative design for measuring influenza vaccine effectiveness: a systematic review. Expert Review of Vaccines. 2014;13(12):1571–1591. doi:10.1586/14760584.2014.966695
    OpenUrlCrossRefPubMed
Back to top
Posted November 18, 2024.
Download PDF

Supplementary Material

Data/Code
Email
Bias in control selection associated with the use of rapid tests in influenza vaccine effectiveness studies
Eero Poukka, Caitriona Murphy, Loretta Mak, Samuel M. S. Cheng, Malik Peiris, Tim K. Tsang, Sheena G. Sullivan, Benjamin J. Cowling
medRxiv 2024.11.16.24317422; doi: https://doi.org/10.1101/2024.11.16.24317422
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area