Abstract
Importance Active monitoring of health outcomes after COVID-19 vaccination provides early detection of rare outcomes that may not be identified in prelicensure trials.
Objective To conduct near real-time monitoring of health outcomes following COVID-19 vaccination in the United States (US) pediatric population aged 6 months to 17 years.
Design We evaluated 21 pre-specified health outcomes; 15 were sequentially tested through near real-time surveillance, and 6 were monitored descriptively within a cohort of vaccinated children. We tested for increased rate of each outcome following vaccination compared to a historical comparator cohort.
Setting This population-based study was conducted under the US Food and Drug Administration public health surveillance mandate using three commercial claims databases.
Participants Children aged 6 months to 17 years were included if they received a monovalent COVID-19 vaccine dose before early 2023 and had continuous enrollment in a medical health insurance plan from the start of an outcome-specific clean window to the COVID-19 vaccination dose.
Exposure Exposure was defined as receipt of a monovalent BNT162b2, mRNA-1273, or NVX-CoV2373 COVID-19 vaccine dose. The primary analysis evaluated dose 1 and dose 2 combined, and secondary analyses evaluated each dose separately. Follow-up time was censored at death, disenrollment, end of risk window, end of study period, or a subsequent dose administration.
Main Outcomes Twenty-one prespecified health outcomes.
Results The study included 4,102,016 enrollees aged 6 months to17 years. Thirteen of 15 outcomes sequentially tested did not meet the threshold for a statistical signal. In the primary analysis, myocarditis or pericarditis signals were detected following BNT162b2 vaccine in children aged 12-17 years old and seizures/convulsions signals were detected following vaccination with BNT162b2 and mRNA-1273 in children aged 2-4/5 years. However, in a post-hoc sensitivity analysis, the seizures/convulsions signal was sensitive to background rates selection and was not observed when 2022 background rates were selected instead of 2020 rates.
Conclusions and Relevance Of the two signaled outcomes, the myocarditis or pericarditis signals are consistent with previously published reports. The new signal detected for seizures/convulsions among younger children should be further investigated in a robust epidemiological study with better confounding adjustment.
Key Points
Question
Did active monitoring detect statistical signals for health outcomes following monovalent COVID-19 vaccination in the US children aged 6 months to 17 years?
Findings
In this study including 4,102,106 vaccinated enrollees from three commercial claims databases, myocarditis or pericarditis signaled after BNT162b2 (12-17 years) and a new signal was detected for seizures/convulsions after BNT162b2 (2-4 years) and mRNA1273 COVID-19 vaccinations (2-5 years).
Meaning
Near real-time monitoring of vaccines can rapidly identify potential safety concerns. While the myocarditis or pericarditis signal is consistent with existing evidence, the new seizures/convulsions signal should be interpreted cautiously given study limitations.
Introduction
Three vaccines against COVID-19 are currently available for use in children in the United States (US) including the Pfizer-BioNTech (BNT162b2) and Moderna (mRNA-1273) COVID-19 mRNA vaccines for those aged 6 months to 17 years and the protein-based Novavax COVID-19 vaccine (NVX-CoV2373) for those aged 12 to 17 years.1–3 As of May 2023, the Centers for Disease Control and Prevention (CDC) reported 31.78 million children had received at least one COVID-19 vaccine dose and 26.2 million children had completed the primary series out of approximately 73 million children aged 6 months to 17 years in the US.4 The US Food and Drug Administration (FDA), utilizing the Biologics Effectiveness and Safety (BEST) Initiative, has been monitoring the safety of COVID-19 vaccines in children since their authorization by applying a near real-time monitoring framework to evaluate the safety of COVID-19 vaccines. This process is a signal detection or screening method and the first step in safety monitoring of these vaccines. This framework is designed to be sensitive enough to rapidly detect less common safety signals. However, results of such study design do not establish a causal relationship between the vaccines and health outcomes and need to be interpreted with caution because of the limited adjustment for confounding and other forms of bias.
The BEST analysis of COVID-19 vaccines in children initially focused on the BNT162b2 COVID-19 monovalent vaccine authorized for use in children aged 5 to 17 years. Surveillance has since been expanded as additional pediatric age groups and vaccine brands received authorization through late 2022. Results of the initial, more limited, safety surveillance in children have been previously published.5 In this report, we present results from the expanded monitoring of health outcomes in children after exposure to the ancestral monovalent COVID-19 vaccines in the US targeting the original COVID-19 strain.
Methods
Study Objective
This study evaluated 21 pre-specified health outcomes after exposure to BNT162b2, mRNA-1273, or NVX-CoV2373 monovalent COVID-19 vaccines in children 6 months to 17 years old by applying a near real-time monitoring framework using health care data from three commercial claims databases in the US. Fifteen outcomes underwent sequential testing, and 6 outcomes were only monitored descriptively due to lack of historical rates.
Data Sources
The study used commercial administrative claims data from Optum, Carelon Research, and CVS Health containing longitudinal medical and pharmacy claims data supplemented with vaccination data from participating local and state Immunization Information Systems (IIS) (Supplemental Table 1).6
Study Population and Period
The study included pediatric enrollees aged 6 months to 17 years who received a monovalent COVID-19 vaccine from the earliest date of its Emergency Use Authorization by age group through April 2023 (Optum), March 2023 (Carelon Research), and February 2023 (CVS Health) (Supplemental Table 2). The surveillance concluded on these respective dates because of limited accrual of exposures and health outcomes. The sequential analyses for most outcomes did not reach the expected number of events that was pre-specified based on anticipated vaccination uptake in a 6-month period post-authorization of individual vaccine products. Another contributing factor to halting surveillance was the de-authorization of the original monovalent mRNA-1273 and BNT162b2 doses for persons of all ages on April 18, 2023.7
The inclusion criteria for the study included enrollment on the vaccination date and continuous enrollment in a participating medical health insurance plan from the start of an outcome-specific clean window to the COVID-19 vaccination date so that only new incident diagnosis of an outcome in the post-vaccination risk window would contribute to the analysis (Supplemental Table 3).
Exposures and Follow Up
Exposure was defined as the receipt of a BNT162b2, mRNA-1273, or NVX-CoV2373 monovalent COVID-19 vaccine dose identified using brand and dose-specific Current Procedural Terminology/ Healthcare Common Procedure Coding System codes8 and National Drug Codes (Supplemental Table 4). Dose number was assigned based on the chronological order in which vaccinations were observed since administration codes were not available in pharmacy claims. The primary analyses included all follow-up time accrued after dose 1 and dose 2 combined. Secondary analyses included stratification by dose number and follow-up time accrued after the individual dose (including monovalent third/booster doses) through censoring at subsequent vaccination, death, disenrollment, end of risk window, or end of study period.
Health Outcomes
Twenty-one pre-specified health outcomes were defined using claims-based algorithms9–10. Outcomes were selected through clinical consultation and literature review. Fifteen health outcomes were assessed using sequential testing comparing rates to historical outcome rates (additional information on historical rates below), and 6 were only monitored descriptively due to lack of historical rates.10 The outcome myocarditis, pericarditis, or co-occurring myocarditis and pericarditis (hereafter referred to as myocarditis/pericarditis) was assessed using four different definitions with varying risk windows and care settings (included in Supplemental Table 3) based on evidence from prior surveillance efforts and clinical input.
Descriptive Monitoring
We estimated outcome rates in the vaccinated population stratified by age, sex, region, urban/rural status, data source, and vaccine brand on a monthly basis.
Sequential Testing
Monthly sequential testing was conducted using the Poisson Maximized Sequential Probability Ratio Test (PMaxSPRT) to detect statistical signals by generating the incidence rate ratio (IRR) comparing outcome rates following vaccination to database-specific historical (expected) rates for 15 health outcomes.11
We estimated annual historical rates for 2019 and 2020 as well as during the COVID-19 pandemic between April and December 2020. Historical rates were adjusted for claims processing delay to account for observation delay and standardized by age and sex where case counts permitted.12 Selection of the historical comparator rate was based on the overlap between the 95% confidence intervals (CI) for the periods prior to and during the COVID-19 pandemic. If rates in these two historical periods differed substantially, we selected either the minimum or more stable rate as a more conservative approach.
Tests were stratified by age based on age-group-specific authorizations by vaccine brand as well as availability of background rates using historical comparator data. For BNT162b2, this included ages 6 months-4 years, 6 months-1 year and 2-4 years (for seizure/convulsions only), 5-11 years, 12-15 years, and 16-17 years. For mRNA-1273, this included 6 months-5 years, 6 months-1 year and 2-5 years (for seizure or convulsions only), 6-11 years, 12-15 years, and 16-17 years. For NVX-CoV2373, the age groups were 12-15 years and 16-17 years.
Sequential testing for each outcome commenced when a minimum of three cases accrued in the risk window. One-tailed tests were used with the null hypothesis that the observed rate was no greater than the historical comparator rate beyond a pre-specified test margin for each outcome-dose-age group being sequentially tested, with an alpha level of 1%. A stringent alpha level was selected to increase the specificity of signals detected from testing multiple outcomes across different analyses. The log likelihood ratio was calculated comparing the likelihood of the observed IRR and the null hypothesis.
At each test, if the log likelihood ratio exceeded a pre-specified critical value, the null hypothesis was rejected, and a signal was declared. Surveillance continued until a signal was detected, the pre-specified maximum surveillance length was reached, or the end of study period was reached.10
Signal Characterization
Signal characterization was conducted after a signal was identified to provide data quality assessment.13 We conducted quality checks to rule out database errors or changes in the patterns of diagnosis codes used to identify events in the study period; estimated the relative risk of outcomes within demographic strata by age and sex; examined the timing of outcomes occurrence during the pre-specified risk window; and assessed whether the signal was sensitive to changes in background rates selection by conducting a post-hoc sensitivity analysis using the updated 2022 background rates as the historical comparator in sequential testing.
Medical Record Review
Medical record review was conducted for the myocarditis/pericarditis outcome following identification of a signal. Brighton Collaboration case definitions were used to adjudicate cases.14 Records meeting the confirmed or probable Brighton classifications were considered true myocarditis/pericarditis cases for the validation analyses.
Results
Descriptive Monitoring
A total of 8,444,355 monovalent COVID-19 vaccine doses were administered to 4,102,016 enrollees aged 6 months -17 years (Figure 1). This included 8,121,591 BNT162b2 (Dose 1: 3,843,778, Dose 2: 3,235,442, Dose 3/ Monovalent Booster: 1,033,036 and Unknown/Unclear: 9,335), and 322,628 mRNA-1273 doses (Dose 1: 173,857, Dose 2: 140,734, Dose 3/Monovalent Booster: 5,284 and Unknown/Unclear: 2,753) administered to children aged 6 months-17 years, as well as 136 NVX-CoV2373 doses (Dose 1: 63, Dose 2: 43, Dose 3/Monovalent Booster and Unknown/Unclear: 30) administered to children aged 12-17 years (Table 1a). Demographic characteristics of the vaccinated population except age reported at first dose were largely similar across vaccine brands. While majority (93.4%) of BNT162b2-vaccinated individuals were 5 years old and older; majority (97.6%) of mRNA-1273-vaccinated individuals were younger than 5 years old. Across all vaccine brands, 2,058,142 (50.2%) were males and 3,901,370 (95.2%) lived in an urban area (Table 1b). We observed low case counts for outcomes that were monitored descriptively only (<80 cases of any given outcome in all three databases combined) (Supplemental Table 5).
Sequential Testing
Among 15 outcomes that were sequentially tested, two outcomes met the statistical threshold for a signal including myocarditis/pericarditis in ages 12-15 and 16-17 years, and seizures/convulsions in ages 2-4/5 years.
Myocarditis/pericarditis signaled for all definitions in the primary analysis following BNT162b2 COVID-19 vaccination among children aged 12-15 and 16-17 years in all three databases. Additionally, dose-specific signals for one or more definitions of the outcomes were detected in ages 12-17 years following dose 1, dose 2, and dose 3 of BNT162b2 vaccine in at least one of the three databases (Table 2).
In the primary analysis, seizures/convulsions met the statistical threshold for a signal in children aged 2-4 years following BNT162b2 vaccination in all three databases, and in children aged 2-5 years following mRNA-1273 vaccination in two of the three databases. Dose-specific signals for seizures/convulsions were detected in two of the three databases following dose 1 and dose 2 BNT162b2 vaccination in ages 2-4 years and following dose 2 mRNA-1273 vaccination in ages 2-5 years (Table 2).
Sequential testing did not initiate for any of the 15 outcomes for mRNA-1273 in ages 5-17 years and NVX-CoV2373 in ages 12-17 years, because no outcomes were observed in any databases.
Signal Characterization
Since myocarditis/pericarditis is a known adverse outcome following COVID-19 mRNA vaccinations, further signal characterization activities were not conducted. In the evaluation of the seizures/convulsions signal among children aged 2-4/5 years, none of the pre-specified data quality checks, including claims duplication, timing of events within pre-specified risk window, and IRR estimates, raised any data quality concerns.
There were 72 observed seizures/convulsions cases among children aged 2-4/5 years and >50% of these cases met the definition of febrile seizures. No differences in rates of seizures/convulsions by sex were identified. The timing of cases did not indicate substantial clustering with cases distributed across the 0-7-day risk window; 31.9% of the seizures/convulsions cases occurred within the 0-1 days following COVID-19 vaccination. The median time between vaccination and diagnosis of seizures/convulsions was 2 days. (Supplemental Figure 1).
The seizures/convulsions signal was sensitive to changes in the selection of comparator rates. Evaluation of the annual background rate of seizures/convulsions indicated that the rates used in the primary analyses (2020) were substantially lower than rates in 2022. Background rates in 2022 ranged from approximately 2.2-2.4 times the 2020 rates across three databases (Supplemental Table 6). A post-hoc sensitivity analysis using 2022 background rates as the comparator in sequential testing did not identify any seizures/convulsions signals.
Medical Record Review
Of the 153 cases of myocarditis/pericarditis after COVID-19 vaccination among children aged 12-17 years, medical record review was conducted for a sample of 40 cases whose records could be obtained. Twenty-nine of these cases (72.5%) were confirmed as true cases of myocarditis/pericarditis, of which 27 patients were male, and 19 were hospitalized with a median length of hospital stay of 2 days (interquartile range: 1, 3). The median time from vaccination to presentation of myocarditis/pericarditis event was 3 days (interquartile range: 2, 5). Medical record review and further validation efforts for seizures/convulsions are currently underway.
Discussion
Our near real-time monitoring of 21 pre-specified health outcomes following monovalent COVID-19 vaccines detected signals for myocarditis/pericarditis in the age group 12-17 years and seizures/convulsions in the age group 2-4/5 years. We did not detect signals for other outcomes that were sequentially tested.
The myocarditis/pericarditis signal is consistent with peer-reviewed publication reports demonstrating an elevated risk of this outcome following mRNA vaccines among younger males aged 12 to 29 years.15–17 Myocarditis/pericarditis is a rare event with a reported average incidence of 39.3 cases per one million vaccine doses administered in children aged 5-17 years within 7 days after BNT162b2 vaccination.18–19 We did not detect a signal for myocarditis/pericarditis in children younger than 12 years old which is consistent with reports from other surveillance systems.20–21
The seizures/convulsions signal in children aged 2-4/5 years has not been previously reported for this age group in active surveillance studies of mRNA COVID-19 vaccines. However, there are reports from the Vaccine Adverse Events Reporting System (VAERS) database which is a passive reporting system and has limitations. In an analysis of VAERS data, only 8 seizures were identified following approximately one million mRNA vaccinations through August 2022 in the age group 6 months to 5 years. Six of the 8 seizures were afebrile on medical evaluation.22 Pre-licensure vaccine safety data indicate that among young children, seizures/convulsions following mRNA COVID-19 vaccines are rare; clinical trial of 3,013 BNT162b2 vaccine recipients in children aged 6 months to 4 years reported only 5 febrile convulsions cases and only one of those (in a 6-month-old participant) was considered possibly related to the vaccine or may also have been caused by a concurrent viral infection.23 Among studies conducted in children with childhood epilepsy (<18 years), no increased risk of medically attended seizures was identified following immunization with COVID-19 vaccines. Seizure risk after COVID-19 vaccination was lower in children who were seizure free for more than six months before vaccination. However, the incidence of general adverse events after vaccination was low with no severe adverse events recorded. 24–25 Generally, there is limited evidence linking the mRNA COVID-19 vaccines to a seizure onset among vaccinated children aged 2-4/5 years.
The new seizures/convulsions signal observed in our study should be interpreted with caution and further investigated in a more robust epidemiological study. Our study utilized a broad seizures/convulsions outcome definition with a 0-7-day risk window because of its applicability to older children. However, in children younger than 5 years, vaccine-related seizures typically manifest as a febrile seizure.26 Although the majority of seizures/convulsions cases met the febrile seizure definition, there was no statistically significant clustering observed at days 0-1. Since febrile seizures can be common in young children for a variety of reasons; the analysis may have identified febrile seizures unrelated to the vaccination later in the risk window.
The post-hoc sensitivity analyses using the 2022 background rates as comparators in sequential testing did not yield any seizures/convulsions signals which suggest that our results are sensitive to comparator rate selection. The decision to use 2020 seizures/convulsions rates as comparators was to maximize sensitivity in the primary analysis. However, seizures/convulsions rates in this age group in 2022 were twice as high as 2020 rates. There could be a couple of potential reasons for elevated outcome rates in 2022 compared to 2020. First, there was an increased incidence of respiratory infections (influenza and respiratory syncytial virus) which are shown to be associated with febrile seizure in younger children, during the study period (mid-2022 to mid-2023) compared to 2020.27–30 Second, in 2020, there were likely fewer emergency department visits for seizure-related events compared to 2022 because of COVID-19 pandemic healthcare resource limitations.31
Our study has a number of strengths. First, the study included a large, geographically diverse population from three US commercial health insurance databases. Due to availability of more complete information from claims supplemented with IIS data and a short data lag from health encounters, we monitored monovalent COVID-19 vaccines safety in a near real-time manner. Additionally, a subset of the identified myocarditis/pericarditis cases were confirmed through medical record review.
The study also has some limitations. We used a near–real-time surveillance method which may be sensitive to comparator rate selection and does not include controlling for bias and confounding. Therefore, results from this study do not establish a causal relationship between the vaccines and health outcomes, and signals should be further evaluated. Secondly, this study only includes data from a commercially insured pediatric population and may not be nationally representative. Furthermore, the study may have limited power to detect small increases in risk of outcomes in certain subgroups or in the cases of more recently authorized vaccines, such as NVX-CoV2373 in children aged 12-17 years as well as mRNA-1273 in those aged 5-17 years.
Conclusion
Our study detected a signal for myocarditis/pericarditis in older children which is consistent with existing literature, and a new signal for seizures/convulsions in young children that is being further evaluated in a more robust study. FDA concludes that the known and potential benefits of COVID-19 vaccination outweigh the known and potential risks of COVID-19 infection. This study was conducted under the FDA BEST Initiative which plays a major role in the larger US federal government vaccine safety monitoring efforts and further supports regulatory decision-making regarding COVID-19 vaccines.
Data Availability
The study protocol was publicly posted previously and a brief report was published describing the preliminary results of BNT162b2 COVID-19 Vaccine in Children Aged 5 to 17 Years on JAMA Pediatrics, as referenced in the manuscript. Data analyses and related documents can be made available where needed, by contacting the corresponding author. De-identified participant data will not be shared without approval from the data partners.
Funder
The US Food and Drug Administration provided funding for this study and contributed as follows: led the design of the study, interpretation of the results, writing of the manuscript, decision to submit, and made contributions to the coordination of data collection and analysis of the data.
Disclosures
Annemarie Kline, Cheryl McMahill-Walraven, Djeneba Audrey Djibo worked on grants, subcontracts, or contracts from Harvard Pilgrim Health Care Institute, Brown University (National Institute on Aging/ IMPACT Collaboratory), Reagan Udall Foundation for the FDA, Academy of Managed Care pharmacy’s Biologics and Biosimilar Collective Intelligence Consortium (BBCIC), TherapeuticsMD, Reachnet (Louisiana Public Health Institute), IQVIA, Pfizer, Carelon Research (formerly Healthcore), and Patient Centered Outcomes Research Institute as employees of CVS Health and report stock or stock options from CVS Health; Carelon Research; Kandace Amend, Robin Clifford, John Seeger, and Jennifer Song reports stock or stock options in UnitedHealth Group; Daniel C. Beachler is an employee of Carelon Research who has previously contracted with Pfizer Inc. for separate projects. No other authors report relevant disclosures.
Additional Contributions
We thank Tainya C. Clarke, Sylvia Cho and Carla E. Zelaya of the US Food and Drug Administration; Anchi Lo, Bing Lyu, Nimesh Shah, Derek Phan, Gyanada Acharya, Kamakshi Sirpal, Laurie Feinberg, Lloyd Marks, Minisha Kochar, Sandia Akhtar, Shruti Parulekar, Stella Zhu, William (Trey) Minter, Yeerae Kim, Yixin Jiao, and Zhiruo (Cassie) Wan of Acumen; Nancy B Shaik, Ana M Martinez-Baquero, Smita Bhatia, Vaibhav Sharma, Carla Brannan of CVS Health and Charlalynn Harris and Danielle Cooper formerly CVS Health; Shiva Vojjala, Ramya Avula, Shiva Chaudhary, Shanthi P Sagare, Ramin Riahi, Navyatha Namburu, and Grace Stockbower of Carelon Research; Michael Goodman, Michael Bruhn, and Ruth Weed of IQVIA; Grace Yang, Sarah Sargen, Alexandra Stone, Wafa Tarazi, Megan Ketchell, Kathryn Federici, Amaka Ume, Emily Myers, Eli Wolter, Jackson Slaney, Bobby Smith, Lauren Peetluk, and Elizabeth Bell of Optum.
Acknowledgements
References
