Author Response
Reviewer #2 (Public Review):
According to the authors, the goal is to identify a method to study changes in hospital presentation and outcomes of new COVID-19 variants using publicly available population-level data on variant relative frequency to infer SARS-CoV variants likely responsible for clinical cases. This would assist in answering questions asked by public health authorities as to differences in disease severity and risk factors and vaccine protection.
Authors use patients' data collected prospectively in 30 countries in their pre-Omicron period (Omicron variant is less than 10% of SARS-CoV2 variants) to the Omicron period (Omicron variant prevalence is >90% of circulating variants). The following factors are analyzed and adjusted for: age/gender, symptoms, comorbidities, vaccination, and outcomes during pre and Omicron periods.
Their model shows that overall, patients were younger, had less symptoms and that the mortality rate was lower in the Omicron period (even if it doesn't reflect in some country reports). No conclusion can be made on vaccination status.
Major weaknesses and strengths:
- The study is presented as a multi-center international study that includes more than 100,000 patients from 30 countries, however, 96.6% of the study patients originated from 2 countries, South Africa (54%) and the United Kingdom (42.6%) (and the relative contribution of South Africa to the study data was hugely different in the 2 study periods, pre-Omicron and Omicron period).
The huge imbalance in the number of patients recruited by center could create many bias in data interpretation. For example, some countries do not report any increase in patients aged less than 12 years old in the omicron period. Country specific medians suggest that the younger age of patients after the Omicron variant experience in the combined dataset is at least partially explained by an increase of data contributed by South Africa, relative to the proportion of data contributed by other countries.
In total only 11 countries contributed data on more than 100 hospitalized cases.
The differences in study data contribution between countries, with more than 90% of all records being from the United Kingdom and South Africa, required both an adapted analytical approach, that transparently presented country-level data rather than only aggregated estimates, and careful discussion of our findings. Indeed, we agree with the reviewer that this imbalance in country-level data contribution and the varying contribution of some countries to the two study periods could lead to erroneous inferences if ignored (i.e. if only aggregated results were reported); for this reason, we presented country-specific data in the Results section. In our descriptive analyses, to achieve this goal without jeopardising intelligibility, we present findings for a subset of countries, those with at least 50 observations per study period; note that this criterion was modified based on another comment from this reviewer. This approach also addresses the reviewer’s concern, which we share, that the varying relative contribution of different countries to study periods could lead to spurious aggregated patterns. In fact, we highlight this problem in the following paragraph of the Results section:
“The median (IQR) ages of patients during the pre-Omicron and Omicron periods were 62 (43 – 76) and 50 (30 – 72) years, respectively; however, country-specific medians suggest that the younger age of patients after Omicron variant emergence in the combined dataset is at least partially explained by an increase in the proportion of data contributed by South Africa, relative to the proportion of data contributed by other countries (Table S6).”
Recruitment of patients is unclear.
We don't really know which patients are selected to be part of the study.
The authors mention the use of the ISARIC (International Severe Acute Respiratory and Emerging Infections Consortium) COVID-19 database (l. 173). This would imply that patients with severe respiratory symptomatic COVID-19 are recruited in the study. It could explain why patients recruited from Brazil or the Netherlands have the same proportion of patients presenting with shortness of breath in the pre- and Omicron period.
Due to the time-sensitivity and scale of this work, involving hundreds of investigators in 30 countries, although the study only included hospitalised patients with SARS-CoV-2 infection, the approach used for patient recruitment in each institution was defined by local investigators. Whilst the sampling strategy was not uniform across sites, one should keep in mind that: (i) recommendations on sampling strategy were shared with local investigators; and (ii) most of the partner institutions involved in this work had previously contributed data to the ISARIC platform and are experienced in patient recruitment and clinical and epidemiological research.
More generally, recruitment approaches could influence the interpretation of our findings in two ways: by reducing the representativeness of the study population in each country; and by inducing bias that could affect the association of interest (the association between study period and fatality risk). Regarding the former, it is possible that in some countries hospitals contributing to this effort admitted patients with more severe disease compared to the local population of COVID-19 hospitalised patients, the target population. Regarding the second potential problem, bias, hospital-based studies might suffer from collider bias, where both the exposure of interest and the outcome directly influence recruitment (selection) to the study or are associated with selection or recruitment through confounders; this is a well-described problem in hospital-based studies that assess COVID-19 outcomes (see Griffith et al. Collider bias undermines our understanding of COVID-19 disease risk and severity. Nature Communications 2020. for a discussion on how different COVID-19 clinical factors can induce bias when different sampling frames are used). Note that collider bias is not the only mechanism of selection bias affecting effect measures; as explained by Miguel Hernán (in Invited Commentary: Selection Bias Without Colliders. American Journal of Epidemiology 2017) between-exposure stratum heterogeneity in the association between the outcome and selection could bias the association between the exposure and the outcome (relative to the effect measure in the target population). However, recruitment approaches used by partner institutions are unlikely to have systematically changed during the study period, and we are unaware of evidence suggesting any association that might have existed between recruitment procedure and outcome differed in the two study periods for most, or indeed some, partner institutions.
We have now modified the Discussion section to highlight this potential weakness of our study:
“Another weakness of our study is that recruitment procedure was not standardised and was defined locally. Whilst this likely affected the generalisability of our descriptive estimates (fatality risk and frequencies of symptoms and comorbidities) to local populations of hospitalised COVID-19 cases (Lash and Rothman, Selection Bias and Generalizability. in Modern Epidemiology 4th Edition 2021; Rothman et al. Why representativeness should be avoided. International Journal of Epidemiology 2013), it might not have affected the association between study period and fatality risk, at least not beyond the well-described potential for collider bias in hospital-based studies on COVID-19 outcomes (Griffith et al. Collider bias undermines our understanding of COVID-19 disease risk and severity. Nature Communications 2020).”
In Nepal, patients were more often recruited from critical care setting (l.572).
However, the authors mention elsewhere that patients recruited for the study were:
• Omicron variant infections in hospitalised patients (I. 161),
• Patients with confirmed or suspected COVID-19 (l.183),
• "some patients were admitted for a medical condition other than covid19 but tested incidentally during hospitalization (l.243)"
• In some countries, information on whether covid-19 was the main reason for hospitalization was also collected. 69.0% of patients admitted during the omicron periods were admitted due to covid-19, patients for whom this information was available were primarily from South Africa (94.9%), (L.310)
• For 35.5% of patients admitted to hospital date of symptoms onset was missing and it was assumed that these were not hospital acquired infections (l.233)
• Information on whether covid-19 was the main reason for hospitalization was collected during the study period and suggest that for a non-negligible proportion of patients, others clinical conditions might have prompted hospitalization.
• In their discussion the authors state that "Finally it is also possible that the question on the primary reason for hospitalization might have been interpreted differently in different countries and even in different hospitals in the same country."
In the few clinical studies from United Kingdom and South Africa 40% to 70% of admissions were qualified as "incidental" COVID-19.
This comment relates to the previous comment and to the sampling strategy used in the study. Please, see our response to the previous comment.
Regarding incidental infections, we have now included information on recent studies (Klann et al. Distinguishing Admissions Specifically for COVID-19 From Incidental SARS-CoV-2 Admissions: National Retrospective Electronic Health Record Study. J Med Internet Res; Voor in ’t holt et al. Admissions to a large tertiary care hospital and Omicron BA.1 and BA.2 SARS-CoV-2 polymerase chain reaction positivity: primary, contributing, or incidental COVID-19. International Journal of Infectious Diseases 2022).
“One possible explanation for this finding would be if incidental SARS-CoV-2 infections, i.e. infections that were not the primary reason for hospitalisation, were more frequent during the Omicron period; the high transmissibility of this variant, and the consequent peaks in numbers of infections, together with its reported association with lower severity, provides support for this hypothesis. However, in the subset of patients with data on the reason for hospitalisation there was no increase in the proportion of admissions thought to be incidental infections and indeed proportions in both study periods were consistent with frequencies of incidental infections in recent studies in the United States (Klann et al. Distinguishing Admissions Specifically for COVID-19 From Incidental SARS-CoV-2 Admissions: National Retrospective Electronic Health Record Study. J Med Internet Res) and the Netherlands (Voor in ’t holt et al. Admissions to a large tertiary care hospital and Omicron BA.1 and BA.2 SARS-CoV-2 polymerase chain reaction positivity: primary, contributing, or incidental COVID-19. International Journal of Infectious Diseases 2022), although in the latter, non-incidental infections included patients for whom COVID-19 was a contributing but not the main cause of hospitalisation.”
Absence of data standardization.
There doesn't seem to be standardized questionnaires across all countries. Some countries do not report on symptoms, others do not report on vaccination status. In total, it seems that less than a third of patients have full data (symptoms, co-morbidities, vaccination, and outcome), and such patients are reported by few countries.
South Africa (that represents 54% of patients) didn't systematically report on symptoms. Hence data showed for symptoms might reflect in volume mainly the United Kingdom patients. In the United Kingdom vaccination rates during the omicron period was 70.3% as compared to 27.9% for South Africa. The authors find that patients with Omicron variant display less symptoms, (which confirms previous findings) however it could have been as plausible that patients from South Africa being less vaccinated exhibit more symptoms.
Analysis for each group of data is based on different patients' group according to the data available for such group.
Data from South Africa used in this analysis are part of the DATCOV national hospital surveillance database. The case report form (CRF) used by the National Institute for Communicable Diseases in South Africa was adapted from the ISARIC CRF; although most sections of that CRF were used for the data collection in the country, information on symptoms was not systematically collected. However, as mentioned above, in our analysis, we also report country-level frequencies of symptoms, rather than only presenting aggregated estimates. We agree with the reviewer that we cannot exclude the possibility that in South Africa a different pattern occurred. Based on this comment, we have now included the following statement in the Discussion section:
“Finally, missing information on symptoms for patients from South Africa prevented our descriptive analysis of changes in clinical presentation in an African setting.”
Vaccination data.
Vaccination data are available for less than 50% of the patients and there is considerable inter-country variation in vaccination rates, as we know but also in the recruitment of patients for the study.
As an example, Table 1 shows the vaccination status by country and study period for 24 countries: Brazil has a vaccination rate of 84.6% and India of 34.8% but on respectively 13 and 23 observations. There are less than 30 observations in 19 countries for pre omicron and less than 30 observations in 15 countries for the omicron period.
No conclusion can be made.
Our study was not designed to assess vaccine effectiveness against the Omicron and non-Omicron variants as controls (e.g. patients hospitalised with respiratory infection caused by pathogens other than SARS-CoV-2) were not recruited. Whilst we descriptively report the frequency of previous vaccination by country and age groups (see Figure S3 in the Supplementary Appendix, with numbers of records in each category presented for transparency), the primary objective in using vaccination data was to control confounding by this factor. The point made by the reviewer, that missing data on vaccination reduced sample size for this comparison, is valid and we have included the following statement in the Discussion section:
“We also observed that history of COVID-19 vaccination was more frequent during the Omicron period, although for most countries the number of patients with vaccination information was limited, especially after stratification by age. Whilst this pattern would be expected if current vaccines were less effective against the Omicron variant compared to previously circulating variants, as suggested by a recent study in England analysing symptomatic disease, there were changes in vaccination coverage in many settings during the second half of 2021 and early 2022, including in response to the reports of Omicron variant cases. Since non-COVID-19 patients (e.g., patients with respiratory infections caused by other pathogens) were not systematically recruited for this multi-country study, it is not possible to estimate vaccine effectiveness during the two study periods and assess its change.”
Major findings of the study:
Major findings of the study match previous individual-based reports:
1-in many settings patients hospitalized with Omicron less often presented with commonly reported symptoms compared to patients infected with pre-omicron variants.
- In a mixed-effects logistic model on 14-day fatality risk that adjusted for sex, age categories and vaccination status hospitalization during the Omicron period were associated with lower risk of death. Similar results were obtained when using 28-days fatality risk and when excluding patients who reported being admitted to hospital due to a medical condition other than covid-19.
- History of COVID-19 vaccination was more frequent during the Omicron period, but the authors cannot make any conclusion on vaccine effectiveness
How to interpret these data? The impact in terms of disease severity of new variants has been shown to be context specific due to regional differences in terms of variability of previous exposure, vaccinations rates and population comorbidity level frequency. As a result of recruitment bias and small recruitment in some countries, several countries have different findings described that do not fit with the conclusions.
As mentioned by the authors, the strength of the project is to have succeeded in engaging so many countries to work together which could definitely assist in the future in understanding new variants characteristics shared globally and identify country specific impact on these variants according to the history of previous variant exposure, vaccine coverage, population morbidity and access to health.
Reviewer #3 (Public Review):
The authors combine outcomes data from patients hospitalised with COVID-19 across 30 countries to investigate differences in likelihood of death from the Omicron variant vs pre-Omicron variants. Data are from the ISARC COVID-19 database; variant status is inferred from country-specific GISAID data. The principal finding is a 36% reduced risk of 14-day death in the Omicron period (OR 0.64 (0.59 - 0.69)) compared with the pre-Omicron period, after multiple adjustment.
The strengths of this paper are the large N and large number of participating countries from different regions, and also the careful and thorough analytical approaches. The main findings are stress-tested through a range of sensitivity analyses using different variant-dominance thresholds and statistical approaches and found to be robust. The figures are clear, well-chosen and easily interpretable.
The principal weaknesses, as acknowledged in the discussion, are the imbalance in the data sources (96.6% of the observations came from GBR or SA), and the lack of fidelity of data on vaccination (vaccination status is limited to a binary 'one or more vaccinations received Y/N' variable). This latter means that conclusions about the innate severity of Omicron vs pre-Omicron variants cannot be drawn.
Nonetheless the findings represent a useful contribution to the literature on the severity of COVID-19 variants, and the approach establishes a template for rapid international collaboration, using GISAID data to infer variant status, that will be useful for formulating policy in response to new variants in the future.
The limited data on timing of vaccination and number of previous doses imply that residual confounding could partially explain the observed association; we mention this limitation in the Discussion section. Although our data alone cannot provide sufficient evidence for differences in innate severity between variants, mechanistic studies (see Shuai et al. Attenuated replication and pathogenicity of SARS-CoV-2 B.1.1.529 Omicron. Nature 2022, and Halfmann et al. SARS-CoV-2 Omicron virus causes attenuated disease in mice and hamsters. Nature 2022) suggest the Omicron variant might be less virulent. We modified the following paragraph in the Discussion section:
“All these factors might have contributed to the observed association, possibly to different degrees in different countries, reason for which this result should not be assumed to necessarily relate to the differences in variant virulence previously suggested by mechanistic studies (Shuai et al. Attenuated replication and pathogenicity of SARS-CoV-2 B.1.1.529 Omicron. Nature 2022; Halfmann et al. SARS-CoV-2 Omicron virus causes attenuated disease in mice and hamsters. Nature 2022).”
