Abstract
Age, sex, and comorbidities are known risk factors for severe COVID-19 but are frequently considered independently and without accurate knowledge of the magnitude of their effects on risk. Single-nucleotide polymorphisms (SNPs) associated with risk of severe COVID-19 have appeared in the literature, but their application in predictive risk testing has not been validated. Reliance on age and sex alone to determine risk of severe COVID-19 will fail to accurately quantify risk. Here, we report the development and validation of a clinical and genetic model to predict risk of severe COVID-19 using confirmed SARS-CoV-2 positive participants from the UK Biobank. Our new model out-performed an age and sex model and had excellent discrimination and was well calibrated in the validation dataset. We also report validation studies of our prototype model and polygenic risk scores based on 8-SNP and 6-SNP panels identified in the literature. Accurate prediction of individual risk will be important in regions where vaccines are not widely available or where people refuse or are disqualified from vaccination, especially given uncertainty about the extent of infection transmission among vaccinated people and the emergence of SARS-CoV-2 variants of concern.
Introduction
The COVID-19 pandemic continues to dominate global public health, with countries having varying success with infection control measures and social distancing protocols,1 Coupled with this are the logistical challenges with the distribution of vaccines2 and the emergence of SARS-CoV-2 variants of concern.3,4 Of those who become infected with SARS-CoV-2, 10%–15% will develop severe COVID-19 requiring hospitalisation and 5% will require intensive care.5 At all stages of the pandemic, there has been an urgent need for accurate quantification of risk of severe COVID-19 to inform protection from infection for those at increased risk.
Epidemiological analyses have recognized that sex and increasing age are risk factors for severe COVID-19 and that common medical comorbidities contribute to individual risk.6-8 Our previous analysis showed that the effects of sex and age are attenuated when comorbidities (such as respiratory disease, hypertension, diabetes, autoimmune disease and cancer history) are taken into account.9 The effect of human genetic variation on COVID-19 severity has been examined by the COVID-19 Host Genetics Initiative, which has now released several meta-analyses of the available genome-wide association studies of COVID-19 severity.10,11 Using population controls, Ellinghaus et al.12 identified two loci (3p21.31 and 9q34.2) as being strongly associated with respiratory failure from COVID-19 and Shelton et al.13 identified the 3p21.31 locus as being associated with severe COVID-19. Also using population controls, Pairo-Castineira et al.14 identified 8 single-nucleotide polymorphisms (SNPs) that achieved genome-wide significance for intensive care admission and identified 6 SNPs (two of which were also in the panel of 8 SNPs) associated with risk of hospitalization.
The emergency authorization of SARS-CoV-2 vaccines15 does not diminish the value of accurate prediction of individual risk of severe COVID-19. Extensive vaccine disqualification criteria (such as pre-existing conditions, pregnancy, and age), vaccine hesitancy, uncertainty as to whether the vaccines are effective against emerging variants of concern,4 and an unknown extent to which vaccines prevent the transmission of infection mean that many people will be at risk of severe COVID-19 should they become infected with SARS-CoV-2.
We previously developed a prototype risk model9 based upon early data from the UK Biobank16,17 and the COVID-19 Host Genetics Initiative meta-analysis of hospitalized vs non-hospitalized COVID-19 cases (which was at that time almost exclusively UK Biobank samples).10,18 Our prototype model appeared to perform well but was based on a small sample size (1,018 cases and 564 controls) from the first wave of the pandemic.9 We decided not to attempt validation in this dataset because of our concern about the representativeness of the data (the SARS-CoV-2 testing data was ascertained early in the pandemic when the limitations on testing availability in the United Kingdom meant that mild and asymptomatic cases were not identified).
In the interim, the UK Biobank has released further data from participants confirmed to be infected with SARS-CoV-2. This latest data release (2,205 cases and 5,416 controls) has a larger proportion of non-hospitalized people, providing more confidence that they are a more representative non-hospitalized control population. In this paper, we perform validation studies of our prototype model and evaluate two polygenic risk scores (PRSs) based on the 8-SNP and 6-SNP panels identified by Pairo-Castineira et al.14 We also took the opportunity afforded by this updated dataset to develop and validate a new model to predict risk of severe COVID-19.
Methods
UK Biobank data and eligibility
Since our first paper on the development of a risk prediction model for severe COVID-19,9 the UK Biobank16,17 has accumulated a large number of additional SARS-CoV-2 test results.19 For this analysis, we downloaded an updated results file on 8 January 2021. As in our first paper, eligible participants were active UK Biobank participants with a positive SARS-CoV-2 test result and who had SNP and hospital data available.9 Of the 47,990 UK Biobank participants with at least one SARS-CoV-2 test result, 8,672 (18.1%) had a positive test result, and of these, 7,621 met our eligibility criteria.
As we did previously,9 we used source of test result as a proxy for severity of disease, where inpatient results were considered severe disease (cases) and outpatient results were considered non-severe disease (controls). If a participant had more than one test result, we classified them as having severe disease if at least one of their results was from an inpatient setting. Of the 7,621 eligible participants, 2,205 (28.9%) were cases and 5,416 (71.7%) were controls.
Data extraction
In addition to the COVID-19 results data file that we downloaded on 8 January 2021, we used the previously extracted UK Biobank hospital and baseline assessment data (see Table 1 for the list of variables extracted). We used Plink version 1.920,21 to extract SNP data from the UK Biobank imputation dataset that we had previously downloaded. We extracted genotypes of the 64 SNPs that were used to calculate the SNP score in our prototype model9 and the 12 SNPs from Tables 1 and 2 of the paper by Pairo-Castineira et al.14 We also identified 43 SNPs from the B1_ALL (hospitalized versus non-hospitalized cases of COVID-19) results of the COVID19-hg GWAS meta-analyses round 4, conducted by the COVID-19 Host Genetics Initiative consortium.10,22 These SNPs were selected by pruning variants with a P value of greater than 10−5 and linkage disequilibrium variants that had an R2 of greater than 0.5 for all populations. Of these 43 SNPS, 40 were available for extraction in the UK Biobank imputation dataset. The full list of SNPs considered in the current paper are listed in Supplementary Table 2.
Validation of prototype model from Dite et al. 9
For the validation of our prototype risk model,9 we used the 1,234 cases and 4,805 controls that weren’t included in our previous paper. We constructed relative risk scores for both the clinical model and the combined clinical and SNP score model using the exponent of the sum of the intercept and the beta coefficients for each risk factor in the prototype model.9
Validation of PRS based on the 8-SNP and 6-SNP panels from Pairo-Castinera et al
To validate PRS based on the SNPs in Table 1 (risk of intensive care admission) and Table 2 (risk of hospitalization) in the paper by Pairo-Castineira et al.14 we used all eligible participants: 2,205 cases and 5,416 controls. We constructed the 8-SNP and 6-SNP PRS using the estimates of the odds ratio (OR) per effect allele and effect allele frequencies provided in Pairo-Castineira et al.14 For the SNPs in their Table 1, we used the GenoMICC risk allele frequencies, and for the SNPs in their Table 2 that didn’t overlap with their Table 1, we used the GenoMICC ORs and used allele frequencies from SNPnexus.23 For the construction of the PRS,24 we assumed independent and additive risks on the log OR scale. For each SNP, we calculated the unscaled population average risk (µ) as:
Next, for each SNP, adjusted risks (with a population average risk equal to 1) were calculated as:
, where N is the number of effect alleles.
The PRS was then calculated as the product of the adjusted risk values for each of the
SNPs.
Development and validation of the new model
To develop a new model to predict risk of severe COVID-19, we used all of the available data and randomly divided it into a 70% training dataset and a 30% validation dataset (ensuring that the datasets were balanced for case and control status). Because our missing data is assumed to be missing at random (if not missing completely at random), we used multiple imputation with 20 imputations to address the missing data for body mass index (linear regression) and the SNP data (predictive mean matching) for the development of the new model in the training dataset. To more closely reflect the availability of data in the real world, we did not use imputed data in the validation dataset.
The clinical variables considered for inclusion in the new model were age, sex, body mass index (BMI), ethnicity (Caucasian vs other), ABO blood type and the following chronic health conditions: asthma, autoimmune disease (rheumatoid arthritis, lupus or psoriasis), haematological cancer, non-haematological cancer, cerebrovascular disease, diabetes, heart disease, hypertension, immunocompromised, kidney disease, liver disease and respiratory disease (excluding asthma). Dummy variables were used for the categorical classifications of age and ABO blood type.
The SNPs selected for consideration in the development of the new model came from three sources: (i) the 64 SNPs from our prototype model,9 (ii) the 12 SNPs from the paper by Pairo-Castineira et al.14 (iii) the 40 SNPs selected from the results of the COVID-19 Host Genetics Initiative’s COVID19-hg GWAS meta-analyses round 4 meta-analysis of non-hospitalized versus hospitalized cases of COVID-19.10,22 To avoid reliance on potentially inaccurate summary statistics to construct a PRS, we used unadjusted logistic regression in the multiple imputation training dataset to identify the subset of SNPS that were associated with risk of severe COVID-19 with P<0.05 (see Supplementary Table 2) and used these as individual risk factors (with a per allele effect) to build our new model.
Statistical methods
Development of new model
We used multivariable logistic regression in the multiple imputation training dataset to develop the new model to predict risk of severe COVID-19. We began with a model that included all of the clinical variables and the SNPs with unadjusted associations with severe COVID-19. We then used backwards stepwise selection to develop the most parsimonious model. For the removed variables we made a final determination on their inclusion or exclusion by adding them one at a time to the parsimonious model. To directly compare the effect sizes of the variables in the final model, regardless of the scale on which they were measured, we used the odds per adjusted standard deviation.25 We used the intercept and beta coefficients from the new model to calculate the COVID-19 risk score for all eligible UK Biobank participants.
Model performance
As well as assess the performance of the prototype model and the 8-SNP PRS and 6-SNP PRS, we assessed the performance of the new model in the imputed training dataset and in the non-imputed validation dataset. The association between risk score and severe COVID-19 was assessed using logistic regression to estimate the OR per quintile of risk score. We assessed model discrimination using the area under the receiver operating characteristic curve (AUC). Where warranted, we plotted the receiver operating characteristic curve of the model.
For models that showed good discrimination, we assessed calibration using logistic regression of the log odds of the risk score to estimate the intercept and the slope (beta coefficient). An intercept close to 0 indicates good calibration, while an intercept of less than 0 indicates overall overestimation and an intercept of greater than 0 indicated overall underestimation of risk.
In terms of the dispersion of the risk score, a slope of close to 1 indicates good estimation across the spectrum of risk. A slope of less than 1 means that the predicted probabilities don’t vary enough (i.e. underestimation of true high risk and overestimation of true low risk). Conversely, a slope of greater than one means that the predicted probabilities vary too much (i.e. underestimation of true low risk and overestimation of true high risk). Where helpful, we also used a calibration plot to illustrate the fit of a model.
We used Stata (version 16.1)26 for analyses; all statistical tests were two-sided and P<0.05 was considered nominally statistically significant.
Ethics approval
The UK Biobank has Research Tissue Bank approval (REC #11/NW/0382) that covers analysis of data by approved researchers. All participants provided written informed consent to the UK Biobank before data collection began. This research has been conducted using the UK Biobank resource under Application Number 47401.
Data availability
We used data from the UK Biobank (under Application Number 47401) for these analyses and do not have permission to share the data. Researchers wishing to access the data used in this study can apply directly to the UK Biobank at https://www.ukbiobank.ac.uk/register-apply/.
Code availability
Stata 16.1 code for the analysis is available from the corresponding author on request.
Results
In the results file downloaded on 8 January 2021, there were 2,205 eligible cases with severe COVID-19 and 5,416 eligible controls with non-severe COVID-19.
Validation of prototype model from Dite et al. 9
Characteristics of the new UKB participants (1,234 cases and 4,805 controls) with positive SARS-CoV-2 test results are shown in Supplementary Table 1.
The odds ratio (OR) per quintile showed that the clinical risk score was strongly associated with severe COVID-19 (OR=1.70; 95% confidence interval [CI]=1.62, 1.79; P<0.001) and that the combined clinical and SNP risk score was less strongly associated with severe COVID-19 (OR=1.45; 95% CI=1.38, 1.52; P<0.001); there was no association with severe COVID-19 for the SNP score (OR=0.98; 95% CI=0.94, 1.03; P=0.5). The discrimination of cases and controls was excellent for the clinical score (AUC=0.711; 95% CI=0.694, 0.727), lower for the combined clinical and SNP score (AUC=0.657; 95% CI=0.639, 0.674) and poor for the SNP score alone (AUC=0.491; 95% CI=0.473, 0.509).
Assessment of model calibration showed that overall, risk was overestimated for both the clinical risk model (α=−1.72; 95% CI=−1.80, −1.65; P<0.001) and the clinical and SNP model (α=−1.63; 95% CI=−1.71, −1.54; P<0.001). For the clinical model, there was no evidence of poor dispersion (β=1.03, 95% CI=0.94, 1.12, P=0.5), while the predictions of the combined clinical and SNP model varied too much (β=0.59, 95% CI=0.52, 0.65, P<0.001).
Validation of PRS based on the 8-SNP and 6-SNP panels from Pairo-Castinera et al. 14
The OR per quintile of risk was not associated with severe COVID-19 for both the 8-SNP PRS (OR=1.03; 95% CI=1.00, 1.07; P=0.08) and the 6-SNP PRS (OR=1.03; 95% CI=1.00, 1.07; P=0.07). Both PRS showed poor discrimination of cases and controls: 8-SNP PRS AUC=0.513 (95% CI=0.499, 0.528) and 6-SNP PRS AUC=0.513 (95% CI=0.499, 0.527).
Development and validation of the new model
Table 1 shows the characteristics of the 1,544 cases and 3,791 controls in the 70% training dataset and the 661 cases and 1,625 controls in the 30% validation data set. In the training dataset, the mean age was 69.8 years (SD=8.6) for cases and 64.6 years (SD=8.4) for controls, and the mean BMI was 29.3 kg/m2 (SD=5.3) for cases and 28.0 kg/m2 (SD=4.9) for controls. In the validation dataset, the mean age was 69.7 years (SD=8.7) for cases and 64.4 years (SD=8.4) for controls, and the mean BMI was 29.4 kg/m2 (SD=5.6) for cases and 28.3 kg/m2 (SD=5.0) for controls.
Training
In the age and sex model, being male and being in one of the four older age groups conferred a substantially increased risk of severe COVID-19 (Table 2), with an OR=1.60 for being male and ORs ranging from 2.74 for the age groups from 65–69 years to 4.95 for the 80+ years group. Direct comparison of the effect size of each variable showed that the age group 75–79 years was the strongest risk factor (with an odds per adjusted standard deviation of 1.58), followed by the 70–74 and 80–84 groups (with odds per adjusted standard deviations of 1.42 and 1.34, respectively).
The new model was developed from the variables in Table 1, which include the clinical variables and the 14 SNPs identified as having unadjusted per allele ORs with P-values <0.05 (see Supplementary Table 2). The variables retained in the new model are shown in Table 3 and comprise three age groups (70–74, 75–79 and 80–84 years), sex, ethnicity, body mass index, six comorbidities and seven SNPs. Compared with the age and sex model, the ORs for sex and age group were attenuated in the new model, with an OR=1.27 for being male, the 70–74 years age group not being at increased risk, and ORs for the other age groups ranging from 1.77 for the 70–74 years group to 2.76 for the 80+ years group. Direct comparison of the effect size of each variable showed that respiratory disease was the strongest risk factor with an odds per adjusted standard deviation of 1.35, followed by the three age groups with odds per adjusted standard deviations of 1.20 to 1.29). The other clinical risk factors and SNPs had odds per adjusted standard deviation in the range 1.07 to 1.13 (or the equivalent protective effect).
The age and sex model had good discrimination of cases and controls with an AUC of 0.676 (95% CI=0.659, 0.692) but the new model with an AUC of 0.752 (95% CI=0.737, 0.767) was a substantial improvement (χ2=149.40, df=1, P<0.001).
Validation
In the non-imputed validation dataset, both models were associated with severe COVID-19. The OR per quintile for the age and sex model was 1.49 (95% CI=1.40, 1.59; P<0.001), while the new model had a substantially higher OR per quintile of 1.77 (95% CI=1.64, 1.90; P<0.001).
In terms of discrimination between cases and controls, the age and sex model had an AUC of 0.671 (95% CI=0.646, 0.696) but the new model with an AUC of 0.732 (95% CI=0.708, 0.756) was a substantial improvement (χ2=41.23, df=1, P<0.001). The receiver operating characteristic curves for both models are shown in Figure 1.
Both models were well calibrated with no evidence of overall overestimation or underestimation for the age and sex model (α=−0.02; 95% CI=−0.18, 0.13; P=0.7) or the new model (α=−0.08; 95% CI=−0.21, 0.05; P=0.3). There was also no evidence of under or over dispersion for the age and sex model (β=0.96, 95% CI=0.81, 1.10, P=0.6) and for the new model (β=0.90, 95% CI=0.80, 1.00, P=0.06). Calibration plots for both models are shown in Figure 2.
Probability of severe COVID-19 in whole UK Biobank
We calculated the probability of severe COVID-19 for all UK Biobank participants who met our eligibility criteria for this study; the distributions are shown in Figure 3, and the distribution of the new model by 5-year age group are shown in Supplementary Figure 1. Using the age and sex model, the mean probability was 0.32 (SD=0.13) and ranged from a minimum of 0.15 to a maximum of 0.56. Using the new model, the mean probability was 0.27 (SD=0.16) and the range was from 0.04 to 0.98, a much wider range than for the age and sex model.
Discussion
An accurate test to predict risk of severe COVID-19 can inform prioritization of vaccine doses to those most at risk27 and will be useful in regions in which vaccination is not widespread enough to provide herd immunity (either through unavailability or vaccine hesitancy), if available vaccines are not effective against variants of SARS-CoV-2, or if available vaccines are not indicated for some people. On an individual level, knowledge of personal risk can empower people to make informed choices about their day-to-day activities, including targeted social distancing in the workplace28 or other crowded places.
The validation of the clinical component of our prototype model confirmed that it performed well with good discrimination (AUC=0.711), but overall, it overestimated risk. The SNP score component of the prototype model was not confirmed in the validation dataset and is likely due to the prototype model having been developed in dataset with a high prevalence of severe COVID-19.
We were also unable to confirm the unadjusted association with severe COVID-19 in the imputed UK Biobank training dataset for any of the SNPs identified by Ellinghaus et al.12 and Pairo-Castineira et al.,14 both of whom used population controls in their analyses (see Supplementary Table 2). When tested as an 8-SNP PRS and a 6-SNP PRS, the loci from Pairo-Castineira et al.14 were uninformative. Pairo-Castineira et al.14 noted that using population controls rather than SARS-CoV-2 positive controls would likely bias any associations towards the null. However, review of the Manhattan plots of the results of the COVID19-hg GWAS meta-analyses round 4 for hospitalized vs non-hospitalized and for hospitalized vs population controls reveals that, other than the 3p21.31 locus, the regions of interest differ considerably.22 If the use of population controls were to attenuate any associations, then we would expect the loci identified using population controls to be present with stronger associations in the hospitalized vs non-hospitalized analysis. Instead, it may be the case that using population controls identifies loci associated with propensity to become infected with SARS-CoV-2 rather than severity of COVID-19 once infected.
Given the failure to confirm SNP-based risk scores in the form of a count of risk alleles or as a PRS, we incorporated SNPs in the new model without relying on published summary statistics and without assumptions as to the identity of the risk allele. We included the SNPs as individual risk factors and estimated the per allele OR for each. By doing so, we were able to identify the subset of SNPs (and clinical risk factors) that were informative for predicting risk. From our initial list of 116 SNPs (Supplementary Table 2), we considered 14 for inclusion in our model and retained seven (rs112641600, rs10755709, rs118072448, rs7027911, rs71481792, rs112317747, and rs2034831), none of which were in the 3p21.31 locus identified by others.12-14,22
Functionally, most of the SNPs retained in our new model are associated with genes that play a role in infection pathways or immunity. The immune function and chromatin remodelling family of GATA transcription factors are associated by the inclusion of SNPs near HIVEP1 (rs10755709), which encodes a viral-infection regulation transcription factor, and GATA3 (rs71481792).29,30 ALPK1 and TIFA are closely downstream of rs112641600 and both have adaptive and innate signal transduction roles and pro-inflammatory functions.31 MSR1, upstream of rs118072448, is a macrophage scavenger receptor and implicated in a broad range of disease types including host viral defence32 and PSAT1 is associated with glutamine metabolic reprogramming by SARS-CoV-2 and viral mRNA translation.33
In the development of the new model, the strongest risk factor was respiratory disease (with an odds per standard deviation of 1.35; Table 3). The older age groups (70–74, 75–79, and 80+ years) and being male all had odds per standard deviations of 1.20 to 1.29. The other risk factors (the seven SPNs, ethnicity, body mass index, cancer history (haematological and non-haematological), cerebrovascular disease, diabetes, hypertension, and kidney disease) all had odds per adjusted standard deviations in the range 1.07 to 1.13 (or the equivalent protective effect). These risk factors are all important to risk prediction, and characterization of the SNP genotypes is as important as ascertaining clinical information.
In the non-imputed validation dataset, the new model performed very well with an AUC of 0.732 (compared with an AUC of 0.752 in the training dataset). Importantly, the new model was well calibrated, showing no evidence of problems with the overall estimation of risk or the dispersion of risk predictions. The validation of the new model also illustrates the importance of considering risk factors beyond age and sex in predicting risk of severe COVID-19. The new model was a substantial improvement over the age and sex model, in terms of the OR per quintile (OR=1.77 and OR=1.49, respectively) and the discrimination of cases and controls (AUC=0.732 and AUC=0.671, respectively). The new model also allows stratification across a wide range of risk (Figure 3B) so that, for example, a healthy person aged 75 years might have a lower risk of severe COVID-19 than a 50-year-old person with several risk factors.
A limitation of this study is that, through necessity, we used hospitalization as a proxy for COVID-19 severity and the outcome measure may have been misclassified for some participants. This would have attenuated the observed associations and it is possible that some risk factors have been omitted unnecessarily. Nevertheless, we are confident in the variables retained. We were also unable to develop models for other important endpoints such as intensive care admission or death.
The progression of the COVID-19 pandemic has seen people experience chronic symptoms, and some of these people will have had only a mild original infection.5 Identifying people who are at increased risk of chronic disease is an obvious direction for future research. Another direction for future research is to investigate whether our model for the prediction of severe COVID-19 is applicable for the new SARS-CoV-2 variants of concern, which have been reported to have increased transmissibility, virulence and antigenicity and cause more severe disease.3,4 Further validation of our new model is required in independent datasets, especially those in which the SARS-CoV-2 variant has been characterized.
Clear benefits of our new model for predicting risk of severe COVID-19 are that the required clinical data is simple to collect and that the genetic information is amenable to high-throughput genotyping, with rapid turnaround that is essential for the present pandemic. In the light of the uncertainty of the future of the COVID-19 pandemic, accurate knowledge of individual risk of severe COVID-19 can make an important contribution to healthcare on a population level and personal level.
Data Availability
We used data from the UK Biobank (under Application Number 47401) for these analyses and do not have permission to share the data. Researchers wishing to access the data used in this study can apply directly to the UK Biobank at https://www.ukbiobank.ac.uk/register-apply/.
Author contributions
All authors were involved in the design and conceptual development of this study. NMM and RA identified the SNPs from the Host Genetics Initiative meta-analysis results. GSD undertook data management and conducted the statistical analyses. GSD and RA wrote the original draft, and all authors reviewed and edited the manuscript. All authors have approved the final manuscript.
Competing interests
GSD, NMM, and RA are employees of Genetic Technologies Limited. Genetic Technologies Limited had no role in the conceptualization, design, data analysis, decision to publish or preparation of the manuscript.
Aspects of this manuscript are covered by Provisional Patent Application AU_2021900392 (pending), Methods of assessing risk of developing a severe response to Coronavirus infection. GSD, NMM, and RA are named inventors on the patent application, which is assigned to Genetic Technologies Limited.
Supplementary figure
Supplementary tables
Acknowledgements
We wish to thank Mr Lawrence Whiting for his invaluable expertise in the management of large data files from the UK Biobank.