ABSTRACT
A disruption of the crosstalk between gut microbiota and the lung (gut-lung axis) has been implicated as a driver of severity during respiratory-related diseases. Lung injury causes systemic inflammation, which disrupts gut barrier integrity, increasing the permeability to gut microbes and their products. This exacerbates inflammation, resulting in positive feedback. To test the possibility that a disrupted gut contributes to Coronavirus disease 2019 (COVID-19) severity, we used a systems biology approach to analyze plasma from COVID-19 patients with varying disease severity and controls. Severe COVID-19 is associated with a dramatic increase in tight junction permeability and translocation of bacterial and fungal products into blood. This intestinal disruption and microbial translocation correlate strongly with increased systemic inflammation and complement activation, lower gut metabolic function, and higher mortality. Our study highlights a previously unappreciated factor with significant clinical implications, disruption in gut barrier integrity, as a force that contributes to COVID-19 severity.
INTRODUCTION
Coronavirus Disease 2019 (COVID-19), the disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, can manifest with diverse clinical presentations. While the majority of infected individuals exhibit asymptomatic or mild respiratory tract infection, a significant population, especially those who are older or suffering from pre-existing metabolic-associated diseases, face severe manifestations such as acute respiratory distress syndrome (ARDS), multi-organ failure, and death.1-5 A state of hyper-inflammation and hyperactivated immune responses, characterized by an ensuing cytokine storm and increased complement activation, has been associated with COVID-19 severity.1,6-10 However, the pathophysiological mechanisms that contribute to these phenomena remain mostly unknown. Understanding these mechanisms is a crucial step in designing rational clinical and therapeutic strategies.
A disruption of the crosstalk between gut microbiota and the lung (gut-lung axis) has been implicated as a driver of severity during respiratory-related diseases, including ARDS.11-14 Systemic inflammation caused by a lung infection or injury can lead to a disruption of the gut barrier integrity and increase the permeability to gut microbes and microbial products. This microbial translocation can exacerbate systemic inflammation and lung injury – resulting in positive feedback.11-14 In addition, SARS-CoV2 can directly infect gut cells,15 and viral infections of the gut cause changes in gut structure16 and breakdown of the epithelial barrier.17,18 Such disruption of the gut-lung axis is more likely to occur in older individuals and individuals with metabolic- and/or aging-associated diseases, as these individuals often experience changes in the composition of the gut microbiota (microbial dysbiosis),19,20 which facilitate a higher susceptibility to falling into the vicious cycle between microbial translocation and systemic inflammation.21-24
Even as microbial translocation impacts systemic inflammation directly, it may also impact it indirectly by modulating circulating levels of gut- and gut microbiota-associated products such as metabolites and lipids. Plasma metabolites and lipids can reflect the functional status of the gut and the metabolic activity of its microbiota.25-28 They also are biologically active molecules in their own right, regulating several immunological functions, including inflammatory responses.29,30 A third class of microbial products that can translocate from the gut is glycan-degrading enzymes. Glycans on circulating glycoproteins and antibodies (IgGs and IgAs) are essential for regulating several immunological responses, including complement activation.31 The glycan-degrading enzymes are released by several members of the gut microbiome and their translocation can alter the circulating glycome, leading to higher inflammation and complement activation.32 Indeed, altered glycosylation of plasma glycoproteins (including immunoglobulin G, IgG) has been associated with the onset and progression of inflammatory bowel disease (IBD).32-37 Furthermore, modulation of the gut microbiota via fecal microbiota transplantation affects IgG and serum glycosylation.38 Here, we hypothesized that a vicious cycle between SARS-CoV2 infection, systemic inflammation, disrupted intestinal barrier integrity, and microbial translocation contributes to COVID-19 severity.
To test this hypothesis, we applied a multi-omic systems biology approach to analyze plasma samples from 60 COVID-19 patients with varying disease severity and 20 age-controlled (most were 50 to 65 years old) and gender-matched (∼50% female) SARS-CoV2 negative controls. We investigated the potential links between gut barrier integrity, microbial translocation, systemic inflammation, and COVID-19 severity. Our data indicate that severe COVID-19 is associated with a dramatic increase in tight junction permeability and translocation of bacterial and fungal products into the blood. This disrupted intestinal barrier integrity and microbial translocation correlates strongly with increased systemic inflammation, increased immune activation, decreased intestinal function, disrupted plasma metabolome and glycome, and higher mortality rate.
RESULTS
Characteristics of the study cohort and study overview
We collected plasma samples from 60 individuals testing positive for SARS-CoV2 (by RT-PCR) and 20 SARS-CoV2 negative controls. The 60 SARS-CoV2 positive individuals were selected to represent three diseases states: 20 with mild symptoms (outpatients); 20 with moderate symptoms (inpatients hospitalized on regular wards); and 20 with severe symptoms (inpatients hospitalized in an intensive care unit (ICU)) (Figure 1a). Individuals were selected to have a median age between 52.5 to 58.5 years to avoid age bias in disease outcome. The study cohort was also chosen to have a 35 to 60% representation of female gender per disease status group (Supplementary Table 1). Samples from hospitalized patients (moderate and severe groups) were collected at the time of diagnosis when the patient was admitted (Supplementary Table 1). Eight individuals of the cohort (two from the moderate group and six from the severe group) died from COVID-19 (Supplementary Table 1). The plasma samples from all individuals were used in a multi-omic, systems biology approach that measured: markers of tight junction permeability and microbial translocation using ELISA and Limulus Amebocyte Lysate assays; inflammation and immune activation/dysfunction markers using ELISA and multiplex cytokine arrays; untargeted metabolomic and lipidomic analyses using mass spectrometry (MS); and plasma glycomes (from total plasma glycoproteins, isolated immunoglobulin G (IgG), and isolated immunoglobulin A (IgA)) using capillary electrophoresis and lectin microarray (Figure 1a and Supplementary Table 2).
Levels of the 35 (out of 50) gut- and gut microbiota-associated plasma metabolites that are disrupted during COVID-19. Red indicates upregulation, blue indicates downregulation; color intensity indicates larger difference. Green indicates FDR<0.05; color intensity indicates lower FDR.
Results of logistic regression models of tight junction permeability and microbial translocation markers for ability to distinguish hospitalized from non-hospitalized individuals.
(a) An overview of the study design; figures in black indicate deceased; moderate and severe patients were hospitalized; severe indicates patients in the intensive care unit. (b) Levels of plasma zonulin, a marker of tight junction permeability, are higher during moderate and severe COVID-19 compared to mild COVID-19 or controls. Kruskal–Wallis test was used for statistical analysis. False discovery rate (FDR) was calculated using the Benjamini-Hochberg method. Symbols in black indicate deceased. (c) Zonulin levels are higher in hospitalized COVID patients (n=40) who eventually died from COVID-19 (n=8) compared to survivors (n=32). Nominal P-value was calculated using the Mann–Whitney U test. (d-g) Levels of markers of translocation and inflammation, LBP (d), β-Glucan (e), sCD14 (f), and MPO (g), are higher during severe COVID-19 compared to mild COVID-19 or controls. Kruskal–Wallis test was used for statistical analysis. FDR was calculated using Benjamini-Hochberg method.
Severe COVID-19 is associated with high tight junction permeability and microbial translocation
We first asked whether severe COVID-19 is associated with changes in tight junction permeability and microbial translocation. We measured the plasma levels of eight established drivers and markers of intestinal barrier integrity (Supplementary Table 2). We found that severe COVID-19 is associated with a dramatic increase in zonulin levels (Figure 1b). Zonulin (haptoglobin 2 precursor) is the only known physiological mediator of tight junction permeability in the digestive tract, where higher levels of zonulin drive increases in tight junction permeability.39 Notably, hospitalized individuals with higher plasma levels of zonulin were more likely to die compared to hospitalized individuals with lower levels of zonulin (Figure 1c).
These higher levels of zonulin should enable the translocation of microbes and their products from the gut into the blood, including parts of the cell wall of bacteria and fungus.40,41 To test this supposition, we measured plasma levels of common bacterial and fungal markers. Exposure to bacterial endotoxin can be determined by measuring plasma lipopolysaccharide (LPS) binding protein (LBP). LBP is an acute-phase protein that binds to LPS to induce immune responses.42 Indeed, we observed a systematic induction of LBP in individuals with severe COVID-19 compared to individuals with mild COVID-19 or controls (Figure 1d). We also found higher levels of β-glucan, a polysaccharide cell wall component of most fungal species and a marker of fungal translocation,43 in individuals with severe COVID-19 compared to those with mild COVID-19 or controls (Figure 1e). In addition, there was a significant induction (FDR=0.025) of the tight junction protein occludin in the severe group compared to controls (data not shown). There also was a strong trend (FDR = 0.051) toward higher levels of the protein 3-alpha (REG3α), a marker of intestinal stress,44 comparing the severe and mild groups (data not shown). We did not observe an increase in intestinal fatty-acid binding protein (I-FABP), a marker of enterocyte apoptosis, suggesting that the increased levels of tight junction permeability and microbial translocation are not associated with enterocyte death.
These high levels of tight junction permeability and microbial (both bacterial and fungal) translocation are expected to lead to microbial-mediated myeloid inflammation. Indeed, levels of soluble CD14 (sCD14; monocyte inflammation marker) (Figure 1f) and myeloperoxidase (MPO; neutrophil inflammation marker) (Figure 1g) were significantly increased during severe COVID-19. Levels of soluble CD163 (sCD163) were also increased significantly (FDR=0.04) in the severe group compared to controls (data not shown). These data indicate that COVID-19 severity and associated mortality are linked to higher tight junction permeability and higher translocation of bacterial and fungal products to the blood.
Microbial translocation is linked to systemic inflammation
Higher levels of microbial translocation should lead to higher systemic inflammation and immune dysfunction. We measured the levels of 31 markers of systemic inflammation and immune activation/dysfunction (Supplementary Table 2) including: 23 cytokines and chemokines (such as IL-6, IL-1β, MCP-1, IP-10, and TNFα), markers of inflammation and thrombogenesis (such as C-reactive protein (CRP) and D-dimer), a marker of complement activation (C3a), a marker of oxidative stress (GDF-15), and three immunomodulatory galectins (galectin-1, -3, and -9). As anticipated, many of these markers were increased in patients with severe COVID-19 compared to patients with mild COVID-19 or controls (Figure 2a-left). In particular, we observed inductions in several cytokines (such as IL-6 and IL-1β) and inflammatory markers (such as CRP and d-dimer). In addition to the expected changes, we also observed significant inductions in the immunomodulatory lectins galectin-3 (Figure 2b) and galectin-9 (Figure 2c). Levels of Gal-9 were higher in the plasma of hospitalized patients who eventually died compared to survivors (Figure 2d). Last, notable dysregulations were observed in levels of C3a (Figure 1e; indicative of complement activation) and GDF-15 (Figure 1f; indicative of oxidative stress), with the levels of GDF-15 higher in deceased hospitalized patients compared to survivors (Figure 1g).
(a left) Heat-map depicting plasma levels of 23 inflammation and immune activation/dysfunction markers whose levels are statistically (FDR<0.05) different between the four disease states. Statistical significance was determined using the Kruskal–Wallis test. FDR was calculated using Benjamini-Hochberg method. SC rho = coefficient of correlation with COVID-19 severity. Heat colors show standardized Z-scores across samples; red indicates upregulation and blue indicates downregulation. (a right) Coefficients of correlation between zonulin, LBP, or β-Glucan and each of the 23 inflammation and immune activation/dysfunction markers. All red-colored correlations had statistical significance of FDR<0.05, whereas the grey-colored correlation was non-significant. Correlations were evaluated using Spearman’s rank correlation tests, and FDR was calculated using the Benjamini-Hochberg method. (b-d) Levels of representative variables, galectin-3 (Gal-3) (b) and galectin-9 (Gal-9) (c), were higher during severe COVID-19 compared to mild COVID-19 or controls, with levels of Gal-9 higher among deceased hospitalized patients compared to survivors (d). (e-g) Levels of C3a (e) and GDF-15 (f) were higher during severe COVID-19 compared to mild COVID-19 or controls, with levels of GDF-15 higher among deceased hospitalized patients compared to survivors (g). Kruskal–Wallis and Mann-Whitney tests were used for statistical analysis. FDR was calculated using Benjamini-Hochberg method. (h-i) Examples of correlations in (a) between LBP and IL-6 (h) or β-Glucan and IL-6 (i). Spearman’s rank correlation tests were used for statistical analysis.
Next, we examined the correlations between the markers of intestinal barrier integrity (zonulin) or microbial translocation (LBP and β-glucan) and the 31 markers of systemic inflammation and immune activation. As shown in Figure 2a-right, higher levels of zonulin, LBP, or β-glucan were strongly positively correlated with higher levels of many of the markers of systemic inflammation and immune activation, including IL-6 (Figure 2h-i). These data support our hypothesis that disruption of intestinal barrier integrity, which results in microbial translocation, is linked to higher systemic inflammation and immune activation during severe COVID-19.
Severe COVID-19 is associated with a plasma metabolomic profile that reflects disrupted gut function
A second set of factors that reflect the functional state of the gut and its microbiota are the plasma metabolites. Importantly, many of these are not solely biomarkers of gut function/dysfunction, but also are biologically active molecules which can directly impact immunological and inflammatory responses. We performed untargeted metabolomic analysis (using LC-MS/MS) on the plasma samples of the same cohort. Within the 80 plasma samples, we identified a total of polar 278 metabolites. We observed a significant metabolomic shift during severe COVID-19 (Figure 3a, a list of the top 50 dysregulated metabolites is in Supplementary Figure 1). Indeed, in principal component analysis of the full metabolomic dataset, the first component was able to completely distinguish controls (and mild patients) from those with severe disease. Pathway analysis of the COVID-19-dysregulated metabolites showed disruption in tRNA charging, citrulline metabolism, and several other amino acid (AA) metabolic pathways (Figure 3b, the top 10 dysregulated metabolic pathways are shown; Supplementary Table 3 shows the top 50 dysregulated metabolic pathways with FDR<0.05). Importantly, changes in AA metabolism, including citrulline, arginine, methionine, and tryptophan (see Figure 3b), are not only markers for gut dysfunction but also can influence the AA-metabolizing bacterial communities and disrupt the gut-microbiome immune axis.45,46 AA are absorbed and metabolized by enterocytes and gut microbiota. Consumption of AA by the gut microbiome is important for bacterial growth and is involved in the production of key microbiome-related metabolites.46 These metabolites can influence epithelial physiology and be sensed by immune cells to modulate the mucosal immune system.47,48
(a) Principal component analysis (PCA) of the 278 metabolites identified in the plasma of the study cohort. Each symbol represents a study participant. (b) Ingenuity Pathway Analysis (IPA) of the plasma metabolites modulated between the disease states with FDR<0.05. The graph shows the top 10 dysregulated metabolic pathways with FDR<0.05. Percentages beside each pathway represent the ratio of dysregulated metabolites among the total number of metabolites assigned to this particular pathway in IPA. (c) Volcano plots depicting plasma metabolites dysregulated in the moderate group compared to the mild group (left) or the severe group compared to the mild group (right). NS= non-significant. The gut-associated metabolites (from Table 1) are indicated by the larger symbols, and a selected set are identified by name. (d-f) As representative examples, levels of Citrulline are lower (d), levels of succinic acid are higher (e), and the ratio between kynurenine/tryptophan [Kyn/Trp] is higher (f) during severe COVID-19 compared to mild COVID-19 or controls. Kruskal–Wallis test was used for statistical analysis. FDR was calculated using Benjamini-Hochberg method. (g-h) For key metabolites in the tryptophan catabolism pathway, levels of tryptophan are lower (g), and levels of kynurenic acid are higher (h) in deceased COVID-19 hospitalized patients compared to survivors. Nominal P-value was calculated using the Mann–Whitney U test.
Next, we focused on 50 of the metabolites (out of the total of 278) that are known to be associated with the function of the gut and its microbiota (Supplementary Table 4 lists the 50 metabolites and their references). Levels of most of these gut-associated plasma metabolites (35 out of 50) were dysregulated during severe COVID-19 compared to mild disease or controls (Table 1 and Figure 3c). Within this metabolic signature of COVID-19-associated gut dysfunction is citrulline, which is also identified as a top metabolic pathway dysregulated by severe COVID-19 (Figure 3c). Citrulline is an amino acid produced only by enterocytes and an established marker of gut and enterocyte function.25 Its levels are significantly decreased during severe COVID-19 (Figure 3d). Also, within this metabolic signature is succinic acid, a well-established marker of gut microbial dysbiosis, whose levels are increasing during severe COVID-19 (Figure 3e).
Notable changes were also observed in several metabolites involved in the catabolism of the AA tryptophan (Figure 3b, c). Higher levels of tryptophan catabolism, indicated by high levels of kynurenine and low levels of tryptophan (i.e. the [Kyn/Trp] ratio), is an established marker of gut microbial dysbiosis.49,50 Indeed, we observed a higher [Kyn/Trp] ratio in individuals with severe COVID-19 than in those with mild disease or controls (Figure 3f). Furthermore, lower levels of tryptophan and higher levels of kynurenic acid were associated with mortality among hospitalized COVID-19 patients (Figure 3g-h). Together, these data indicate that a metabolic signature associated with severe COVID-19 is compatible with disrupted gut functions and dysregulated gut-microbiome axis.
Plasma metabolomic markers of COVID-19-associated gut dysfunction associate with higher inflammation and immune dysfunction
As noted above, many plasma metabolites are bioactive molecules that can directly impact immunological and inflammatory responses. Therefore, we sought to identify links between the 35 dysregulated gut-associated plasma metabolites (Table 1) and the dysregulated markers of microbial translocation, inflammation, and immune activation (Figure 2a). We observed strong links between levels of the dysregulated gut-associated metabolites and levels of markers of microbial translocation (Figure 4a) as well as levels of inflammation and immune activation (Figure 4b). Notable correlations were observed between lower levels of citrulline and higher IL-6 (Figure 4c), higher levels of succinic acid and higher IL-6 (Figure 4d), and higher [Kyn/Trp] ratio and higher IL-6 (Figure 4e). These data highlight the potential links between disrupted metabolic activities, especially those related to the gut and its microbiota, and systemic inflammation and immune dysfunction during COVID-19.
Correlation heat-maps depicting the correlations between COVID-19-modulated, gut-associated metabolites and (a) markers of tight junction permeability and microbial translocation or (b) markers of inflammation and immune dysfunction. SC rho = coefficient of correlation with COVID-19 severity. Red-colored correlations = positive correlations with FDR<0.05, blue-colored correlations = negative correlations with FDR<0.05, gray-colored correlations = non-significant (FDR>0.05). (c-e) Examples of the correlations between citrulline and IL-6 (c), succinic acid and IL-6 (d), or [Kyn/Trp] ratio and IL-6 (e). Spearman’s rank correlation tests were used for statistical analysis. FDR was calculated using Benjamini-Hochberg method.
Severe COVID-19 is associated with disrupted lipid metabolism
Intermediary metabolites and sulfur-containing amino acids (e.g. methionine, a regulated COVID-19 pathway, Figure 3b) are potent modulators of lipid metabolism. Therefore, we performed lipidomic analysis on the plasma samples of the same cohort. We identified a total of 2015 lipids using untargeted MS. Similar to the plasma metabolome, the plasma lipidome shifted significantly during severe COVID-19 (Figure 5a). These 2015 lipids were divided into 24 lipids classes (Supplementary Table 5); out of these 24 classes, 16 were significantly (FDR<0.05) modulated by severe COVID-19 (11 were downregulated whereas five were upregulated) (Figure 5b). Pathway analysis of this severe-COVID-19-associated lipidomic signature showed that glycerophospholipid and choline metabolism were the most significantly dysregulated pathways (Figure 5c). The gut microbiota is heavily involved in these two interconnected pathways.51 Gut microbial dysbiosis can alter the digestion and absorption of glycerophospholipids, leading to several diseases.51-54 These data provide yet another layer of evidence that severe COVID-19 is associated with systemic dysregulations that are linked to disrupted gut function.
(a) Principal component analysis (PCA) of 2015 lipids identified in the plasma of the study cohort. (b) The 2015 identified lipids were assigned to 24 classes (Supplementary Table 5). Heat-map depicts the 16 lipid classes dysregulated by severe COVID-19 (FDR<0.05). Statistical significance was determined using the Kruskal–Wallis test. FDR was calculated using Benjamini-Hochberg method. SC rho = coefficient of correlation with COVID-19 severity. Heat colors show standardized Z-scores across samples; red indicates upregulation, and blue indicates downregulation. (c) Lipid pathway analysis of the plasma lipids modulated between the disease states with FDR<0.05 was performed using LIPEA (Lipid Pathway Enrichment Analysis; https://lipea.biotec.tu-dresden.de/home). The graph includes all dysregulated pathways with FDR<0.05. Percentages beside each pathway represent the ratio of dysregulated lipids among the total number of lipids assigned to this particular pathway by LIPEA.
Severe COVID-19 is associated with altered plasma glycomes that are linked to inflammation and complement activation
Finally, we examined plasma glycomes. It has been reported that translocation of glycan-degrading enzymes released by several members of the gut microbiome can alter circulating glycomes.32 Within the plasma glycome, glycans on circulating glycoproteins and antibodies (IgGs and IgAs) play essential roles in regulating several immunological responses, including complement activation.31 For example, galactosylated glycans link Dectin-1 to Fcγ receptor IIB (FcγRIIB) on the surface of myeloid cells to prevent inflammation mediated by complement activation.31 A loss of galactose decreases the opportunity to activate this anti-inflammatory checkpoint, thus promoting inflammation and complement activation, including during IBD.32,34-37 Indeed, IgG glycomic alterations associate with IBD disease progression and IBD patients have lower IgG galactosylation compared to healthy controls.33
We applied several glycomic technologies to analyze the plasma glycome (total plasma, isolated IgG, and isolated IgA). First, we used capillary electrophoresis to identify the N-linked glycans of total plasma glycoproteins and isolated plasma IgG (this identified 24 and 22 glycan structures, respectively; their names and structures are in Supplementary Tables 6 and 7). We also used a 45-plex lectin microarray to identify other glycans on total plasma glycoproteins and isolated IgA. The lectin microarray enables sensitive analysis of multiple glycan structures by employing a panel of 45 immobilized lectins (glycan-binding proteins) with known glycan-binding specificity (Supplementary Table 8 lists the 45 lectins and their glycan-binding specificities).55
We first observed a significant (FDR<0.05) glycomic alteration during severe COVID-19 in levels of IgA glycans, plasma N-glycans, plasma total glycans, and IgG glycans (Figure 6a). These changes are exemplified by an apparent loss of the anti-complement activation galactosylated glycans from IgG and total plasma glycoproteins (Figure 6b-c, respectively). When we examined the correlations between the plasma glycome and markers of tight junction permeability/microbial translocation or inflammation/immune activation (Supplementary Figure 2), as expected, we observed significant negative correlations (FDR<0.05) between levels of terminal galactose on IgG or plasma glycoproteins and markers of permeability/ translocation (Figure 6d) or markers of inflammation (Figure 6e). These data highlight the potential links between the disrupted plasma glycome and systemic inflammation during COVID-19.
(a) Heat-map depicting glycans dysregulated by severe COVID-19 (FDR<0.05). Names of the glycan-binding lectins are provided in parentheses. Statistical significance was determined using the Kruskal– Wallis test. FDR was calculated using Benjamini-Hochberg method. SC rho = coefficient of correlation with COVID-19 severity. Heat colors show standardized Z-scores across samples; red indicates upregulation, and blue indicates downregulation. (b-c) Levels of terminal digalactosylated N-glycans in IgG (b) or total plasma glycoproteins (c) are lower during severe COVID-19 compared to mild COVID-19 or controls. Kruskal–Wallis test. FDR was calculated using Benjamini-Hochberg method. (e-d) Correlation heat-maps depicting the correlations between galactosylated N-glycans (rows) and markers of tight junction permeability and microbial translocation (d) or markers of inflammation and immune dysfunction (e). SC rho = coefficient of correlation with COVID-19 severity. Red-colored correlations = positive correlations with FDR<0.05, blue-colored correlations = negative correlations with FDR<0.05, and gray-colored correlations = non-significant. Spearman’s rank correlation tests were used for statistical analysis. FDR was calculated using Benjamini-Hochberg method.
Multivariable logistic models, using cross-validation Lasso technique, selected gut-associated variables whose combination associates with the risk of hospitalization during COVID-19
Our data thus far support the hypothesis that gut dysfunction fuels COVID-19 severity. We sought to examine whether markers of tight junction permeability and microbial translocation (Supplementary Table 2) can distinguish between hospitalized COVID-19 patients (moderate and severe groups combined) and non-hospitalized individuals (mild and controls combined). We applied the machine learning algorithm Lasso (least absolute shrinkage and selection operator) regularization to select markers with the highest ability to distinguish between the two groups. The analysis employed samples with complete data sets (n=79; one sample did not have complete data). Lasso selected zonulin, LBP, and sCD14 as the three markers to be included in a multivariable logistic regression model that distinguishes hospitalized from non-hospitalized individuals with area under the ROC curve (AUC) of 99.23% (Figure 7a; 95% confidence interval: 98.1% -100%). This value was higher than the AUC values obtained from logistic models using each variable individually (Table 2).
(a) The machine learning algorithm, Lasso (least absolute shrinkage and selection operator) regularization, selected three markers (zonulin, LBP, and sCD14) that, when combined, can distinguish hospitalized (n=40; severe and moderate groups combined) from non-hospitalized (n=40; mild and control groups combined) individuals. The receiver operator characteristic (ROC) curve showing an area under the curve (AUC) of 99.23% from the multivariable logistic regression model with the three variables combined. (B) Coefficients from the multivariable logistic model were used to estimate a hospitalization risk score for each individual and then tested for the ability of these scores to accurately classify hospitalized (n=40) from non-hospitalized (n=39; one sample did not have a complete dataset) individuals at an optimal cut-point. The model correctly classified hospitalized (97.5% sensitivity) and non-hospitalized (94.9% specificity), with an overall accuracy of 96.2%. Squares represent individuals the model failed to identify correctly. (c) Logistic regression model using the L-kynurenine/L-tryptophan [Kyn/Trp] ratio, a marker of gut microbiome dysbiosis, is able to distinguish hospitalized from non-hospitalized individuals. ROC curve showing the area under the curve (AUC) is 91.3%.
Next, we used the multivariable logistic model to estimate a risk score of hospitalization for each individual. We then examined the ability of these risk scores to classify hospitalized from non-hospitalized individuals. As shown in Figure 7b, the model correctly classified 97.5% of hospitalized (sensitivity) and 94.9% of non-hospitalized (specificity) individuals, with an overall accuracy of 96.2%. Furthermore, we examined the ability of the L-kynurenine/L-tryptophan [Kyn/Trp] ratio, an established marker of gut microbial dysbiosis described above, to distinguish hospitalized from non-hospitalized individuals. Logistic model showed that [Kyn/Trp] ratio alone can distinguish hospitalized from non-hospitalized with an AUC value of 91.9% (Figure 7c; 95% confidence interval: >85% -98.7%). This analysis further highlights the plausible link between severe COVID-19 and disrupted gut function, orchestrated by an increase in tight junction permeability, microbial translocation, possible microbial dysbiosis, and dysregulated digestion and metabolism.
DISCUSSION
We used a systems biology approach to provide multiple layers of evidence that severe COVID-19 is associated with a disruption in intestinal barrier integrity, higher microbial translocation, and intestinal dysfunction. These data highlight a previously unappreciated factor, disruption in gut barrier integrity, as a force that likely fuels COVID-19 severity. Our results do not imply that microbial dysbiosis and translocation are the primary triggers of severe COVID-19, as the complex clinical syndrome of severe COVID-19 likely embodies multiple pathophysiological pathways. Also, our in vivo analyses do not unequivocally demonstrate a causal relationship between gut dysfunction and COVID-19 severity. However, the robust literature indicating that a disrupted intestinal barrier and microbial dysbiosis and translocation fuel inflammation and disease severity during ARDS11-14 supports our hypothesis and is consistent with our findings.
SARS-CoV-2 infection can affect the gastrointestinal tract (GI) tract and cause GI symptoms.56,57 Recently, it has been suggested that the severity of GI symptoms (mainly vomiting and diarrhea) correlates inversely with COVID-19 severity (for unclear reasons).58 On the other hand, our observations suggest that disruption in gut function and higher microbial translocation correlate positively with COVID-19 severity. These are not necessarily mutually exclusive findings, but rather indicate that the interplay between the gut and SARS-CoV2 infection in modulating disease severity is complex. The potential role of the gut should be further explored, in multiple cohorts and settings, longitudinally during different stages of infection, and using gut biopsies and stool samples.
Our data raise several critical questions, including are there long-term implications of a disrupted gut barrier and intestinal function in survivors of severe COVID-19? In survivors of SARS-CoV-1 infection, long-term health complications (including metabolic dysfunctions) were observed for many years after convalescence.59,60 HIV+ individuals also can suffer complications of gut microbial translocation for years after viral suppression.61-63 The current ‘long-haulers’ after severe COVID-19 may also be on a path towards long term consequences due to persistent, microbial translocation. Thus, understanding the long-term implications of the disrupted gut function that we observed during severe COVID-19 should be a clinical priority. An accompanying priority should be to consider how to modify clinical practice to prevent or reduce gut disruption. Currently, for example, a large number of patients are receiving antibiotic therapy during their COVID-19 treatment.1,5 However, massive use of antibiotics can alter gut microbiota and gut function, leading to higher susceptibility to inflammatory disorders.64 Thus, for any clinical practices that alter the gut, their overall impact on disease course should be carefully considered, especially for older patients and those with metabolic diseases, who are likely to have a lower threshold for intestinal dysfunction.
Our study raises the possibility that some of the markers we identified through our multi-omic approach may be able to predict the risk of disease progression if measured immediately after diagnosis. The plasma is a pool of biological molecules, including polar metabolites, lipids, and glycoproteins, secreted or shed from multiple tissues. Thus, the levels of these molecules likely indicate the overall status of multiple organs, making them excellent candidates for biomarker discovery. Indeed, various plasma metabolites and glycomic features of plasma glycoproteins have been investigated as biomarkers for several diseases, including IBD, diabetes, cardiovascular diseases, and cancer.65-71 In addition, markers of intestinal barrier permeability have been used as predictors of multiple organ dysfunction during critical illness.72 The striking differences we observed in the levels of specific metabolites, lipids, and glycoproteins between COVID-19 disease states suggests they may also have value for risk prediction or therapeutic response. Future longitudinal, controlled studies will be needed to assess these possibilities.
Our study also reveals several potential therapeutic targets for severe COVID-19, including zonulin. Zonulin is the only known physiological modulator of the intestinal tight junctions.73 Microbial dysbiosis and translocation enhance zonulin release, which in turn induces tight junction permeability, leading to more microbial translocation. This microbial translocation triggers inflammation, which promotes further gut leakiness.40,41 Increased intestinal permeability and serum zonulin levels have been observed during many inflammatory diseases, including Crohn’s disease and celiac disease.74,75 Preventing zonulin-mediated increase in intestinal permeability by a zonulin receptor antagonist AT1001 (larazotide acetate) decreased the severity and incidence of several inflammation-associated diseases in pre-clinical and clinical studies.76-82 The strikingly high level of serum zonulin we observed during severe COVID-19, which were associated with inflammation and mortality, raise the question of whether modulators of tight junction permeability (such as with AT-1001) can lessen COVID-19 severity.
Another question is whether there is a genetic predisposition to intestinal barrier permeability and COVID-19 severity. Zonulin is the uncleaved precursor to haptoglobin (HP).73 There are two common alleles for haptoglobin, HP1 and HP2. Individuals with the HP2 allele produce high levels of zonulin, whereas individuals with the HP1 allele do not.41 High-zonulin-producers (HP2 allele) are more likely to develop IBD83,84 and experience morbidity from inflammatory diseases.85,86 The worldwide frequency of each haptoglobin phenotype varies by region.84,87 How the HP genotype and zonulin polymorphisms88 are linked to severe COVID-19 and associated mortality should be the subject of future studies.
A second potential therapeutic target revealed by this work is citrulline. Citrulline levels were reduced in both moderate and severe COVID-19, and the citrulline metabolism and biosynthesis pathways were among the top metabolic pathways disrupted in severe COVID-19. Citrulline is an intermediate in arginine metabolism,89 and a marker of gut and enterocyte function.25,90 Disrupted citrulline metabolism, as we observed during severe COVID-19, has been associated with microbial dysbiosis and dysregulated intestinal function.91 In addition to its role as a biomarker, citrulline has an important role in preserving gut barrier function. In an intestinal obstruction mouse model, pretreatment with a citrulline-rich diet preserved gut barrier integrity.92 Thus, our data, and the work of others from diverse fields, suggest that greater understanding of the interaction between the gut, intestinal microbiota, and amino acid metabolism during COVID-19 might inform pharmaceutical and diet approaches to improve COVID-19 outcomes.
A significant strength of our multi-omics approach is its ability to uncover connections between severe COVID-19 and biomolecules of different classes. The carbohydrate structures (glycans) attached to circulating proteins, including antibodies, and their receptors (lectins) are increasingly being appreciated for their essential roles in a variety of immune functions. Among the glycobiological molecules regulated by severe COVID-19 are galectins (increasing) and galactosylated glycans on circulating glycoproteins (decreasing). Both may point to a potential glycomic contributions to the severity of COVID-19. First, galectins (secreted, GalNAc-binding proteins) have emerged as significant modulators of cytokine expression by immune cells during several diseases, including viral infections.93-95 Importantly, small molecule inhibitors for galectins, especially for Gal-3, can reduce inflammation and cytokine release.96,97 Therefore, galectins represent potential therapeutic targets to reduce cytokine storm during COVID-19.98,99 Second, galactosylated glycans on circulating antibodies link Dectin-1 on phagocytes to FcγRIIB on myeloid cells to prevent the inflammation mediated by complement activation.31 Loss of galactose, as observed during COVID-19, decreases the opportunity to activate this important anti-complement activation checkpoint, thereby promoting inflammation. As using highly-galactosylated immune complexes can prevent inflammation mediated by complement activation,31 these and similar glycomic approaches may represent another therapeutic opportunity to reduce inflammation during COVID-19.
Caveats of our study include the sample size and sampling of blood. As noted above, correcting for potential confounders will require larger cohorts from varying geographic and demographic settings. Independent test sets and samples from non-COVID1-9 hospitalized and ICU-admitted patients will also be needed. In addition, longitudinal analyses will be required to examine the long-term implications of our findings and their potential value as prognostic biomarkers. Analysis of gut biopsies, stool, and bronchial lavage will be needed to determine the precise contributions of the gut-lung axis in COVID-19. Finally, mechanistic studies in vitro and in animal models of SARS-CoV2 infection will be needed to examine the direct versus the indirect impact of the infection on intestinal barrier integrity and function.
In summary, while this study was exploratory in nature, our data strongly suggest for the first time: (1) severe COVID-19 is associated with disrupted intestinal barrier integrity, higher microbial translocation, and gut dysfunction; (2) severe COVID-19 is associated with a dramatic shift in levels of several biologically active molecules, which likely contribute to disease severity by inducing inflammation. Our study is beginning to shed light on the potentially critical role of a previously unappreciated factor, disruption of intestinal barrier integrity, in the pathophysiology of severe COVID-19. By understanding these unappreciated underpinnings of COVID-19, this work may serve to identify biomarkers for risk stratification and build a foundation for developing strategies to prevent or reduce the severity of COVID-19.
METHODS
Study cohort
Analyses were performed using plasma samples from 60 individuals tested positive for SARS-CoV2, and 20 SARS-CoV2 negative controls collected at Rush University Medical Center (RUMC). The 60 SARS-CoV2 positives were selected to represent three diseases states: 20 with mild symptoms (outpatients); 20 with moderate symptoms (inpatients hospitalized on regular wards); and 20 with severe symptoms (inpatients hospitalized in an intensive care unit (ICU)) (Figure 1a). Individuals were selected to have a median age between 52.5 to 58.5 years. The study cohort was also chosen to have a 35 to 60% representation of female gender per disease status group (Supplementary Table 1). Eight participants of the cohort (two from the moderate group and six from the severe group) died from COVID-19 (Supplementary Table 1). All research protocols of the study were approved by the institutional review boards (IRB) at Rush University and The Wistar Institute. All human experimentation was conducted in accordance with the guidelines of the US Department of Health and Human Services and those of the authors’ institutions.
Measurement of plasma markers of tight junction permeability and microbial translocation
Plasma levels of soluble CD14 (sCD14), soluble CD163 (sCD163), LPS Binding Protein (LBP), and FABP2/I-FABP were quantified using DuoSet ELISA kits (R&D Systems; catalog # DY383-05, # DY1607-05, # DY870-05, and # DY3078, respectively). The plasma level of zonulin was measured using an ELISA kit from MyBiosorce (catalog # MBS706368). Levels of occludin were measured by ELISA (Biomatik; catalog # EKC34871). β-glucan detection in plasma was performed using Limulus Amebocyte Lysate (LAL) assay (Glucatell Kit, CapeCod; catalog # GT003). Plasma levels of Reg3A were measured by ELISA (RayBiotech; catalog # ELH-REG3A-1).
Measurement of plasma markers of inflammation and immune activation
Plasma levels of GM-CSF, IFN-β, IFN-γ, IL-10, IL-13, IL-1β, IL-33, IL-4, IL-6, TNF-α, Fractalkine, IL-12p70, IL-2, IL-21, IL-22, IL-23, IP-10, MCP-2, MIP-1α, SDF-1a, IFN-α2a, IL-12/IL-23p40, and IL-15 were determined using customized MSD U-PLEX multiplex assay (Meso Scale Diagnostic catalog# K15067L-2). Plasma levels of C-Reactive Protein (CRP), Galectin-1, Galectin-3, and Galectin-9 were measured using DuoSet ELISA kits (R&D Systems; catalog # DY1707, # DY1152-05, # DY2045, and # DY1154, respectively). Levels of Growth Differentiation Factor-15 (GDF-15) were measured by ELISA using GDF-15 Quantikine ELISA Kit (R&D Systems; catalog # DGD150). Plasma levels of Myeloperoxidase (MPO), d-dimer, and C3a were measured by ELISA (Thermo Fischer; catalog # BMS2038INST, # EHDDIMER, #BMS2089, respectively)
Untargeted measurement of plasma metabolites
Metabolomics analysis was performed as described previously.100 Briefly, polar metabolites were extracted from plasma samples with 80% methanol. A quality control (QC) sample was generated by pooling equal volumes of all samples and was injected periodically during the analysis sequence. LC-MS/MS was performed on a Thermo Scientific Q Exactive HF-X mass spectrometer with HESI II probe and Vanquish Horizon UHPLC system. Hydrophilic interaction liquid chromatography was performed at 0.2 ml/min on a ZIC-pHILIC column (2.1 mm × 150 mm, EMD Millipore) at 45°C. Solvent A was 20 mM ammonium carbonate, 0.1% ammonium hydroxide, pH 9.2, and solvent B was acetonitrile. The gradient was 85% B for 2 min, 85% B to 20% B over 15 min, 20% B to 85% B over 0.1 min, and 85% B for 8.9 min. All samples were analyzed by full MS with polarity switching. The QC sample was also analyzed by data-dependent MS/MS with separate runs for positive and negative ion modes. Full MS scans were acquired at 120,000 resolution with a scan range of 65-975 m/z. Data-dependent MS/MS scans were acquired for the top 10 highest intensity ions at 15,000 resolution with an isolation width of 1.0 m/z and stepped normalized collision energy of 20-40-60. Data analysis was performed using Compound Discoverer 3.1 (ThermoFisher Scientific). Metabolites were identified by accurate mass and retention time using an in-house database generated from pure standards or by MS2 spectra using the mzCloud spectral database (mzCloud.org) and selecting the best matches with scores of 50 or greater. Metabolite quantification used peak areas from full MS runs and were corrected based on the periodic QC runs.
Untargeted measurement of plasma lipids
Lipidomics analysis was performed as described previously.101 Briefly, plasma samples were spiked with EquiSplash mix (Avanti Polar Lipids). Lipids were extracted with 2:1:1 chloroform: methanol: 0.8% sodium chloride. Samples were resuspended in 1:9 chloroform: methanol after drying the organic phase under nitrogen. LC-MS runs were performed on a Thermo Scientific Q Exactive HF-X mass spectrometer with HESI II probe and Vanquish Horizon UHPLC system. Reversed-phase liquid chromatography was performed at 0.35 ml/min on an Accucore C30 column (2.1 mm x 150 mm, ThermoFisher Scientific) at 50°C. Solvent A was 50:50 acetonitrile: water, and solvent B was 88:10:2 isopropanol: acetonitrile: water solvents. Both solvents contained 5 mM ammonium formate and 0.1% formic acid. The gradient was 0% B for 3 min, 0% to 60% B over 7 min, 60% to 85% B over 10 min, 85% to 100% B over 10 min, 100% B for 5 min, 100% to 0% B over 0.01 min, and 0% B for 4.99 min. All samples were analyzed by data-dependent MS/MS with separate runs for positive and negative ion modes. Full MS scans were acquired at 120,000 resolution with a scan range of 300-2,000 m/z in positive mode and 250-2,000 m/z in negative mode. Data-dependent MS/MS scans were acquired for the top 20 ions at 15,000 resolution with an isolation width of 0.4 m/z. Stepped normalized collision energy of 20-30 was used for positive ion mode and 20-30-40 was used for negative ion mode. Data analysis was performed using LipidSearch 4.2 (ThermoFisher Scientific). Lipid species were identified from MS/MS spectra using an in-silico fragmentation database and were filtered by expected adducts and identification quality. Lipid species quantification used peak areas and was corrected based on EquiSplash deuterated lipids for represented classes.
IgG isolation
Bulk IgG was purified from plasma using Pierce Protein G Spin Plate (Thermo Fisher; catalog # 45204). IgG purity was confirmed by SDS gel.
IgA isolation
Bulk IgA was purified from IgG depleted plasma using CaptureSelect IgA Affinity Matrix (Thermo Fisher; catalog # 194288010). IgA was concentrated using Amicon® filters (Milipore catalogue #UFC805024) and purity was confirmed by SDS gel.
N-glycan analysis using capillary electrophoresis
For both plasma and bulk IgG, N-glycans were released using peptide-N-glycosidase F (PNGase F) and labeled with 8-aminopyrene-1,3,6-trisulfonic acid (APTS) using the GlycanAssure APTS Kit (Thermo Fisher; catalog # A33952), following the manufacturer’s protocol. Labeled N-glycans were analyzed using the 3500 Genetic Analyzer capillary electrophoresis system. Total plasma N-glycans were separated into 24 peaks (Supplementary Table 6) and IgG N-glycans into 22 peaks (Supplementary Table 7). The relative abundance of N-glycan structures was quantified by calculating the area under the curve of each glycan structure divided by the total glycans using the Applied Biosystems GlycanAssure Data Analysis Software Version 2.0.
Glycan analysis using lectin array
To profile plasma total and IgA glycomes, we used the lectin microarray as it enables analysis of multiple glycan structures. The lectin microarray employs a panel of 45 immobilized lectins with known glycan-binding specificity (lectins and their glycan-binding specificity are detailed in Supplementary Table 8). Plasma proteins or isolated IgA were labeled with Cy3 and hybridized to the lectin microarray. The resulting chips were scanned for fluorescence intensity on each lectin-coated spot using an evanescent-field fluorescence scanner GlycoStation Reader (GlycoTechnica Ltd.), and data were normalized using the global normalization method.
Statistical analysis
Kruskal-Wallis and Mann–Whitney U tests were used for unpaired comparisons. Spearman’s rank correlations were used for bivariate correlation analyses. Severity correlation coefficient (SC rho) tested correlation versus patient groups with the severity groups quantified as follows: control=1, mild=2, moderate=3, severe=4. FDR for each type of comparison was calculated using the Benjamini–Hochberg approach within each data subset separately and FDR<5% was used as a significance threshold. Principal Component Analysis was performed on log2-transcformed z-scored data. Pathway enrichment analyses were done on features that passed significant SC rho at FDR<5%. Enrichments for the metabolites were tested using QIAGEN’s Ingenuity® Pathway Analysis software (IPA®, QIAGEN Redwood City, www.qiagen.com/ingenuity) using the “Canonical Pathway” option. Enrichments for the lipids were done using LIPEA (https://lipea.biotec.tu-dresden.de/home) with default parameters. To explore biomarkers that could be distinguish clinical outcome (hospitalization vs. non-hospitalization), specific set of microbial translocation variables were identified among those with FDR<0.05. Variables for the multivariable logistic model were selected from the identified specific set of biomarkers using Lasso technique with the cross-validation (CV) selection option by separating data in 5-fold. Due to the exploratory nature of this study with moderate sample size, variable selection was determined using 100 independent rounds runs of CV Lasso with minimum tuning parameter lambda. The markers that were selected 80 or more times from 100 runs were used as final set of variables in our model. The ability of the final logistic model was assessed by AUC with 95% confidence interval. Statistical analyses were performed in R 4.0.2 and Prism 7.0 (GraphPad).
Data Availability
All data are available in the manuscript or upon request.
SUPPLEMENTARY MATERIALS
Supplementary Figure 1: Top 50 metabolites dysregulated by COVID-19. Heat-maps depicting the top 25 metabolites induced and the top 25 metabolites reduced by COVID-19 (FDR<0.05). Statistical significance was determined using the Kruskal–Wallis test. FDR was calculated using Benjamini-Hochberg method. SC rho = coefficient of correlation with COVID-19 severity. Heat colors show standardized Z-scores across samples; red indicates upregulation, and blue indicates downregulation.
Supplementary Figure 2: The COVID-19-associated plasma glycomic signatures are associated with microbial translocation and systemic inflammation. Correlation heat-maps depicting the correlations between COVID-19 modulated plasma glycans and (a) markers of tight junction permeability and microbial translocation or (b) markers of inflammation and immune dysfunction. SC rho = coefficient of correlation with COVID-19 severity. Red-colored correlations = positive correlations with FDR<0.05, blue-colored correlations = negative correlations with FDR<0.05, and gray-colored correlations = non-significant correlations.
Supplementary Table 1. Demographic and clinical characteristics of the study cohort.
Supplementary Table 2: A list of plasma markers measured in this study.
Supplementary Table 3: Top 50 metabolic pathways disrupted by severe COVID-19.
Supplementary Table 4: List of the gut-associated and gut microbiota-associated metabolites detected in our study using untargeted LC-MS/MS (50 of the 278 metabolites identified in plasma).
Supplementary Table 5. The two thousand fifteen lipids identified in this study were assigned to 24 lipid classes.
Supplementary Table 6. The structures and names of N-glycans identified in plasma by capillary electrophoresis. These glycan structures can be grouped into 15 groups: bisecting GlcNAc (B group), sialic acid (non-siaylated (S0), mono-sialylated (S1), di-sialylated (S2), tri-sialylatedand (S3), tetra-sialylated (S4), and total sialylated (ST)), galactose (agalactosylated (G0), mono-galactosylated (G1), di-galactosylated (G2), and total galactosylated (GT)), core fucose (FC group), branched fucose (FB group), high branched (HB group), and low branch (LB group).
Supplementary Table 7. The structures and names of N-glycans identified in IgG by capillary electrophoresis. These glycan structures were grouped into 9 groups, depending on the presence or absence of four key monosaccharides: bisecting GlcNAc (B group), sialic acid (mon-sialylated (S1), di-sialylated (S2), and total siaylated (ST)), terminal galactose (agalactosylated (G0), mono-galactosylated (G1), di-galactosylated (G2), and total galactose (GT)), and fucose (F group).
Supplementary Table 8. Lectins used in the 45-plex lectin microarray and their glycan-binding specificity.
AUTHOR CONTRIBUTIONS
M.A-M, A.L, and A.Ke conceived and designed the study. L.B.G. carried out the majority of experiments. H.D., X.Y, H.W, Q.L, and A.Ko performed all bioinformatics and biostatistical analyses. M.D. ran the lectin array experiments. C.S.P reviewed and selected gut-associated metabolites. N.F.Z, R.A.B, M.W.S, CBF, A.Ke, and A.L selected study participants and interpreted data. A.R.G, N.G, and H.T performed metabolic and lipidomic analyses. L.B.G. and M.A-M wrote the manuscript, and all authors edited it.
COMPETING INTERESTS STATEMENT
The authors have no competing interests.
ACKNOWLEDGMENTS
This study is supported by a supplement to R01 DK123733 (R01 DK123733-01S1) for M.A-M, A.L, and A.Ke and R24 AA026801-02S1 for A.Ke. M.A-M is also supported by The Foundation for AIDS Research (amfAR) impact grant # 109840-65-RGR, NIH grants (R01 AG062383, R01NS117458, R21 AI143385, R21 AI129636, and R21 NS106970), and the Penn Center for AIDS Research (P30 AI 045008). The Wistar Proteomics and Metabolomics Shared Resource is supported in part by NIH Cancer Center Support Grant CA010815. The Thermo Q-Exactive HF-X mass spectrometer was purchased with NIH grant S10 OD023586. We would like to thank Rachel E. Locke, Ph.D. for providing comments.
REFERENCES
- 1.↵
- 2.
- 3.
- 4.
- 5.↵
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.↵
- 12.
- 13.
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.
- 23.
- 24.↵
- 25.↵
- 26.
- 27.
- 28.↵
- 29.↵
- 30.↵
- 31.↵
- 32.↵
- 33.↵
- 34.
- 35.
- 36.
- 37.↵
- 38.↵
- 39.↵
- 40.↵
- 41.↵
- 42.↵
- 43.↵
- 44.↵
- 45.↵
- 46.↵
- 47.↵
- 48.↵
- 49.↵
- 50.↵
- 51.↵
- 52.
- 53.
- 54.↵
- 55.↵
- 56.↵
- 57.↵
- 58.↵
- 59.↵
- 60.↵
- 61.↵
- 62.
- 63.↵
- 64.↵
- 65.↵
- 66.
- 67.
- 68.
- 69.
- 70.
- 71.↵
- 72.↵
- 73.↵
- 74.↵
- 75.↵
- 76.↵
- 77.
- 78.
- 79.
- 80.
- 81.
- 82.↵
- 83.↵
- 84.↵
- 85.↵
- 86.↵
- 87.↵
- 88.↵
- 89.↵
- 90.↵
- 91.↵
- 92.↵
- 93.↵
- 94.
- 95.↵
- 96.↵
- 97.↵
- 98.↵
- 99.↵
- 100.↵
- 101.↵
- 102.
- 103.
- 104.
- 105.
- 106.
- 107.
- 108.
- 109.
- 110.
- 111.
- 112.
- 113.
- 114.
- 115.
- 116.
- 117.
- 118.
- 119.
- 120.
- 121.
- 122.
- 123.
- 124.
- 125.
- 126.
- 127.
- 128.
- 129.
- 130.
- 131.
- 132.
- 133.
- 134.