Abstract
Colorectal cancer is a common condition with an uncommon burden of disease, heterogeneity in manifestation, and no definitive treatment in the advanced stages. Against this backdrop, renewed efforts to unravel the genetic drivers of colorectal cancer progression are paramount. Early-stage detection contributes to the success of cancer therapy and increases the likelihood of a favorable prognosis. Here, we have executed a comprehensive computational workflow aimed at uncovering the discrete stagewise genomic drivers of colorectal cancer progression. Using the TCGA COADREAD expression data and clinical metadata, we constructed stage-specific linear models as well as contrast models to identify stage-salient differentially expressed genes. Stage-salient differentially expressed genes with a significant monotone trend of expression across the stages were identified as progression-significant biomarkers. Among the biomarkers identified are: CRLF1, CALB2, STAC2, UCHL1, KCNG1 (stage-I salient), KLHL34, LPHN3, GREM2, ADCY5, PLAC2, DMRT3 (stage-II salient), PIGR, HABP2, SLC26A9 (stage-III salient), GABRD, DKK1, DLX3, CST6, HOTAIR (stage-IV salient), and CDH3, KRT80, AADACL2, OTOP2, FAM135B, HSP90AB1 (top linear model genes). In particular the study yielded 31 genes that are progression-significant such as ESM1, DKK1, SPDYC, IGFBP1, BIRC7, NKD1, CXCL13, VGLL1, PLAC1, SPERT, UPK2, and interestingly three members of the LY6G6 family. Significant monotonic linear model genes included HIGD1A, ACADS, PEX26, and SPIB. The stage-salient genes were benchmarked using normals-augmented dataset, and cross-referenced with existing knowledge. In addition, the signature of a multicellular immuno-cyte community specific to colorectal cancer relative to normal tissue was identified. The candidate biomarkers were used to construct the feature space for learning an optimal model for the digital screening of early-stage colorectal cancers. A feature space of just seven biomarkers, namely ESM1, DHRS7C, OTOP3, AADACL2, LPHN3, GABRD, and LPAR1, was sufficient to optimize a RandomForest model that achieved >98% balanced accuracy (and performant recall) on blind validation with external datasets. Survival analysis yielded a panel of three stage-IV salient genes, namely HOTAIR, GABRD, and DKK1, for the design of an optimal multivariate model for patient risk stratification. Integrating the above results, we have developed COADREADx, a web-server for assisting the screening and prognosis of colorectal cancers. COADREADx has been deployed at: https://apalanialab.shinyapps.io/coadreadx/ for academic research and further refinement.
Introduction
Colorectal adenocarcinoma (COADREAD), or colorectal cancer, is the third most commonly diagnosed cancer in males and the second in females, with an estimated 1.9 million cases and 930 000 deaths occurring in 2020 (compared to 1.4 million cases and 693,000 deaths in 2012) [1]. There are numerous lifestyle and environmental drivers of colorectal cancer in addition to family history, making the bulk of its incidence sporadic [2]. The main lifestyle and environmental influences include a lack of balanced diet [3], physical inactivity, obesity [4], consumption of alcohol and tobacco [5], etc. Familial forms of colorectal cancer include familial adenomatous polyposis (FAP) and hereditary nonpolyposis colorectal cancer (HNPCC), also called Lynch syndrome. Genetic susceptibility in FAP is associated with mutations in the APC tumor suppressor gene (TSG) [6], while HNPCC is associated with mutations in the genes MSH2 and MLH1 involved in the DNA repair pathway [2]. Since survival rates in colorectal cancer plummet with late-stage of presentation, effective surveillance via access to screening models is necessary. Early-stage diagnosis of colorectal cancer is essential to secure an advantageous prognosis, which could help in clinical management of the disease.
The Cancer Genome Atlas (TCGA) research network has found mutational and integrative signatures in the multidimensional COADREAD dataset [7], but so far our knowledge with respect to the stage-wise progression of colorectal cancer has been incomplete and inadequate. Given that the AJCC staging of colorectal cancer is based on histopathology (viz. the TNM staging) [8], we studied the evidence for a molecular basis of cancer progression in discrete stages, and developed data-driven workflows for discerning the molecular signatures of colorectal cancer through RNA-Seq transcriptomics. We extended the protocol introduced in Sarathi and Palaniappan [9], and identified stage-salient biomarkers. In addition, a new class of biomarkers called progression-significant DEGs, which are genes with a significant monotone trend of differential expression, were also identified. It is noted that the early-stage (i.e, stage-I and stage-II salient) biomarkers could be useful in development of diagnostics and prognostic models, whereas progression-significant biomarkers could pinpoint potential therapeutic targets to halt or reverse the course of cancer (before it does metastasize to a point of no return). A network analysis grounds the findings in a larger context, lending more evidence for the molecular origins of stage-wise discrete cancer progression. It is known that gene expression profiles of certain markers define cell-type identity [10], and even tissue microenvironment [11], it is reasonable to suppose that a community structure of cell-types drives colorectal cancer progression. Molecular gene signatures characterize the cell composition of the tumor, and it could be argued that the tumor progression through stages is in part or whole determined by the complex and collective changes in gene expression. Based on the above results, we have developed models for the early-stage screening as well as risk stratification of colorectal cancer.
These models are bundled into COADREADx, a pilot tool for the digital diagnostic and prognostic screening of colorectal cancers. A user-friendly interface to COADREADx is available at: https://apalanialab.shinyapps.io/coadreadx/ for academic use. All original datasets used in the study were obtained from the public-domain, and all the intermediate results generated from the study are available as Supplementary Information (doi: 10.6084/m9.figshare.20489211.v4).
Materials and methods
The workflow is summarized in Fig 1 and discussed in detail below. The identification of stage-salient biomarkers follows the computational protocol developed earlier in our lab [9].
Data preprocessing
Normalized and log2-transformed Illumina HiSeq RNA-Seq gene expression data for Colorectal Adenocarcinoma (COADREAD) processed by the RSEM pipeline [12] were obtained from TCGA via the firebrowse.org portal [13]. The patient barcode (uuid) of each sample encoded in the variable called ‘Hybridization REF’ was parsed and used to annotate the controls and cancer samples. To annotate the stage information of the cancer samples, we obtained the corresponding clinical dataset from firebrowse.org and merged the clinical data with the expression data by matching the “Hybridization REF” in the expression data with the aliquot barcode identifier in the clinical data. The cancer staging is encoded in the attribute “pathologic_stage” of the clinical data. The sub-stages (A,B,C) were collapsed into the parent stage, resulting in four stages of interest (I, II, III, IV). We retained a handful of clinical variables related to demographic features, namely age, sex, height, weight, and vital status. Using this merged dataset, we filtered out genes that showed little change in expression across all samples (defined as σ < 1). We also removed cancer samples that were missing stage annotation (value ‘NA’ in the “pathologic stage”) from our analysis. The data pre-processing was done with R (www.r-project.org) and the final data set was processed through voom in limma to prepare for linear modeling [14].
Linear modelling
Linear modeling of expression across cancer stages relative to the baseline expression (i.e, in normal tissue controls) was performed for each gene using the R limma package [15]. The following linear model was fit for each gene’s expression based on the design matrix shown in Fig 2A: where the independent variables are indicator variables of the sample’s stage, the intercept α is the baseline expression estimated from the controls, and βi are the estimated stagewise log fold-change (lfc) coefficients relative to controls. The linear model was subjected to empirical Bayes adjustment to obtain moderated t-statistics [16]. To account for multiple hypothesis testing and the false discovery rate, the p-values of the F-statistic of the linear fit were adjusted using the method of Hochberg and Benjamini [17]. The linear trends across cancer stages for the top significant genes were visualized using boxplots to ascertain the regulation status of the gene relative to the control.
Pairwise contrasts
To perform contrasts, a slightly modified design matrix shown in Fig 2B was used, which would give rise to the following linear model of expression for each gene: where the controls themselves constitute one of the indicator variables, and the βi are all coefficients estimated only from the corresponding samples. Our first contrast of interest, between each stage and the control, was achieved using the contrast matrix shown in Table 1. Four contrasts were obtained, one for each stage vs control. A threshold of |lfc| > 2 was applied to each contrast to identify genes differentially expressed with respect to the control. Genes could be differentially expressed in any combination of the stages. In the first pass, we identified genes whose |lfc| > 2 for any stage. For the genes that passed, we identified the stage that showed the highest |lfc| for each gene and assigned the gene as specific to that stage for the rest of our analysis.
Significance analysis
We applied four-pronged criteria to establish the salience of the stage-specific differentially expressed genes.
Adj. p-value of the contrast with respect to the control < 0.001
(ii)-(iv) P-value of the contrast with respect to other stages < 0.05. Six such contrasts are possible.
To obtain the above p-values (ii) - (iv), we used the contrast matrix shown in Table 2, which was supplied as an argument to the contrastsFit function in limma.
To deal with any sparsity of progression-significant genes salient to any stage,we defined the “pval_pdt” of a given gene in a certain stage as the product of the p_values of its expression contrast in that stage vs each of the other stages (eg, pval_pdt of gene x in stage 1 is (pval(gene x in st1 vs st2))*(pval(gene x in st1 vs st3))*(pval(gene x in st1 vs st4)).
Monotonic Expression
The linear model in eqn. (1) would not be sufficient to identify genes with an monotonic, trend of expression in sync with disease progression, which could uncover stage-agnostic expression of progression-significant driver genes. Towards this end, we used a model of gene expression where the cancer stage was treated as a numeric variable: where X takes a value in [0,1,2,3,4] corresponding to the sample stage: [control, I, II, III, IV], respectively. It was noted the mean gene expression could show the following patterns of monotonic expression across cancer stages:
monotonic upregulation, where mean expression follows: control < I < II < III < IV.
monotonic downregulation, where mean expression follows: control > I > II > III > IV.
The sets of genes conforming to either (i) or (ii) were identified to yield monotonically upregulated and monotonically downregulated genes. These two sets were merged, and the final set of genes was evaluated using the adj. p-values from the model given by eqn. (3) to yield genes with significant monotonic patterns of expression.
Models for cancer screening and prognosis
(i) Validation of biomarkers with normals-augmented dataset
To study the reliability of findings when a reasonable number of controls are used, we augmented the TCGA cohort with the COADREAD dataset from RNAseqDB [18] that couples TCGA data with 339 normals from the Genotype-Tissue Expression (GTEx) database [19]. The consolidated dataset was subjected to the same biomarker protocol to identify stage-salient genes, and the results compared with those obtained with the TCGA dataset.
(ii) Development of diagnostic model
The different classes of biomarkers discussed above, including stage-salient genes and monotonically expressed genes, could be used as the feature space to train machine learning (ML) algorithms to solve the binary classification problem of cancer v/s normal samples [20]. Towards this, we split the TCGA dataset in the ratio 0.8:0.2 stratified on the outcome class (‘cancer’ or ‘normal’), and extracted the features of interest. To reduce the dimensionality of the feature space, feature selection techniques such as Boruta [21] and recursive feature elimination [22] were applied to the train dataset and a consensus reduced feature space was obtained.
Different ML algorithms were trained on this feature space and hyperparameters optimized by cross-validation. The performance of the ML algorithms was evaluated on the holdout testset to determine the best ML model. The best-performing ML model was then validated on external out-of-domain cohorts.
(iii) Development of Prognostic model
To study the prognostic significance of the identified stage-salient genes, we used the patient ‘OS’ (‘Overall Survival’) attribute in the clinical metadata of the TCGA cohort. Survival analysis was performed according to the protocol outlined in Muthamilselvan and Palaniappan [23].
Univariate Cox regression analysis of the top stage-salient genes was executed to screen the prognostically significant ones, using the R survival library [24]. Genes with p-value < 0.05 were regarded as candidate genes for building a multivariate Cox regression model. This was done using backward variable selection based on the model’s Akaike Information Criterion (AIC) metric [25]. The procedure yielded an optimal prognostic signature of size n, given by the following equation: where the βi are the coefficients for the expression of the ith gene. The median risk score from the above distribution was used to classify TCGA COADREAD patients into high-risk and low-risk groups, as implemented in R survminer library [26]. Kaplan–Meier analysis was then performed to assess significance in survival rate variations between the high-risk and low-risk groups, and thereby qualify the biomarker signature.
Benchmarking
Principal component analysis (PCA) was performed using prcomp in R. We used the rand function to choose 100 random genes. In order to visualize significant outlier genes with a large effect size, volcano plots were obtained by plotting the (-log10)-transformed p-value vs. the log fold-change of gene expression. Heat maps of significant stage-salient differentially expressed genes were visualized using R heatmap and clustered using hclust. Novelty of the identified stage-salient genes was ascertained by screening against curated databases, including the Cancer Gene Census (CGC; cancer.sanger.ac.uk) [27], Network of Cancer Genes NCG7.0 [28], and the Clinical Trials Registry (www.clinicaltrials.gov). STRINGdb was used to translate the findings into network-level insights [29]. To perform immuno-cyte infiltration analysis, we used Cibersort and estimated the proportion of tumor-infiltrating immune cells in TCGA COADREAD samples based on gene expression signatures [10, 30]. Cibersort’s inbuilt LM22 signature estimated the proportion of 22 standard immune cell types; setting the number of permutations to 100 allowed the calculation of sample-wise statistical significance with respect to the estimated values. The immuno-cyte patterns of significant samples were analyzed to provide a snapshot of immune ecotypes at play in significant tumor and normal samples, which would increase our basic understanding of colorectal cancer pathologies and advance rational therapies. The cell-type correlation matrix computed from the proportions of cell-types across significant samples was used to identify substantial co-occurrence patterns. The relative abundance of immunocytes between tumor and normal samples was compared to pinpoint significant differentially elevated or depressed tumor-infiltrating immune cells.
Results
The gene expression matrix from TCGA consisted of 20,502 genes x 428 samples. Upon data pre-processing, the gene expression matrix consisted of 18,212 genes x 409 samples, with an additional vector denoting the sample stage. This dataset is made available as Supplementary File S1. Table 3 shows the distribution of TCGA samples with the corresponding AJCC staging. Table 4 shows the summary of demographic characteristics, where it is seen that the average age was ∼ 65 years, and average BMI was ∼ 29, hinting at the etiological roles of ageing and obesity.
Numeric attributes presented as mean ± standard deviation. Nominal attributes (gender and vital status) presented as counts (number of samples). Body mass index (BMI) was calculated only for those instances with both height and weight data.
After preprocessing with voom in limma, [14], the dataset yielded 9433 significant genes (adj. P < 1E-5) in the linear modeling, suggesting the existence of a linear trend in their expression across cancer stages. Such an observation could be explained by cancer hallmarks that typically worsen with progression, for e.g., genome-wide instabilitya cancer hallmark, [31]. Some top-ranked upregulated genes from the linear modeling included CDH3, KRT80, ETV4 and ESM1. CDH13 was notably a top upregulated gene obtained from the linear modeling of hepatocellular carcinoma (only after GABRD and PLVAP) in an earlier analysis [9]; these observations point to a consistent role for members of the cadherin gene family in cancer progression in gastrointestinal cancers. The top downregulated genes included OTOP2, OTOP3, AADACL2 and DHRS7C. Table 5 shows the log-fold changes of the top ten genes in with respect to normal samples,and Boxplots of the expression of the top 9 genes indicated a progressive net increase in expression across cancer stages relative to control for up-regulated genes, while repressed expression across cancer stages relative to control was the hallmark of downregulated genes (Fig 3). A constant trend of regulation across stages underscores the stage-specific basis of cancer progression. It is noted that the linear trend identified needs to be validated with a model for monotonic expression (see Methods), and some stage-specific genes might exhibit maximal differential expression in stages other than stage 4 (Fig 4).
A mixture of both upregulated and downregulated genes was obtained, shown separately here.
The samples were visualized using a PCA of the top 100 genes from the linear model (Fig 5A). Separate and distinct clusters of the controls and cancer samples suggested that considerable changes in gene expression in cancer samples. Hence linear modeling yields cancer-specific genes (Supplementary File S2). In contrast, the PCA plot of randomly sampled 100 genes (Fig 5B) failed to distinguish the cancer and control samples, highlighting the significance of linear models in the analysis of cancers.
Differences in gene expression constitute the basis of cell-type identities, and it may not be surprising that gene expression differences drive cancer progression through the AJCC stages. In the first pass, we eliminated 15,970 genes with |lfc|<2 in all stages (Table 1). We binned the remaining genes into different partitions, to obtain stage-specific genes of varying sizes (Fig 6). To establish salience, we applied the second contrast (Table 3) and checked for filter criteria (ii) - (iv) stated in the Methods section. Genes that passed all filters were identified as stage-salient DEGs. This process yielded 71 stage-I salient, 2 stage-II salient, 0 stage-III salient and 59 stage-IV salient genes (Supplementary File S3).
Considering the sparsity of genes passing the filters for stages 2 and 3, we applied the pval_pdt, described in the Methods section, and extracted the top 10 genes for each stage. For stages 1 and 4, all these top 10 genes figured in the 71 and 59 genes that had been identified as stage-salient DEGs, respectively. For stage 2, we took the 2 genes that passed the filtering and appended genes with the lowest pval_pdt to obtain 10 genes. For stage 3, we used the 17 genes with pval_pdt < 0.125E-3. The top 10 genes from each stage are shown in Table 6, and the entire set of 157 stage-salient DEGs are presented in Supplementary File S3. It is significant that GABRD emerges as a stage-IV salient gene in COADREAD, reinforcing its identification as a stage-IV salient gene in hepatocellular carcinoma [9], and suggesting a driver role in the metastasis of gastrointestinal cancers more generally.
Visualizing the lfc expression of stage-salient genes revealed systematic progressive expression across stages (Fig 7). The heatmap was clustered using stage-wise expression differences w.r.to controls and showed an early-stage (stages 1 & 2) vs late-stage (stages 3 & 4) separation, arguing for the role of progression-significant genes in driving colorectal cancer. Visualizing the clustering of these 40 genes algorithm (Fig 8), we observed that a lot of the stage 4 genes are proto-oncogenes, steadily over-expressed in the cancer phenotype unto metastasis, whereas most of the early-stage (stages 1 and 2) genes are tumor suppressor genes, which are differentially down-regulated in the cancer phenotype. Even though these observations are selective, it is tempting to visualize the implications for the progression pathway of colorectal cancer – initially disabling the damage-control mechanisms innate to the cell and then progressively spiraling out of control.
The results of the numeric model (eqn.3) sorted by significance are presented in Supplementary File S4. The monotonic analysis yielded 1944 monotonically expressed genes (MEGs; 1389 upregulated and 555 downregulated). These are factors with a constant expression trend agnostic of stage. Applying an adj.p-value cutoff <0.05 yielded 1800 significant MEGs (noted in Supplementary File S5). Examining the overlap of these significant MEGs with stage-salient DEGs yielded 31 progression-significant driver genes (Table 7; expression visualized in Supplementary File S6). As expected, most of these biomarkers (27) are stage-4 salient DEGs, and most of them (27) are also consistently upregulated, signifying unchecked cellular damage progressing to metastasis. Significant MEGs that are also significant (adj.p-val < 1E-5) in the linear and numeric models (1186 and 997 genes, respectively) are presented in Supplementary File S7. Some of the top 200 genes from the linear model (by adj. p-value) are also significant MEGs; these 18 genes can be found in Supplementary File S8. The intersection between the top 200 genes from the numeric model and the significant monotonically expressed genes yielded 39 genes (presented in Supplementary File S9). A total of 36 genes were found common to the top 200 of both the linear and the numeric (ordinal) models (Supplementary File S10). Three stage-salient DEGs figured in the top 200 genes from the numeric model, namely CES3, LPHN3, and WSCD1. Two of the top 200 genes of the linear model were also stage-salient MEGs, namely GABRD and ESM1.
31 genes sorted by the direction of fold-change (up- or downregulation) and corrected significance from the numeric model are shown. Only four genes in this group are monotonically downregulated, namely PIGR, ADH6, ATOH1, and CXCL13, while all the rest are potential proto-oncogene MEGs. It is seen that there are four stage-III salient DEGs (PIGR, DSG3, C2orf48, BIRC70) while all the rest are stage-IV salient DEGs.
Normals-augmented validation
To examine any negative results with the inclusion of more controls in teasing out stage-specific markers, we augmented the dataset using RNAseqDB, which added 339 normal colorectal samples. We noted that the RNAseqDB preprocessing protocol eliminated non-coding transcripts from consideration, ignoring possible expression salience of non-coding RNA biomarkers like HOTAIR. Application of our whole protocol to this controls-augmented dataset yielded a linear model, 1925 stage-specific DEGs (755 stage-I, 418 stage-II, 163 stage-III and 589 stage-IV), and 105 stage-salient markers (40 stage-I, 6 stage-II, 2 stage-III and 57 stage-IV). These are presented in Supplementary File S11. We found a substantial consensus of stage-salient genes between the two datasets, with 70 biomarkers in common (Table 8; highlighted in Supplementary File S11). Notably six of the top stage-I salient genes and nine of the top stage-IV salient genes were identified as salient to the respective stages with the normals-augmented dataset as well, providing robust validation for these biomarkers.
In addition, we identified a colonic cancer dataset with stage-annotation from the Gene Expression Omnibus (GEO) database [32], namely GSE39582, provided by the Carte d’identité des tumeurs, Ligue Nationale contre le Cancer, France [33]. The dataset had a large number of stage-II (271) and stage-III samples (210), relative to stage-I (38) and stage-IV (60) samples. However, only two normal samples were available, so the dataset was augmented with 308 normal colonic tissue samples from the GTEx. The augmented dataset was subjected to batch correction using ComBat [34], and antilog2 was taken to obtain the necessary counts for input to voom and the protocol described in the Methods was applied. The results are presented in Supplementary File S12. Five stage-IV salient genes, namely CYP24A1, FGF19, NKD1, COL9A3, and EDNRA are common to both the analyses. In addition, six stage-I salient genes, namely CPXM2, NPR3, PALM, PRDM6, TAGLN, and TPM2 are identified as stage-IV salient here. However the concordance between the markers from the reference TCGA dataset and GSE39582 is not extensive, and merits discussion. Foremost, GSE39582 is limited to colon cancer samples, which might differ in some features from rectal cancers, thereby missing some variation that is captured in the TCGA COADREAD dataset. Second, we would like to note that out-of-domain cohorts might be sensitive to distribution shifts in gene expression, which require measurement calibration with an adequate number of normals from the same (new) cohort. Since there were few normal samples in the original GSE39582 dataset, this might significantly skew the extension of gene signatures established with the reference TCGA cohort. The addition of 308 normal colonic samples available in the GTEx does not mitigate this issue, since (i) these are from an entirely different cohort, and (ii) normal rectal tissue samples remain unaccounted for. In addition, the applicability of candidate biomarker signatures to new cohorts might be bounded by bioinformatic problems pertaining to data curation and processing. The contrarian findings prompted us to seek robust validation of the models developed below.
The pval_pdt measure was applied to identify the top ten stage-2 salient and stage-3 salient genes. A substantial stage-wise consensus could be observed. The intersection of the top-10 stage-salient genes in each dataset is shown as ‘Top-10 overlap.’
Development of a diagnostic aid for colorectal cancer screening
We combined the 157 stage-salient genes, top ten genes from linear modeling, and the 18 genes that were both linear and monotonically expressed into a single expression feature-space of 185 genes. The TCGA dataset was split into a train dataset of 287 cancer and 41 normal samples, and a holdout testset of 71 cancer and 10 normal samples. Application of the feature selection techniques yielded a consensus feature space of just seven essential features, viz. four of the top ten linear modelling genes (ESM1, DHRS7C, OTOP3, AADACL2), two stage-salient genes (stage-2 salient LPHN3 and stage-4 salient GABRD) and one linearly monotonic gene (LPAR1). Using these features, four different ML models were trained, and hyperparameters optimized. The models were ranked on their performance on the training and holdout test sets (Table 9), and the Random Forest and 2-layer Neural Network models were identified for blind external validation.
Performance in terms of balanced accuracy (average of the accuracy on either class) is reported. All models achieved ‘perfection’ on the holdout testset, with marginal performance difference on the training set itself.
Two external datasets were chosen for blind validation: (i) Rectal_cancer_MSK [35] with 113 cancer samples, obtained from https://www.cbioportal.org/; and (ii) 308 normal colon samples from the GTEx. It is noted that the microarray-based GEO datasets benchmarked in our study, namely GSE25071, GSE21510, and GSE39582 were limited in the coverage of the gene-space, lacking expression values for some of the seven features used in the models, and not further considered. The hyperparameter-optimized Random Forest and 2-layer neural network models were re-built on the full TCGA dataset and evaluated on the external datasets (Table 10). All the cancer samples were correctly predicted by the Random Forest model, yielding ‘perfect’ recall. There were just eleven misclassified instances out of the 421 samples in the combined external dataset, and all such instances were normal colon tissue samples, leading to a balanced accuracy of 98.27%. The Random Forest model outperformed the 2-layer Neural network model on all the metrics considered, including sensitivity, specificity, F1-score, and Mathews correlation coefficient (MCC).
The Random Forest model was clearly superior to the Neural Network 2-layer model on the external validation. Bal. acc. refers to balanced accuracy (average of sensitivity (recall) and specificity).
Development of a prognostic model for colorectal cancer
All the 157 stage-salient genes were subjected to univariate Cox regression analysis, and the significant biomarkers (P < 0.05) are presented in Supplementary File S13. Of the top stage-salient genes, five emerged significant, namely JPH3, HOTAIR, CST6, GABRD, and DKK1 (all P < 0.03). HOTAIR, CST6, GABRD, and DKK1 are stage-IV salient, while JPH3 is stage-I salient (Fig 12). Multivariate Cox regression analysis with feature selection yielded an optimal panel of three genes, namely HOTAIR, GABRD, and DKK1, with a model p-value ∼ 5e-04, and individual significances ∼ 0.0086, 0.0053, and 0.0238, respectively (i.e, all p-values < 0.05). The multivariate risk model was given by:
The hazard rate for all the prognostic factors significantly exceeded 1.0, indicating that the constituents of the biomarker panel elevated the prognostic risk, suggesting possible oncogenic roles in line with their overexpression. The distribution of risk scores yielded a median maxstat value of 2.74 for patient risk stratification. Further, the Kaplan-Meier curve of the multivariate model suggested that the high-risk group was significantly associated (p-value <0.0014) with a poorer overall survival than the low-risk group (Fig. 12d). The model yielded an acceptable Concordance index (C-index) ∼ 0.71±0.05, suggesting further application as a novel prognostic panel [36–38]. It is significant (and perhaps not surprising) that the identified prognostic panel is entirely composed of stage-IV salient biomarkers, suggesting that the distance to metastasis is the single dominant factor in the stratification and determination of prognosis of colorectal cancer.
Discussion
To clarify the sum of findings from our studies, we began by looking at the canonical CRC drivers, APC and MSH2, which are both implicated in familial CRC. APC and MSH2 are both significantly differentially expressed (adj.p-values ∼ 7.35e-13 and 2.06e-18 respectively). The expression patterns of these two genes (Fig 10) showed that APC was downregulated in the cancer phenotype, flagging its key role as a known TSG.
We then looked at the hub-driver genes identified in a previous study of CRC network analyses [39], and found that GRIN2A and EIF2B5 were significantly differentially expressed in the cancer samples (adj.p-values ∼ 2.14e-37 and 2.32e-13, respectively). GRIN2A is a TSG with least expression in stage 2 (Fig 11A), reinforcing its role as a hub driver gene for stage 2 progression. EIF2B5 is an oncogene with maximal expression in stage 3 (Fig 11B), again according with its identified role as a major hub driver gene for progression to advanced stages of colorectal cancer.
An analysis of the top genes from our linear model uncovered certain interesting observations. The top gene hit, CDH3 (Cadherin 3 or P-Cadherin), has been found to be overexpressed in a great majority of Pancreatic Ductal Adenocarcinomas (PDACs) [40], lending support to its key role in gastrointestinal cancers. Further, hypomethylation of the CDH3 promoter has been found in addition to (and the cause of) increased expression of CDH3 in both Breast Cancer [41] and Advanced Colorectal Cancer [42]. This can be due to the fact that over-expression of P cadherin leads to high motility of cells, which enables the cancer cells to metastasize.
There is emerging evidence for the role of KRT80 in head and neck squamous cell carcinoma [43], but it is not a known cancer driver (https://www.intogen.org/search?gene=KRT80). The gene OTOP2 has been identified as a TSG, as it was significantly downregulated in the cancer phenotype. Another independent study also found that wild type p53 regulated OTOP2 transcription in cells, and increased levels of OTOP2 suppressed tumorigenesis in vitro [44]. OTOP3 belongs to the same family of otopetrin proton channels, but there is no published evidence for its role in any cancer (https://www.intogen.org/search?gene=OTOP3).
AADACL2 is not a known cancer driver, but there is evidence for its role in a comorbid breast-colorectal cancer phenotype [45]. ETV4, another top candidate in our linear model, has shown significant promise as a therapeutic target. A previous study found that ETV4 knockdown in metastatic murine prostate cancer cells abrogates the metastatic phenotype but does not affect tumor size [46]. According to our model, ETV4 shows maximal expression in stage 4 and is concordant with a molecular basis for cancer stages. ETV4 is also a designated cancer gene in the COSMIC census [47].
ESM1 was found to be clearly overexpressed in clear cell renal cell carcinoma [48], and is also one among the 59 stage-4 salient genes from our study. Moreover, ESM1 is also an MEG identified in our study, placing it as a very significant driver of CRC progression. DHRS7C has been recently implicated in signaling pathways involved in glucose metabolism [49]. It exerts its effects via mTORC2, a complex known to be at the heart of metabolic reprogramming [50].
Mysteriously DHRS7C was seen downregulated in colorectal cancer, given that its upregulation is necessary for glucose uptake. These observations merit experimental investigations to ascertain the precise nature of the molecular biology in question.
Some studies reveal that the LIM-domain-containing JUB serves as an oncogene in CRC by promoting the epithelial-mesenchymal transition (EMT), a critical process in the metastatic transition [51]. The gene MTHFD1L coding for methylenetetrahydrofolate dehydrogenase 1–like is significantly overexpressed in the colorectal cancer phenotype. Studies show that MTHFD1L contributes to the production and accumulation of NADPH to levels that are sufficient to combat oxidative stress in cancer cells. The elevation of oxidative stress through MTHFD1L knockdown or the use of methotrexate, an antifolate drug, sensitizes cancer cells to sorafenib, a targeted therapy for hepatocellular carcinoma [52].
Comparing the transcriptomic stage-specific patterns of colorectal cancer samples identified here with their methylomic stage-specific patterns [53], we uncovered interesting connections. Some of the stage-salient genes here were also identified as stage-specific differentially methylated genes, namely: BAI3, TPM2, ZSCAN18, ZNF415 (Stage-I); PLAC2, DMRT3 (Stage-II); PIGR, TUBAL3 (Stage-III); and CST6 (Stage-IV). GABRD was earlier found to be significantly differentially methylated in all stages except stage-IV, suggesting that methylation precedes the stage-4 salient change in gene expression observed in this study. In the other direction, GPX3, identified as a stage-I salient gene here, was detected as differentially methylated in stage-2, suggesting the interpretation that change in its expression is necessary for cancer metastasis and mesenchymal transition. The details for the above analysis are presented in Supplementary File S14.
These are cancer driver genes with known experimental evidence. In the case of FAM135B, FEV, CBFB, and CTNND2, the regulatory status inferred here is at odds with the documented cancer role, and could point to anomalous regulation tractable to experimental investigation.
Stage-1 salient DEGs
The genes CALB2 and TMEM59L cluster together in Fig 8, showing the least expression in stage-I, suggesting the hypothesis that they are tumor suppressor genes whose expression is required to prevent tumorigenesis. This is supported by evidence in literature, specifically that cells in which CALB2 is silenced do not respond to 5-flourouracil, a popular treatment for CRC, indicating that CALB2 expression is essential for 5-flourouracil induced apoptosis [54]. Another study found that heterozygosity in SNP513 of Intron 9 of the gene CALB2 might be a predictive marker for CRC [55]. It has also been noted that increased TMEM59L expression was a pro-apoptotic indicator of cell death during oxidative stress in neuronal cells [56]. Regarding SOX2 and SOX10, it is noteworthy that the Cancer Genome Atlas Network observed SOX9 as a novel gene with significant recurrent mutations in COADREAD [7].
Stage-2 salient DEGs
KLHL34 was found to be hypermethylated in Locally Advanced Rectal Cancer, and knockdown of KLHL34 lowered colony formation, increased cytotoxicity, and increased radiation induced caspase 3 activity in LoVo cells [57]. CCBP2, encoding the Chemokine decoy receptor D6, has an inhibitory effect on breast cancer malignancies due to its action to sequester pro-malignant chemokines [58]. The lncRNA PLAC2 induces cell cycle arrest in glioma by binding to Ribosomal Protein RP L36 in a mechanism involving STAT1 [59]. GPC5 was found to be overexpressed in the lung cancer phenotype [60], in lymphoma, and in gastric cancer [61]. The work by Wang et al. [61] also showed that the overexpression of miR-217 impaired GPC5-induced promotion of proliferation and invasion in GC cells.
Stage-3 salient DEGs
Copy number polymorphisms of TRY6 gene have been found in Breast Cancer [62]. HABP2 gene overexpression has been observed in lung adenocarcinoma and has been proposed as a novel biomarker for the same [63].
Stage-4 salient DEGs
The lncRNA HOTAIR was found to be significantly overexpressed in HCC, and a potential biomarker for lymph node metastasis in HCC [64], and later implicated in different cancers [65]. Another widely-cited study [66] showed that enforced HOTAIR gene expression in epithelial cancer cells induces chromatin reprogramming and an increased metastatic state, while inhibition of HOTAIR inhibits cancer invasiveness. These accounts of the role of HOTAIR in metastasis accord with our findings that HOTAIR is a stage-4 salient significantly monotonically expressed biomarker. GWAS analysis identified a strong association of C6orf15 with occurrence of follicular lymphoma [67]. Promoter methylation of cell free DNA of the CST6 gene was found to be a potential plasma biomarker for Breast Cancer [68]. Expression of VGLL1 and its intronic miRNA miR-934 are associated with sporadic and BRCA1-associated triple negative basal-like breast carcinomas [69]. Expression of DKK1, an inhibitor of osteoblast differentiation, was found to be associated with the presence of bone lesions in patients with multiple myeloma [70]. TMEM40 has been found to be a potential biomarker in patients with Bladder cancer, serving as an oncogene and a possible therapeutic target [71]. The emergence of the C,E,and F members of the Lymphocyte Antigen 6 (LY6) family [72,73] as monotonically expressed proto-oncogenes holds promise for immunotherapy. There is a substantial evidence base for GABRD [74], which is a key component of both the screening and prognostic models developed here. Consistent expression trends in GABRD and other stage-salient MEG DEGs provide unmistakable evidence for the existence of molecular signatures in CRC progression.
Benchmarking with curated databases
We found 13 of the top 200 genes from the linear model documented in the CGC v84 as known cancer genes (Table 11). Two genes, MACC1 and SALL4, were specifically documented for colorectal cancer. HSP90AB1 had been earlier identified as a top MEG in HCC [9]. Screening the 157 stage-salient genes against the NCG7.0, which is a curated database of cancer drivers and healthy drivers, yielded 28 genes, of which eight were in the top 40 stage-salient genes (Supplementary File S15). All the hits were documented to carry mutations in their coding region (vs noncoding region). Three were canonical oncogene drivers, namely HOXC11, SOX2, and KCNJ5, while the rest 25 are putative oncogenes and putative tumor suppressors in almost equal measure. Two stage-salient genes, namely CNTN1 and BAI3 (ADGRB3) were documented as putative tumor suppressor genes involved in gastric adenocarcinoma, providing specific support for our findings. PIGR is identified as an essential healthy driver [75], signifying that mutations in this gene confer an exceptional protective effect, and its down-regulation could drive tumorigenic processes. Intriguingly, the stage-salient genes C5ORF23 (NPR3), SOX2, and KCNJ5 are the only instances where the documentation is dissonant with our primary findings; these three were marked as putative oncogenes, though they are identified as down-regulated here. Further investigations in this direction are warranted to set the literature straight.
Documentary evidence for drugs targeting any of these genes is absent, emphasizing the value of the present study in pinpointing novel candidates for diagnosis, therapy and prognosis. To perform a systematic analysis of therapeutic interventions based on these targets, we consulted ClinicalTrials.gov for clinical trials targeting stage-salient genes. Ten genes from the top stage-salient genes are being pursued in clinical trials, either as target or endpoint, colorectal or other cancers. Details of clinical trials along with the current status/phase of each trial are provided in Supplementary File S16. DKK1 and HOTAIR are the only stage-4 salient genes implicated as targets/endpoints in clinical trials. DKK1 is involved in three clinical trials for colorectal and gastric cancers. HOTAIR is the target of a clinical trial for thyroid cancer (NCT03469544) [76]. HOTAIR is documented in the NONCODE database (http://www.noncode.org/) as disease-associated, specifically with colorectal cancer (ID: NONHSAG011264.3), validating its role in oncogenic processes. It is notable that GABRD is not a target in any of the registered clinical trials, flagging a prime potential interest for future efforts. LPHN3, a stage-2 salient gene, is targeted in four clinical trials aimed against metastatic colorectal cancer, to explore possible therapeutic efficacy in thwarting cancer progression prior to irreversible outcomes. FADS6 (a stage-II salient gene) is an endpoint in a clinical trial to treat colorectal adenomatous polyps, which is a precursor to malignant lesions. CALB2 and C5orf23 (NPR3) are each involved in one clinical trial related to colorectal cancer. Some stage-salient genes are being pursued in treatment of cancers in other cell types/tissues, underlining the role played by certain genes in contributing to general cancer hallmark processes [31]. Specifically NAT2 is a target in nine different clinical trials against diverse cancers, significantly highlighting its essential role in driving hallmark processes in unrelated cancers.
Insights from Network Analysis
Stage-wise network analysis of colorectal cancer progression has shed light on certain genes potentially underlying progression [77]. The strength of the computational evidence for the candidate biomarkers identified herein urged a network analysis to examine the findings in a larger context. The intersection between the sets of all stage-salient biomarkers and the significant MEGs might highlight monotonically enriched pathways essential to the pathophysiology of colorectal cancers. Hence the 31 stage-salient MEGs were chosen to reconstruct the STRING network, with 50 interactors in the first shell and 10 interactors in the second shell. This yielded a PPI with 235 edges with an extremely significant enrichment p-value < 1.0e-16 (Fig 12). A Gene Ontology [78] analysis of this reconstructed network showed enrichment for the Wnt-Frizzled-LRP5/6 complex component at p-value < 1E-04. An analysis with KEGG [79] showed enrichment for 2-oxocarboxylic acid metabolism at p-value ∼ 0.001, indicating a Warburg-shift in metabolism. An analysis with Reactome [80] showed significant enrichment of SMAD2/3 and SMAD4 MH2 Domain Mutants in Cancer (p-value < 0.01). These observations in toto provide striking evidence for the involvement of these biomarkers in driving CRC progression.
Isolated nodes in the network included GABRD, DLX3, ISM2, LY6G6C & E, SPYDC, UPK2, C2orf48, PIGR, KRTAP3-1, C7orf52 (NAT16), SPERT, and PLAC1. All the isolated nodes are proto-oncogenic (see Table 7), hence could provide targets for inhibition in personalized cancer medicine. An outlier component (not in the giant connected component) was made of the CXCL chemokine family, stemming from CXCL13 - a recently recognized immune checkpoint with a key role in tumor progression [81,82]. This component could constitute a novel target for upregulation in CRC immunotherapy. A drug-repurposing search with the DrugGeneBadger [83] for each of the 31 stage-salient MEGs yielded drugs (small molecules with q-values < 0.05) to pharmacologically alter the expression of these identified biomarkers. The search revealed that curcumin is effective against at least 13 of these targets, and piperlongumine against eight of these targets. Six biomarkers (HOTAIR, ISM2, KRTAP3-1, SPDYC, LY6G6F, and NKD1) found no drug available in LINCS1000 [84] to modulate their expression, and these constitute potential novel targets for drug discovery against metastatic transition in CRC.
A network specific to colon cancer could be obtained using the results for GSE39582. Among the 503 Stage-IV salient genes, 262 were also monotonically significant (Supplementary File S17). We reconstructed a StringDB network seeded with these 262 monotonic stage-salient genes. The resulting interaction network with 316 nodes and 521 edges was significant (p-value ∼ 1e-15). Fig. 13 shows the giant connected component of this network; the full network is available in Supplementary File S17. Enrichment analysis of the network with Gene Ontology indicated significance for Arp2/3 complex-mediated actin nucleation (p-value ∼ 1e-4), which is known to contribute to invasive colorectal cancer [85]. A KEGG analysis showed enrichment for oxidative phosphorylation (p-value ∼ 1e-20), with a prominent clustering of NDUF and COX gene families. A Reactome analysis showed a minor enrichment of enzymatic protein conjugation processes (UBE2I, UBA2, SAE1) that monitor intracellular proteins and cell states (p-value ∼ 0.02). These findings indicate an enrichment of proliferation-independent metabolism-rewiring pathways necessary for colorectal cancer progression, and could be contrasted with the analyses in Marisa et al. [33].
Immune-cell infiltration analysis
Deconvolution of the TCGA samples based on the LM22 immno-cyte signature with 100 permutations yielded 107 samples with significance (P < 0.05), including eleven controls. These samples, with their TCGA identifiers, are presented in Supplementary File 18. The significant samples were analyzed for the relative abundance of the 22 immune cell types. A heatmap of the sample-wise immune cell-type proportions was generated (Supplementary File 19 Fig.A), and the clustering patterns of the cell-types across samples was visualized using a dendrogram. We observed the following clusters: mast cells resting and plasma cells; mast cells activated and neutrophils; T cells CD8, T cells follicular helper, and macrophages M1; T cells CD4 memory resting and B cells naive. The macrophages M0 and M2 were clear outgroups in the dendrogram. A normalized stacked bar chart of the sample-wise immnuo-cyte fractions revealed substantial variations in immune cell-type composition between normal and cancer samples (Supplementary File 19, Fig.B). To investigate further, we analyzed the differences in distribution of cell proportions between normal and tumor samples for each immune cell-type (Supplementary File 19, Fig.C; data presented in Supplementary File S18). Eight of the 22 immunocyte types showed significant distribution differences (adj. P < 0.05). Specifically, we found the infiltration of four immune cell-types preferentially enriched in tumor samples, namely macrophages M0, T cells CD4 memory activated, mast cells activated, and neutrophils, while four other immune cell-types were preferentially depleted in tumor samples, namely macrophages M2, T cells CD4 memory resting, mast cells resting, and plasma cells. In particular, macrophages M0 exhibited both the largest effect size (> 2.0) and the greatest significance (< 1E-07) of infiltration in tumor samples. The preferential enrichment of mast cells activated and T cells CD4 memory activated versus the preferential depletion of mast cells resting and T cells CD4 resting suggested that tumorigenesis activates resting immune cell-types, potentiating their infiltration of the tumor microenvironment. To integrate these observations, we computed the correlation matrix of the immune cell-types based on their sample-wise proportions over both normal and tumor samples (Supplementary File 19, Fig.D). The largest positive correlations were exhibited by T cells follicular helper with T cells CD8 (Pearson’s ρ ∼ 0.52), and with macrophages M1 (Pearson’s ρ ∼ 0.45), reinforcing their clustering in the dendrogram. Intriguingly, the largest negative correlation (in magnitude) was exhibited by macrophages M0 and T cells CD4 memory resting (Pearson’s ρ ∼ −0.51) (Supplementary File 19, Fig.D). Given that macrophages M0 are preferentially enriched in tumor samples whereas T cells CD4 resting and mast cells resting (Pearson’s ρ ∼ −0.47 with macrophages M0) are both preferentially depleted, these observations cohere and could hold preliminary significance for immunotherapy. Discovery of multicellular community structures could pave the way for personalized immunotherapy in CRC treatment [11, 86].
COADREADx
Based on the external validation, the Random Forest model was identified as the best model for screening early-stage cancer. Coupled with the prognostic model, these could aid the risk stratification of patient samples. With this application in mind, we have deployed COADREADx, an experimental web service for the screening of patient samples as ‘cancer’ or ‘normal’, and subsequent prognostication in the case of ‘cancer’. COADREADx has been implemented using R-Shiny (https://shiny.rstudio.com/), and is available for academic use at: https://apalanialab.shinyapps.io/coadreadx/. A help document with sample input files for different use-cases, and a companion how-to video have been made available on the landing page. To aid the effective interpretation of COADREADx predictions, the prediction probability for the predicted diagnostic class is provided, yielding a level of confidence in the prediction.
Similarly the risk stratification of ‘cancer’ samples is accompanied by the quantile of the estimated risk-score as well as its fold-change from the median value of the risk score distribution. These values suggest the strength of evidence for the predicted risk class.
In summary, we have performed a novel de novo analysis of the TCGA COADREAD gene expression dataset, and identified multiple interesting classes of biomarkers. The biomarkers have been validated with alternative datasets, network analysis and immune cell infiltration analysis. Some of the biomarkers could suggest novel hypotheses for targeted therapy and immunotherapy. Using purifying techniques, we have carved feature spaces from these biomarkers to build screening and prognostic models of colorectal cancer. The screening model has been externally validated, while the prognostic model has been bootstrapped for confidence. Both the models have been deployed as a web-server, COADREADx, which has been configured to return confidence estimates for all its predictions. Phenomena of distribution drift and shift in new samples and out-of-domain cohorts challenge the applicability of COADREADx, which might need refinement in the light of such data. Enabling risk stratification is vital to treatment strategy and clinical management of the cancer. Thus experimental validation and further improvement of COADREADx is necessary to demonstrate its clinical utility for screening and prognosis purposes. It is reckoned that the availability of such software-as-medical-devices could ease the accessibility to effective surveillance technologies for early detection of colorectal cancer [20].
Conclusions
We have executed multiple workflows towards computational validation of stage-salient signatures of colorectal cancer progression. We have identified stage-agnostic progression-significant monotonically expressed biomarkers. Modulating the expression of progression-significant biomarkers (for e.g, by inhibiting the overexpressed ones or activating the expression of downregulated ones) represents a promising potential strategy to effectively intervene in the progression of colorectal cancer. The candidate biomarkers identified have been benchmarked against curated databases and the literature. A binary classification model for early-stage screening of colorectal cancer was created using seven consensus biomarkers (namely ESM1, DHRS7C, OTOP3, AADACL2, LPHN3, GABRD, and LPAR1), and yielded > 98% balanced accuracy on external validation. A survival analysis protocol yielded a prognostic panel of three stage-IV salient genes (namely HOTAIR, GABRD, and DKK1) for patient risk stratification, suggesting that high-risk prognosis is entirely dependent on the oncogenic expression of these metastasis-salient genes, and inviting experimental confirmation. By benchmarking our findings in multiple ways, we have evaluated the assumptions underlying our computational models. The weight of the evidence presented herein suggests the central role of molecular factors in cancer progression. In summary, we have developed a set of tools for colorectal cancer screening and prognosis, COADREADx, based on the candidate biomarkers identified in our study. COADREADx is available for academic use at: https://apalanialab.shinyapps.io/coadreadx/. Our work provides a pilot study for further exploration of signature panels on the overall path to securing the best possible intervention for the condition. The hypothesis-agnostic overall study design provides a framework for the investigation of other cancers, and more generally, conditions that are progressive (and degenerative).
Data Availability
All data produced in the present work are either contained in the manuscript or available online at: https://doi.org/10.6084/m9.figshare.20489211.v4 COADREADx is available for nonprofit use at: https://apalanialab.shinyapps.io/coadreadx
Supporting Information
S1 File: Dataset
S2 File: Sorted linear model - full information
S3 File: Stage-specific DEGs
S4 File: Sorted numeric model - full information
S5 File: significant MEGs
S6 File: Expression visualization of MEGs
S7 File: MEGs significant in Linear Model and Numeric Model (adj. p-val < 1E-05)
S8 File: MEGs that are in the Linear Model top200
S9 File: MEGs in the Numeric Model top200
S10 File: Overlap between the Top 200 of Linear & Numeric Models
S11 File: Stage-salient genes identified in the normals-augmented RNAseqDB COADREAD data
S12 File: Stage-salient genes identified in an external cohort (GSE39582) for benchmarking
S13 File: Univariate Cox survival analysis of all the 157 stage-salient genes
S14 File: Consensus between the stage-salient genes and the stage-specific DMGs identified using ChAMP software
S15 File: Summary of the stage-salient genes documented in the Network of Cancer Genes
S16 File: Summary of the stage-salient genes that have been used as targets or endpoints in clinical trials, as documented in ClinicalTrials.gov
S17 File: Monotonically expressed genes (MEG) identified in the external dataset GSE39582, and STRINGdb network reconstruction using the overlap between MEGs and stage-4 salient DEGs.
S18 File: Deconvolution results from Cibersort immuno-cyte profiling analysis, yielding significant samples, as well as the raw data used in the related figures.
S19 File: Figures A, B, C, D: Analysis of immune cell-type infiltration between tumor and normals, with visualization of immune ecotypes and differential distribution of immune cell-type populations.
Acknowledgments
We would like to thank the reviewers for helpful comments. We are grateful to SASTRA deemed University for resources, infrastructure, and support. Computing in our lab is also supported on a generous grant from Google TPU Research Cloud (TRC).
Footnotes
The manuscript has been significantly expanded to include a section on CRC survival analysis and prognostic modeling. A diagnostic model has also been constructed with >98% balanced accuracy on external validation. Both the diagnostic and prognostic models have been translated into COADREADx, an experimental web-server for the nonprofit automatic screening and prognosis of colorectal cancer. Benchmarking of the results has also been extended, and a co-author in the light of the above revisions has been added.
References
- 1.↵
- 2.↵
- 3.↵
- 4.↵
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.↵
- 23.↵
- 24.↵
- 25.↵
- 26.↵
- 27.↵
- 28.↵
- 29.↵
- 30.↵
- 31.↵
- 32.↵
- 33.↵
- 34.↵
- 35.↵
- 36.↵
- 37.
- 38.↵
- 39.↵
- 40.↵
- 41.↵
- 42.↵
- 43.↵
- 44.↵
- 45.↵
- 46.↵
- 47.↵
- 48.↵
- 49.↵
- 50.↵
- 51.↵
- 52.↵
- 53.↵
- 54.↵
- 55.↵
- 56.↵
- 57.↵
- 58.↵
- 59.↵
- 60.↵
- 61.↵
- 62.↵
- 63.↵
- 64.↵
- 65.↵
- 66.↵
- 67.↵
- 68.↵
- 69.↵
- 70.↵
- 71.↵
- 72.↵
- 73.↵
- 74.↵
- 75.↵
- 76.↵
- 77.↵
- 78.↵
- 79.↵
- 80.↵
- 81.↵
- 82.↵
- 83.↵
- 84.↵
- 85.↵
- 86.↵