Machine learning-based classification of Alzheimer’s disease and its at-risk states using personality traits, anxiety, and depression

2.1 Participants

For our cross-sectional study, we used baseline data from participants recruited through the DELCODE study. For detailed information on the DELCODE study, see Jessen et al. (2018). We included a large cohort of 733 participants that were assigned to four different groups based on their entry diagnosis: HC, SCD, aMCI, and mild AD. All participants were aged 60 years or older, fluent in German, able to give informed consent, and had a study partner present.

Participants for the study were recruited either through local newspaper advertisements or from memory clinics. Healthy controls self-identified as cognitively healthy and passed a telephone screening for SCD. These individuals were included as HC if their memory test performance was within 1.5 standard deviations (SD) of the age-, gender-, and education-adjusted normal performance on all Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) subtests and if they did not meet the SCD criteria (Jessen et al. 2014). Conversely, individuals expressing cognitive decline concerns to the memory center physician were categorized as either SCD or aMCI, based on a comprehensive semi-structured interview following the SCD-plus criteria (Jessen et al. 2014) and their CERAD performance. SCD participants outperformed the −1.5 SD below normal, while aMCI patients underperformed (>1.5 SD) on the "recall word list" subtest, thus excluding non-amnestic MCI participants. They did not meet the criteria for dementia, and their inclusion was based on the memory clinic diagnoses, which adhered to the current research criteria for MCI as defined by the National Institute on Aging-Alzheimer’s Association (Albert et al. 2011; McKhann et al. 2011).

Assignment to the AD group was based on both clinical diagnosis and on the Mini Mental Status Examination (MMSE). Only participants with mild AD (>18 points and <26 points on the MMSE) were included. Aside from HC, all participant groups (SCD, MCI, AD) were memory clinic referrals and underwent clinical assessments at their respective memory centers. These assessments consisted of a medical history review, psychiatric and neurological examinations, neuropsychological testing, blood laboratory analysis, and routine MRI scans. Cognitive function was measured using the CERAD neuropsychological test battery, which was administered at all memory centers.

2.2 MRI data acquisition

Structural and functional MRI data were acquired on 3T Siemens scanners following the DELCODE study protocol (Jessen et al. 2018; Düzel et al. 2019). A T1-weighted MPRAGE image (TR = 2.5 s, TE = 4.37 ms, flip-α = 7°; 192 slices, 256 x 256 in-plane resolution, voxel size = 1 x 1 x 1 mm) was acquired for co-registration and improved spatial normalization.

The MPRAGE was followed by a 7:54 min resting-state fMRI (rs-fMRI) acquisition, during which T2*-weighted echo-planar images (EPI; TR = 2.58 s, TE = 30 ms, flip-α = 80°; 47 axial slices, 64 x 64 in-plane resolution, voxel size = 3.5 x 3.5 x 3.5 mm) were acquired in odd-even interleaved-ascending slice order. Participants were instructed to lie inside the scanner with eyes closed, but without falling asleep. Directly after, phase and magnitude fieldmap images were acquired to improve correction for artifacts resulting from magnetic field inhomogeneities via unwarping. This was followed by brief co-planar T1-weighted inversion recovery EPIs.

The complete study protocol included other scanning sequences not used in the analyses reported here (Jessen et al. 2018).

2.3 fMRI data preprocessing and analysis

Data preprocessing and computation of mPerAF maps were performed using Statistical Parametric Mapping (SPM12; Wellcome Trust Center for Neuroimaging, University College London, London, UK) and the RESTplus toolbox (Jia et al. 2019), following a recently described protocol (Kizilirmak et al. 2022). EPIs were corrected for acquisition time delay (slice timing), head motion (realignment), and magnetic field inhomogeneities (unwarping), using voxel-displacement maps (VDMs) derived from the fieldmaps. The MPRAGE image was spatially co-registered to the mean unwarped image and segmented into six tissue types, using the unified segmentation and normalization algorithm implemented in SPM12. The resulting forward deformation parameters were used to normalize unwarped EPIs into a standard stereotactic reference frame (Montreal Neurological Institute, MNI; voxel size = 3 x 3 x 3 mm). Normalized images were spatially smoothed using an isotropic Gaussian kernel of 6 mm full width at half maximum.

PerAF is a voxel-wise, scale-independent measure of low-frequency (0.01-0.08 Hz) BOLD signal fluctuations relative to the mean BOLD signal intensity for each time point, averaged across the whole time series (Jia et al. 2020). The global-mean-adjusted PerAF (mPerAF) was computed from rs-fMRI using an adapted version² of the RESTplus toolbox (Jia et al. 2019). A DMN mask (Shirer et al. 2012) was applied, representing a composite of functionally defined regions of interest (ROIs), and the resulting mPerAF maps served as voxel-wise mean-centered predictor variables.

2.4 Clinical and risk factor assessments

Trained study physicians administered the baseline clinical assessments in the DELCODE study. These assessments followed a fixed order and were completed within a single day. Caregivers of participants with AD were allowed to help complete the questionnaires. Clinical assessments included the Geriatric Depression Scale short form (GDS; Sheikh & Yesavage 1986), the Geriatric Anxiety Inventory short form (GAI-SF; Byrne & Pachana 2011), and the Big Five Inventory short form (BFI-10; Rammstedt & John 2007; Rammstedt et al. 2017). Scores on the five personality scales (each calculated as the mean of the two respective items) were included as five standardized predictors. The sum scores of GDS and GAI-SF were included as standardized predictors, respectively.

2.5 ApoE genotyping

The single nucleotide polymorphisms (SNPs) rs7412 and rs429358, which define the ε2, ε3, and ε4 alleles of the ApoE gene, were determined using a TaqMan® SNP Genotyping Assay (ThermoFisher Scientific). ApoE ε4 non-carriers (ε2/ε2, ε2/ε3, ε3/ε3) were coded as 0, heterozygotes (ε2/ε4, ε3/ε4) were coded as 1, and homozygotes (ε4/ε4) were coded as 2.

2.6 Cerebrospinal fluid biomarker assessment

Cerebrospinal fluid biomarkers (tTau, pTau181, and Aβ42/40 ratio; collectively referred to as CSF biomarkers) were measured using commercially available kits according to manufacturers’ specifications: V-PLEX Aβ Peptide Panel 1 (6E10) Kit (K15200E) and V-PLEX Human Total Tau Kit (K151LAE) (Mesoscale Diagnostics LLC, Rockville, USA), and Innotest Phospho-Tau(181P) (81581; Fujirebio Germany GmbH, Hannover, Germany).

2.7 Assessment of confounding features

Chronological age was included as a standardized predictor (mean = 0, SD = 1). The acquisition site predictor used in the DELCODE study included ten distinct sites across Germany, which were represented as dummy-coded predictors using ten binary variables. Gender was included as a dummy-coded predictor with two binary predictors.

2.8 Prediction of outcome from predictor variables and performance assessment

Predictor variables were combined into eight feature sets (Figure 1). In this study, we will employ the terms "predictor(s)" and "feature(s)" interchangeably, as well as "group(s)" and "class(es)", to represent the same concept.

1. Base model: age, gender, site

2. mPerAF: base model, mPerAF

3. Personality: base model, BFI-10

4. Depression, anxiety: base model, GDS, GAI-SF

5. Personality extended: base model, BFI-10, GDS, GAI-SF

6. ApoE: base model, ApoE genotype

7. CSF: base model, tTau, pTau181, Aβ42/40 ratio

8. All w/o CSF: base model, mPerAF, BFI-10, GDS, GAI-SF, ApoE genotype

To predict the outcome variable (participant group) with the feature sets, we employed Support Vector Classification (SVC) using linear Support Vector Machines (SVMs) with soft-margin parameter C = 1 and 10-fold cross-validation. All SVM analyses were implemented using LibSVM in MATLAB via custom scripts available on GitHub (https://github.com/JoramSoch/ML4ML).

Predictive performance of participant classification was assessed using decoding accuracy (DA), that is, the average proportion of correctly classified participants across all groups, and class accuracy (CA), that is, the same proportion, separately for each group, each ranging between 0 and 1.

For each feature set, statistically significant differences from chance-level prediction for DA and CAs were tested, and pairwise comparisons of each feature set against the base model were performed. This was done using one-tailed paired t-tests for the classification performance of each feature set against the base model, with each pair consisting of a subsample evaluated using both feature sets. Bonferroni-Holm correction was applied for multiple testing. Additionally, a subsample-by-subsample correlation matrix of DAs across all permutations was computed and incorporated into a general linear model of the pairwise accuracy differences across all subsamples. All scripts used to perform the analyses are available at https://github.com/jmkizilirmak/DELCODE162.

2.9 Handling of missing values and unbalanced class sizes

Participants with missing data for age, gender, site, mPerAF, BFI-10, GDS, GAI-SF, and ApoE genotype were excluded from analysis (N = 663; 179 HC, 308 SCD, 113 aMCI, 63 AD). Due to additional missing CSF biomarker values, additional exclusions were made for the "CSF" feature set (N = 341; 75 HC, 155 SCD, 71 aMCI, 40 AD) and the “CSF” feature set was excluded from inferential comparisons to maintain statistical power. Supplementary information provides an alternative analysis with equal sample sizes (N = 311; Table S4) across all feature sets, as well as an analysis with SCD and aMCI groups merged into an "at-risk for AD" group (Table S2).

Subsampling was used to ensure equal numbers of participants in each group when performing SVC (Puechmaille 2016). The size of each subsample was based on the smallest group (rounded off to the nearest 10). A total of 30 subsamples were created, and each subsample was subjected to 1000 permutations of group membership to establish a null distribution. Permutations were performed to calculate the p-value of the prediction accuracy.

3 Results

Classification results are reported in Table 2 and inferential statistical comparisons are reported in Table 3. DAs are visualized in Figure 2 and CAs in Figure 3. The four best performing feature sets sorted by decoding accuracy are depicted as a confusion matrix in Figure 4.

• Download figure
• Open in new tab

Figure 2. Decoding accuracies of the evaluated feature sets.

The 90% confidence intervals were obtained by averaging the confidence intervals of the 30 subsamples (single dots) on which SVCs were performed.

• Download figure
• Open in new tab

Figure 3. Class accuracies of the evaluated feature sets.

The dotted line represents the chance level. Error bars represent the average 90% confidence interval across all 30 subsamples.

• Download figure
• Open in new tab

Figure 4. Confusion matrices of best performing feature sets by decoding accuracy

3.1 Base model: Low predictive value of combining age, gender, and site

The “base model” produced the lowest overall DA (DA = .345, p = .047) and no CA was significantly different from chance for any group (Figure 3).

3.2 mPerAF: Low but above-chance performance of resting-state DMN activity

Feature set “mPerAF” performed significantly above chance (DA = .352, p = .010), along with significant CAs for both HC (CA = .417, p = .026) and AD (CA = .446, p = .016). CAs for SCD (CA = .287, p = .299) and aMCI (CA = .258, p = .419) were statistically indifferent from chance.

3.3 Personality trait and affective state scores: Highest prediction accuracies for HC and across groups

Feature set “Personality” was consistently outperformed by “Personality extended”, which produced the overall highest DA (DA = .414, p = .001). Combining scores of geriatric depression and anxiety demonstrated the overall highest class accuracy for healthy controls (CA = .628, p = .003) and the overall third-highest DA (.392, p = .003).

3.4 ApoE: Third-highest decoding accuracy

Feature set “ApoE” showed the third-best performance (DA = .402, p = .002). It also demonstrated significantly above chance CAs for HC (CA = .522, p = .021) and AD (CA = .522, p = .023).

3.5 Relatively poor performance of combined predictors without CSF biomarkers

Across all groups and in terms of DA, prediction accuracies of feature set “All w/o CSF” were consistently lower than those of “Personality” and “Personality extended” and it was not in the top three CAs for any participant group.

3.6 CSF biomarkers predict AD best, but perform poorly for HC

Feature set “CSF” exhibited the highest CAs for the groups of SCD (CA = .348, p = .301) and AD (CA = .647, p = .009), as well as the second-highest DA (.405, p = .017). CAs for HC (CA= .431, p = .156) and aMCI (CA = .194, p = .675) were non-significant above chance.

3.7 Comparison of feature sets and summary

The highest performance in terms of DA (Figure 4) were achieved by the feature sets “Personality extended” (DA = .414, p = .001) followed by “CSF” (DA = .405, p = .017), “ApoE” (DA = .402, p = .002), and “Depression, anxiety” (DA = .392, p = .003). All feature sets—except “mPerAF”—performed significantly better than the base model in predicting group membership (Table 3).

4.1 Inferiority of the combined predictor and poor prediction accuracy of resting-state activity of the DMN

Our hypothesis that combining non-invasive predictors (feature set "All w/o CSF") would outperform CSF biomarkers in prediction accuracy was not supported by our data. The classification accuracies of the "All w/o CSF" feature set were comparably low and similar to the "mPerAF" feature set, suggesting that the inclusion of mPerAF paradoxically reduced classification performance. While DMN resting-state mPerAF performed above chance, its performance did not significantly differ from the "Base model".

The predictive ability of resting-state fMRI of the DMN for AD has yielded inconsistent findings. While certain studies have reported consistent alterations in DMN activity and connectivity in AD (Mevel et al. 2011) and the added value of combining different MRI modalities to classify AD (Schouten et al. 2016), other research suggests that neuropsychiatric measures may have higher predictive ability (Gill et al. 2020).

It is important to note that most DMN studies have focused on functional connectivity rather than voxel-wise amplitude measures like mPerAF. The divergent results could be attributed to our approach of evaluating all groups simultaneously, resembling a fully automated diagnostic process, as opposed to making binary decisions between distinct groups. Furthermore, unequal sample sizes can introduce bias in classification, and various approaches have been proposed to address this issue (Jo et al. 2019).

4.2 A combination of personality, anxiety, and depression scores yield a relatively high overall prediction accuracy

Personality alone demonstrated class accuracies statistically significant above chance for the groups of HC and AD, but not for SCD and aMCI, partially confirming our hypothesis. "Personality" was surpassed by the feature set "Personality extended". However, the accuracy of correctly classifying the aMCI group was equally high, while class accuracies for the SCD and aMCI groups remained nonsignificant, partially supporting our hypothesis. These results indicate that depression and anxiety contribute additional predictive value to the decoding accuracy of the BFI-10. However, the highest class accuracy for HC was achieved by a feature set containing scores of depression and anxiety, but containing no scores of personality traits. This could be due to the absence of individuals with AD pathology in the HC group, while the SCD and aMCI groups contain an unknown proportion of individuals at risk of progression to AD. Previous studies have indicated that depressive episodes can be prodromal manifestations of neurodegeneration in AD (Panza et al. 2010; Singh-Manoux et al. 2017; Hansen et al. 2021). The predictive ability of the feature set "Depression, anxiety" for (HC) may be primarily attributed to the GDS as some of the GAI-SF items overlap with those of the BFI-10 neuroticism scale, suggesting depression scores to be well-suited in distinguishing between healthy individuals and participants with cognitive impairment. AD participants were best classified using a combination of CSF biomarkers, consistent with previous findings (Olsson et al. 2016; Lleó et al. 2019; Düzel et al. 2022). The predictive value of combining CSF biomarkers, personality traits and scores of depression and anxiety should be investigated further.

4.3 Poor classification accuracy for SCD and aMCI with any feature set

Predictions for participant groups with SCD or aMCI were mostly above chance level but not statistically significant. (Table 2). This trend persisted after merging SCD and aMCI into an “at-risk for AD” group (Table S2). Because underlying conditions causing SCD or aMCI in DELCODE participants were unknown at baseline, it is reasonable to assume that a proportion of participants did not actually have preclinical AD (see section 2.1). Neither SCD nor aMCI are specific to AD and can be caused by a variety of conditions, including normal aging. Conversion rates to AD vary, which may explain the poor class accuracies for these groups. (Chételat et al. 2005; Johns et al. 2012; Bessi et al. 2018), which may explain the poor class accuracies for these groups.

4.4 Limitations

Our study has several limitations. CSF biomarkers were only measured in a portion of the sample, resulting in different sample sizes for feature sets and exclusion of the "CSF" feature set from inferential analysis. Anosognosia is known to be a common occurrence in the early stages of AD (Leicht et al. 2010; Orfei et al. 2010; de Ruijter et al. 2020) and may also have confounded the assessments of the GDS, the GAI-SF (Starkstein 2014), and the BFI-10 (Agϋera-Ortiz et al. 2019). Additionally, caregiver influence on self-reports may have affected the accuracy of assessments in the aMCI and AD groups. Another important limitation relates to the demographics of the groups. Despite being composed of confounding variables only, the "Base model" performed above chance. This can be attributed to the association between age and dementia risk (Terracciano & Sutin 2019). On average, AD participants were older than HC or those with SCD (Table 1). However, because age was included in all feature sets, its predictive value was consistently accounted for.

4.5 Conclusions

Our results show that no single combination of the evaluated features achieved consistently superior class accuracies for all assessed participant groups. The combination of depression and anxiety scores was most effective in classifying healthy controls, supporting previous findings that regard late-life depression as a prodrome of Alzheimer’s disease, while CSF biomarkers were most effective in classifying participants with mild Alzheimer’s disease. The highest overall prediction accuracies across all participant groups were achieved by a combination of personality traits with scores of depression and anxiety, closely followed by CSF biomarkers and the ApoE genotype. These findings indicate that a combination of CSF biomarkers, personality, depression and anxiety scores, and the ApoE genotype may have complementary value for classification of AD and associated at-risk states. Further investigation is needed, particularly regarding the predictive value of personality traits and associated affective states as low-cost and easily assessable screening tools. Moreover, our findings highlight the challenge of accurately classifying SCD and aMCI groups using machine learning approaches when the underlying conditions of these cognitive impairments are unknown. Addressing this challenge requires adhering to consensus on terminology and conceptual frameworks.