Revealing Individual Neuroanatomical Heterogeneity in Alzheimer’s Disease
===========================================================================

* Serena Verdi
* Seyed Mostafa Kia
* Keir Yong
* Duygu Tosun
* Jonathan M. Schott
* Andre F. Marquand
* James H. Cole
* the Alzheimer’s Disease Neuroimaging Initiative

## Abstract

Alzheimer’s disease is clinically heterogeneous, in symptom profiles, progression rates and outcomes. This clinical heterogeneity is linked to underlying neuroanatomical heterogeneity. To explore this, we employed the emerging technique of neuroanatomical normative modelling to index regional patterns of variability in cortical thickness in individual patients from the large multi-site Alzheimer’s Disease Neuroimaging Initiative. We aimed to characterise individual differences and outliers in cortical thickness in patients with Alzheimer’s disease, people with mild cognitive impairment and cognitively normal controls. Furthermore, we assessed the relationships between cortical thickness heterogeneity and cognitive function, amyloid-beta, tau, ApoE genotype. Finally, we examined whether individual neuroanatomical normative maps were predictive of conversion from mild cognitive impairment to diagnosed Alzheimer’s disease. Data on cortical thickness from the 148 brain regions of the Destrieux FreeSurfer atlas was obtained from T1-weighted MRI scans of 1492 participants scanned at 62 different sites. A neuroanatomical normative model was developed to index normal cortical thickness distributions using a separate healthy reference dataset (n= 33,072), employing hierarchical Bayesian regression to predict cortical thickness per region using age and sex. These regional normative models were then fine-tuned to the ADNI dataset after which cortical thickness z-scores per region were calculated, resulting in a z-score ‘map’ for each participant. Regions with z-scores < -1.96 were classified as outliers. Patients with Alzheimer’s disease had a median of 12 outlier regions out of a possible 148. Individual patterns of outlier regions were highly variable, with the highest overlap in the parahippocampal gyrus at only 47% of patients. For 62 regions, over 90% of these patients had cortical thicknesses within the normal range. Patients with Alzheimer’s disease had significantly more outlier regions than people with mild cognitive impairment or controls [*F*(2, 1022) = 95.39), *P* = 2.0 x ×10−16]. They were also statistically more dissimilar to each other than were people with mild cognitive impairment or cognitive normal controls [*F*(2, 1024) = 209.42, *P* = 2.2×10−16]. Having a greater number of outlier regions was associated with worse cognitive function, CSF protein concentrations and an increased risk of converting from mild cognitive impairment to Alzheimer’s disease within three years (HR =1.028, 95% CI[1.016,1.039], *P* =1.8 ×10−16). Individualised normative maps of cortical thickness highlight the heterogeneity of Alzheimer’s effects on the brain. Regional outlier estimates have the potential to be a marker of disease and could be used to track an individual’s disease progression or treatment response in clinical trials.

Keywords
*   Normative modelling
*   ADNI
*   Alzheimer’s disease
*   heterogenous
*   hierarchical Bayesian regression

## Introduction

Alzheimer’s disease is the commonest cause of dementia, being characterised by progressive deterioration in cognitive functioning and independence 1. The Alzheimer’s disease spectrum comprises substantial clinical and biological differences between patients recognised in clinical and research criteria. Such differences comprise variations in genetic basis 2–4, symptom profile, age-at-onset, trajectory and severity 5,6, biomarker readouts (e.g., CSF amyloid-beta levels) 7, comorbidities 8,9, and in atrophy patterns 10. Despite this, conventional statistical analyses in Alzheimer’s disease research focus on group averages. This fundamental statistical assumption posits that Alzheimer’s disease will affect different patients in similar ways 11, characterising the ‘average’ patient. To reach the goal of precision medicine for Alzheimer’s we need to look beyond the average and design statistical approaches that reflect the heterogeneity between patients.

Neuroimaging methods are the gold standard of understanding of the *in vivo* brain 12. Structural imaging has been described as the ‘imaging workhorse of neurodegeneration’, being commonly recommended in Alzheimer’s disease diagnostic guidelines 13. With this in mind, large structural neuroimaging datasets are increasingly available for dementia, such as Alzheimer’s Disease Neuroimaging Initiative (ADNI), Open Access Series of Imaging Studies (OASIS) and National Alzheimer’s Coordinating Center (NACC) 14–16 as well as in the general population (e.g., UK Biobank and the Human Connectome Project). These datasets provide the ability to chart variation across cohorts and facilitating individual prediction.

Large neuroimaging datasets have particularly supported the development and application of data-driven methods in Alzheimer’s disease research. This has further revealed that differences in brain structure are very common in patients 10,17. Moreover, they have enabled the estimation of disease subtypes from neuroimaging data, as a way to disentangle heterogeneity by grouping patients by distinctive neurobiological and cognitive characteristics 10,17–20 and disease progression 21. Such subtypes have the potential to stratify patient groups for clinical decision-making, such as regarding treatment strategy, services and therapies tailored to clinico-radiological phenotype and/ or trial enrolment 22–24.

Importantly, there are challenges associated with the clinical translation of neuroimaging derived subtypes 18. These include validity of subtypes, how distinct subtypes are from each other and how stable subtypes are over the disease course 17,25. Moreover, by design, clustering assumes homogeneity within each cluster, despite individual-level variation present, therefore limiting the representation of heterogeneity in the sample 26. Individual-level variation is perhaps most apparent in patients with atypical, non-amnestic Alzheimer’s disease who comprise up to a third of young-onset Alzheimer’s disease and face particular barriers to diagnosis and appropriate care 24. Arguably, assessing the neurobiology of Alzheimer’s disease at the individual patient level will provide more precise understanding of their disease, likely outcomes and facilitate tailored treatment strategies.

While the concept of patient-centred, individualised precision medicine for dementia is well established, only research has been published using this approach with neuroimaging data in Alzheimer’s. These include techniques based on deep autoencoders 27 and machine learning classification 17. However, these studies included only limited validation of individualised predictions, did not fully characterise the spatially distributed nature of alterations in Alzheimer’s disease, and failed to assess how neuroanatomical variability related to cognitive performance, CSF biomarkers, disease progression, or genetic factors. Furthermore, these studies did not deal with crucial confounding effects of MRI scanner on heterogeneity.

One technique for capturing individual-level variability in the brain is neuroanatomical normative modelling. This can provide individual statistical inferences with respect to an expected ‘normative’ distribution or trajectory over time. Specifically, neuroanatomical normative modelling adopts this concept by modelling the relationship between neurobiological variables (e.g., neuroimaging features) which are represented regionally and covariates (e.g., demographic variables such as age and sex) to map centiles of variation across a cohort (i.e., z-scores). An individual can then be located within the normative distribution to establish to what extent they deviate from the expected pattern in each measure. By applying this approach to derive neuroanatomical normative models at brain regions, a map can be generated of where and to what extent an individual’s brain differs from the norm 28,29. This technique has shown to be suitable for precise mapping of individual patterns of variation in brain structure across multiple psychiatric and neurodevelopmental disorders, thereby parsing the neuroanatomical heterogeneity present 30,31. These include attention-deficit hyperactivity disorder 32, autism 33,34, bipolar disorder and schizophrenia 35. Such findings motivate the application of neuroanatomical normative modelling to Alzheimer’s disease 36.

Here, we examine individual patterns of variation in brain structure in patients with Alzheimer’s disease using neuroanatomical normative modelling. Using the well-characterised Alzheimer’s Disease Neuroimaging Initiative dataset, we applied a recent implementation of the normative modelling framework, hierarchical Bayesian regression. This technique has shown to efficiently accommodate inter-site variation and provides computational scaling, which is useful when using large studies, or combining smaller studies together, that are acquired across multiple sites in a federated learning framework 37–39. We aimed to quantify spatial patterns of neuroanatomical heterogeneity using cortical thickness measures in patients with Alzheimer’s disease, people with mild cognitive impairment and cognitively normal controls, by calculating deviations from normative ranges for each brain region and then identifying statistical outliers. Here we aimed to 1) assess the extent of neuroanatomical variability between individual patients based on overlapping or distinct patterns of outliers in cortical thickness, 2) quantify group differences in between-subject similarity, 3) relate the quantity of neuroanatomical outliers to cognitive performance and disease biomarkers, and 4) examine whether the number of outliers related to subsequent disease progression in people with mild cognitive impairment.

## Materials and methods

### Participants

Participants were derived from two datasets (1) a reference dataset comprised of healthy people across the human lifespan (2) a clinical target dataset which included people with Alzheimer’s disease or mild cognitive impairment in addition to age-matched cognitively normal controls. The reference dataset was made by combining data on healthy people from multiple publicly available sources, including Open Access Series of Imaging Studies (OASIS), Adolescent Brain Cognitive Development (ABCD) study and UK Biobank (UKB) 16,40,41, detailed in **Table 2** 39. The clinical data used in the preparation of this article was obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database; [http://adni.loni.usc.edu](http://adni.loni.usc.edu)). The ADNI was launched in 2003 as a public-private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD). Inclusion criteria were the availability of a baseline T1-weighted MRI, Alzheimer’s disease participants had to meet the National Institute of Neurological and Communicative Disorders and Stroke–Alzheimer’s Disease and Related Disorders Association criteria for probable Alzheimer’s disease 42, and were screened to exclude genetic risk for familial Alzheimer’s. MCI participants were reported a subjective memory concern either autonomously or via an informant or clinician, participants had no significant levels of impairment in other cognitive domains. Written informed consent was obtained from all participants before experimental procedures were performed.

### MRI acquisition

For the clinical dataset, T1-weighted images were acquired at multiple sites using 3T MRI scanners. Detailed MRI protocols for T1-weighted sequences by vendor are available online ([http://adni.loni.usc.edu/methods/documents/mri-protocols/](http://adni.loni.usc.edu/methods/documents/mri-protocols/)). The quality of raw scans was evaluated by University of California San Francisco prior to our exclusion criteria. Scans were excluded based on technical problems and significant motion artefacts and clinical abnormalities ([https://adni.bitbucket.io/reference/docs/UCSFFSX51/UCSF%20FreeSurfer%20Methods%20and%20QC_OFFICIAL.pdf](https://adni.bitbucket.io/reference/docs/UCSFFSX51/UCSF%20FreeSurfer%20Methods%20and%20QC_OFFICIAL.pdf)).

### Estimation of cortical thickness

T1-weighted scans from both the reference and ADNI datasets were processed using a mix of both FreeSurfer versions 5 and 6. Cortical thickness values were generated using the *recon-all* cross-sectional approach 43. This cortical thickness algorithm calculates the mean distance between vertices of a corrected, triangulated estimated grey/white matter surface and grey matter/CSF (pial) surface 44, which generated the cortical thickness of each region of the Destrieux atlas regions 45. This included both mean and median cortical thickness values and 148 regions cortical thickness values for each participant.

Quality control of FreeSurfer processing for the reference dataset relied on automated filtering median-centred absolute Euler number higher than 25, as we have done in prior work 38,39. The exclusion of outliers based on Euler numbers has been shown to be a reliable quality control strategy in large neuroimaging cohorts 46,47. For ADNI, quality control was based on visual review of each cortical region performed by UCSF. Only scans which passed this quality control were included in the dataset.

### Neuroanatomical Normative modelling

A hierarchical Bayesian regression model was trained on multi-site data to generate normative models per region using the covariates age and sex. This was based on the population variation in the reference dataset (training data), as per Kia and colleagues which adaptively pools parameter estimates across sites via a shared prior over regression parameters across sites 39. This simultaneously accounts for inter-site variation and allows sites to borrow strength from one another in a fully Bayesian framework. The advantage of training the models on the large independent dataset, compared to just using ADNI, is that ADNI consists of many sites with small sample sizes. This would result in unstable estimates of normative distributions that could be strongly influenced by outliers or sampling bias. Here, by training on over n=33,000 from only nine datasets (with 60 sites), the model produces much more stable distribution estimates across the entire lifespan. Next, these estimates were conditioned to our specific context, using an adapted transfer learning approach 39. The parameters of the reference normative model were recalibrated to the ADNI dataset using 70% of healthy controls per ADNI site, where 70% was used to give stable estimates of the transferred model parameters, given that many of the scan sites in ADNI have quite small sample sizes. The remaining 30% of healthy controls plus MCI and patients with Alzheimer’s disease were used to assess the heterogeneity in neuroanatomical presentation. This process generated regional and mean cortical thickness z-scores for each participant in the clinical dataset, relative to the normative range of the reference dataset. All modelling steps are performed using PCNToolkit (v0.20) ([https://github.com/amarquand/PCNtoolkit](https://github.com/amarquand/PCNtoolkit)).

### Statistical analysis

### Group cortical thickness differences

Cortical thickness group comparisons were conducted using t-tests at each region and corrected for multiple comparisons using the False Discovery Rate. Significant p-values were mapped onto the Destrieux atlas using the R package *ggseg* 48.

### Outlier definition and statistics

Outliers in terms of low cortical thickness were identified for each region, defined as Z < - 1.96. We only used the lower bound threshold for outliers as we were primarily interested in cortical thinning associated with neurodegeneration. Here the number of outliers were summed across 148 regions for each participant, to give a total outlier count across regions. Linear regression was used to test for group differences in mean cortical thickness Z-score and total outlier count. Additionally, group comparisons at *each* region were conducted using ANOVA and corrected for multiple comparisons using the False Discovery Rate. Hamming distance, a quantitative measure of similarity between binary thresholded cortical thickness outlier vectors was used to measure dissimilarity between individuals. Mean Hamming distances were then compared between groups.

Lastly, to explore spatial patterns of cortical thickness outliers per group, the proportion of participants within each group whose cortical thickness was an outlier (i.e., Z <-1.96) was calculated for each region. This enabled visualisation of the extent to which patterns of outlier regions overlap or are distinct. This was mapped using the Destrieux atlas via the R package *ggseg*. All statistical analyses were implemented in R version 3.6.2.

### Outlier associations with cognitive function and CSF markers

Linear regression adjusting for age and sex were used to examine the relationship between total outlier score and cognitive composite scores (memory or executive function) 49 or CSF markers (amyloid-beta and phospho-tau). Linear regression adjusting for age and sex were used to assess the effects of total outlier score, diagnostic group, and their interaction on memory and executive function, measured as ADNI MEM and ADNI EF scores, respectively 49. Conversely, we assessed the association between the CSF markers of amyloid-beta and phospho-tau and the total outlier score, adjusting for age and sex. We also use total scores from the Mini-Mental State Examination (MMSE) to stratify outlier maps in both MCI and patients with Alzheimer’s disease groups.

### MCI to Alzheimer’s disease conversion analysis

Follow-up diagnosis status data, up to three years from baseline scan, were obtained from 454 people with mild cognitive impairment. In total, 76 people with MCI at baseline had converted to Alzheimer’s disease within three years. We then ran a survival analysis using Cox proportional hazards regression to assess whether total outlier relating to the risk of converting from MCI to Alzheimer’s disease, controlling for age and sex. We use a Kaplan– Meier plot to illustrate how either a low or high outlier score (split via median) can contribute to the risk of converting.

### Data availability

Statistical analysis scripts are available on GitHub ([https://github.com/serenaverdi/ADNI_normative-modelling](https://github.com/serenaverdi/ADNI_normative-modelling)). The neuroanatomical normative model was generated using the PCNtoolkit software package ([https://github.com/amarquand/PCNtoolkit](https://github.com/amarquand/PCNtoolkit)). ADNI data used in this study are publicly available and can be requested following ADNI Data Sharing and Publications Committee guidelines: [http://adni.loni.usc.edu/data-samples/access-data/](http://adni.loni.usc.edu/data-samples/access-data/)

## Results

### Participants

In the reference dataset, a total of n=33,072 T1-weighted MRI scans were collated across 60 sites, which included males n= 16,170, females n=16,902, with a mean age of 40 years and a range of 8-97 years (This sample is described in detail in Kia et al 2021 and summarised in **Supplementary material, Table 1)** 39. The clinical ADNI dataset amounted to total 1,492 participants which were acquired across 62 sites **(Table 1)**. Here 70% of controls were removed from the clinical dataset and were used as a calibration dataset to adapt the normative model to the new sites, these were randomly selected and stratified across sites and gender to make sure all sites and genders are present in the adaptation set. This left a total of 1,027 participants in the final clinical dataset.

View this table:
[Table 1.](http://medrxiv.org/content/early/2022/07/03/2022.06.30.22277053/T1)

Table 1. Demographics of ADNI sample.
Statistical differences were assessed using ANOVA and chi-squared tests. *MMSE had maximum score of 30.

### Patients with Alzheimer’s disease have smaller cortical thicknesses than MCI and control groups

To understand mean effects across the cohort, mean cortical thicknesses were compared across participant groups. Age- and sex-adjusted mean cortical thickness significantly differed between groups overall [*F*(2,1487) = 137.9, *P* = 2.0 × 10−16]. Pairwise comparisons (Tukey post-hoc) were all significant [*P*<0.001], with mean cortical thickness being lowest in Alzheimer’s disease (mean = 2.28 SD = 0.13) and highest in controls (mean = 2.42 SD = 0.11), with MCI being intermediate (mean = 2.38, SD = 0.12) **(Supplementary material, Fig.1)**.

Region-level pairwise group comparisons (total of 148 regions) provided evidence cortical thickness measures were on average lower in 133 regions in Alzheimer’s versus cognitively normal controls, in 111 regions in Alzheimer’s versus MCI and in 78 regions in MCI versus cognitively normal controls, after false discovery rate correction **(Supplementary material, Fig.1)**

Next, cortical thickness z-scores, derived from comparison to the normative model, were then compared across participant groups. In this way, we could compare the degree to which each group differed from the separate reference cohort, used to define the normative model. Consistent with comparisons of mean cortical thickness, age- and sex-adjusted z-scores differed between groups overall [*F*(2, 1022) = 69.49), *P* = 2.0 × 10−16]. Pairwise comparisons (Tukey post-hoc) were all significant [*P*<=0.003], with Z-scores being lowest in Alzheimer’s disease (mean = -1.27, SD = 1.41), highest in controls (mean = 0.07, SD = 1.04), and intermediate in MCI (mean =-0.28, SD =1.17) (**Supplementary material, Fig.2 A**).

Furthermore, age- and sex-adjusted total outlier counts differed between groups overall [*F*(2, 1022) = 95.39), *P* = 2.0 x ×10−16]. Pairwise comparisons (Tukey post-hoc) were all significant [*P*<=0.003], with total outlier counts being highest in Alzheimer’s disease (median = 12, IQR = 28), lowest in controls (median = 2, IQR = 6) and intermediate in MCI (median = 4, IQR = 9) **(Supplementary material, Fig.2 B)**.

Region-level pairwise group comparisons (total of 148 regions) showed higher numbers of outliers in cortical thickness in 79 regions in Alzheimer’s versus cognitively normal controls, in 63 regions in Alzheimer’s versus MCI and 1 region in MCI versus cognitively normal controls, after false discovery rate correction. Region-level group differences in outlier count were mostly evident within temporoparietal and to a lesser extent frontal and occipital regions **(Fig.1, Panel A)**.

![Fig.1](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/03/2022.06.30.22277053/F1.medium.gif)

[Fig.1](http://medrxiv.org/content/early/2022/07/03/2022.06.30.22277053/F1)

Fig.1 Outlier regions
**(A):** Mapped are significant group differences of region outliers **(B)** Mapped are the percentage of region outliers proportional to participant group.

### Patients with Alzheimer’s disease are less similar to each other than people with MCI or with normal cognition

Hamming distance matrices indicated greater within-group dissimilarity in patients with Alzheimer’s disease, relative to MCI or control participants, who were most similar to each other in spatial patterns of outliers **(Fig.2)**.

![Fig.2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/03/2022.06.30.22277053/F2.medium.gif)

[Fig.2:](http://medrxiv.org/content/early/2022/07/03/2022.06.30.22277053/F2)

Fig.2: Outlier dissimilarity (A) Outlier distance heatmaps
Both x and y axes represent participants within each group. Yellow indicates higher hamming distance (greater dissimilarity between participants in this brain region), as opposed to if participants are identical in this brain region, the Hamming distance would be 0, represented by white in the colour bar **(B) Outlier distance density:** illustrates the spread of outlier dissimilarity (calculated by Hamming distance) within each group.

Median hamming distance significantly differed between groups overall, [*F*(2, 1024) = 209.42, *P* = 2.2×10−16]. Pairwise comparisons (Tukey post-hoc) were all significant [*P*<0.001], with being highest in Alzheimer’s disease (median = 32, IQR = 32), and lowest in controls (median = 6, IQR = 8), with MCI being intermediate (median =10, IQR = 14).

### Patients with Alzheimer’s disease have spatially higher proportions of cortical thickness outliers

The proportion of outliers defined within each group differed in regional patterns between Alzheimer’s, MCI and control groups. This is illustrated in **Fig.1, Panel B** and in **Supplementary material, Fig.3**. For a breakdown of proportions see **Supplementary material, Table 2**. A greater number of regions and a higher proportion of the group were outliers in patients with Alzheimer’s disease, as expected. In fact, 145 regions in the Alzheimer’s group had over the expected 2.5% of patients with an outlier (based on the Z < - 1.96 threshold). The left parahippocampal gyrus was the region with the highest outlier percentage (47% of the Alzheimer’s group). The second highest region outlier proportion was in the right parahippocampal gyrus (36% of the Alzheimer’s group). For the MCI group, 138 regions in the MCI group had outliers (over the expected 2.5% of the group). The left parahippocampal gyrus is the region with the highest outlier percentage (14% of the MCI group). The second highest region outlier proportion was in the right parahippocampal gyrus (12% of the MCI group). For cognitively normal controls, only 66 regions in the control group had outliers above the expected 2.5%. The left occipital temporal lateral sulcus was the region with the highest outlier percentage (6% of controls).

### Outliers are associated with cognitive function and CSF amyloid-beta and phosphorylated tau

Total outlier score across the whole sample was significantly associated with memory performance [*β* = -0.015, *P* = 2.2×10−16] and executive function [*β* = -0.02, *P* = 2.2×10−16] in a linear regression model. To check for Simpson’s paradox (the association between two variables within a sample), we also model a group by total outlier score interaction term, which was not significant for memory performance [*F*(2, 914) = 1.11, *P* = 0.329] but was for executive function [*F*(2, 914) = 4.04, *P* = 0.017] **(Fig.3, Panel A; B)**. Lower MMSE score showed different spatial patterns of outliers in both MCI (Fig.4, Panel A) and AD (Fig.4, Panel B) groups.

![Fig 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/03/2022.06.30.22277053/F3.medium.gif)

[Fig 3.](http://medrxiv.org/content/early/2022/07/03/2022.06.30.22277053/F3)

Fig 3. Total outlier count associations with cognitive function and CSF markers.
Fitted lines are from a linear regression model per diagnostics group for **(A)** Memory function, **(B)** Executive function, **(C)** CSF amyloid-beta, and **(D)** phospho-tau.

![Fig 4:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/03/2022.06.30.22277053/F4.medium.gif)

[Fig 4:](http://medrxiv.org/content/early/2022/07/03/2022.06.30.22277053/F4)

Fig 4: Outlier maps according to MMSE.
Mapped are the percentage of region outliers proportional to MMSE scoring subgroup in **(A)** MCI participants **(B)** Alzheimer’s disease participants.

In addition, total outlier score was significantly associated with amyloid-beta [*β* = 0.002, *P* = 0.022] and phospho-tau [*β* = 0.1301, *P* = 1.04×10-8], which was not influenced by either group amyloid-beta [*F*(2, 576) = 0.96, *P* = 0.38] or phospho-tau interaction [*F*(2, 576) = 1.362, *P* = 0.257] **(Fig.3, Panel C; D)**.

### Variability in cortical thickness is not solely due to disease stage or other clinical factors

To explore whether individual differences in outlier maps was driven by disease-related characteristics (such as ApoE genotype, demographics) or by disease stage, we examined sets of participants closely matched for ApoE genotype status, age, sex and MMSE score. **Fig.5** presents four individual female patients with Alzheimer’s all aged 71-72 years, heterozygous for ApoE ε4, with similar MMSE scores, all of whom were amyloid-positive on CSF. These individual patients might be considered similar from biological or clinical perspectives, yet their patterns of outliers in cortical thickness are markedly variable; for example, variously suggesting lateralised (Patient 3) and occipital atrophy (Patient 1). However, MMSE score and age does explain some of the variance in total outlier score [adjusted *R**2*= 0.1942, *P* = 2.2×10−16]. Conversely, similar patterns are present within precuneus/post cingulate regions.

![Fig. 5:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/03/2022.06.30.22277053/F5.medium.gif)

[Fig. 5:](http://medrxiv.org/content/early/2022/07/03/2022.06.30.22277053/F5)

Fig. 5: Outlier maps of individuals with similar disease-related characteristics.
Outlier regions mapped for individual participants, matched on diagnostic group, sex, age, ApoE ε4, genotype and MMSE score.

![Movie.1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/03/2022.06.30.22277053/F6.medium.gif)

[Movie.1:](http://medrxiv.org/content/early/2022/07/03/2022.06.30.22277053/F6)

Movie.1: Individual outlier maps (available to play in separate file).
Each regional outlier map looping through each participant of this study

### Greater numbers of outliers are associated with risk of conversion from mild cognitive impairment to Alzheimer’s Disease

A survival analysis indicated that for every 10 points of total outlier score, risk of converting from MCI to Alzheimer’s disease within 3 years increased by 31.4% (HR = 1.028, 95% CI [1.016,1.039], *P* = 1.8×10−16) **(Fig.6, Panel A)**. This is illustrated within a Kaplan–Meier plot, which shoes how a high outlier score can contribute to the risk of converting in comparison to a low outlier score **(Fig.6, Panel B)**.

![Fig.6:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/03/2022.06.30.22277053/F7.medium.gif)

[Fig.6:](http://medrxiv.org/content/early/2022/07/03/2022.06.30.22277053/F7)

Fig.6: Conversion from mild cognitive impairment to Alzheimer’s Disease
**(A)** Kaplan–Meier plot of MCI to Alzheimer’s disease conversion: the two lines represent a median split of total outlier score, with <4 classed as low outlier score (blue), and ≥4 classed as high outlier score (red). Crosses indicate censoring points (i.e., age at last diagnosis assessment). Filled colour represent the 95% confidence intervals. **(B)** Mapped are the proportion of regional outliers among people with MCI who converted Alzheimer’s disease.

## Discussion

In this study, we defined individual spatial patterns of cortical thickness outliers using a novel neuroanatomical normative modelling approach. Applying this approach to cognitively normal controls, MCI participants and patients with Alzheimer’s disease illustrated that Alzheimer’s disease does not affect different people in a uniform way. Moreover, our analysis provides a way to quantify and visualise these individual differences in patterns of cortical atrophy. The total outlier score (the sum of the total number of outliers an individual has), provides an individualised marker of cortical atrophy for subsequent analyses. Overall, these results provide evidence of, 1) heterogeneous patterns of cortical thickness between patients with Alzheimer’s disease, 2) associations of cortical atrophy with cognitive performance and CSF amyloid-beta and phospho-tau, and 3) the potential of individualised markers of cortical atrophy to predict survival time before conversion from the MCI stage to diagnosed Alzheimer’s disease.

The current findings align with the established understanding of Alzheimer’s disease. We observed a high total outlier count in patients with Alzheimer’s disease, consistent with the evidence of cortical thinning as a consequence of Alzheimer’s disease neuropathology 50. Moreover, neuroimaging literature has shown that cortical thinning is significantly associated with poor cognitive performance 51,52, a decrease in CSF amyloid-beta and an increase in CSF phospho-tau 53, which is also consistent with the present study. Atrophy has also been associated with the risk of progression from MCI to Alzheimer’s disease 54, alongside a combination of other biomarkers 55. Importantly, all these previous studies examined the correlates of common patterns of cortical atrophy. Conversely, here we were able to specifically consider individual variability in patterns of cortical thickness. This offers precise, personalised neuroanatomical information, as opposed to assessing group average relationships. Potentially, the total outlier score could be used as an individual patient marker of poor brain health. Indeed, related measures have recently been adopted as a clinical measure, i.e., brain volume/thickness patient Z-scores, but has been calculated using different normative modelling techniques 56,57. However, these techniques have tended to base their normative population on relatively smaller reference samples; have limited normative modelling pipelines to just whole brain, or within specific regions; did not fully account for site related variation (i.e., site effects) and did not fully relate these to clinical outcomes and cognitive scores

We evaluated group differences in cortical thickness outliers at 148 brain regions, to explore whether consistent neuroanatomical patterns were present across individuals. We observed more outliers in patients with Alzheimer’s disease in temporal regions such as the hippocampus and the cingulate cortex. These are areas known to be sensitive to neurodegeneration in Alzheimer’s disease 58, and are responsible for clinical symptoms in Alzheimer’s disease 59,60. However, our approach enabled us to go beyond these group-average regional differences. In the Alzheimer’s disease group, the highest proportion of outliers in a single region was less than 50%. This suggests the individual spatial patterns of outliers in Alzheimer’s only partially overlap between patients; if atrophy were homogenous, we might expect 100% of outliers to be here. Heterogeneous atrophy in the temporal lobe is consistent with subtyping studies 10,61. Previous efforts have estimated the extent to which multiple distinct atrophy patterns were expressed within each participant, which includes temporal atrophy 62. This considerably extends efforts made using autoencoders to calculate individual deviation score in a dataset which includes ADNI patients – here medial temporal regions were also susceptible to scores deviating from the norm 27.

Going further, our study reveals that *each* patient not only differs in the number of outliers they have, but the regional patterns of outliers differ markedly. **(Movie. 1)**. The latter is reflected in large levels of dissimilarity between Alzheimer’s disease individuals **(Fig. 2)**. Potentially, one reason for the variable patterns of atrophy is simply disease stage, whereby more atrophy appears with greater disease progression. However, our results indicate that this is not the case, as when closely examining patients of very similar demographics and clinical characteristics, being at a comparable disease stage (e.g., based on MMSE score), heterogenous patterns of cortical atrophy were still present. Crucially, our approach does not preclude the presence of subtypes, although the Z-score maps can be used as input to a cluster algorithm or similar. However, the goal here was to go beyond subtypes, producing informative spatial maps at the individual level.

We also found that lower composite scores of both memory and executive function were significantly associated with higher total outlier score **(Fig. 3)**. While this is consistent with previous studies showing relationships between neuroanatomical variation and cognitive performance,10,63,64 this also suggests that individualised measures of neuroanatomy, rather than relying on typical regions (e.g., hippocampus), are sensitive to cognitive function. Potentially, this approach of deriving individualised neuroanatomical measures could benefit clinical trials, where therapeutic effects could be detected across the whole brain, instead of just pre-identified regions of interest.

Cognitively normal controls also showed some outliers suggesting a degree of within-group heterogeneity in unaffected people. The implication of this is that the assumption of homogeneity in case-control studies should be made with caution, even in control groups. Statistical designs for basic research and for clinical trials should better reflect this heterogeneity in brain structure. Although the reference dataset includes over 30,000 individuals, it is still not truly representative of a healthy population 65. Also, patients who volunteer for research studies do not necessarily reflect the clinical population; the threshold for contraindications for research scans is higher than clinical scans. Future neuroanatomical normative modelling studies could supplement the reference dataset with MRI scans acquired from routine clinical visits or other less selective sources.

There are some limitations to the current study. The sample taken from this study is cross-sectional, reflecting a snapshot in time, yet heterogeneity has been shown to differ temporally 66. Therefore, it will be important for future studies to incorporate serial neuroimaging to define patient-level longitudinal trajectories. In turn, mapping neuroanatomical variability using neuroanatomical normative modelling at different time points has the potential to improve predictions of disease progression or treatment response, at the level of the individual patient. Potentially, data-driven staging methods (e.g., SusStain 21) may provide clinically useful information of longitudinal trends of individual heterogeneity whilst taking account of an individual’s disease stage.

The demographic composition ADNI is another issue. In particular, ADNI is comprised of more early-stage dementia participants 67. Examining late-stage patients with Alzheimer’s disease may offer interesting insights into the heterogeneity in spatial patterns of atrophy across the disease course. Clinical observations have suggested that late-stage patients with Alzheimer’s disease have widespread atrophy across the brain; therefore, we may hypothesise such patients will have less heterogenous patterns of atrophy. However, regardless of the heterogenous *patterns* of atrophy, the total outlier score can still provide information about the extent of cortical atrophy in a given individual.

Another limitation of ADNI is the limited representation of cognitive domains beyond memory, executive function and language. Between a quarter to a third of the Alzheimer’s disease group exhibit parieto-occipital outliers, comparable to separate parieto-occipital predominant subtypes associated with prominent visuospatial dysfunction 18, Further characterisation of how outlier distribution relates to non-memory/executive symptoms may be of particular clinical relevance, for example given the implications of visuospatial dysfunction for diminished autonomy, falls risk and appropriate services 24,68,69.

The current approach omitted subcortical and cerebellar regions and could be extended to encompass more of the brain. However, we selected the Destrieux atlas was selected as it contains a higher number of cortical parcellations compared to alternatives in FreeSurfer, to afford better spatial resolution 45. Importantly, FreeSurfer is widely used making it easier to amass a large reference dataset of processed data, and readily enabling other research groups to employ the same neuroanatomical normative modelling approach.

## Conclusion

Alzheimer’s disease affects different patients’ brains in different ways. This neuroanatomical heterogeneity aligns with common observations of clinical heterogeneity. While perhaps intuitive, this fact is often overlooked in neuroimaging research and clinical trial design. We provide a quantitative approach to estimate this variability at the individual patient level based on spatial patterns of an individual’s cortical thickness normative deviations. Individualised maps of neuroanatomical outliers were related to cognitive performance and CSF biomarkers. Furthermore, the number of outliers, based on individual patterns, helped predict conversion from MCI to Alzheimer’s disease. These individual neuroanatomical maps, derived from normative models, have the potential to be a marker of Alzheimer’s disease state. These could index disease progression or even to evaluate the effectiveness of potential disease-modifying treatments, tailored to the individual patient.

## Supporting information

Supplementary Material [[supplements/277053_file04.docx]](pending:yes)

## Data Availability

Statistical analysis scripts are available on GitHub (https://github.com/serenaverdi/ADNI_normative-modelling). The neuroanatomical normative model was generated using the PCNtoolkit software package (https://github.com/amarquand/PCNtoolkit). ADNI data used in this study are publicly available and can be requested following ADNI Data Sharing and Publications Committee guidelines: http://adni.loni.usc.edu/data-samples/access-data/

## Funding

This work was supported by the EPSRC-funded UCL Centre for Doctoral Training in Intelligent, Integrated Imaging in Healthcare (i4health) (EP/S021930/1) and the Department of Health’s National Institute for Health Research funded University College London Hospitals Biomedical Research Centre. In addition, A.F.M. gratefully acknowledges funding from the Dutch Organization for Scientific Research via a VIDI fellowship (grant number 016.156.415); J.M.S. acknowledges the support of Alzheimer’s Research UK, Brain Research UK, Weston Brain Institute, Medical Research Council and the British Heart Foundation.

K.Y. is an Etherington PCA Senior Research Fellow and is funded by the Alzheimer’s Society, grant number 453 (AS-JF-18-003).

## Competing interests

The authors report no competing interests.

## Supplementary material

Supplementary material is available in a separate document.

## Acknowledgements

Data collection and sharing for this project was funded by ADNI (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defence award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.;Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.;Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (*[www.fnih.org](http://www.fnih.org)*). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

## Abbreviations

ADNI =
:   Alzheimer’s Disease Neuroimaging Initiative;
AD =
:   Alzheimer’s disease;
ApoE =
:   Apolipoprotein E;
CSF=
:   Cerebrospinal fluid;
CN =
:   cognitively normal;
MCI =
:   Mild Cognitive impairment;
MMSE=
:   Mini-Mental State Examination

*   Received June 30, 2022.
*   Revision received June 30, 2022.
*   Accepted July 3, 2022.


*   © 2022, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  1.Richards M, Brayne C. What do we mean by Alzheimer’s disease? BMJ. 2010;341(7778):865–867. doi:10.1136/BMJ.C4670
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1136/BMJ.C4670&link_type=DOI) 

2.  2.Cohen-Mansfield J. Heterogeneity in dementia: Challenges and opportunities. Alzheimer Disease and Associated Disorders. 2000;14(2):60–63. doi:10.1097/00002093-200004000-00002
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/00002093-200004000-00002&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10850743&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000087424800002&link_type=ISI) 

3.  3.Ringman JM, Goate A, Masters CL, et al. Genetic Heterogeneity in Alzheimer Disease and Implications for Treatment Strategies. Current Neurology and Neuroscience Reports. 2014;14(11). doi:10.1007/s11910-014-0499-8
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11910-014-0499-8&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25217249&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

4.  4.Frisoni GB, Altomare D, Thal DR, et al. The probabilistic model of Alzheimer disease: the amyloid hypothesis revised. Nat Rev Neurosci. 2022;23(1):53–66. doi:10.1038/S41583-021-00533-W
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/S41583-021-00533-W&link_type=DOI) 

5.  5.Ryan J, Fransquet P, Wrigglesworth J, Lacaze P. Phenotypic heterogeneity in dementia: A challenge for epidemiology and biomarker studies. Frontiers in Public Health. 2018;6. doi:10.3389/fpubh.2018.00181
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3389/fpubh.2018.00181&link_type=DOI) 

6.  6.Goyal D, Tjandra D, Migrino RQ, Giordani B, Syed Z, Wiens J. Characterizing heterogeneity in the progression of Alzheimer’s disease using longitudinal clinical and neuroimaging biomarkers. Alzheimer’s and Dementia: Diagnosis, Assessment and Disease Monitoring. 2018;10:629–637. doi:10.1016/j.dadm.2018.06.007
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.dadm.2018.06.007&link_type=DOI) 

7.  7.Pais M, Loureiro J, Do Vale V, et al. Heterogeneity of Cerebrospinal Fluid Biomarkers Profiles in Individuals with Distinct Levels of Cognitive Decline: A Cross-Sectional Study. Journal of Alzheimer’s Disease. 2021;81(3):949–962. doi:10.3233/JAD-210144
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3233/JAD-210144&link_type=DOI) 

8.  8.Bunn F, Burn AM, Goodman C, et al. Comorbidity and dementia: A scoping review of the literature. BMC Medicine. 2014;12(1). doi:10.1186/s12916-014-0192-4
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12916-014-0192-4&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25358236&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

9.  9.Santiago JA, Potashkin JA. The Impact of Disease Comorbidities in Alzheimer’s Disease. Frontiers in Aging Neuroscience. 2021;13:38. doi:10.3389/FNAGI.2021.631770/BIBTEX
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3389/FNAGI.2021.631770/BIBTEX&link_type=DOI) 

10. 10.Dong A, Toledo JB, Honnorat N, et al. Heterogeneity of neuroanatomical patterns in prodromal Alzheimer’s disease: links to cognition, progression and biomarkers. Brain. 2017;140(3):735–747. doi:10.1093/brain/aww319
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/brain/aww319&link_type=DOI) 

11. 11.Mehta RI, Schneider JA. What is “Alzheimer’s disease”? The neuropathological heterogeneity of clinically defined Alzheimer’s dementia. Curr Opin Neurol. 2021;34(2):237–245. doi:10.1097/WCO.0000000000000912
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/wco.0000000000000912&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

12. 12.Young PNE, Estarellas M, Coomans E, et al. Imaging biomarkers in neurodegeneration: Current and future practices. Alzheimer’s Research and Therapy. 2020;12(1):1–17. doi:10.1186/S13195-020-00612-7/FIGURES/4
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/S13195-020-00612-7/FIGURES/4&link_type=DOI) 

13. 13.Dementia. Dementia: Assessment, management and support for people living with dementia and their carers. Published online 2018.
    
    

14. 14.Weiner MW, Aisen PS, Jack CR, et al. The Alzheimer’s Disease Neuroimaging Initiative: Progress report and future plans. Alzheimer’s and Dementia. 2010;6(3). doi:10.1016/j.jalz.2010.03.007
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jalz.2010.03.007&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20451868&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000277931700002&link_type=ISI) 

15. 15.Marek K, Chowdhury S, Siderowf A, et al. The Parkinson’s progression markers initiative (PPMI) – establishing a PD biomarker cohort. Annals of Clinical and Translational Neurology. 2018;5(12):1460–1477. doi:10.1002/acn3.644
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/acn3.644&link_type=DOI) 

16. 16.LaMontagne P, Benzinger T, Morris J, et al. OASIS-3: Longitudinal Neuroimaging, Clinical, and Cognitive Dataset for Normal Aging and Alzheimer Disease. medRxiv. Published online 2019. doi:10.1101/2019.12.13.19014902
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAxOS4xMi4xMy4xOTAxNDkwMnYxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMDcvMDMvMjAyMi4wNi4zMC4yMjI3NzA1My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

17. 17.Habes M, Grothe MJ, Tunc B, McMillan C, Wolk DA, Davatzikos C. Disentangling Heterogeneity in Alzheimer’s Disease and Related Dementias Using Data-Driven Methods. Biological Psychiatry. 2020;88(1):70–82. doi:10.1016/j.biopsych.2020.01.016
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.biopsych.2020.01.016&link_type=DOI) 

18. 18.en Kate M, Dicks E, Visser PJ, et al. Atrophy subtypes in prodromal Alzheimer’s disease are associated with cognitive decline. Brain. 2018;141(12):3443–3456. doi:10.1093/brain/awy264
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/brain/awy264&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

19. 19.Varol E, Sotiras A, Davatzikos C. HYDRA: Revealing heterogeneity of imaging and genetic patterns through a multiple max-margin discriminative analysis framework. Neuroimage. 2017;145:346–364. doi:10.1016/J.NEUROIMAGE.2016.02.041
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.neuroimage.2016.02.041&link_type=DOI) 

20. 20.Wen J, Varol E, Sotiras A, et al. Multi-scale semi-supervised clustering of brain images: Deriving disease subtypes. Medical Image Analysis. 2022;75:102304. doi:10.1016/J.MEDIA.2021.102304
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.media.2021.102304&link_type=DOI) 

21. 21.Young AL, Marinescu R v., Oxtoby NP, et al. Uncovering the heterogeneity and temporal complexity of neurodegenerative diseases with Subtype and Stage Inference. Nature Communications. 2018;9(1). doi:10.1038/s41467-018-05892-0
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41467-018-05892-0&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

22. 22.Lam B, Masellis M, Freedman M, Stuss DT, Black SE. Clinical, imaging, and pathological heterogeneity of the Alzheimer’s disease syndrome. Alzheimer’s Research and Therapy. 2013;5(1). doi:10.1186/alzrt155
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/alzrt155&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23302773&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

23. 23.Zhang B, Lin L, Wu S. A Review of Brain Atrophy Subtypes Definition and Analysis for Alzheimer’s Disease Heterogeneity Studies. Journal of Alzheimer’s Disease. 2021;80(4):1339–1352. doi:10.3233/JAD-201274
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3233/JAD-201274&link_type=DOI) 

24. 24.Graff-Radford J, Yong KXX, Apostolova LG, et al. New insights into atypical Alzheimer’s disease in the era of biomarkers. Lancet Neurol. 2021;20(3):222–234. doi:10.1016/S1474-4422(20)30440-3
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1474-4422(20)30440-3&link_type=DOI) 

25. 25.Mohanty R, Mårtensson G, Poulakis K, et al. Comparison of subtyping methods for neuroimaging studies in Alzheimer’s disease: a call for harmonization. Brain Communications. 2020;2(2). doi:10.1093/BRAINCOMMS/FCAA192
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/BRAINCOMMS/FCAA192&link_type=DOI) 

26. 26.Marquand AF, Wolfers T, Mennes M, Buitelaar J, Beckmann CF. Beyond Lumping and Splitting: A Review of Computational Approaches for Stratifying Psychiatric Disorders. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging. 2016;1(5):433–447. doi:10.1016/j.bpsc.2016.04.002
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.bpsc.2016.04.002&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27642641&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

27. 27.Pinaya WHL, Scarpazza C, Garcia-Dias R, et al. Using normative modelling to detect disease progression in mild cognitive impairment and Alzheimer’s disease in a cross-sectional multi-cohort study. Scientific Reports 2021 11:1. 2021;11(1):1–13. doi:10.1038/s41598-021-95098-0
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41598-021-95098-0&link_type=DOI) 

28. 28.Marquand AF, Kia SM, Zabihi M, Wolfers T, Buitelaar JK, Beckmann CF. Conceptualizing mental disorders as deviations from normative functioning. Molecular Psychiatry. 2019;24(10):1415–1424. doi:10.1038/s41380-019-0441-1
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41380-019-0441-1&link_type=DOI) 

29. 29.Marquand AF, Rezek I, Buitelaar J, Beckmann CF. Understanding Heterogeneity in Clinical Cohorts Using Normative Models: Beyond Case-Control Studies. Biological Psychiatry. 2016;80(7):552–561. doi:10.1016/j.biopsych.2015.12.023
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.biopsych.2015.12.023&link_type=DOI) 

30. 30.Rutherford S, Kia SM, Wolfers T, et al. The Normative Modeling Framework for Computational Psychiatry. Published online 2021. doi:10.1101/2021.08.08.455583
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYmlvcnhpdiI7czo1OiJyZXNpZCI7czoxOToiMjAyMS4wOC4wOC40NTU1ODN2MiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzA3LzAzLzIwMjIuMDYuMzAuMjIyNzcwNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

31. 31.Rutherford S, Fraza C, Dinga R, et al. Charting brain growth and aging at high spatial precision. Elife. 2022;11. doi:10.7554/ELIFE.72904
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7554/ELIFE.72904&link_type=DOI) 

32. 32.Wolfers T, Beckmann CF, Hoogman M, Buitelaar JK, Franke B, Marquand AF. Individual differences v. the average patient: Mapping the heterogeneity in ADHD using normative models. Psychological Medicine. 2019;50(2):314–323. doi:10.1017/S0033291719000084
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1017/S0033291719000084&link_type=DOI) 

33. 33.Zabihi M, Oldehinkel M, Wolfers T, et al. Dissecting the Heterogeneous Cortical Anatomy of Autism Spectrum Disorder Using Normative Models. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging. 2019;4(6):567–578. doi:10.1016/j.bpsc.2018.11.013
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.bpsc.2018.11.013&link_type=DOI) 

34. 34.Bethlehem RAI, Seidlitz J, Romero-Garcia R, Trakoshis S, Dumas G, Lombardo M V. A normative modelling approach reveals age-atypical cortical thickness in a subgroup of males with autism spectrum disorder. Communications Biology. 2020;3(1). doi:10.1038/s42003-020-01212-9
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s42003-020-01212-9&link_type=DOI) 

35. 35.Wolfers T, Doan NT, Kaufmann T, et al. Mapping the Heterogeneous Phenotype of Schizophrenia and Bipolar Disorder Using Normative Models. JAMA Psychiatry. 2018;75(11):1146–1155. doi:10.1001/jamapsychiatry.2018.2467
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamapsychiatry.2018.2467&link_type=DOI) 

36. 36.Verdi S, Marquand AF, Schott JM, Cole JH. Beyond the average patient: how neuroimaging models can address heterogeneity in dementia. Brain. 2021;144(10):2946–2953. doi:10.1093/BRAIN/AWAB165
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/BRAIN/AWAB165&link_type=DOI) 

37. 37.M Bayer JM, Dinga R, Mostafa Kia S, et al. Accommodating Site Variation in Neuroimaging Data Using Hierarchical and Bayesian Models a Preprint. bioRxiv. Published online 2021:2021.02.09.430363.
    
    

38. 38.Kia SM, Huijsdens H, Dinga R, et al. Hierarchical Bayesian Regression for Multi-site Normative Modeling of Neuroimaging Data. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2020;12267 LNCS:699-709. doi:10.1007/978-3-030-59728-3_68/FIGURES/4
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/978-3-030-59728-3_68/FIGURES/4&link_type=DOI) 

39. 39.Kia S, Huijsdens H, Rutherford S, et al. Federated Multi-Site Normative Modeling using Hierarchical Bayesian Regression. Published online 2021. doi:10.1101/2021.05.28.446120
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYmlvcnhpdiI7czo1OiJyZXNpZCI7czoxOToiMjAyMS4wNS4yOC40NDYxMjB2MSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzA3LzAzLzIwMjIuMDYuMzAuMjIyNzcwNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

40. 40.Casey BJ, Cannonier T, Conley MI, et al. The Adolescent Brain Cognitive Development (ABCD) study: Imaging acquisition across 21 sites. Developmental Cognitive Neuroscience. Published online 2018. doi:10.1016/j.dcn.2018.03.001
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.dcn.2018.03.001&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=29567376&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

41. 41.Littlejohns TJ, Holliday J, Gibson LM, et al. The UK Biobank imaging enhancement of 100,000 participants: rationale, data collection, management and future directions. Nature Communications. Published online 2020. doi:10.1038/s41467-020-15948-9
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41467-020-15948-9&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32457287&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

42. 42.McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of alzheimer’s disease: Report of the NINCDS-ADRDA work group⋆ under the auspices of department of health and human services task force on alzheimer’s disease. Neurology. 1984;34(7):939–944. doi:10.1212/wnl.34.7.939
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1212/WNL.34.7.939&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=6610841&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

43. 43.Fischl B. FreeSurfer. Neuroimage. Published online 2012. doi:10.1016/j.neuroimage.2012.01.021
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.neuroimage.2012.01.021&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22248573&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000306390600031&link_type=ISI) 

44. 44.Fischl B, Dale AM. Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proc Natl Acad Sci U S A. Published online 2000. doi:10.1073/pnas.200033797
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoicG5hcyI7czo1OiJyZXNpZCI7czoxMToiOTcvMjAvMTEwNTAiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8wNy8wMy8yMDIyLjA2LjMwLjIyMjc3MDUzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

45. 45.Destrieux C, Fischl B, Dale A, Halgren E. Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. Neuroimage. 2010;53(1):1. doi:10.1016/J.NEUROIMAGE.2010.06.010
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.neuroimage.2010.06.010&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20547229&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000280818900001&link_type=ISI) 

46. 46.Dale AM, Fischl B, Sereno MI. Cortical surface-based analysis: I. Segmentation and surface reconstruction. Neuroimage. Published online 1999. doi:10.1006/nimg.1998.0395
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1006/nimg.1998.0395&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9931268&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000078608900001&link_type=ISI) 

47. 47.Monereo Sánchez JA, de Jong A JJ, Drenthen A GS, et al. Quality control strategies for brain MRI segmentation and parcellation: practical approaches and recommendations - insights from The Maastricht Study. bioRxiv. Published online 2021.
    
    

48. 48.Mowinckel AM, Vidal-Piñeiro D. Visualisation of Brain Statistics with R-packages ggseg and ggseg3d. arXiv. Published online 2019. doi:10.1177/2515245920928009
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/2515245920928009&link_type=DOI) 

49. 49.Gibbons LE, Carle AC, Mackin RS, et al. A composite score for executive functioning, validated in Alzheimer’s Disease Neuroimaging Initiative (ADNI) participants with baseline mild cognitive impairment. Brain Imaging Behav. 2012;6(4):517–527. doi:10.1007/S11682-012-9176-1
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11682-012-9176-1&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22644789&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

50. 50.Julkunen V, Niskanen E, Koikkalainen J, et al. Differences in cortical thickness in healthy controls, subjects with mild cognitive impairment, and Alzheimer’s disease patients: A longitudinal study. Journal of Alzheimer’s Disease. Published online 2010. doi:10.3233/JAD-2010-100114
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3233/JAD-2010-100114&link_type=DOI) 

51. 51.Mouton PR, Martin LJ, Calhoun ME, Dal Forno G, Price DL. Cognitive decline strongly correlates with cortical atrophy in Alzheimer’s dementia. Neurobiol Aging. 1998;19(5):371–377. doi:10.1016/S0197-4580(98)00080-3
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0197-4580(98)00080-3&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9880038&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000077725500001&link_type=ISI) 

52. 52.Bakkour A, Morris JC, Wolk DA, Dickerson BC. The effects of aging and Alzheimer’s disease on cerebral cortical anatomy: Specificity and differential relationships with cognition. Neuroimage. 2013;76:332–344. doi:10.1016/J.NEUROIMAGE.2013.02.059
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/J.NEUROIMAGE.2013.02.059&link_type=DOI) 

53. 53.Fortea J, Vilaplana E, Alcolea D, et al. Cerebrospinal fluid β-amyloid and phospho-tau biomarker interactions affecting brain structure in preclinical Alzheimer disease. Annals of Neurology. 2014;76(2):223–230. doi:10.1002/ANA.24186
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/ana.24186&link_type=DOI) 

54. 54.Querbes O, Aubry F, Pariente J, et al. Early diagnosis of Alzheimer’s disease using cortical thickness: impact of cognitive reserve. Brain. 2009;132(8):2036–2047. doi:10.1093/BRAIN/AWP105
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/brain/awp105&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19439419&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000268330500005&link_type=ISI) 

55. 55.Popescu SG, Whittington A, Gunn RN, et al. Nonlinear biomarker interactions in conversion from mild cognitive impairment to Alzheimer’s disease. Human Brain Mapping. 2020;41(15):4406–4418. doi:10.1002/HBM.25133
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/HBM.25133&link_type=DOI) 

56. 56.Goodkin O, Pemberton H, Vos SB, et al. The quantitative neuroradiology initiative framework: Application to dementia. British Journal of Radiology. Published online 2019. doi:10.1259/bjr.20190365
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1259/bjr.20190365&link_type=DOI) 

57. 57.Stelmokas J, Yassay L, Giordani B, et al. Translational MRI Volumetry with NeuroQuant: Effects of Version and Normative Data on Relationships with Memory Performance in Healthy Older Adults and Patients with Mild Cognitive Impairment. Journal of Alzheimer’s Disease. 2017;60(4):1499–1510. doi:10.3233/JAD-170306
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3233/JAD-170306&link_type=DOI) 

58. 58.Popuri K, Ma D, Wang L, Beg MF. Using machine learning to quantify structural MRI neurodegeneration patterns of Alzheimer’s disease into dementia score: Independent validation on 8,834 images from ADNI, AIBL, OASIS, and MIRIAD databases. Human Brain Mapping. Published online 2020. doi:10.1002/hbm.25115
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/hbm.25115&link_type=DOI) 

59. 59.Leech R, Sharp DJ. The role of the posterior cingulate cortex in cognition and disease. Brain. Published online 2014. doi:10.1093/brain/awt162
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/brain/awt162&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23869106&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000330434000006&link_type=ISI) 

60. 60.Jack CR, Petersen RC, Xu Y, et al. Rates of hippocampal atrophy correlate with change in clinical status in aging and AD. Neurology. Published online 2000. doi:10.1212/wnl.55.4.484
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1212/wnl.55.4.484&link_type=DOI) 

61. 61.Noh Y, Jeon S, Lee JM, et al. Anatomical heterogeneity of Alzheimer disease Based on cortical thickness on MRIs. Neurology. 2014;83(21):1936–1944. doi:10.1212/WNL.0000000000001003
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1212/WNL.0000000000001003&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25344382&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

62. 62.Zhang X, Mormino EC, Sun N, Sperling RA, Sabuncu MR, Yeo BTT. Bayesian model reveals latent atrophy factors with dissociable cognitive trajectories in Alzheimer’s disease. Proc Natl Acad Sci U S A. 2016;113(42):E6535–E6544. doi:10.1073/pnas.1611073113
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoicG5hcyI7czo1OiJyZXNpZCI7czoxMjoiMTEzLzQyL0U2NTM1IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMDcvMDMvMjAyMi4wNi4zMC4yMjI3NzA1My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

63. 63.Park JY, Na HK, Kim S, et al. Robust Identification of Alzheimer’s Disease subtypes based on cortical atrophy patterns. Scientific Reports. 2017;7. doi:10.1038/srep43270
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/srep43270&link_type=DOI) 

64. 64.Sun N, Mormino EC, Chen J, Sabuncu MR, Yeo BTT. Multi-modal latent factor exploration of atrophy, cognitive and tau heterogeneity in Alzheimer’s disease. Neuroimage. 2019;201. doi:10.1016/j.neuroimage.2019.116043
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.neuroimage.2019.116043&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31344486&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 

65. 65.Varoquaux G, Cheplygina V. How I failed machine learning in medical imaging -- shortcomings and recommendations. 2021;(Section III):1–10.
    
    

66. 66.Ferreira D, Verhagen C, Hernández-Cabrera JA, et al. Distinct subtypes of Alzheimer’s disease based on patterns of brain atrophy: Longitudinal trajectories and clinical applications. Scientific Reports. Published online 2017. doi:10.1038/srep46263
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/srep46263&link_type=DOI) 

67. 67.Aisen PS, Petersen RC, Donohue MC, et al. Clinical core of the Alzheimer’s Disease Neuroimaging Initiative: progress and plans. Alzheimers Dement. 2010;6(3):239. doi:10.1016/J.JALZ.2010.03.006
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jalz.2010.03.006&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20451872&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000277931700006&link_type=ISI) 

68. 68.Martin K, Thomson R, Blizzard L, Wood A, Garry M, Srikanth V. Visuospatial ability and memory are associated with falls risk in older people: a population-based study. Dement Geriatr Cogn Disord. 2009;27(5):451–457. doi:10.1159/000216840
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1159/000216840&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19420939&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000266098300007&link_type=ISI) 

69. 69.Perry RJ, Hodges JR. Relationship between functional and neuropsychological performance in early Alzheimer disease. Alzheimer Dis Assoc Disord. 2000;14(1):1–10. doi:10.1097/00002093-200001000-00001
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/00002093-200001000-00001&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10718199&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F03%2F2022.06.30.22277053.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000085675200001&link_type=ISI)