Abstract
Connected speech samples elicited by a picture description task are widely used in the assessment of aphasias, but it is not clear what their interpretation should focus on. Although such samples are easy to collect, analyses of them tend to be time-consuming, inconsistently conducted, and impractical for non-specialist settings. Here, we analysed connected speech samples from patients with the three variants of primary progressive aphasia (semantic, svPPA N = 9; logopenic, lvPPA N = 9; non-fluent, nfvPPA N = 9), progressive supranuclear palsy (PSP Richardson’s syndrome N = 10), corticobasal syndrome (CBS N = 13), and age-matched healthy controls. There were three principal aims: (i) to determine the differences in quantitative language output and psycholinguistic properties of words produced by patients and controls; (ii) to identify the neural correlates of connected speech measures; and (iii) to develop a simple clinical measurement tool. Using data-driven methods, we optimised a 15-word checklist for use with the Boston Diagnostic Aphasia Examination ‘cookie theft’ and Mini Linguistic State Examination ‘beach scene’ pictures and tested the predictive validity of outputs from Least Absolute Shrinkage and Selection Operator (LASSO) models using an independent clinical sample from a second site. The total language output was significantly reduced in patients with nfvPPA, PSP and CBS relative to those with svPPA and controls. The speech of patients with lvPPA and svPPA contained a disproportionately greater number of words of both high frequency and high semantic diversity. Results from our exploratory voxel-based morphometry analyses across the whole group revealed correlations between grey matter volume in (i) bilateral frontal lobes with overall language output, (ii) the left frontal and superior temporal regions with speech complexity, (iii) bilateral frontotemporal regions with phonology, and (iv) bilateral cingulate and subcortical regions with age of acquisition. With the 15-word checklists, the LASSO models showed excellent accuracy for within-sample k-fold classification (over 93%) and out-of-sample validation between patients and controls (over 90%), and moderately good (59% - 74%) differentiation between the motor disorders (nfvPPA, PSP, CBS) and lexico-semantic groups (svPPA, lvPPA). In conclusion, we propose that a simple 15-word checklist provides a suitable screening test to identify people with progressive aphasia, while further specialist assessment is needed to differentiate accurately some groups (e.g., svPPA versus lvPPA and PSP versus nfvPPA).
1. Introduction
Speech is an integral part of effective communication and is often disturbed by brain damage such as stroke or neurodegeneration. Breakdown in speech production is important clinically as it can be diagnostic for different types of aphasia. Clinicians use conversations and narratives to detect communication difficulties in people with a speech and/or language impairment. Connected speech elicited by a picture description task, in particular, has been used to distinguish healthy controls from patients with diverse neurodegenerative diseases, as well as between specific subtypes of stroke aphasia and Primary Progressive Aphasia (PPA).1–3 To aid differential diagnosis and improve understanding about the nature of speech and language changes in PPA, many speech and linguistic measures have been previously investigated (e.g., acoustic/prosodic, lexico-semantic, morpho-syntactic, pragmatic/discourse) and subsequently quantified (e.g., speech rate, syllable duration, words per minute, psycholinguistic word properties) in connected speech analyses. However, transcription and quantification of speech properties require advanced linguistic expertise and are time-consuming. A simple analytical tool for analysing connected speech would be of great benefit. For example, if a simple target word list can be used (validated by in-depth, systematic analysis of connected speech with high diagnostic differentiation between progressive aphasias), this could be a practical and efficient clinical tool for assessing and diagnosing people with a neurodegenerative language impairment.
An important first step to this objective is to determine the distribution of words produced by each patient group and consider the variety of speech features and psycholinguistic properties. Both qualitative and quantitative differences in connected speech have been reported in PPA. For example, the number of content words is reduced in patients with the semantic variant (svPPA), with over-reliance on highly frequent words; in other words, the content of their speech becomes “lighter” with overuse of words that are more frequent, less concrete, less imageable, and more semantically diverse.4,5 Even though relatively less is known about the psycholinguistic properties of words produced by the non-fluent (nfvPPA) and logopenic (lvPPA) variants, articulatory and prosodic features, such as syllable duration, speech rate and word length, and grammatical complexity have been reported to differentiate between these two variants.6–8
Language impairments are also common in progressive supranuclear palsy (PSP) and corticobasal syndrome (CBS). Both conditions have features that overlap with nfvPPA9,10 such as dysfluency and syntactic impairments in production and comprehension.11 Similarities across these three groups have been reported in acoustic and lexical measures of connected speech during a picture description task.12 Connected speech alterations have been found in PSP patients13–15 including reduced speech rate, reduced total number of words and sentences, higher number of pronouns, and impaired grammatical complexity.16,17 Only a few studies have investigated connected speech in CBS, with one describing an overall reduction in connectedness (i.e., the number of connected events as a proportion of mentioned events) during a narrative discourse18 and another reporting reduced speech production rate and lexical-semantic errors during a picture description task.19
The differing methods of connected speech analysis in previous investigations pose a challenge in determining which measures, amongst an exhaustive list of word properties and features related to speech/language quantification, are useful for distinguishing between neurodegenerative diseases with a primary or associated language impairment. Here, we sought to address this knowledge gap with the following aims: 1) to determine which speech-related properties differentiate between svPPA, lvPPA, nfvPPA, PSP, CBS, and healthy controls during picture description using a principal component analysis to understand and simplify the patterns of change in quantifiable speech and psycholinguistic properties of connected speech; 2) to examine the neural correlates of connected speech in these conditions; and 3) to use a data-driven approach to develop an easy-to-use and practical word checklist.
2. Materials and methods
2.1 Participants
Seventy-four people (24 healthy controls, nine svPPA, nine lvPPA, nine nfvPPA, 10 PSP, 13 CBS) from the Mini Linguistic State Examination (MLSE)20 study were included in the development dataset. Controls were recruited through the National Institute for Health Research “Join Dementia Research” register and via local advertisement; other participants were recruited from tertiary referral services at Addenbrooke’s Hospital, Cambridge (N = 46), and Salford Royal Foundation Trust and its associated clinical providers (N = 4). Patients from a second site in the MLSE Study20 at St. George’s Hospital, London made an out-of-sample test set with svPPA (N = 7), lvPPA (N = 13), nfvPPA (N = 5), PSP (N = 2), and CBS (N = 6). Clinical diagnoses of PPA, PSP, and CBS were based on current consensus criteria.21–23 In our development sample, one nfvPPA patient declared a native language of Italian. Two svPPA patients from our out-of-sample site declared a native language of Gujarati and Indian Patois. All three patients who declared a non-English native language were pre-morbidly highly fluent in English.
2.2 Connected speech acquisition, transcription, reliability, and analysis
Participants completed the MLSE and the Boston Diagnostic Aphasia Examination (BDAE)24 and were asked to describe both the BDAE ‘cookie theft’ and MLSE ‘beach scene’ pictures. The instruction for both pictures were as follows: “Look carefully at this picture and describe aloud what is happening. Try to use sentences. I will stop you after one minute. Ready?” The examiners politely allowed time for the participants to finish their description after the one-minute mark. Connected speech samples were video recorded and transcribed by a speech-language pathologist (SKH), blinded to the clinical diagnoses, using the f4transcript notation software version 7.0, which has been previously reported to make the manual writing of speech samples from audio or video recordings more efficient. Speech samples were formatted for analysis with the Frequency in Language Analysis Tool25 which has specific codes for false speech. For example, false starts, grammatical errors, grammatical clause boundaries, prosodic indicators, non-lexical interjections such as filler words and pauses, repetitions, unintelligible segments, and neologisms were coded and excluded from the analyses.
To assess transcription reliability, we randomly selected 2 transcripts from each diagnostic group. We assessed the reliability of two different transcribers (SKH and KAP) by dividing the number of matching words between the two transcripts by the total words in the transcript used in our analyses (transcribed by the first author, SKH). This method is consistent with previously reported reliability analyses of transcriptions.26 We found a high percent agreement between the two transcripts with an average of 92% (range 81% to 98%) and 100% for the words in the 15-item checklist (below).
Using the transcribed speech samples free of false speech, we calculated the simplest measurements of connected speech (i.e., word counts, ratios, timing) to test whether these can differentiate groups as well as other measures of connected speech that tend to be more time-consuming to score and analyse (e.g., acoustic features). The total number and type counts for words, total time and words per minute were calculated for each participant. Additionally, the number and type counts for word bigrams (i.e., two word combinations such as “the mother”) and word trigrams (i.e., three word combinations such as “sink is overflowing”), type-to-token ratios for words, word bigrams, and word trigrams, proportion of function relative to content words, and combination ratio (i.e., a measure of connected language calculated as word trigram count divided by word count)27 were extracted using an automated script for language quantification with the use of FLAT.25 These measures of speech fluency for the two picture description tasks were included in our first principal component analysis (PCA).
Next, for the psycholinguistic word properties, each distinct word produced across all participants was extracted for analysis. We then excluded function words (e.g., articles, demonstratives, prepositions) and, for each content word, we looked up the ratings from various databases for length, log frequency,28 semantic diversity,29 semantic neighborhood density,30 concreteness,31 age of acquisition,32 orthographic and phonological Levenshtein distance.33,34 Where ratings for pluralised words were unavailable, word properties for the singular version were extracted. Although ratings for familiarity and imageability were initially obtained, these measures were excluded in the main analysis due to the unavailability of ratings for a high proportion of words. Of the available data, imageability ratings were strongly correlated with concreteness ratings (R = 0.94, p < 0.001) and familiarity ratings were moderately correlated with log frequency ratings (R = 0.45, p < 0.001). These word properties were included in our second PCA.
2.3 Statistical analysis
We used PCA as a dimensionality reduction method to investigate the distinct speech characteristics underlying connected speech performance. First, we calculated the average counts per participant for the quantifiable measures of speech fluency (e.g., number and type of words, type to token ratio, word per minute) using the transcribed speech samples which were then entered into a varimax-rotated PCA. A Kaiser-Meyer-Olkin test determined the suitability of our dataset. We selected three components based on Cattell’s criterion. Using principal component scores per participant, we conducted a one-way analysis of variance (ANOVA) to test for group differences.
Next, to understand the underlying pattern of variations in the lexico-semantic word properties produced by all patients and controls, all unique content words produced by patients and controls in both picture descriptions were compiled into a single ‘speech corpus’ and the psycholinguistic properties of each word were entered into a varimax-rotated PCA. After selecting three components using Cattell’s criteria, principal component scores for the words produced by each participant were extracted and then averaged across individual participants. Using these averaged principal component scores per participant, we tested the differences between group and task (i.e., ‘cookie theft’ versus ‘beach scene’) using a two-way ANOVA. Past studies have found that comparison of mean values can be relatively insensitive for detecting patients’ altered word usage, whereas distribution analyses can be more sensitive (e.g., where there are more pronounced changes in one part of the distribution).4,5 Thus, using data from the word properties PCA, principal component scores were split into quartiles (ranging from −4 to 2-, greater than −2 to 0, greater than 0 to 2, and greater than 2 to 4). For each participant, we counted the number of times each participant produced words in each range of a principal component (e.g., −4 to −2 in PC 1) and each point in the psycholinguistic dimensional space (e.g., −4 to −2 in PC 1 and 2 to 4 in PC 2). We then generated contour plots that mapped the proportion of words produced by each participant which were then averaged across groups. Using a method previously applied by Hoffman et al.5 we generated difference plots by subtracting the mean of control data from that of each patient group’s data to visualise the differences between control versus patient maps. We explored differences between groups across the variation in word properties in two ways. First, we took the mean value of the proportion of words produced by each patient group and compared them to the control data in each of the dimensional spaces using two-tailed t-tests. Secondly, for a more sensitive method, we conducted a distribution analysis by quantifying the number of words produced by controls and patients in each of the principal components’ quartiles. A repeated measures ANOVA was performed with quartiles as within-subject and group as between-subject factors.
Post hoc analyses were conducted using Tukey’s honestly significant difference (HSD) test for multiple comparison. All statistical analyses were performed in R statistical software (version 2023.03.0).
2.4 Neuroimaging acquisition and voxel-based morphometry analysis
All participants underwent T1-weighted structural MRI of the brain. Participants from Cambridge were scanned using a 3T Siemens Skyra MRI scanner. Whole-brain T1-weighted structural images were acquired using the following parameters: iPAT2; 208 contiguous sagittal slices; field of view (FOV) = 282 x 282 mm2; matrix size 256 x 256; voxel resolution = 1.1 mm3; TR/TE/ TI = 2000 ms/2.93 ms/850 ms, respectively; and flip angle 8°. Participants from Manchester were scanned using a 3T Philips Achieva MRI scanner. Whole-brain T1-weighted images were acquired using the following parameters: SENSE = 208 contiguous sagittal slices; FOV = 282 x 282 mm2; matrix size 256 x 256; voxel resolution = 1.1mm3; TR/TE/TI = 6600 ms/2.99 ms/850 ms, and flip angle 8°.
Whole-brain grey matter changes were indexed using voxel-based morphometry (VBM) analyses of structural T1-weighted MRI, integrated into Statistical Parametric Mapping software (SPM12: Wellcome Trust Centre for Neuroimaging, https://www.fil.ion.ucl.ac.uk/spm/software/spm12/). A standard pre-processing pipeline was implemented involving: (i) brain segmentation into three tissue probability maps (grey matter, white matter, cerebrospinal fluid); (ii) normalisation (using Diffeomorphic Anatomical Registration Through Exponentiated Lie Algebra, DARTEL);35 (iii) study-specific template creation using grey matter tissue probability maps; (iv) spatial transformation to Montreal Neurological Institute (MNI) space using transformation parameters from the corresponding DARTEL template; and (v) image modulation and smoothing using 8mm full-width-half-maximum Gaussian kernel to increase signal-to-noise ratio. Segmented, normalised, modulated and smoothed grey matter images were used for VBM analyses.
We examined the associations between whole-brain grey matter intensity and PCA-generated principal component scores, which were averaged across two picture description tasks, using t-contrasts. Age and total intracranial volume were included as nuisance covariates. Clusters were extracted using a threshold of p < 0.001 uncorrected for multiple comparisons with a cluster threshold of 100 voxels. We chose 100 voxels as our cluster threshold as we were interested in smaller subcortical regions that have been reported to be associated with speech production and are often atrophic in the disease groups.
2.5 Word checklist analysis
To determine target words that could best differentiate between groups, we used Least Absolute Shrinkage and Selection Operation (LASSO) logistic regression.36 Given the large number of predictors (i.e., 500+ unique words used by the whole group), relatively small sample size per group, and multicollinearity of the words (e.g., the likelihood that a participant would say “overflowing” and “sink”), the LASSO method is highly appropriate for automated feature selection and shrinkage. While multiple correlated words are entered into the model, only the most important predictor variables (i.e., the least number of words that best differentiate between groups) will be selected. As a first step, we pooled together all of the different words that patients and controls produced which resulted in over 500+ tokens per picture. We then streamlined this collection by carrying out LASSO regressions for each picture including all unique words produced per picture as predictors for the following comparisons: (i) controls versus each patient group, and (ii) each patient group against one another. Whether or not a participant produced a word such as “overflowing” was coded as 1 for produced and 0 for not produced. We accounted for differences in dialect (e.g., score of 1 if the participant said boy, chap, lad, or bloke) and morpho-syntax such as verb tense (e.g., stealing/stolen) and singular/plural forms (e.g., plate/plates).
Next, the words that had been selected in each pairwise comparison (by logistic LASSO regression) were compiled, resulting in a pool of 33 words for the BDAE ‘cookie theft’ and 46 words for the MLSE ‘beach scene’ pictures. We re-ran the LASSO regressions for each pairwise comparison using these truncated lists and the resulting words were further rank ordered by (i) the number of times they appear in the pairwise comparisons, (ii) their beta coefficients, and (iii) the magnitude of difference in the overall proportion by group (e.g., magnitude would be 1 if all of the controls produced the word ‘overflow’ but none of the svPPAs did). The top 15 words resulting from this rank ordering were entered into a series of four-fold cross-validated LASSO logistic regressions with each predicting the diagnostic distinction of interest (e.g., controls versus patients). The scoresheets using the 15 words are shown in Appendix 1.
To evaluate the robustness of the model in predicting group classification with the word checklists, we conducted out-of-sample predictive validity testing with connected speech data from St. George’s Hospital. There were no differences in demographics between patients from the two test sites except for PSP patients from St. George’s having lower scores on the revised Addenbrooke’s Cognitive Examination (ACE-R) compared to those from Cambridge (p = 0.02). We tested the 15-word checklist with the St. George’s data assigning a score of 1 if the participant produced the target word and a 0 if the word was omitted. Morpho-syntactic variations were scored as correct if the root matched the target word (e.g., overflowing for overflow, digging for dig). As an index of accuracy for our binomial models (i.e., pairwise comparisons), we report classification performance on the test data using function confusion.glmnet from the glmnet package in R for the following comparisons: controls versus all patients, patients belonging to the “motor” group (i.e., nfvPPA, PSP, CBS) versus “lexico-semantic” group (i.e., svPPA, lvPPA), and each patient group against one another. Of note, PSP and CBS patients were grouped into one due to small sample size (i.e., 2 PSP) in our out-of-sample test set.
To test the hypothesis that supplementing the checklist with cognitive scores might improve the differentiation between groups, we ran another LASSO logistic regression with the 15 words (coded the same way as noted above), as well as subtest scores from the ACE-R and MLSE. We estimated the LASSO model using a within-sample four-fold cross-validation with the Cambridge training set and tested the generalisability of our model with the St. George’s data as out-of-sample test.
3. Results
3.1 Demographics
Demographic and clinical features are shown in Table 1. There were no significant differences in all groups for age, gender, and handedness, as well as symptom duration for patients. There were significant differences between groups in education; post hoc tests confirmed that controls left education later than patients with nfvPPA, CBS, and PSP (p < 0.05). Significant group differences emerged on total MLSE and ACE-R scores. Controls performed better on the MLSE when compared with patients with svPPA, lvPPA, nfvPPA, and CBS (p < 0.001), PSP performed better than lvPPA (p = 0.001) and nfvPPA (p = 0.007), and CBS performed better than lvPPA (p = 0.03). On the ACE-R, controls performed better than all patient groups (p < 0.05), and nfvPPA, PSP, and CBS performed better than lvPPA (p < 0.05), and PSP performed better than svPPA (p = 0.001). As shown in Table 1, in our development sample, all participants were white. Two svPPA patients from our out-of-sample site were non-white.
3.2 Quantification of speech fluency
Average counts per participant for the quantifiable properties of words and word combinations per picture were entered into a PCA with varimax rotation. Three principal components were identified using Cattell’s criteria which explained 86.5% of the variance (Kaiser-Meyer-Olkin = 0.70). The loadings of each measure are shown in Supplementary Table 1.
Type and token counts for words, word bigrams, and word trigrams, word per minute, type-to-token ratio of words, and combination ratio loaded most heavily on principal component (PC) 1 and thus we labelled this PC as ‘speech quanta’. Type-to-token ratio of words, word bigrams, and word trigrams loaded most heavily on PC 2 which we labelled as ‘lexical richness’. Word per minute, an index of speech fluency, and combination ratio, the degree to which an individual produced longer, more-complex combinations as opposed to single word fragments, loaded heavily on PC 3 and we adopted the working label of ‘speech complexity’.
Group performance patterns on all three PCs are visually summarised in Figure 1A. For PC 1, the results from a one-way ANOVA revealed group differences (F(1,142) = 71.19, p < 0.001), driven by controls and svPPA patients having higher scores than those with nfvPPA (p < 0.001), PSP (p < 0.01), and CBS (p < 0.05). Additionally, controls had higher scores than patients with lvPPA (p = 0.01), who in turn had higher scores than those with nfvPPA (p < 0.001). A one-way ANOVA did not reveal group differences for PC 2 (F(1,142) = 1.26, p = 0.26). For PC 3, the results from a one-way ANOVA revealed group differences (F(1,142) = 12.77, p < 0.001), driven by controls having higher scores than those with nfvPPA (p < 0.001), PSP (p < 0.001), and CBS (p = 0.002).
(A) Scores of quantitative measures of speech fluency. For PC 1 (‘speech quanta’), the results from a one-way ANOVA revealed significant group differences (F(1,142) = 71.19, p < 0.001), driven by controls (N = 24) and patients with svPPA (N = 9) having higher scores than those with nfvPPA (N = 9), PSP (N = 10), and CBS (N = 13), and controls having higher scores than those with lvPPA (N = 9), and patients with lvPPA having higher scores than those with nfvPPA. PC 2 (‘lexical richness’) resulted in no group differences (F(1,142) = 1.26, p = 0.26), and for PC 3 (‘speech complexity’) significant group differences were found (F(1,142) = 12.77, p < 0.001), driven by controls having higher scores than patients with nfvPPA (p < 0.001), PSP (p < 0.001), and CBS (p = 0.002). (B) Scores of quantitative measures of word properties across groups. For PC 1 (‘length’), the results from a two way ANOVA revealed significant group differences (F(5,134) = 4.29, p < 0.001), driven by svPPA and lvPPA patients producing words that were shorter, phonologically and orthographically less complex than controls (p < 0.05). For PC 2 (‘semantic richness’), significant differences were found for group (F(5,134) = 16.62, p < 0.001) and task (F(1,134) = 22.05, p < 0.001). Patients with svPPA and lvPPA produced more words that were characterised as more frequent and semantically diverse than those with nfvPPA, PSP, CBS, and controls (p < 0.01). For PC 3 (‘acquisition age’), significant differences were found for group (F(5,134) = 7.09, p < 0.001) and task (F(1,134) = 50.24, p < 0.001) for PC 3. Post hoc analyses revealed that (i) nfvPPA patients produced words that were characterised as significantly earlier acquired than those with svPPA (p < 0.001), PSP (p = 0.05), and controls (p < 0.001), and (ii) CBS patients used words that were significantly earlier acquired than those with svPPA (p = 0.002) and controls (p = 0.01). Results from post hoc analyses using Tukey’s honestly significant difference (HSD) test for multiple comparisons are shown as asterisks indicating level of significance: * p ≤ 0.05; ** p ≤ 0.01; *** p ≤ 0. 001. CBS, corticobasal syndrome; lvPPA, logopenic variant of primary progressive aphasia; nfvPPA, non-fluent variant of primary progressive aphasia; PC, principal component; PSP, progressive supranuclear palsy; svPPA, semantic variant of primary progressive aphasia.
Correlations between the speech fluency PC scores and total and subdomain scores of the MLSE can be found in Supplementary Table 2.
3.3 Quantification of word properties
Ratings of psycholinguistic features for all words produced by controls and patients were entered into a PCA with varimax rotation. Three principal components were identified using Cattell’s criteria, each representing a group of covarying psycholinguistic features. These three components explained 85.5% of the variance (Kaiser-Meyer-Olkin = 0.75). The loadings of each measure are shown in Supplementary Table 3. Length, phonological and orthographic Levenshtein distance loaded most heavily on PC 1 and we adopted the working label of ‘length’. Concreteness, log frequency, semantic neighbourhood density, and semantic diversity loaded heavily on PC 2 which we labelled as ‘semantic richness’. Age of acquisition loaded most heavily on PC 3 which we labelled as ‘acquisition age’.
The three scores, obtained from the psycholinguistic PCA results, per participant along with the elicitation task were into a two-way ANOVA which revealed significant group differences in PC 1 (F(5,134) = 4.29, p < 0.001), driven by svPPA and lvPPA patients producing words that were shorter, phonologically and orthographically less complex than controls (p < 0.05) (see Figure 1B).
For PC 2, significant differences were found for group (F(5,134) = 16.62, p < 0.001) and task (F(1,134) = 22.05, p < 0.001). The task effect was driven by more frequent and semantically diverse words produced for the ‘cookie theft’ than the ‘beach scene’ picture. Post hoc analyses revealed that svPPA and lvPPA patients produced more words that were characterised as more frequent and semantically diverse than those with nfvPPA, PSP, CBS, and controls (p < 0.01).
Significant differences were found for group (F(5,134) = 7.09, p < 0.001) and task (F(1,134) = 50.24, p < 0.001) for PC 3. The words used to describe the ‘cookie theft’ were found to be later acquired. Post hoc analyses revealed that nfvPPA patients produced words that were characterised as significantly earlier acquired than those with svPPA (p < 0.001), PSP (p = 0.05), and controls (p < 0.001). Similarly, CBS patients used words that were significantly earlier acquired than those with svPPA (p = 0.002) and controls (p = 0.01).
Correlations between the word properties PC scores and total and subdomain scores of the MLSE can be found in Supplementary Table 2.
3.3.1 Differences in multivariate word properties
Moving beyond the simplistic mean statistic, we looked at the bivariate distributions of words across the psycholinguistic space and how these might shift in each patient group (e.g., patients produce fewer words in one part of the space and might substitute more words in another part of the space). Figure 2 shows the contour plot for controls (left), depicting the averaged proportion of words produced within the principal component space, and the difference plots where the mean of the control data for the three principal components from the ‘word properties’ PCA (Section 3.3) were subtracted from that of the patient data.
The top and bottom left plots show the contour distributions across principal component (PC) 1: Length, PC 2: Semantic richness, and PC 3: Acquisition age produced by healthy controls. Difference plots comparing patients with healthy controls are shown to the right of the contour plots of healthy controls. In the control plots, yellow tones show where the greatest proportions of words were found within the principal component space. For controls versus patients, the red and blue tones represent principal component spaces where patients produced more words than controls and where controls produced more than patients, respectively. Taking the mean value of the proportion of words produced by each patient group (svPPA N = 9, lvPPA N = 9, nfvPPA = 9, PSP N = 10, CBS N = 13), we compared them to the control data (N = 24) in each of the dimensional spaces using two-tailed t-tests. The arrows indicate where in the maps there were significant differences between controls and patients (p-values are shown as asterisks indicating level of significance: * p < 0.05; ** p < 0.01; *** p < 0. 001).
Relative to controls, svPPA and lvPPA patients produced a greater proportion of words in the higher semantic richness (i.e., more semantically diverse and frequent) and lower length (i.e., shorter, less phonologically and orthographically complex) space. In contrast, nfvPPA, PSP, and CBS patients produced a greater proportion of words with lower semantic richness and acquisition age (i.e., earlier acquired) space.
3.3.2 Distribution analysis of word properties PCA
Another way to go beyond the simplistic mean statistic is to undertake a formal distribution analysis for each principal component. This has been shown in previous work to be much more sensitive to changes in the content words produced by patients.37,38 As shown in Figure 3, principal component scores for PC 1 to PC 3 from the word properties PCA were divided into quartiles and the number of words produced in each quartile was computed for each participant followed by a group mean.
Each plot shows the number (N) of words produced in each quartile (Q) by patient groups (svPPA N = 9, lvPPA N = 9, nfvPPA = 9, PSP N = 10, CBS N = 13) for principal component (PC) 1 ‘length’, PC 2 ‘semantic richness’, and PC 3 ‘acquisition age’. For PC 1, a six groups x four quartiles repeated measures ANOVA showed a significant effect of group only for both ‘cookie theft’ (F(5,283) = 37.16, p < 0.001) and ‘beach scene’ (F(5,272) = 39.18, p < 0.001). For PC 2, significant effects were found for group (F(5,280) = 33.68, p < 0.001), quartile (F(1,280) = 4.67, p = 0.03), and group-by-quartile interaction (F(5,280) = 4.36, p < 0.001) for ‘cookie theft’. For ‘beach scene’, significant effects were found for group (F(5,270) = 28.94, p < 0.001), quartile (F(1,270) = 5.53, p = 0.02), and group-by-quartile interaction (F(5,270) = 8.29, p < 0.001). For PC 3, significant effects were found for group (F(5,283) = 36.15, p < 0.001), quartile (F(1,283) = 17.17, p < 0.001), and group-by-quartile interaction (F(5,283) = 2.47, p = 0.03) for ‘cookie theft’. For ‘beach scene’, significant effects were found for group (F(5,265) = 31.04, p < 0.001), quartile (F(1,265) = 21.67, p < 0.001), and group-by-quartile interaction (F(5,265) = 2.47, p = 0.03). The effect of quartile from post hoc analyses using Tukey’s honestly significant difference (HSD) test for multiple comparisons is shown as asterisks indicating level of significance: * p ≤ 0.05; ** p ≤ 0.01; *** p ≤ 0. 001. BDAE, Boston Diagnostic Aphasia Examination; CBS, corticobasal syndrome; lvPPA, logopenic variant of primary progressive aphasia; MLSE, Mini Linguistic State Examination; nfvPPA, non-fluent variant of primary progressive aphasia; PSP, progressive supranuclear palsy; svPPA, semantic variant of primary progressive aphasia.
For PC 1, a six groups x four quartiles repeated measures ANOVA showed a significant effect of group only for both ‘cookie theft’ (F(5,283) = 37.16, p < 0.001) and ‘beach scene’ (F(5,272) = 39.18, p < 0.001). For PC 2, a six groups x four quartiles repeated measures ANOVA showed significant effects of group (F(5,280) = 33.68, p < 0.001), quartile (F(1,280) = 4.67, p = 0.03), and group-by-quartile interaction (F(5,280) = 4.36, p < 0.001) for ‘cookie theft’. For ‘beach scene’, a six groups x four quartiles repeated measures ANOVA showed significant effects of group (F(5,270) = 28.94, p < 0.001), quartile (F(1,270) = 5.53, p = 0.02), and group-by-quartile interaction (F(5,270) = 8.29, p < 0.001). For PC 3, a six groups x four quartiles repeated measures ANOVA showed significant effects of group (F(5,283) = 36.15, p < 0.001), quartile (F(1,283) = 17.17, p < 0.001), and group-by-quartile interaction (F(5,283) = 2.47, p = 0.03) for ‘cookie theft’. For ‘beach scene’, a six groups x four quartiles repeated measures ANOVA showed significant effects of group (F(5,265) = 31.04, p < 0.001), quartile (F(1,265) = 21.67, p < 0.001), and group-by-quartile interaction (F(5,265) = 2.47, p = 0.03). Our results are summarised in Supplementary Table 4.
3.4 Neural correlates of connected speech properties
Associations between grey matter intensity and principal component scores from both quantitative measures of speech fluency and word properties are shown in Figure 4 and Supplementary Table 5. In the entire group (i.e., patients and controls), PC 1 (‘speech quanta’) scores correlated with grey matter intensities of the bilateral middle and superior frontal gyri, right inferior frontal gyrus, insula, putamen, and caudate. PC 3 (‘speech complexity’) scores correlated with grey matter intensities of the left insula, inferior, middle, and superior frontal gyri, extending medially, superior temporal gyrus, and parts of the limbic system. No significant correlations were found for PC2 (‘lexical richness’) scores.
This figure shows regions of grey matter intensity that uniquely correlate with principal component scores in the whole group including controls (N = 24) and patients (svPPA N = 9, lvPPA N = 9, nfvPPA = 9, PSP N = 10, CBS N = 13) using t-contrasts. Clusters were extracted using a threshold of p < 0.001 uncorrected for multiple comparisons with a cluster threshold of 100 voxels with age and total intracranial volume included as nuisance covariates. CBS, corticobasal syndrome; lvPPA, logopenic variant of primary progressive aphasia; nfvPPA, non-fluent variant of primary progressive aphasia; PSP, progressive supranuclear palsy; svPPA, semantic variant of primary progressive aphasia.
For the word properties PCA, PC 1 (‘length’) scores correlated with grey matter intensities of the left insula, middle and superior temporal gyri, bilateral parahippocampal and fusiform gyri, right inferior and middle temporal gyri, and limbic structures. PC 3 (‘acquisition age’) scores correlated with grey matter intensities of the bilateral cingulate gyri and right caudate and putamen. No significant correlations were found for PC 2 (‘semantic richness’) scores.
When excluding healthy controls, PC 1 (‘length’) scores correlated significantly with a single cluster including the left insula, middle and superior temporal gyri (see Supplementary Table 6). No significant correlations were found for the other PC scores. Supplementary Table 7 shows the results when using a cluster-forming height threshold of p < 0.005 paired with a cluster extent threshold of p < 0.05 FWE-corrected.
3.5 Word checklist
Using the word checklist for each picture (Appendix 1), the LASSO logistic regression selected a group of words that together predicted group membership (see Supplementary Table 8). Of note, the LASSO regression for svPPA versus lvPPA, and nfvPPA versus PSP resulted in zero words for both pictures; in other words, none of these words could differentiate between these groups. These results motivated our hierarchical classification as shown in Figure 5, where the “motor” group included patients with nfvPPA, PSP, and CBS, and the “lexico-semantic” group included those with svPPA and lvPPA. The within-sample k-fold validation accuracies for ‘cookie theft’ were as follows: 96% for patients versus controls and 92% for “motor” versus “lexico-semantic” groups. Out-of-sample test accuracy with the St. George’s data (N = 34) resulted in 91% for patients versus controls and 74% for “motor” versus “lexico-semantic” groups.
This figure summarises the validations for (A) BDAE ‘cookie theft’ 15-word checklist, (B) BDAE ‘cookie theft’ 15-word checklist with cognitive measures of ACE-R and MLSE, (C) MLSE ‘beach scene’ 15-word checklist, and (D) MLSE ‘beach scene’ 15-word checklist with cognitive measures of ACE-R and MLSE. The curved brackets under the group names (e.g., Patients, “Motor”, “Lexico-Semantic”) illustrate the number of participants who were classified correctly out of the total N. The percentages in red indicate the within-sample four-fold and out-of-sample validation accuracies. ACE-R, Addenbrooke’s Cognitive Examination – Revised; BDAE, Boston Diagnostic Aphasia Examination; CBS, corticobasal syndrome; lvPPA, logopenic variant of primary progressive aphasia; MLSE, Mini Linguistic State Examination; nfvPPA, non-fluent variant of primary progressive aphasia; PSP, progressive supranuclear palsy; svPPA, semantic variant of primary progressive aphasia.
For ‘beach scene’, the within-sample k-fold validation accuracies were as follows: 94% for patients versus controls and 88% for “motor” versus “lexico-semantic” groups. Out-of-sample test accuracy resulted in 97% for patients versus controls and 59% for “motor” versus “lexico-semantic” groups. Of note, the LASSO regression for nfvPPA versus PSP and CBS combined also resulted in zero words for both pictures.
Since we were not able to differentiate individual patient groups using the checklist alone, we tested the hypothesis that supplementing with cognitive measures might improve the differentiation between these groups. To this end, we supplemented the LASSO models with ACE-R and MLSE sub-scores along with the target words and found improved differentiation for within-sample validation for both nfvPPA versus PSP and CBS (91% for ‘cookie theft’ and 90% for ‘beach scene’) and svPPA versus lvPPA groups (100% for both ‘cookie theft’ and ‘beach scene’). Moreover, results from the out-of-sample predictive validity testing showed that the checklists and LASSO models were generalisable more for svPPA versus lvPPA (90% for ‘cookie theft’ and 95% for ‘beach scene’) when compared with nfvPPA versus PSP and CBS (54% for ‘cookie theft’ and 69% for ‘beach scene’).
In Figure 5, the curved brackets under the group names (e.g., Patients, “Motor”, “Lexico-Semantic”) illustrate the number of participants who were classified correctly. Supplementary Figure 1 shows each participant’s scores on the 15-word checklists and ACE-R and highlights the participants who were misclassified. The sensitivity and specificity of the word checklists are shown in Supplementary Figure 2.
4. Discussion
Clinical impressions from listening to patients’ speech are often used to guide diagnosis but there are two main challenges that this study addresses. First, it is not clear what aspects of the speech should be the target of the assessment. Second, although samples of speech are easy to collect, detailed analyses of connected speech are time-consuming and require specialist expertise. In the present study, we undertook detailed transcription and analyses of connected speech elicited by two picture description tasks and established which speech features and/or psycholinguistic properties might show the greatest differentiation across groups. We then identified the atrophy correlates of speech-related features. Finally, using data-driven methods, we established a clinically efficient and effective vocabulary checklist method to aid differential diagnosis between the subtypes of primary progressive aphasia (PPA), progressive supranuclear palsy (PSP), and corticobasal syndrome (CBS).
We found significant differences in both speech features and psycholinguistic properties of words between patients and controls. These features also differentiated svPPA and lvPPA versus the remaining groups which are most typically associated with a tauopathy and/or motor disorders (nfvPPA, CBS, PSP). The total language output was significantly reduced in patients with nfvPPA, PSP, and CBS relative to those with svPPA and controls. Inspection of the proportion of words produced across the lexico-semantic space revealed that patients with svPPA and lvPPA used a greater proportion of words with high semantic richness (i.e., more frequent and semantically diverse) and lower length (i.e., shorter, less phonologically and orthographically complex) such as “do”, “out”, and “get” relative to controls. In contrast, patients with nfvPPA, PSP, and CBS showed the opposite pattern with a greater proportion of words in the lower semantic richness and acquisition age (i.e., earlier acquired) space such as “dog”, “boy”, and “cookie”.
We demonstrated that a straightforward word checklist can provide a “user-friendly” tool, quantifiable in a simple way, with high sensitivity in differentiating healthy controls from patients with a progressive aphasia. The 15-word checklist showed excellent accuracy for within-sample k-fold validation, for differentiating patient groups from controls. Even on an out-of-sample validation dataset, the 15-word checklist was excellent at differentiating patients from controls (out-of-sample test accuracy of 91% and 97% for ‘cookie theft’ and ‘beach scene’) and moderately good at differentiating primary “lexico-semantic” (svPPA, lvPPA) from “motor” (nfvPPA, PSP, CBS) groups (accuracy of 74% and 59% for ‘cookie theft’ and ‘beach scene’). The 15 words did not accurately differentiate patients with svPPA from lvPPA, or nfvPPA from PSP and CBS. This is perhaps unsurprising given the patients’ similar patterns of word usage, total language output, and psycholinguistic properties of the words elicited. Supplementing the 15-word checklist with cognitive measures of ACE-R and MLSE subtest scores increased diagnostic accuracy for nfvPPA versus PSP and CBS for within-sample validation (91% for ‘cookie theft’ and 90% for ‘beach scene’), as well as svPPA versus lvPPA for both within-sample (100% for both ‘cookie theft’ and ‘beach scene’) and out-of-sample validation (90% for ‘cookie theft’ and 95% for ‘beach scene’). With regard to differentiating patients from controls, the best ACE-R subtest was verbal fluency which replicates a recent study that found this simple clinical assessment is excellent at differentiating patients from controls but has limited use for differential diagnosis between patient subgroups.39 We propose that the quick and simple 15-word checklist is a suitable screening test to identify people with progressive aphasia, although further specialist assessment is needed for accurate diagnostic sub-typing. In the following sections, we interpret these findings, consider their clinical implications, and note directions for future research.
Reduced language output from nfvPPA, PSP and CBS
Patients with nfvPPA, PSP and CBS were distinguishable from those with svPPA, lvPPA and controls, based on reduced language output and connected speech fluency (as measured by the ‘speech quanta’ and ‘speech complexity’ PCs). In particular, combination ratio has been previously proposed as a measure of connected language output because it represents the degree to which an individual produces longer, more-complex combinations of words over the total word count.27 Many studies have suggested that measures such as reduced language output, slowed articulation rate, speech-sound errors, and proportion of function to content words, can differentiate patients with nfvPPA from the other variants of PPA in connected speech and other language tasks.2,40–44 Interestingly, even without measures of acoustics/prosody such as speech pauses, articulation rate, and syllable duration (that are technically difficult to code and quantify), we were able to differentiate between nfvPPA, PSP, and CBS versus svPPA, lvPPA, and controls using a simple quantification of connected speech (e.g., type/token count).
Despite a sparse literature on connected speech in PSP and CBS, reduced language output and speech rate have been reported in both groups.12,17,19 In the present study, PSP and CBS patients were comparable to nfvPPA patients in that all groups produced fewer words with reduced speech complexity. Our results support previous findings19,45 that a general reduction in language output may be a characteristic pattern of PSP and CBS patients, like those with nfvPPA. Moreover, overall performance on various cognitive and language assessments has also been reported to be similar for PSP, CBS and nfvPPA patients.11,20,46
Lexico-semantic features
SvPPA and lvPPA patients produced a greater proportion of words that are more frequent and semantically diverse, as well as shorter and less phonologically complex. This finding is consistent with previous reports and highlights two important points.4,7 First, the secondary changes in other psycholinguistic properties such as imageability and length may be related to the under-sampling of the low frequency words used by controls; in other words, svPPA patients generated more “lighter” words that tend to be less imageable and more semantically diverse (e.g., “something”). In addition to under-sampling the low frequency space, svPPA patients have also been found to over-sample the higher frequency space by substituting alternatives to the low frequency target items or picture elements they are unable to name.4 For example, in the present study, svPPA patients tended to replace low frequency words typically produced by controls (e.g., “the sink is overflowing”) with higher frequency words that are less imageable and shorter (e.g., “it’s coming out”).47 Additionally, prior studies have consistently reported that patients with svPPA/semantic dementia replace content words with high frequency, high semantic diversity, and low imageability words not only during picture description, but also in other aspects of language output such as naming and verbal fluency.4,5,8,39,48,49 Frequency and age of acquisition effects in svPPA have also been found beyond tests requiring language output such as lexical decision.50 Less is known about the psycholinguistic properties of words used by patients with lvPPA. Our findings accord with those of Cho et al.51 who reported that lvPPA patients produced shorter and more frequent content words when describing the ‘cookie theft’ picture. Furthermore, our formal distribution analysis with the difference plots (Figure 2) and quantification of words produced in each quartile (Figure 3) revealed contrastive patterns across the patient groups with (1) svPPA and lvPPA producing shorter words with high frequency and semantic diversity and (ii) nfvPPA, PSP, and CBS producing later acquired, lower frequency, and less semantically diverse words.
Grey matter correlates of connected speech features
High scores on the ‘speech quanta’ PC correlated with greater grey matter intensities of bilateral middle and superior frontal gyri, and right inferior frontal gyrus (IFG) extending medially and subcortically to include the insula. Cho et al.40 found increased speech errors and production of partial words in nfvPPA to be associated with cortical thinning in the left middle frontal gyrus. Ash et al.2 found speech sound deficits and reduced speech rate in nfvPPA to be related to atrophy in the insula, a region thought to be important for speech articulation,52,53 and right premotor and supplementary motor regions. Prior studies have also suggested the role of the superior and middle frontal gyri in the grammatical processing of language production and comprehension.54,55 These findings highlight the potential role of the bilateral frontal region in measures of speech production and rate.
High scores on the ‘speech complexity’ PC correlated with grey matter intensities of the left insula, IFG, superior temporal gyrus (STG), and limbic structures. The largest cluster was found for the left insula and IFG, extending into the temporal lobe. Beyond overt speech production, the IFG and insula are reported to be critical in the acoustic measures of speech production such as pause segment duration in motor speech disorders including nfvPPA, ALS, and post-stroke aphasia.52,56,57 Our findings are in line with previously reported associations between superior temporal regions and greater morpho-syntactic demands,58 grammaticality,2 complex sentence production,59 lexical phonology60, and verbal generation in controls and diverse patient groups.45 The STG has also been reported to be implicated in the prefrontal-temporal feedback loop and associated with self-monitoring of speech ouput.61
High scores on the ‘length’ PC correlated with greater grey matter intensities of the bilateral temporal lobe, including medial temporal regions, insula, and right limbic lobe. Notably, when excluding controls, the only cluster that correlated significantly included the left insula, middle and superior temporal gyri (see Supplementary Table 6). During an overt picture naming task, Wilson et al.62 found word length to be positively correlated with signal intensity in the left STG in healthy controls. In addition, Hodgson et al.63 found the middle and superior temporal regions to be not only implicated in phonology but also general semantics and semantic control. The ability to generate longer, phonologically more complex words and word combinations may rely on processing speech sounds, as well as accessing conceptual knowledge and controlled retrieval of meaningful semantic information.
Word checklist for picture description
Validated tools to analyse connected speech samples are scare, and to this end, we optimised simple checklists for two widely used picture-narratives to assess PPA subtypes, PSP, and CBS. We employed a hierarchical structure in our LASSO analysis given the nature of word usage across patient groups. The LASSO models could not differentiate svPPA versus lvPPA, nfvPPA versus PSP, and nfvPPA versus PSP and CBS with the target words alone. Supplementing the checklist with MLSE and ACE-R subtest scores improved the differentiation between these groups with excellent within group four-fold cross validation accuracies. Out-of-sample test accuracy was also found to be high for svPPA versus lvPPA, which emphasises the need for further specialist assessments for aphasic groups that cluster based on shared clinical features (i.e., anomia in svPPA and lvPPA, motor speech and/or agrammatism in nfvPPA, PSP, and CBS).
Clinical tools that are fast, simple, and sensitive to aphasia subtypes including various checklists have previously been proposed for post-stroke aphasia,64 but to our knowledge this is the first study to provide a direct comparison of word usage across PPA subtypes and Parkinson-plus disorders and optimise a checklist for these patient groups. Future studies with connected speech samples could employ similar methodologies such as our LASSO models to generate specific word checklists for other picture description tasks, different languages, and/or diverse patient groups. The present study could also potentially inform the design of future studies in developing targeted pictures that contain the key vocabulary items that help to differentiate specific clinical groups.
Limitations and clinical implications
There are limitations to our study. We only present clinical, not pathological, diagnoses, although clinic-pathological correlations are high for PPA and PSP. Our sample size for the out-of-sample test validation was small particularly for certain groups such as PSP. However, we mitigated the potential limitations of small-sample k-fold cross-validation by conducting predictive validity testing on an unseen dataset. This supports generalisability of our models and word checklists. Future work is warranted to test the generalisability of the word checklists to larger patient samples that span various disease stages, non-English languages, and varying levels of demographics including education and geographical regions.
A major aim of the present study was to ameliorate the problem of connected speech analyses being time-consuming, effortful and inconsistent across clinicians and different clinical/research settings. As a result, our systematic analysis of connected speech did not include other acoustic and articulatory measures investigated in prior studies. There are undoubtedly other features of the patients’ connected speech that can help with differentiation43 which are not captured in our approach, but these require expertise and time-consuming transcription and analyses. Finally, we acknowledge that our imaging analyses were exploratory but nonetheless add to the current literature pertaining to regions engaged in connected speech.
In conclusion, we propose that screening for language deficits in PPA and “motor” disorders like PSP and CBS is achievable with a one-minute sample of connected speech. By focusing on the number and lexico-semantic metrics of the given words, rather than acoustic features, this method is likely to be robust to detect dysarthrophonia from disease, even with reduced bandwidth from remote recordings. The screening test is not a substitute for in-depth neuropsychological assessment, but a contributing tool towards diagnosis. Furthermore, it has the advantage of applicability in resource-limited settings and with limited expertise. Future versions of the test for non-English speakers would further increase the international utility of this approach.
Data availability
Unthresholded statistical parametric images are available freely on request of the corresponding or senior author. Word lists produced by participants are available on request. Participant MRI scans and transcripts may be available, subject to a data sharing agreement required to protect confidentiality and adhere to consent terms.
Funding
This work and the corresponding author (SKH) were supported and funded by the Bill & Melinda Gates Foundation, Seattle, WA, and Gates Cambridge Trust (Grant Number: OPP1144). This study was supported by the Cambridge Centre for Parkinson-Plus; the Medical Research Council (MC_UU_00030/14; MR/P01271X/1; MR/T033371/1); the Wellcome Trust (220258); the National Institute for Health and Care Research Cambridge Clinical Research Facility and the National Institute for Health and Care Research Cambridge Biomedical Research Centre (BRC-1215-20014; NIHR203312); an intramural award (MC_UU_00005/18) to the MRC Cognition and Brain Sciences Unit; and MRC Career Development Award (MR/V031481/1). For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.
Competing interests
The authors report no competing interests.
Supplementary material
ANOVA findings on the effects of group, quartile and group-by-quartile interaction from the distribution analysis of word properties principal component analysis
Voxel based morphometry results showing regions of grey matter intensity that correlate with PCA-generated principal component in the whole group
Voxel based morphometry results showing regions of grey matter intensity that correlate with PCA-generated factors in patients only
Voxel based morphometry results showing regions of grey matter intensity that correlate with PCA-generated factors with a cluster-forming height threshold of p < 0.005 paired with a cluster extent threshold of p < 0.05 FWE-corrected
Scatterplots showing total scores on the CALL 15-word checklists and ACE-R with the following color representations: magenta circles for people misclassified as controls, blue circles for those misclassified as belonging to the “motor” group, and black circles for those misclassified as belonging to the “lexico-semantic” group
Receiver operating characteristic (ROC) curves showing how well the CALL checklists distinguish between controls and patients (left), and between “lexico-semantic” and “motor” groups (right). All data are shown in black, the Cambridge validation dataset is shown in red, and St George’s out-of-sample dataset is shown in blue. When comparing controls relative to all patients using the BDAE ‘cookie theft’ checklist, the area under the curve (AUC) was largest for the Cambridge development dataset (98.92%), followed by all data (98%) and the St George’s out-of-sample dataset (97.06%). Using the MLSE ‘beach scene’ checklist, the AUC was largest for the Cambridge development dataset (98.67%), followed by all data (98.5%) and the St George’s out-of-sample dataset (95.59%). When comparing “lexico-semantic” versus “motor” groups using the BDAE ‘cookie theft’ checklist, the AUC was largest for the Cambridge development dataset (98.87%), followed by all data (90.23%) and the St George’s out-of-sample dataset (73.63%). Using the MLSE ‘beach scene’ checklist, the AUC was largest for the Cambridge development dataset (94.53%), followed by all data (82.65%) and the St George’s out-of-sample dataset (60.07%).
Appendix: Cambridge Language List (CALL)
The scoresheets using the 15 words are shown below in Appendices 1 and 2. The first column shows the 15 target words. In the second column, the examiner marks if the participant produced each specific word or not. While variations in morpho-syntactic word forms (e.g., wearing instead of wear/worn) should be marked as correct, synonyms and related words (e.g., “woman” for “lady”) should not. The third column shows the specific “points” associated with each of the words that help to differentiate between healthy controls and patients. The “points” per word were derived directly from the LASSO coefficients. To make CALL easy to use, we rounded up the coefficient and cut-off values to 1 decimal point and then multiplied all values by ten.
As an example, for the BDAE ‘cookie theft’ picture, production of the word “doing” is credited with 8 “points”, whilst the production of “something” is debited with 6 “penalty points”. The total points for the words that each participant produced are calculated. If the summed value exceeds 60 then a “diagnosis” of control is more likely; a score below 60 denotes that the participant is more likely to be a patient. If an indicative diagnosis of patient results (from the scoring of column 3), then a similar secondary scoring process is undertaken. This time, the fourth column provides the positive (blue) and negative (red) “points” associated with the prediction of “lexico-semantic” vs. “motor” patient group membership. Again, the total positive and negative points for the words that each participant produced are calculated. If the summed value exceeds 21 then a “diagnosis” of “lexico-semantic” patient is more likely; a score below 21 denotes that the participant is more likely to be a “motor” patient. For a worked example of a representative svPPA patient, see Supplementary Table 9.
Appendix 1
Please see below the checklist scoresheets for the (A) BDAE ‘cookie theft’ picture narrative and (B) the MLSE ‘beach scene’ picture narrative differentiating healthy controls versus patients, and between “lexico-semantic” and “motor” groups.
Appendix 2
The checklist scoresheets for additional diagnostic differentiations. As in the main manuscript, we employed a hierarchical classification (i.e., controls versus patients; “lexico-semantic” versus “motor” groups) to the checklists as shown in Appendix 1 as the LASSO regressions for svPPA versus lvPPA, and nfvPPA versus PSP resulted in zero words for both pictures. Under each checklist below, we provide the within-group and out-of-sample validation accuracies to use for reference.
Acknowledgements
We thank our patients and their families for supporting this work.
Footnotes
↵† Joint senior authors.
For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.
This version of the manuscript has been revised based on the Reviewer comments during the peer review process.
References
