DOC screen completion time reflects executive function, speed of processing and fluency in an observational cohort study ======================================================================================================================== * Alisia Southwell * Sajeevan Sujanthan * Tera Armel * Elaine Xing * Arunima Kapoor * Xiao Yu Eileen Liu * Krista L. Lanctot * Nathan Herrmann * Brian J. Murray * Kevin E. Thorpe * Megan L. Cayley * Michelle N. Sicard * Karen Lien * Demetrios J. Sahlas * Richard H. Swartz ## Abstract **Background** The DOC screen was developed to identify Depression, Obstructive sleep apnea, and Cognitive impairment (“DOC” comorbidities) after stroke. Each component has its own score, but additional information may be gained from the time to complete the screen. Cognitive screening completion time is rarely used as an outcome measure. We assessed the added value of using DOC screen completion time as a predictor of impairment on detailed cognitive assessments. **Methods** Consecutive English-speaking new referrals to the stroke prevention clinic were consented to participate in detailed neuropsychological testing (n=437). DOC screen scores and times were compared to cognitive test scores using multiple linear regression and receiver operating characteristic (ROC) analysis. All linear regression analyses controlled for age, sex, years of education, and functional outcome as assessed by the modified Rankin score. **Results** Average completion time for the DOC screen was 3.8 ± 1.3 minutes. After accounting for age, sex and cognitive screen score, completion time was a significant independent predictor, of speed of processing (p = .002, 95% CI: -0.016 to -0.004), verbal fluency (p < .001, CI: -0.012 to -0.006) and executive (p = .004, CI: -0.006 to -0.001), but not memory, function. Completion time above 5.5 minutes (332.5 seconds) was associated with a high likelihood of impairment on gold standard executive (likelihood ratios 3.9-5.2) and speed of processing (likelihood ratio = 5.2) tasks. **Conclusions** DOC screen completion time is easy to collect in routine care and is independently associated with speed of processing, language and executive dysfunctions after stroke. People who take more than 5.5 minutes to complete the DOC screen are likely to have deficits in executive functioning and speed of processing. These domains can be challenging to screen for in stroke survivors, and this measure provides a simple, clinically feasible method to screen for these under-appreciated concerns. Clinical Trials Registration Identifier: [NCT02363114](http://medrxiv.org/lookup/external-ref?link_type=CLINTRIALGOV&access_num=NCT02363114&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F23%2F2023.02.15.23286004.atom) Clinical Trials URL: [https://clinicaltrials.gov/ct2/show/NCT02363114](https://clinicaltrials.gov/ct2/show/NCT02363114) ## Introduction Stroke is the leading cause of neurological disability in adults1 and survival after stroke is increasing.2–4 In addition to physical post-stroke deficits,5 approximately 30 to 50% of stroke survivors are affected by each of **d**epression, **o**bstructive sleep apnea (OSA), and **c**ognitive impairment (DOC).6–9 These DOC comorbidities are all associated with poorer functional outcomes,10 and an increased risk of mortality.11 The DOC screen was developed as a feasible and valid tool to screen and stratify stroke patients into high, intermediate, and low risk groups for DOC comorbidities to facilitate detection and management in high-volume stroke clinic settings.12 The screen is efficient, yet designed to maintain the construct validity of a delayed recall task. Eighty-nine percent of patients in stroke prevention clinics are able to complete the tool in <6 minutes (mean=4.2 minutes, SD=1.5).12 In validation studies, the cognitive component of the DOC score is helpful to quickly stratify people into “cognitively normal”, “cognitively impaired” and “need more assessment” groups, compared to more detailed cognitive testing.12Although the DOC completion time was originally collected as a way to assess feasibility, practitioners can record this measure when administering the DOC screen in clinical settings. Several studies have reported the average time taken to complete other well-known cognitive screens as feasibility demonstrations, including the Montreal Cognitive Assessment (MoCA; means ranging from 9.5 minutes – 11 minutes)13,14 and the Mini Mental State Examination (MMSE; means ranging from 8 minutes – 13.4 minutes).14,15 However, few studies have assessed the utility of using a cognitive screen’s completion time as a metric to evaluate underlying cognitive abilities, such as executive functioning. Executive dysfunction and delays in speed of processing are the most commonly reported cognitive impairments after stroke. The DOC screen specifically examines mood symptoms, cognitive (executive, memory and abstraction) dysfunction and OSA/fatigue – all of which could be associated with cognitive or psychomotor slowing.16 Screen completion time is an immediately available metric, requiring no additional effort from either patients or clinicians, that might reflect executive function. The objective of this study was to determine whether completion time for the DOC screen is a reliable reflection of cognitive dysfunction and whether a single completion time cut-point could indicate impairment. ## Methods All patients were recruited from the DOC feasibility and validity study.12 This study included English speaking (or English fluent) patients newly referred to stroke prevention clinics over a two-year period (n=1504). We excluded patients with severe aphasia, severe motor dysfunction and patients who were not fluent in English. Each eligible participant was administered the DOC screen (**Figure 1**) as a brief screen of depression, obstructive sleep apnea (OSA) and cognitive impairment. All DOC screens were timed from the beginning of the memory registration (first task) until the end of the 5-word free recall (final task). Chart abstractions by trained research members captured demographic and clinical data on all participants from patient charts using previously published and validated methods.17,18 ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/02/23/2023.02.15.23286004/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2023/02/23/2023.02.15.23286004/F1) Figure 1: The DOC screen (freely available for download at [www.docscreen.ca](http://www.docscreen.ca)). To reduce sampling bias, all consecutive patients from stroke prevention clinics who completed the DOC screen were asked to complete more detailed neuropsychological assessments, including a cognitive battery and formal mood assessments as outlined in the DOC feasibly study.12 A complete list of all mood and cognitive assessments completed as part of the DOC study is reported elsewhere.12 In this analysis, cognition was assessed using the 30-minute neuropsychological battery recommended by the NINDS-CSN.19 This cognitive battery consists of the: Controlled Oral World Association Test (phonemic fluency), Animal Naming task (semantic fluency), California Verbal Learning Test (CVLT), Digit Symbol Coding, and Trail Making Tests (TMT-A and TMT-B). All scores were normalized (z-score or scaled score) for age using age-matched norms from each respective test manual. CVLT and Animal Naming were also education-standardized.20,21 The study was approved by the institutional Research Ethics Board. ### Statistical Analysis Statistical analysis was performed using IBM SPSS Statistics for Windows version 24. Descriptive statistics, including means and standard deviations, were calculated for age, screen completion time, and number of years of education. To assess whether screen time reflects cognitive function, independent linear regression models were used to examine the association between DOC completion time and the scaled or z-scores of all neuropsychological subtests. Data from all participants was used in the regression models. A sensitivity analysis was performed using a complete case approach to assess whether missing variables affected the models. All models controlled for age, education, modified Rankin Score (mRS) and sex. Due to the established relationship between the DOC cognitive sub-scores and detailed cognitive assessments,12 we also controlled for the DOC-Cognition score in all models. Significance was set at p < 0.05 for all analyses. To identify whether a single cut point (in seconds) for screen time could be found with high specificity and likelihood ratios for cognitive impairment, receiver operating characteristic (ROC) curves were used. ROC analyses were run for each neuropsychological assessment significantly associated with the DOC screen completion time. A logistic regression with screen time completion and the impairment classification on the NINDS-CSN assessments was applied to the ROC curves. The classification of impairment of NINDS-CSN was defined as scores >2.0 SD from expected norms, on 2 or more cognitive tasks. This required participants to have completed all tests in the detailed cognitive battery, thus a complete case approach was used for all ROC analyses. First, a single, specific cut-point (time in seconds) was defined based on the ROC curve output for patients with an overall classification of impaired on the NINDS-CSN battery. The cut-point was pre-specified to have 95% specificity for impairment. This cut-point was then applied to ROC curves from each individual assessment and evaluated using likelihood ratios (LR). ## Results 437 patients completed cognitive and mood gold standard assessments (Supplemental Table 1). 213 (48.7%) participants were male, the mean (± standard deviation) age was 62.7 ± 15.6 years, and the mean years of education was 15.6 ± 3.9 years (**Table 1**). The DOC screen completion mean was 3.8 ± 1.3 minutes (range: 1.9-9.6 minutes). One hundred and thirty-four (31%) patients had an ischemic stroke, 138 (32%) had a probable/possible TIA, and the remainder (37%) were diagnosed with other conditions (**Table 1**). Non-stroke/TIA diagnoses included patients referred with possible stroke symptoms, but whose further investigations revealed alternative diagnoses, as well as patients without specific stroke/TIA symptoms referred for either vascular risk reduction or assessment of incidental abnormal imaging findings. View this table: [Table 1:](http://medrxiv.org/content/early/2023/02/23/2023.02.15.23286004/T1) Table 1: Demographics for participants completing detailed cognitive and neuropsychological assessments (n = 437). We performed linear regressions with DOC screen completion time (seconds) as a predictor for each neuropsychological assessment score (**Table 2**). In all models we controlled for age, sex, years of education, screening score of cognitive function (DOC-Cognition score), and overall function (mRS). All regression models for screen completion time were significant (p < .001) (Supplemental Table 2). Additionally, model summaries showed that screen completion time was a significant predictor (p < 0.005) of: verbal fluency semantic score (95% Confidence Interval (CI) of Beta-coefficient from linear regression: -.006 to -.001), verbal fluency phonemic Score (95% CI: -.018 to -.006), Digit Symbol Coding (95% CI: -.016 to -.004) and the Trail Making Tests (TMT-A 95% CI: -.017 to -.005; TMT-B 95% CI: -.016 to -.004). In all cases, these were negative correlations (i.e., longer completion times correlated with poorer cognitive scores). DOC screen completion time was not a significant predictor of memory performance on the CVLT Short Delay Free Recall (p=.713, 95% CI: -.003 to .002) or the CVLT Long Delay Free Recall (p=.790, 95% CI: -.002 to .003). Results did not differ in the sensitivity models requiring complete case data (see Table 2 compared to Supplemental Table 3 with complete case data). View this table: [Table 2:](http://medrxiv.org/content/early/2023/02/23/2023.02.15.23286004/T2) Table 2: Linear Regression results showing the effect of DOC screen completion time on individual neuropsychological assessments. Using the single cut-off point approach on the overall impairment ROC curve (Figure 2, Table 3A), the point with 95% specificity for impairment was 332.5 seconds. When this time was applied to ROC models for each individual cognitive task (Table 3B), the same cut-point had high specificity on all executive and speed of processing tasks. The area under the curve was greater than 0.7 for all executive and speed of processing tasks. Likelihood ratios for predicting abnormal results on executive and speed of processing tasks ranged from four to six – that is, people taking more than 332.5 seconds to complete the DOC screen were 4-6 times more likely to have severe impairment on executive and speed of processing tasks than those with faster completion times (see Table 3). View this table: [Table 3:](http://medrxiv.org/content/early/2023/02/23/2023.02.15.23286004/T3) Table 3: – ROC model outputs comparing DOC screen completion time with full neuropsychological assessments, with a cut off set at **332.5 seconds (95% specificity) obtained from the model for overall impairment**. ## Discussion Several studies22 have shown that post-stroke impairments can be separated into independent cognitive factors including language, memory and executive function, with deficits in executive functioning and speed of processing being the most common.23 Screening tests for executive function and speed of processing are limited in routine clinical care. These results demonstrate that DOC screen completion time is an independent predictor of executive function (semantic fluency,24 TMT-B25), speed of processing (Digit symbol coding,26 TMT-A and B27) and verbal fluency28 after stroke, even after controlling for age, sex, education, DOC-cognition score and stroke severity. Completion time did not predict CVLT scores, a verbal test primarily affecting verbal memory (learning/registration and recall).29 Verbal fluency, while reflecting language function, is also reflective of executive function.30 Moreover, we have demonstrated that a 332.5 second (roughly 5.5 minutes) cut-off has 95 % specificity and high likelihood ratios for predicting both overall cognitive and executive function impairment. This can be used as a quick and easily obtainable measure to identify people most likely to be impaired on executive and speed of processing tasks. A few notable neuropsychological measures have used completion time to assess specific cognitive functions. For instance, Trail Making Tests (TMTs) are a set of widely accepted timed neuropsychological measures that provide insight into executive abilities.27 Processing speed is highly associated with performance on TMT Part B, which involves decision-making skills comparable to measures of executive function.25,31 Similarly, Woods et al. discovered that a patient’s question completion time on self-paced questionnaires could be used as a measure of executive functioning.32 Question completion time measures processing and decision-making speeds, providing insight into motivation, effort, and cognitive ability that is not measured by existing tests.32 These studies support the notion that timed measures may be useful as a measure of executive dysfunction in addition to their use as diagnostic screening instruments. The findings presented in our study correspond well to those reported by Woods et al. Their analysis showed that complex tasks, akin to our DOC-Cognitive tasks, were strongly related to executive function and processing speed. Their neuropsychological tests (including TMT-B and Digit Span) also correlated significantly with self-paced question completion time. Their research process was similar to ours, wherein completion time was compared to existing screens to validate completion time as a metric; both studies suggest that completion time of self-paced complex assessments may be valid markers of executive function. Few studies use completion time of a neuropsychological screening tool as a cognitive marker. Most screens (like the MoCA or MMSE) are not routinely timed when applied in clinical settings. Moreover, executive function deficits are not often assessed in stroke patients; these deficits are subtle, challenging to test for, and often go unrecognized.23 Measures that are quick and easy to administer, such as DOC screen completion time, may aid in detecting executive dysfunction. The interpretation of our findings is limited by our sample population. Compared to the total number of patients who were asked to volunteer from the stroke prevention clinic (n=1504), consenting participants (n=437) tended to be slightly younger and with slightly milder neuropsychological deficits (healthy participant bias).12 However, our sample also included a wide range of patients across the full spectrum of severity. As expected from stroke/TIA clinic samples, 62% had a diagnosis of stroke and/or TIA, and the rest had alternative diagnoses common in stroke prevention clinics (mimics, multiple vascular risk factors, abnormal imaging). This heterogeneity reflects the pragmatic nature of the screening and its broad generalizability to the population of patients referred to stoke prevention clinics. TIA patients are well recognized to share similar long-term risk profiles33 and are also at risk for cognitive impairment,34 compared to those with imaging confirmed strokes. The relationship between timing and gold standard testing was found across a range of severity from normal function to severely impaired. It should also be noted that there is not a single perfect cut-off score for DOC completion time that indicates executive dysfunction. To facilitate clinical utility, and because this is intended as a screen in high-volume clinics, we chose to explore a cut-off with high specificity so clinicians could be confident there was a high likelihood of true impairment beyond this time; however, this cut-off will have a low sensitivity and will miss some people with impairments. Previous work has already established that the DOC-cog score can also be a sensitive screen, effectively ruling out cognitive impairment in people who score highly.12 Finally, it is important to note that although screen completion time may be a useful tool to identify people likely to have executive dysfunction, it is still not equivalent to a detailed neuropsychological assessment. ## Conclusion Clinical cognitive screening tools have not commonly used completion time as a metric. We aimed to determine whether the DOC screen completion time could provide clinically relevant information on patients’ cognitive function. DOC screen completion time reflects executive function, speed of processing and verbal fluency. When administering the DOC screen, completion time requires no additional time or patient burden to collect. This convenience is vital in busy stroke prevention clinic settings, where there is minimal time for detailed cognitive assessments. Exploring whether screen time can act as a predictor of future outcomes would provide further support the utility of this measure in clinical settings. ## Data Availability All data is not available. DOC screening was performed as part of clinical care and used for research under REB approval with waiver of consent, on the condition of restricted data sharing. ## Sources of Funding This study was supported by CIHR (Grant Ref No. 137038) and HSF (Project No. 000392). RHS received salary support for research from an Ontario Clinician Scientist (Phase II) Award from the HSF Canada and from the Department of Medicine (Neurology) at Sunnybrook and University of Toronto. In-kind support for data storage was provided by the Ontario Brain Institute and the Ontario Neurodegenerative Disease Research Initiative (ONDRI). ## Disclosures RHS reports ownership shares in FollowMD Inc., a virtual vascular risk reduction clinic. None of the other authors have any conflicts of interest to disclose. ## Acknowledgments We would like to acknowledge the DOC participants who volunteered their time for the study. ## Non-Standard Abbreviations and Acronyms CVLT : California Verbal Learning Test DOC : Depression, Obstructive Sleep Apnea, Cognitive Impairment MMSE : Mini Mental State Examination MoCA : Montreal Cognitive Assessment OSA : Obstructive Sleep Apnea QCT : Question Completion Time TMT : Trail Making Test * Received February 15, 2023. * Revision received February 15, 2023. * Accepted February 23, 2023. * © 2023, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. 1.Feigin VL, Nichols E, Alam T, et al. Global, regional, and national burden of neurological disorders, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol. 2019;18(5):459–480. doi:10.1016/S1474-4422(18)30499-X [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1474-4422(18)30499-X&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F23%2F2023.02.15.23286004.atom) 2. 2.Lakshminarayan K, Berger AK, Fuller CC, et al. Trends in 10-Year Survival of Patients With Stroke Hospitalized Between 1980 and 2000. Stroke. 2014;45(9):2575–2581. doi:10.1161/STROKEAHA.114.005512 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjQ1LzkvMjU3NSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzAyLzIzLzIwMjMuMDIuMTUuMjMyODYwMDQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 3. 3.Rodríguez-Castro E, López-Dequit I, Santamaría-Cadavid M, et al. Trends in stroke outcomes in the last ten years in a European tertiary hospital. BMC Neurol. 2018;18(1):164. doi:10.1186/s12883-018-1164-7 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12883-018-1164-7&link_type=DOI) 4. 4.Waziry R, Heshmatollah A, Bos D, et al. Time Trends in Survival Following First Hemorrhagic or Ischemic Stroke Between 1991 and 2015 in the Rotterdam Study. Stroke. 2020;51(3):824–829. doi:10.1161/STROKEAHA.119.027198 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1161/STROKEAHA.119.027198&link_type=DOI) 5. 5.Verstraeten S, Mark R, Sitskoorn M. Motor and Cognitive Impairment after Stroke: A Common Bond or a Simultaneous Deficit? Stroke Res Ther. 2016;1(1). 6. 6.Herrmann N, Seitz D, Fischer H, et al. Detection and treatment of post stroke depression: results from the registry of the Canadian stroke network. Int J Geriatr Psychiatry. 2011:n/a-n/a. doi:10.1002/gps.2663 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/gps.2663&link_type=DOI) 7. 7.Johnson KG, Johnson DC. Frequency of sleep apnea in stroke and TIA patients: a meta-analysis. J Clin Sleep Med. 2010;6(2):131–137. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.5664/jcsm.27760&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20411688&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F23%2F2023.02.15.23286004.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000276788500004&link_type=ISI) 8. 8.Patel MD, Coshall C, Rudd AG, Wolfe CDA. Cognitive Impairment after Stroke: Clinical Determinants and Its Associations with Long-Term Stroke Outcomes. J Am Geriatr Soc. 2002;50(4):700–706. doi:10.1046/j.1532-5415.2002.50165.x [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1046/j.1532-5415.2002.50165.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11982671&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F23%2F2023.02.15.23286004.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000174963600014&link_type=ISI) 9. 9.Kapoor A, Lanctôt KL, Bayley M, et al. “Good Outcome” Isn’t Good Enough. Stroke. 2017;48(6):1688–1690. doi:10.1161/strokeaha.117.016728 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjQ4LzYvMTY4OCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzAyLzIzLzIwMjMuMDIuMTUuMjMyODYwMDQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 10. 10.Kapoor A, Lanctot KL, Bayley M, Herrmann N, Murray BJ, Swartz RH. Screening for Post-Stroke Depression and Cognitive Impairment at Baseline Predicts Long-Term Patient-Centered Outcomes After Stroke. J Geriatr Psychiatry Neurol. 2019;32(1):40–48. doi:10.1177/0891988718819859 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0891988718819859&link_type=DOI) 11. 11.Swartz RH, Bayley M, Lanctôt KL, et al. Post-stroke depression, obstructive sleep apnea, and cognitive impairment: Rationale for, and barriers to, routine screening. Int J Stroke. 2016;11(5):509–518. doi:10.1177/1747493016641968 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1747493016641968&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27073189&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F23%2F2023.02.15.23286004.atom) 12. 12.1. Romigi A Swartz RH, Cayley ML, Lanctôt KL, et al. The “DOC” screen: Feasible and valid screening for depression, Obstructive Sleep Apnea (OSA) and cognitive impairment in stroke prevention clinics. Romigi A, ed. PLoS One. 2017;12(4):e0174451. doi:10.1371/journal.pone.0174451 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0174451&link_type=DOI) 13. 13.Lees RA, Hendry BA K, Broomfield N, Stott D, Larner AJ, Quinn TJ. Cognitive assessment in stroke: feasibility and test properties using differing approaches to scoring of incomplete items. Int J Geriatr Psychiatry. 2017;32(10):1072–1078. doi:10.1002/gps.4568 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/gps.4568&link_type=DOI) 14. 14.Barnay J-L, Wauquiez G, Bonnin-Koang HY, et al. Feasibility of the Cognitive Assessment scale for Stroke Patients (CASP) vs. MMSE and MoCA in aphasic left hemispheric stroke patients. Ann Phys Rehabil Med. 2014;57(6-7):422–435. doi:10.1016/j.rehab.2014.05.010 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.rehab.2014.05.010&link_type=DOI) 15. 15.Molloy DW. Standardised Mini-Mental State Examination (SMMSE): Guidelines for Administration and Scoring Instructions. Australia; 2014. 16. 16.Schrijvers D, Hulstijn W, Sabbe BGC. Psychomotor symptoms in depression: A diagnostic, pathophysiological and therapeutic tool. J Affect Disord. 2008;109(1-2):1–20. doi:10.1016/j.jad.2007.10.019 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jad.2007.10.019&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18082896&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F23%2F2023.02.15.23286004.atom) 17. 17.Hall R, Khan F, O’Callaghan C, et al. Ontario Stroke Evaluation Report 2013: Spotlight on Secondary Stroke Prevention and Care. Toronto; 2013. [https://www.ices.on.ca/Publications/Atlases-and-Reports/2013/Ontario-Stroke-Evaluation-Report-2013.aspx](https://www.ices.on.ca/Publications/Atlases-and-Reports/2013/Ontario-Stroke-Evaluation-Report-2013.aspx). 18. 18.Kapral MK, Fang J, Hill MD, et al. Sex Differences in Stroke Care and Outcomes. Stroke. 2005;36(4):809–814. doi:10.1161/01.STR.0000157662.09551.e5 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjg6IjM2LzQvODA5IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDIvMjMvMjAyMy4wMi4xNS4yMzI4NjAwNC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 19. 19.Hachinski V, Iadecola C, Petersen RC, et al. National Institute of Neurological Disorders and Stroke–Canadian Stroke Network Vascular Cognitive Impairment Harmonization Standards. Stroke. 2006;37(9):2220–2241. doi:10.1161/01.STR.0000237236.88823.47 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjM3LzkvMjIyMCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIzLzAyLzIzLzIwMjMuMDIuMTUuMjMyODYwMDQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 20. 20.Tombaugh T. Normative Data Stratified by Age and Education for Two Measures of Verbal Fluency FAS and Animal Naming. Arch Clin Neuropsychol. 1999;14(2):167–177. doi:10.1016/S0887-6177(97)00095-4 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0887-6177(97)00095-4&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=14590600&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F23%2F2023.02.15.23286004.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000078678200001&link_type=ISI) 21. 21.Delis DC, Kramer JH, Kaplan E, Ober BE. California Verbal Learning Test | Second Edition. Second. San Antonio: The Psychological Corporation; 2000. 22. 22.Swartz RH, Stuss DT, Gao F, Black SE. Independent Cognitive Effects of Atrophy and Diffuse Subcortical and Thalamico-Cortical Cerebrovascular Disease in Dementia. Stroke. 2008;39(3):822–830. doi:10.1161/STROKEAHA.107.491936 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjg6IjM5LzMvODIyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDIvMjMvMjAyMy4wMi4xNS4yMzI4NjAwNC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 23. 23.Zinn S, Bosworth HB, Hoenig HM, Swartzwelder HS. Executive Function Deficits in Acute Stroke. Arch Phys Med Rehabil. 2007;88(2):173–180. doi:10.1016/j.apmr.2006.11.015 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.apmr.2006.11.015&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17270514&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F23%2F2023.02.15.23286004.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000244148400006&link_type=ISI) 24. 24.Maseda A, Lodeiro-Fernández L, Lorenzo-López L, Núñez-Naveira L, Balo A, Millán-Calenti JC. Verbal fluency, naming and verbal comprehension: three aspects of language as predictors of cognitive impairment. Aging Ment Health. 2014;18(8):1037–1045. doi:10.1080/13607863.2014.908457 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/13607863.2014.908457&link_type=DOI) 25. 25.Arbuthnott K, Frank J. Trail Making Test, Part B as a Measure of Executive Control: Validation Using a Set-Switching Paradigm. J Clin Exp Neuropsychol. 2000;22(4):518–528. doi:10.1076/1380-3395(200008)22:4;1-0;FT518 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1076/1380-3395(200008)22:4;1-0;FT518&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10923061&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F23%2F2023.02.15.23286004.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000088550400009&link_type=ISI) 26. 26.Deloire MS, Bonnet MC, Salort E, et al. How to detect cognitive dysfunction at early stages of multiple sclerosis? Mult Scler J. 2006;12(4):445–452. doi:10.1191/1352458506ms1289oa [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1191/1352458506ms1289oa&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16900758&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F23%2F2023.02.15.23286004.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000239431200010&link_type=ISI) 27. 27.Muir RT, Lam B, Honjo K, et al. Trail Making Test Elucidates Neural Substrates of Specific Poststroke Executive Dysfunctions. Stroke. 2015;46(10):2755–2761. doi:10.1161/STROKEAHA.115.009936 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjEwOiI0Ni8xMC8yNzU1IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDIvMjMvMjAyMy4wMi4xNS4yMzI4NjAwNC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 28. 28.Whiteside DM, Kealey T, Semla M, et al. Verbal Fluency: Language or Executive Function Measure? Appl Neuropsychol Adult. 2016;23(1):29–34. doi:10.1080/23279095.2015.1004574 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/23279095.2015.1004574&link_type=DOI) 29. 29.Jacobs M, Donders J. Criterion validity of the California Verbal Learning Test-Second Edition (CVLT-II) after traumatic brain injury. Arch Clin Neuropsychol. 2007;22(2):143–149. doi:10.1016/j.acn.2006.12.002 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.acn.2006.12.002&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17207963&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F23%2F2023.02.15.23286004.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000245539300002&link_type=ISI) 30. 30.Aita SL, Beach JD, Taylor SE, Borgogna NC, Harrell MN, Hill BD. Executive, language, or both? An examination of the construct validity of verbal fluency measures. Appl Neuropsychol Adult. 2019;26(5):441–451. doi:10.1080/23279095.2018.1439830 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/23279095.2018.1439830&link_type=DOI) 31. 31.MacPherson SE, Cox SR, Dickie DA, et al. Processing speed and the relationship between Trail Making Test-B performance, cortical thinning and white matter microstructure in older adults. Cortex. 2017;95:92–103. doi:10.1016/j.cortex.2017.07.021 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.cortex.2017.07.021&link_type=DOI) 32. 32.Woods DL, Yund EW, Wyma JM, Ruff R, Herron TJ. Measuring executive function in control subjects and TBI patients with question completion time (QCT). Front Hum Neurosci. 2015;9. doi:10.3389/fnhum.2015.00288 [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3389/fnhum.2015.00288&link_type=DOI) 33. 33.Edwards JD, Kapral MK, Fang J, Swartz RH. Long-term morbidity and mortality in patients without early complications after stroke or transient ischemic attack. Can Med Assoc J. 2017;189(29):E954–E961. doi:10.1503/cmaj.161142 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTg5LzI5L0U5NTQiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMy8wMi8yMy8yMDIzLjAyLjE1LjIzMjg2MDA0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 34. 34.Pendlebury ST, Wadling S, Silver LE, Mehta Z, Rothwell PM. Transient Cognitive Impairment in TIA and Minor Stroke. Stroke. 2011;42(11):3116–3121. doi:10.1161/STROKEAHA.111.621490 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjEwOiI0Mi8xMS8zMTE2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDIvMjMvMjAyMy4wMi4xNS4yMzI4NjAwNC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)