Abstract
INTRODUCTION Dementia assessment includes cognitive and behavioral testing with informant validation. Conventional testing is resource intensive, with uneven access. Online unsupervised assessments could reduce barriers to risk assessment. We interrogated the relationship between informant-rated behavioral changes and neuropsychological test performance in older adults in the Brain Health Registry.
METHODS Participants completed online unsupervised cognitive tests, and informants completed the Mild Behavioral Impairment Checklist via a Study Partner portal. Cognitive performance was evaluated in MBI+/- individuals, as was the association between cognitive scores and MBI symptom severity.
RESULTS Mean age of the 499 participants was 67, 61% of which were female. MBI+ participants had lower working memory and executive function test scores. Lower cognitive test scores associated with greater MBI burden.
DISCUSSION Our findings support the feasibility of remote, informant-reported behavioral assessment and support its validity by demonstrating a relationship to cognitive test performance using online unsupervised assessments for dementia risk assessment.
Systematic review The authors searched MEDLINE and Google Scholar for studies linking Mild Behavioral Impairment (MBI) and cognition in non-demented older adults. Most studies have utilized transformed Neuropsychiatric Inventory scores to assess MBI, and relatively few using the novel MBI-checklist (MBI-C), with the largest study using self-report. Exploration of informant reports of MBI is important due to impaired insight that may accompany neuropsychiatric symptoms.
Interpretation Older adults with online, informant reported MBI had poorer performance in memory and executive function measured using online neuropsychological testing compared to those without MBI. These findings are consistent with the current literature and suggest that the MBI-C may serve as a marker for poorer cognitive performance.
Future directions Our data support the role of online testing of cognition and behavior for risk assessment. This approach to evaluate behavior and cognition can be explored further, to determine if it is a scalable, online approach to detection of neurodegenerative disease.
1. Background
Access to dementia assessments is uneven in North America and across the world. This disparity has important clinical repercussions, particularly in regions where specialized resources are limited, and identification is delayed until later manifestations. The development of unsupervised platforms that do not require highly trained administrators may resolve this dual impasse and improve clinical outcomes. Moreover, they may create a low-cost recruitment infrastructure for early intervention trials, where no disease-modifying drug in Alzheimer’s disease (AD) has met all primary endpoints1,2 in part due to poor recruitment of individuals without overt impairment or who are in the earliest stages of disease3,4. Advances in online services open the possibility of assessment portals in any region with an internet connection, for any individual with access to the internet and a computing device. Moreover, cognitive tests have been computerized and can be delivered without an administrator, and with convergent validity with those administered in tertiary cognitive assessment centers5-9.
The addition of behavioral assessments to online platforms may provide additional relevant information. Neuropsychiatric symptoms (NPS) such as agitation, anxiety, apathy, depression, and psychosis are considered core features of dementia and are associated with poorer patient outcomes10. However, NPS can often precede cognitive symptoms, including in 30% of those who develop AD11. Mild behavioral impairment (MBI) is a pre-dementia neurobehavioral syndrome characterized by the de novo emergence and persistence of NPS in older adults representing a change from longstanding patterns of behavior12. MBI is associated with amyloid, tau, and neurodegeneration13-19, and a greater risk of incident cognitive decline and dementia20-26. Incorporating MBI into screening may provide a complementary approach to early detection4. However, informant information is often required to validate the syndrome, and structured assessment tools suitable for widespread dissemination through unsupervised platforms have only recently been developed. The Mild Behavioral Impairment Checklist (MBI-C) incorporates informant information and is the validated case ascertainment instrument developed specifically to capture MBI in accordance with the criteria developed by the International Society to Advance Alzheimer’s Research and Treatment-Alzheimer’s Association (ISTAART-AA)27-30. Translated into over 20 languages, the MBI-C may also allow a broader reach for obtaining online informant reports of behavioral change.
The aim of this study was to investigate informant-based MBI in an online unsupervised platform, the Brain Health Registry (BHR), capable of assessing early dementia risk markers31. We determined the utility of the BHR for converging assessments of cognitive and behavioral symptoms using neuropsychological testing and informant-reported MBI-C. We hypothesized that participants with MBI+ status would have poorer cognitive performance measured by the Lumos test battery. We further hypothesized that individuals with poorer memory, executive function, processing speed, and inhibitory control would have a higher burden of MBI symptoms.
2. Methods
2.1 Brain Health Registry
The BHR31 is an internet-based public registry and cohort that recruits participants using a variety of methods including a website, social media, brochures, and online advertising. All participants are required to give informed consent with an online consent form. Upon completion of the consent form, participants may complete questionnaires regarding personal and family medical history, early childhood history, sleep quality, diet, quality of life scales, psychiatric symptomatology, as well as online cognitive testing via Lumosity32, CogState33, or Memtrax34 tests. Additionally, study partners of BHR participants can register on the BHR study partner portal, on which informant-rated measures are completed35.
2.2 Study Participants
Participants were included if: 1) Lumosity cognitive tests were completed; and 2) their informant completed the MBI-C via the BHR study partner portal within a year of the cognitive tests. Participants were excluded if they reported: 1) developmental or learning disorders; 2) neurological conditions such as movement disorders, multiple sclerosis, traumatic brain injury; 3) current or past psychiatric diagnoses including schizophrenia, major mood or anxiety disorders, or PTSD.
2.3 Study Variables
2.3.1 Lumosity online Forward Memory Span
The assessment of Forward Memory Span is based on the Corsi block-tapping tasks36. The participant is asked to recall the sequence of circles in the same order it was presented. The length of the sequence increases by one every two trials. The session comes to an end when the participant records two incorrect answers at the same span level. This task is used as a measure of visual short-term memory and attention.
2.3.2 Lumosity online Reverse Memory Span
The Reverse Memory Span task is a slightly altered version of the original Corsi block-tapping tasks. It is identical to the forward visual memory span assessment, with the exception that the participant is asked to recall the sequence of circles in the reverse order. This reverse task is used as a measure of visual working memory and attention.
2.3.3 Lumosity online Trail Making Test B
In Trail Making Test (TMT) B, blue circles (numbered 1 to 12) and capital letters (A to L) are arranged in 6 possible layouts with non-overlapping spatial locations. The participant must alternate between numbers and letters for this task, clicking in increasing order. When the blue circle is clicked, it turns orange and a straight line appears to connect the circles. The timer for the task begins when the participants clicks the first circle. If the participant records an incorrect click, an X appears on their screen and they are required to go back to the previous circle. For this study, we included the response time and number of errors as measures of processing speed attention and sequencing ability.
2.3.4 Lumosity online Go/No-Go
In the Go/No-Go assessment, participants are presented with target pictures and distractor stimuli. The target picture is chosen from a set of photos of fruit. Each stimulus appears after a random delay between 1000 and 3000 ms to discourage anticipatory responding. The participant is instructed to respond as quickly as possible within 1500ms. The assessment ends when a participant responds to ten “Go” trials. If the participant submits three incorrect responses (responding to “no-go” or failing to respond to “go”), the participant will restart the task. The participant is given feedback on timing and correctness. This assessment is used to measure response inhibition and speed of information processing.
2.3.5 Mild Behavioral Impairment Checklist
The MBI-C is included in the BHR study partner portal and is therefore completed by an informant. The MBI-C is explicit that symptoms are de novo in later life, represent a change from longstanding patterns of behavior, and are persistent for at least six months. The MBI-C consists of questions in the five MBI domains of apathy, mood and anxiety, agitation and impulsivity, impaired social cognition, and psychosis, with items geared towards capturing NPS in community dwelling, functionally independent, non-demented older adults. The scale takes ∼7-8 minutes to complete, consisting of 34 questions; scoring is from 0-3, representing absent, mild, moderate, and severe changes, with a total score range of 0-10228.
2.4 Statistical Analysis
Continuous demographic variables (age and years of education) were analyzed using independent sample t-tests to compare MBI+ and MBI-groups; sex distribution between the two groups was analyzed using chi-square tests. MBI-C was dichotomized based on a validation in primary care non-demented older adults in which scores of >7 differentiated MBI+ from MBI-with a sensitivity of 0.93, specificity of 0.76 and AUC of 0.9337. As exploratory analyses, cutpoints of >5 and >6 were also analyzed. Univariate Analysis of Covariance (ANCOVA) was used to compare performance on Lumosity cognitive tests between MBI+ and MBI-groups, covarying for age, sex, education, and neuropsychological and neurobehavioral assessment interval. Skewed data were log-transformed, however the TMT response time variable was analyzed with a negative binomial regression due a skewed distribution with an overrepresentation of zeros. For Go/No-Go errors, ordinal logistic regression was performed because the response variable only had three possible values: 0, 1, and 2.
Additionally, negative binomial regressions were fitted to assess Lumosity task prediction of MBI-C total scores. Negative binomial regression is preferred when the data are skewed, as in this sample where the mode on the MBI-C is zero indicating no emergent and persistent NPS. Lumosity task measures as continuous scores were the independent variables in these models. The covariates included were age, sex, education, and neuropsychological and neurobehavioral assessment interval. The p values for Lumosity task measures were calculated using likelihood ratio tests.
Statistical analyses were performed using SPSS v26 and R 3.6.2.
3 Results
Participant selection is described in Figure 1. The final sample included 499 participants with a mean age of 67 (SD 10.4), of which 308/499 were females (61%) (Table 1). The number of MBI+ participants was 31 (6.2%) (Figure 1). A significantly greater number of men were classified as MBI+ (64%, p=0.002). MBI+ participants had significantly poorer forward memory span (mean sequence length of 4.68 vs. 5.26, p=0.005; Figure 2a), poorer reverse memory span (3.81 vs. 4.85, p<0.0001; Figure 2b), more TMT errors (4.29 vs. 1.85, p=0.01; Figure 2c), and longer TMT completion time (67.67 vs. 45.08 seconds, p<0.0001; Figure 2d). MBI was not associated with the number of errors (p=0.84) and response time (p=0.16) on the Go/No-Go task (Figures 2e & 2f). The effect sizes for these differences were modest. Of the tests that significantly differed between groups, the largest effect size (Cohen’s f) was for reverse memory span (0.20), followed by TMT response time (0.20), memory span (0.13), and TMT accuracy (0.11). See Tables 2-4 for statistical reporting. Analyses using cutpoints of >5 and >6 for MBI-C show very similar results and are included in supplemental tables (Supplemental tables 1-6).
Negative binomial regressions utilizing Lumosity scores to predict MBI score determined that worse memory span (X2(1, N=499)=6.6, p=0.01), worse reverse memory span X2(1, N=498)=5.4, (p= 0.02), more TMT errors (X2(1, N=499)=5.8, p=0.02) and longer TMT response time (X2(1, N=499)=9.6, p=0.002) were all associated with higher MBI-C total scores. Go/No-Go errors (X2(1, N=497)=0.16, p=0.69) and Go/No-Go response time (X2(1, N=497)=0.97, p=0.33) were not associated with MBI-C score (Table 5).
4 Discussion
In a sample of 499 participant dyads in BHR, we demonstrated the feasibility of delivering unsupervised online assessments of behavioral and cognitive markers of dementia risk. Utilizing the validated cut off score of >7 on the MBI-C, MBI+ status was associated with significant differences in memory and executive function, measured using memory span, reverse memory span, TMT errors and TMT speed. Further, significant associations were found between poorer objectively measured cognitive performance, in the domains of memory and executive function, and MBI symptom severity. Effect sizes were small, ranging from 0.11-0.20. The findings do suggest that a simple informant reported behavioral measure completed via an online portal might be a relevant addition to neuropsychological testing, warranting further study in BHR. In other work, MBI+ status has demonstrated significant and meaningful associations with incident cognitive decline and dementia across several studies, settings, and populations20-26. Thus, while convergent with tests of memory and executive function, behavioral and cognitive markers of risk may be distinct, potentially offering complementary measures of risk.
Our data indicate that the BHR is an effective platform to conduct remote assessments of cognitive functioning with convergence of behavioral and cognitive tests. Poorer performance on unsupervised online neuropsychological testing has been associated with self-report MCI and AD8. Online participant testing is an efficient and reliable tool for neuropsychological testing, which can identify performance decrements in executive dysfunction and memory32. Similarly, online informant reports such as those collected in the BHR study partner portal are valuable and informative. Online study-partner reported cognitive decline is comparable to data collected in clinic, is associated with objectively defined participant cognition35, and is associated with amyloid, clinical diagnosis of dementia due to AD, and in-clinic cognitive screening test scores5. In our study, online informant-reported behavioral symptoms associated with differences in memory and executive function collected via the participant portal. This harmonized utilization of participant and study partner portals is effective and can allow continuation of research activities even during trying times such as the recent pandemic, where consistent in person visits between clinicians and patients were not feasible. Although the COVID-19 pandemic has highlighted the need for alternative infrastructure to allow continued care, the tools that have been developed may permit the assessment of older adults who for physical, social, or cognitive reasons could not previously access care. This approach also allows outreach to areas less accessible to academic centers.
The cognitive domain differences detected with the MBI-C include memory and executive function, which are relevant and important for AD risk38. Early decline in memory and executive function has been shown to be associated with the preclinical disease process, thus, early detection of reductions in cognitive functioning may be useful in identifying populations at risk39,40. Both memory and executive function are important endpoints in AD trials41. Longitudinal cohorts have demonstrated that an acceleration of decline in memory performance occurs 3-4 years before a diagnosis of MCI and 7 years before a diagnosis of AD, while for executive function an accelerated decline occurs 2-3 years before AD diagnosis42,43. The finding of small but significant associations between MBI and poorer memory and executive function performance is consistent with the evolving description of the cognitive profile of MBI44-46. These findings are consistent with a previous study using the UK online PROTECT study portal in which self-reported MBI, measured with the MBI-C, was associated with faster decline in attention and working memory at one year in older adults with normal cognition20. A subsequent analysis of cognitively normal PROTECT participants, with a median follow up time of 3 years, demonstrated an association between baseline informant-reported MBI+ status and decline in measures of working memory and fluid intelligence26. In an overlapping sample, AD genetic risk was determined using polygenic risk scores. AD genetic risk was associated with worse cognition in the informant-reported MBI+ group but not in the MBI-group. The strongest association was in those with more severe MBI, aged ≥6547. These convergent findings support leveraging online cognitive and behavioral measures to explore dementia risk. In a recent study of National Alzheimer Coordinating Center data, the combination of informant-reported MBI and subjective cognitive decline (SCD) had a greater risk of incident cognitive and functional decline at three years compared to either MBI or SCD alone24. Further, in a subsequent study, those with SCD and MBI together had a shorter median time to incident MCI compared to those with SCD in the absence of MBI48. Taken together the results suggest that individuals with subtle neuropsychological test score differences and MBI together may be at higher risk for cognitive and functional decline.
As the case ascertainment instrument developed to measure MBI in accordance with the ISTAART-AA MBI criteria, the MBI-C was designed to: 1) operationalize the MBI concept; 2) measure a selected list of NPS which may help identify preclinical or prodromal dementia; and 3) predict risk of several dementias28. This instrument has been validated in an online cohort of cognitively normal older adults27, a primary care sample with SCD29 or MCI49, and a cognitive neurology clinic population with SCD and MCI50.
However, limitations in our study may affect interpretation and generalizability. These limitations include high education levels and restriction to participants and study-partners who can successfully complete tasks online35. Since the BHR is an online self-report database, the lack of a clinical diagnosis within the sample group is a potential source of error. BHR participants may have undiagnosed and/or unreported neurodegenerative disease or psychiatric disorders, which may be associated with greater MBI score. While online cognitive testing has been validated8,33, further research is needed, given the lack of supervision or control for test environment, external factors, distractors, or cues. Further, we were not able to control for important disease related factors such as severity and time since symptom onset. Although MBI was associated with statistically significant differences in Lumosity neuropsychological test scores, effect sizes were small, and the clinical significance of these differences is difficult to interpret in the largely cognitively healthy population enrolled in BHR. Our data are promising but not conclusive, and further research is required. Whether these small differences in cognitive test scores represent greater risk for incident cognitive decline and dementia can be addressed in the future using longitudinal data and a cohort that includes participants with cognitive impairment.
In summary, in this BHR study combining self- and informant-rated measures, we observed the convergence of behavioral risk markers for dementia and cognitive differences, reflected by neuropsychological tests incorporating memory and executive function. The findings lend additional support to online unsupervised administration of cognitive and neuropsychiatric measures, as a low-cost approach to improve access to neurocognitive assessments, potentially identifying at-risk older adults.
Data Availability
Data used in the manuscript is used from the Brain Health Registry online platform.
HIGHLIGHTS
Online portals for older adults and study partners can be used for dementia risk assessment
This approach is useful when in-person assessments are not feasible (physical, social, cognitive or health systems-realted (e.g., the COVID-19 pandemic))
The relationship between mild behavioral impairment (MBI) and cognition in older adults was explored
MBI was associated with small magnitude, but significantly poorer performance in memory and executive function, and may serve as a complementary measure of risk
Acknowledgements
Data used in this study were collected using the BHR, which is funded by the NIH, Alzheimer’s Association, Alzheimer’s Drug Discovery Foundation, California Department of Public Health, Connie and Kevin Shanahan, The Drew Foundation, General Electric, Global Alzheimer’s Platform Foundation, Larry L. Hillbolm Foundation, The Ray and Dagmar Dolby Family Fund, The Rosenberg Alzheimer’s Project and Patient-Centered Outcomes Research Institute.
FK, HC, AW, TW have no interests to declare. ZI is funded by the Canadian Institutes of Health Research, and has received consulting fees/honoraria from Otsuka/Lundbeck, outside the submitted work. His institution has received funds from Acadia, Biogen, Roche, and Sunovion, also outside the submitted work. RLN is a co-investigator for the BHR. RSM has received grant funding from the National Institute of Mental Health and has received research support from Johnson & Johnson. MWW receives support for his work from the following: National Institute of Health, Department of Defense, Patient-Centered Outcomes Research Institute, California Department of Public Health, University of Michigan, Siemens, Biogen, Larry L. Hillbolm Foundation, Alzheimer’s Association, The State of California, Johnson & Johnson, Kevin and Connie Shanahan, GE, Vrije Universiteit Medical Center Amsterdam, Australian Catholic University, The Stroke Foundation and the Veterans Administration. He has served on the Advisory Boards for Eli Lilly, Cerecin/Accera, Roche, Alzheon, Inc., and BHR.