ABSTRACT
Objective Functional dependence is a multifactorial health condition that affects well-being and life expectancy. To better understand the mechanisms underlying functional dependence, we aimed to identify the variables that best prospectively classify adults with and without limitations in basic and instrumental activities of daily living.
Methods A filtering approach was used to select the best predictors of functional status from 4,248 candidate predictors collected in 39,927 participants aged 44 to 88 years old at baseline. Several machine learning models using the selected baseline variables (2010-2015) were compared for their ability to classify participants by functional status (dependent vs. independent) at follow up (2018-2021) on a training dataset (n = 31,941) of participants from the Canadian Longitudinal Study on Aging. The best performing model was then examined on a test dataset (n = 7,986) to confirm its sensitivity, specificity, and accuracy.
Results Eighteen baseline variables were identified as the best predictors of functional status at follow up. Logistic regression was the best performing model for classifying participants by functional status and achieved balanced accuracy of 81.9% on the test dataset. Older age, phycological distress, slow walking speed, perceived health, being retired, having a chronic condition, and never going for walks at baseline were associated with greater odds of being functionally dependent at follow-up. In contrast, the absence of functional limitations, greater grip strength, being a female and free of chronic conditions at baseline were associated with lower odds of being functionally dependent at follow-up.
Conclusion Functional dependence can be best prospectively estimated by age, psychological distress, physical fitness, physical activity, chronic conditions, and sex. These predictors can estimate functional dependence more than 6 years in advance with high accuracy.
Impact Such early identification of functional dependence allows sufficient time for the implementation of interventions designed to delay or prevent functional decline.
Lay Summary Whether a patient will be dependent in 6 years can be predicted with good accuracy by 18 variables, including age, psychological distress, physical fitness, physical activity, chronic disease, and sex.
INTRODUCTION
In 2020, the world’s population included 1 billion people aged 60 years or older. This number is expected to double by 2050. During the same period, the population aged 80 years or older is expected to triple.1 These demographic trends are associated with a greater number and proportion of people who experience difficulties with physical functioning2,3, as manifested by a reduced ability to perform basic activities of daily living (ADL) (e.g., dressing, bathing, and walking) and instrumental activities of daily living (IADL) (e.g., shopping, managing one’s medications). These limitations have been shown to be a major contributor to increased early mortality4 and decreased quality-adjusted life years5. In addition, functional dependence is associated with patient financial barriers to care6, caregiver burden7,8, costly hospitalization9, and long-term care home admission10,11. As a result, preventing functional dependence is now considered a key component of healthy aging, which is defined as the process of developing and maintaining the functional ability that enables well-being in older age12. However, while home-based interprofessional programs involving rehabilitation professionals have demonstrated efficacy in reducing ADL and IADL disability13, healthcare systems and payers show only modest interest in the prevention of functional limitations, which may in part be due to the lack of direct reimbursement for preventive care related to physical function. Another possible explanation for this lack of interest is the difficulty in determining who should benefit from this preventive care. The ability to predict which individuals are likely to become functionally dependent in the future is essential for identifying and intervening in high-risk cases. To enable and refine this prediction, studies have identified a range of variables associated with functional dependence. A 2007 systematic review of eight studies investigating early indicators of functional dependence found that age, lower functional status, disability, cognitive impairment, length of hospital stays, and depression were predictors of functional decline14. Other research identified additional predictors such as physical activity (n = 12,860)15, adverse childhood experiences (n = 25,775)16, and disadvantaged childhood socioeconomic circumstances (n = 24,440)17 in a 12-year European longitudinal study, as well as poor chewing ability in a 6-year Japanese study (n = 748)18. In another Japanese sample (n = 1,523), walking speed, one-legged stance time as well as time to rise and walk were associated with a decline in functional independence19. Additionally, lower socioeconomic status, unemployment, low fruit and vegetable consumption, and lower cognitive performance were associated with a greater risk of developing functional dependence over a four-year period in a Brazilian sample (n = 412)20. Finally, age, chronic condition, and VO2max were associated with increased odds of becoming functionally dependent over the course of eight years in a Canadian sample (n = 297)21.
However, some of these studies relied on small samples, most of them focused on a reduced number of candidate predictors, and none considered whether alternative modelling strategies would better fit the data. The machine learning approach is well suited to fill this gap, as it allows researchers to compare model performance between several algorithms (e.g., random forests, naive bayes) to select the model that is best able to predict or classify an outcome.22,23,24,25 To our knowledge, machine learning has not yet been applied to investigate the predictors that can prospectively classify functionally dependent and independent adults. However, machine learning has been applied to the prediction and classification of a condition closely related to functional dependence, namely frailty, an age-related state of vulnerability that affects multiple physiological systems.26 In a review that identified 217 frailty measurement tools, 52% included a functional limitation component27, supporting that functional dependence and frailty are highly intertwined. Studies using machine learning to classify frailty28,29,30,31 have produced models with sensitivity (i.e., true positives) ranging from 74% to 99% and specificity (i.e., true negatives) ranging from 70% to 96%. In these models, factors such as income, age, the chair rise test, perceived health, chronic conditions, and balance problems demonstrated the strongest ability to classify frailty. However, it is unclear which indicators best classify functional status.
The objective of the present study was to use machine learning on a large cohort study to robustly identify the predictors that best prospectively classify functionally dependent and independent adults. Knowledge of such predictors would help identify individuals at higher risk of developing future functional limitations, inform the development of preventive interventions, and contribute to healthy aging.
METHODS
Participants
Participants were recruited from the Canadian Longitudinal Study on Aging (CLSA), a nationally representative longitudinal study aimed at measuring the biological, societal, psychosocial, and physical factors related to healthy aging in Canada.32 Baseline data collection was conducted between 2010 and 2015 using two approaches: data collection from a ‘tracking’ cohort of participants was collected via hour-long computer-assisted phone interviews (n = 21,241) and data collection from a ‘comprehensive’ cohort via 90-minute in-person interviews in addition to a data collection site visits (n = 30,097). Additionally, a ‘maintaining contact questionnaire’ was administered by phone to both cohorts. The tracking cohort and comprehensive cohort were used as baseline data in our analyses. A follow-up assessment was from 2018 to 2021.
Participants in the CLSA were recruited through Canada’s provincial health registries, random-digit dialing, and the Canadian Community Health Survey on Healthy Aging.33,34 Exclusion criteria included: residents living in Canada’s three territories and First Nations reserves, full-time members of the Canadian Armed Forces, people living with cognitive impairments, and individuals living in institutions, including 24-hour nursing homes.32 Environmental measures from the Canadian Urban Environmental Health Research Consortium (CANUE)34 were linked with the CLSA dataset. The CANUE dataset contains multiple annual measures of greenness (e.g., tree canopy coverage), neighborhood characteristics (e.g., building density, public transportation), air quality (e.g., fine particulate matter, smoke exposure), and weather (e.g., local climate zones, land surface temperatures).
Data from 39,927 participants was partitioned into data used to train statistical models (training data, 80%; n = 31,941) and data used to test the predictive ability of the models (testing data, 20%; n = 7,986). Data used for classification was stratified by severity of ADL and IADL limitations so that equal proportions of older adults with low and high limitations at follow-up were present in both the training and the test datasets. All modeling on the training data used 10-fold cross-validation, which is generally considered best practice with machine learning.35 Resampling methods such as cross-validation enable the determination of how well a model works without using the test data.
Dependent Variable
The primary dependent variable was limitations in ADL and IADL at follow-up, as assessed by a modified version of the Older Americans’ Resources and Services Multidimensional Functional Assessment Questionnaire (OARS).36,37 Twenty questions were used to assess participants’ ability to complete seven ADL (i.e., can you dress and undress yourself without help, can you eat without help, can you take care of your own appearance without help, can you walk without help, can you get in and out of bed without any help or aids, can you take a bath or shower without help, do you ever have trouble getting to the bathroom in time). Twenty-one questions were used to assess seven IADL (i.e., can you use the telephone without help, can you get to places out of walking distance without help, can you go shopping for groceries or clothes without help, can you prepare your own meals without help, can you do your housework without help, can you take your own medicine without help, can you handle your own money without help). The CLSA ‘Basic and Instrumental Activities of Daily Living Classification’ is based on the following scores: 1 (no limitation), 2 (mild limitation), 3 (moderate limitation), 4 (severe limitation), and 5 (total limitation) in ADL and IADL. The method used to derive this variable assigns extra weight to the ability to prepare meals and was scored to be similar to variables used in Statistics Canada’s Canadian Community Health Survey. Two binary variables from the Basic and Instrumental Activities of Daily Living Classification were computed. The first dependent variable classified participants with no or mild limitation as functionally independent, and participants with moderate, severe, or total limitation as functionally dependent. The second measure classified participants with no limitation as functionally independent and participants with mild, moderate, severe, or total limitation as functionally dependent. This approach of dichotomising functional limitations has been used in previous research.38,39
Predictors Selection
Merging the variables from the tracking and comprehensive datasets resulted in an initial set of 4,248 variables, from which 3,574 variables were manually removed because they were single items of summary variables or outside the scope of the study (e.g., biomarkers). A total of 674 summary variables were considered as possible predictors of functional status (independent vs. dependent). Using a filtering approach40, univariate logistic regressions were conducted between candidate predictors and functional status. A total of 630 variables were filtered out due to low associations with functional status. Subsequently, 26 variables were filtered out for containing redundant measures (e.g., multiple measures for age, income), using skip-logic questions, or through Least Absolute Shrinkage and Selection Operator (LASSO) reducing predictor estimates to 0 in logistic regression models, resulting in 18 predictor variables that were measured at baseline (Table 1): age, chronic conditions, driver’s license, grip strength, hip circumference, home care, limitations in ADL and IADL, memory (immediate and delayed recall), pain, perceived health, psychological distress, retirement, sex, urinary incontinence, visual acuity, walking frequency, and walking speed.
Statistical Analyses
Modeling was conducted using the ‘tidymodels’ package41, a meta-engine for machine learning models in R42, which contains a collection of packages for modeling and machine learning in addition to workflow structures preventing common machine learning errors such as data leakage.43 A series of exploratory models were performed from which the best performing models were selected for subsequent interpretation.35 Only participants with complete data on the dependent variable (n = 39,927) were included in the analyses. Data imputation was performed for all predictor variables using bagged tree models, which can impute missing data for both numeric and categorical variables.
In machine learning, hyperparameters are set prior to the training process to guide the learning process. For example, hyperparameters include regularization strength in regression-based models, which prevents overfitting by adding a penalty to the models’ loss function that quantifies the error margin between a model’s prediction and the actual target value. Model/hyperparameter combinations refer to the various possible configurations of a model based on the values of its hyperparameters. Our analysis used a grid search approach to optimize hyperparameters and tested 125 model/hyperparameter combinations for classifying functional status.
During the training process, the values of the model parameters are determined. For the classification of functional status, the following packages and models were used: XGBoost44, ranger (random forest)45, klaR (naïve Bayes)46, neural network (nnet)47, and glmnet (logistic regression)48. Numeric predictors for XGBoost, neural networks, and logistic regressions were normalized. Categorical predictors were encoded into numerical data using one-hot encoding. Hyperparameters for each model were determined with a grid search methodology using the ‘tune’ package43 and the best performing models were selected for further fitting and evaluation. During the model training and testing processes, model performance was evaluated through the level of misclassification on out-of-sample data using a standard confusion matrix22, which represents different combinations of predicted and actual values forming the basis of metrics for precision, specificity, and accuracy. In addition, area under the curve (AUC) of the receiver operating characteristic (ROC) curve was used to represent how well the model can distinguish between functional statuses.35 A 50% AUC indicates random guessing, and 100% indicates perfect performance. Here, this score was interpreted as was interpreted as: ≥ 60%, poor; ≥ 70%, fair; ≥ 80% good; ≥ 90%, excellent.49 Specificity refers to the proportion of true negative, sensitivity refers to the proportion of true positives, and accuracy is the proportion of true positive and negative results among the total cases examined. By default, binary classification models use a threshold of 0.50 to predict if a given case should be classified as 0 (functionally independent) or 1 (functionally dependent); the optimal threshold value which balances specificity and sensitivity was computed for each model.
Results
Descriptive Results
A total of n = 39,927 participants had reported their functional status and were retained in the final sample, which included English (n = 32,298) and French speakers (n = 7,629), 20,620 females (51.6%). Age ranged from 44 to 88 years at baseline. Detailed descriptive statistics of the sample’s characteristics are presented in Table 1.
When comparing the baseline characteristics between functionally independent and dependent participants at follow-up, several differences between groups were observed. For example, 92.5% of participants classified as independent at follow-up reported no impairments at baseline while 50.4% of those classified as dependent at follow-up reported no impairments at baseline. Additionally, dependent participants reported a greater rate of chronic conditions at baseline (98.5% vs. 90.6%), greater pain (63.0% vs 36.1%), not having a driver licence (26.7% vs. 4.3%), higher rates of urinary incontinence (25.5% vs. 7.31%), poorer perceived health (8.8% vs. 1.2%), higher retirement rates (75.5% vs. 41.0%), weaker grip strength (26.2 ± 9.8 kg vs. 34.0 ± 11.4 kg), and older age (70.9 ± 10.5 years vs. 61.7 ± 9.7 years). The mean number of years between baseline and follow-up was 6.3 years (range: 3.9 to 9.6 years).
Model Performance
Models classifying participants with no or mild limitation as functionally independent participants and participants with moderate, severe, or total limitation as functionally dependent provided the strongest accuracy.
The best two models demonstrated approximately equivalent performance on the training data: logistic regression (AUC = 89.2%, accuracy = 97.3%, specificity = 99.7%, sensitivity = 13.0%) and XGBoost (AUC = 89.1%, accuracy = 97.4%, specificity = 99.8%, sensitivity = 12.8%) (Table 2, Figure 1). When the threshold of 0.50 was adjusted to optimise a balance between sensitivity and specificity, model performance increased to above 80% sensitivity and 80% specificity for each of these two models (Table 2). When considering the overall classification accuracy using balanced sensitivity and specificity, logistic regression was the best performing model with 82.0% accuracy (vs. 81.7% for the XGBoost model). Model performance for logistic regression on the test data was approximately equivalent to the training data (balanced accuracy = 81.9%, specificity = 79.0%, sensitivity = 84.7%), suggesting the model did not overfit on the training data.
Best Classifiers of Functional Dependence
The baseline variables that best classified functionally dependence at follow-up were older age (OR = 1.83), psychological distress (OR = 1.22), slower walking speed (OR = 1.19), being retired (OR = 1.16), having a chronic condition (OR = 1.14), and never going for walks (OR = 1.12) (Figure 2). In contrast, the following baseline variables were associated with lower odds of being dependent at follow-up were the absence of functional limitations at baseline (OR = 0.79), stronger grip strength (OR = 0.81), the absence of chronic conditions (OR = 0.87), being a female (OR = 0.89), having an unrestricted driver’s license (OR = 0.91), and better memory (OR = 0.96 and 0.91) (Figure 2).
DISCUSSION
This study applied machine learning techniques to data from a large cohort study (n = 39,927) to identify the variables from baseline assessments (2011-2015) that best classify functionally dependent and independent adults at follow-up (2018-2021).
Eighteen variables were identified as the best predictors of functional status: age, chronic conditions, driver’s license, grip strength, hip circumference, home care, limitations in ADL and IADL, memory (immediate and delayed recall), pain, perceived health, psychological distress, retirement, sex, urinary incontinence, visual acuity, walking frequency, and walking speed. Using these variables, the best model showed a high proportion of true negatives (79%) and true positives (85%), demonstrating high accuracy in correctly identifying functionally dependent and independent adults (82%). The following baseline variables were the best classifiers of functional dependence at the follow-up assessment: age, psychological distress, physical fitness, chronic conditions, and sex. None of the environmental variables from the CANUE dataset (e.g., greenness, neighborhood characteristics, air quality, and weather) showed strong evidence of an association with functional status This was surprising given the hypothesized link between environmental quality and frailty.50 However, it is possible that longer exposures to sub-optimal environments may be required for effects to become identifiable.
Our model outperforms some recent models focusing on frailty28,29, which is closely related to functional dependence.27 For example, Aponte-Hao et al.28 used two years of electronic medical records from Canadian primary care practices to predict frailty with the Rockwood Clinical Frailty Scale51 and obtained 78% sensitivity and 74% specificity. Tarekgen et al.29 used one-year of data from the Piedmontese Longitudinal Study to predict five indicators of frailty (mortality, urgent hospitalization, disability, fracture, and emergency admission) with sensitivity ranging from 75% to 81% and specificity from 70% to 80%. While comparison with these studies is difficult due to differences in the assessed concepts (frailty vs. functional status), variability in the time between assessments, differences in national contexts, and the nature of the available predictors in the dataset, our model obtained reasonable performance with greater distance between data measurements and fewer prediction variables (n = 18) than previous studies [n = 7528; n = 5829; n = 2730]. This latter result is clinically important as this parsimony facilitates future work of personalized prediction of those at risk of becoming functionally dependent.
While our models outperformed some previous work, higher model performance has been reported in the frailty literature.30,31,52 For example, Leme and De Oliveira31 predicted frailty using four years of data from the English Longitudinal Study of Aging to predict frailty using the Fried phenotype measure53 and obtained 83% sensitivity and 88% specificity with the strongest predictors of frailty including age, the chair rise test, perceived health, and balance problems. Interestingly, while age and perceived health were among the strongest predictors in our analysis, the chair-rise test, income, and balance problems were not. These latter variables may be related to dimensions of frailty that are unrelated to functional dependence. National context (Canada vs. Brazil) may partly explain the discrepancy with household income. The inclusion of several physical fitness measures in the present study (e.g., walking frequency, walking speed, grip strength) may have accounted for the relationship between physical capacity and frailty. In another example, Koo et al.30 used 27 predictors in the Korean Frailty and Aging Cohort, over a one-year period, to predict scores on the Fried Frailty Index and achieved a score in which sensitivity and specificity contribute equally (F1 score) of 95.3%. Notably, these higherperforming models were obtained with shorter intervals between data collection waves [6.3 years on average in the present study vs. one year29 and four years30]. When classification is performed in a single observational event (e.g., using a motion capture system during body movements to predict frailty), classification performance can increase to 97.5%52. However, it is important to keep in mind that although greater classification performance can be achieved when the time between predictor and outcome measures is shorter, the purpose of modeling studies is not to assess health status. The value of modeling studies lies in their ability to identify as many years in advance as possible who is at risk of becoming frail or functionally dependent, allowing sufficient time to intervene and delay or even prevent this functional decline.
Strengths and Limitations
This study has several strengths including (i) the use of a large cohort study, (ii) the investigation of functional limitations that fills a gap in the literature, (iii) the exploration of predictors of functional limitations at multiple levels (individual, social, environmental), and (iv) similar high accuracy across the tested models, suggesting well-selected variables and robust results. Several limitations should also be noted. First, the length of time between baseline and follow-up assessments varied due to the nature of CLSA data collection and it is unclear whether sufficient time between timepoints was sufficient for causal relationships to manifest. Second, a strong class imbalance was present in the data with only 2.9% of participants classified as functionally dependent. Synthetic minority oversampling techniques (SMOTE)54 could have been used to generate a balanced dataset with synthetic data. However, this technique is not without its limitations and can introduce noise into a model.55 In our analyses, we opted to balance the proportion of functionally dependent adults in the training and test dataset without artificially creating data.
Conclusions
To our knowledge, we present the first study that identifies and models the variables that best prospectively classify functionally dependent and independent adults. The identified predictors can predict functional dependence more than 6-year in advance (range 3.9 to 9.6 years) with high accuracy. Because functional ability is a key component of healthy aging12, preventing its decline should be a priority to healthcare providers, especially in the context of an aging population.1 Early identification of individuals at risk allows healthcare professionals sufficient time to implement interventions aimed at delaying or preventing functional decline, and to refer patients to the appropriate specialists when needed, before functional decline develops.
While all healthcare providers can incorporate such preventive care into their treatment plans, rehabilitation professionals play a central role in maintaining and optimizing functional abilities. In addition, predictors such as muscle strength, walking speed and frequency, and pain management fall within their scope of practice.56,57 Screening for these predictors should be considered when assessing patients, especially older adults, to allow for early intervention. Patient education regarding the identification of predictors of functional decline and the importance of addressing them should not be overlooked, as this may help motivate patients to actively take part in their care. In short, prospective identification and early intervention of functional dependence could have an important impact on preventing loss of autonomy, reducing significant costs to individuals and the healthcare system, as well as reducing caregiver burden and long-term care admissions.
Data Availability
The dataset used in this study is available for researchers who meet the criteria for accessing CLSA (www.clsa-elcv.ca) and the R scripts are publicly available in Zenodo: https://doi.org/10.5281/zenodo.12584730
Declarations
Data and Code Availability
In accordance with good research practice58 the R scripts are publicly available in Zenodo.59 This research has been conducted using the CLSA dataset (Baseline Tracking Dataset version 4.0, Baseline Comprehensive Dataset version 7.0, Follow-Up 2 dataset version 1.1) under application number 2304007. This dataset is available for researchers who meet the criteria (www.clsa-elcv.ca).
Funding
This research was made possible using the data collected by CLSA. Funding for CLSA is provided by the Government of Canada through the Canadian Institutes of Health Research (CIHR; LSA 94473), the Canada Foundation for Innovation (CFI), and the following Canadian provinces: Alberta, British Columbia, Manitoba, Newfoundland, Nova Scotia, Ontario, Quebec. Matthieu Boisgontier is supported by Natural Sciences and Engineering Research Council of Canada (NSERC; RGPIN-2021-03153), the CFI, Mitacs, and the Banting Discovery Foundation. Zachary van Allen is supported by a Mitacs Accelerate Postdoctoral Fellowship and the Banting Discovery Foundation.
Authorship Contribution Statement
Based on the Contributor Roles Taxonomy (CRediT)60 individual author contributions to this work are as follows:
Zachary van Allen: Conceptualization; Methodology; Formal Analysis; Writing (Original Draft); Writing (Review and Editing).
Nathalie Dionne: Writing (Original Draft); Writing (Review and Editing).
Matthieu Boisgontier: Conceptualization; Methodology; Writing (Original Draft); Writing (Review and Editing); Supervision (ZMvA and ND); Project Administration; Funding Acquisition.
Conflict of Interest
The authors declare that there are no conflicts of interest related to the content of this article.
Footnotes
↵* matthieu.boisgontier{at}uottawa.ca