Decoding accelerometry for classification and prediction of critically ill patients with severe brain injury
============================================================================================================

* Shubhayu Bhattacharyay
* John Rattray
* Matthew Wang
* Peter Dziedzic
* Eusebia Calvillo
* Han B. Kim
* Eshan Joshi
* Pawel Kudela
* Ralph Etienne-Cummings
* Robert D. Stevens

## ABSTRACT

Our goal is to explore quantitative motor features in critically ill patients with severe brain injury (SBI). We hypothesized that computational decoding of these features would yield information on underlying neurological states and clinical outcomes. Using wearable microsensors placed on all extremities, we recorded 1,701 hours of continuous, high-frequency accelerometry data from a prospective cohort (*n* = 69) admitted to the ICU with SBI. Models were trained using time-, frequency-, and wavelet-domain features and levels of responsiveness and outcome as labels. The two primary tasks were detection of levels of responsiveness, assessed by motor sub-score of the Glasgow Coma Scale (GCSm), and prediction of functional outcome at discharge, measured with the Glasgow Outcome Scale–Extended (GOSE). Detection models achieved significant (AUC: 0.70 [95% CI: 0.53–0.85]) and consistent (observation windows: 12 min – 9 hours) discrimination of SBI patients capable of purposeful movement (GCSm > 4). Prediction models accurately discriminated SBI patients of upper moderate disability or better (GOSE > 5) with 2– 6 hours of observation (AUC: 0.82 [95% CI: 0.75–0.90]). Results suggest that computational analysis of time series motor activity in patients with SBI yields clinically important insights on underlying neurologic states and short-term clinical outcomes.

## INTRODUCTION

Despite advances in intensive care, the global burden of severe brain injury (SBI), in terms of mortality, long-term disability, and economic costs, is the highest among all major injuries1. Existing approaches to predict SBI outcomes, such as recovery of consciousness and functional independence, are imprecise for individual patients2 and can raise ethical concerns due to the potential for withdrawal of life sustaining+therapies3. At the same time, recent developments in artificial intelligence and big data processing represent an opportunity to optimize SBI patient monitoring with high-resolution, longitudinal waveform data and to improve the precision of SBI prognoses with flexible modeling strategies4. Hence, a key focus in the care of SBI is the discovery and validation of quantitative monitoring modalities that improve upon the accuracy and reliability of clinical characterization and the reliability of predicted outcomes5.

For acute neurological disorders, the assessment of motor function provides an important clinical window into neural systems associated with sensorimotor processing, emotion, coordination, planning, and learning6-8. Neurological damage and intensive care unit (ICU) practices (e.g., sedation, bedrest) are associated with a dramatic reduction in normal physical activity9, resulting in systemic pro-inflammatory signaling10 and an elevated risk of venous thromboembolism, infection, skin and soft tissue damage, delirium, and loss of muscle mass and strength11-14. A corollary is that structured programs designed to increase physical activity for SBI patients in the ICU can significantly reduce neurological complications and may lead to improved functional recovery15. However, it is uncertain whether the incorporation of continuous motion sensing in the ICU could yield clinically significant gains for SBI monitoring and prognosis.

Wearable accelerometers provide an objective and continuous assessment of motor activity over extended periods of time16. In contrast to most other motion sensing modalities17, integration of accelerometers in the ICU is feasible. Advances in microelectromechanical systems (MEMS) technology have made it possible to construct inexpensive, minimally obtrusive wearable accelerometers that can be optimized for the clinical space18. Accelerometers respond to changes in movement frequency and intensity, measure tilt from the gravitational axis, and produce little variation or drift over time18-22. The use of accelerometers to monitor gross physical activity in the ICU has already been tested with varying degrees of success23. Herein, we aim to more specifically determine whether a relationship exists between motion features derived from triaxial accelerometry time-series and neurological motor states and functional outcomes of SBI patients.

In this pilot study of the Neurological Injury Motion Sensing (NIMS) project, we explore the impact and limitations of high-resolution accelerometry in patients with SBI admitted to the ICU. We developed a matrix of wearable accelerometers to quantitatively capture motor activity from the extremities of SBI patients. Applying techniques from time-series analysis, dimensionality reduction, and logistic regression, we extract interpretable time-, frequency-, and wavelet-domain motion features and assess their performance in motor function detection and short- and long-term functional outcome prediction models. We then assess relative significance of the extracted features to determine how specific accelerometry profiles relate to clinically evaluated motor function and global outcomes. Finally, through a retrospective case analysis, we demonstrate how accelerometry-based model outputs can potentially be used to monitor neurological transitions.

## RESULTS

### Study population characteristics

Of the 72 total SBI patients recruited in the ICU, 3 participants were excluded from the study due to withdrawn consent (*n* = 2) or corruption of accelerometry data during upload (*n* = 1), resulting in a study population of *n* = 69. Five patients were lost from one-year follow-up due to unsuccessful contact, and thus the study population at 12 months post hospital discharge was *n* = 64. Detailed characteristics of the study population are summarized in **Table 1**.

View this table:
[Table 1.](http://medrxiv.org/content/early/2021/05/25/2021.05.19.21257319/T1)

Table 1. 
Study population characteristics.

From each of the study participants, we collected triaxial accelerometry data (sampled at 10 Hz) from a wearable matrix of 6 sensors, placed on each elbow, wrist, and ankle and an additional sensor placed on the bed for external movement correction (**Fig. 1a**). The median recording duration per patient was 24.09 hours (IQR: 22.81–25.11 hours), and accelerometry data was recorded fairly uniformly across the stages of ICU stay in terms of proportion completed (**Supplementary Fig. S1 online**). In total, 1,701 hours of multisegmental accelerometry data were recorded.

![Fig. 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/05/25/2021.05.19.21257319/F1.medium.gif)

[Fig. 1.](http://medrxiv.org/content/early/2021/05/25/2021.05.19.21257319/F1)

Fig. 1. Accelerometry processing and feature extraction pipeline and experimental paradigm.
**a** Wearable sensor placement on severe brain injury patients in the ICU (top left) and accelerometry(units: *g*)-to-feature pipeline. Sensor placement acronyms correspond to the right and left elbows (RE and LE), the right and left wrists (RW and LW), and the right and left ankles (RA and LE). *f**s* represents the sampling rate of accelerometry in Hz. Feature type acronyms are decoded in **Table 3. b** Experimental paradigm to derive model probabilities for motor function detection per the motor component score of the Glasgow Coma Scale (GCSm) and functional outcome predictions per the Glasgow Outcome Scale – Extended (GOSE) from extracted motion features. LOL corresponds to linear low-rank projection (LOL).

During their stay in the ICU (median: 19 days, IQR: 11–29 days), study participants were evaluated with the Glasgow Coma Scale (GCS)24, 25 a median 9.25 times per day (IQR: 7.17–11.50 times per day). In total, we extracted scores from 14,240 GCS evaluations, 13,190 of which (92.63%) took place in the ICU and 653 of which (4.59%) coincided with accelerometry capture times. The trajectory of the motor component scores of the GCS (GCSm), along with corresponding times of accelerometry capture, of each patient included in our analysis is provided in **Supplementary Figure S2 online**.

### Motor function detection performance

In this work, we use clinically evaluated GCSm scores extracted from electronic health records (EHR) as the primary markers of functional motor states. The scores of the 6-point GCSm are defined by best motor responses to physical stimuli and are outlined in **Table 1**.

We trained and evaluated threshold-level GCSm detection models from automated accelerometry-based motion features extracted from 19 varying observation windows, from 3 minutes to 24 hours, directly preceding the GCSm evaluations (**Fig. 1b**). The count distributions of GCSm scores available for each observation window are listed in **Supplementary Table S1 online**.

The receiver operating characteristic (ROC) curves of the optimally discriminating models at each GCSm threshold, along with their mean areas under the curves (AUC) and optimal observation windows, are shown in **Figure 2a**. Based on the 95% confidence intervals of mean AUC, significant discrimination (AUC > 0.5, α = 0.05) was achieved by the extracted features at every threshold of GCSm except for GCSm > 2. However, only GCSm > 4 detection models achieve significant discrimination from shorter observation window durations (≤ 30 minutes); GCSm > 4 detection models achieve significant discrimination consistently with an observation window of 12 minutes or greater (**Fig. 2b**). The mean AUCs, along with 95% confidence intervals, at each threshold of GCSm is provided for all 19 tested observation windows in **Supplementary Table S2 online**. As GCSm > 1, GCSm > 3, and GCSm > 5 detection models achieve significant discrimination at less than or equal to 3 different observation windows, only GCSm > 4 detection models achieve significant discrimination at a broad range of observation windows (12 min – 9 hours). Binary classification performance metrics of optimally discriminating motor function detection models are provided in **Table 2**. At none of the GCSm thresholds do the models achieve significantly greater accuracy than the proportion of the most represented class based on 95% confidence intervals. Only the GCSm > 4 detection model achieves a higher mean accuracy (0.71) and a significantly greater F1 score (0.78 [95% CI: 0.67–0.87]) than its proportion of the most represented (in this case, positive) class (0.66). Only GCSm > 4 and GCSm > 5 detection models achieved both a mean sensitivity and mean specificity over 0.5, but not significantly.

View this table:
[Table 2.](http://medrxiv.org/content/early/2021/05/25/2021.05.19.21257319/T2)

Table 2. 
Classification performance metrics of optimally discriminating models.

![Fig. 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/05/25/2021.05.19.21257319/F2.medium.gif)

[Fig. 2.](http://medrxiv.org/content/early/2021/05/25/2021.05.19.21257319/F2)

Fig. 2. Discrimination performance of motor function detection models on validation sets.
**a** Receiver operating characteristic (ROC) curves of models pertaining to the observation windows with the highest achieved area under the ROC curve (AUC) per each detection threshold of the motor component score of the Glasgow Coma Scale (GCSm). Shaded areas represent 95% confidence intervals derived using bias-corrected bootstrapping (1,000 resamples) to represent the variation across repeated cross-validation folds (5 repeats of 5 folds) and nine missing value imputations. The values in each box represent the observation window achieving the highest AUC as well as the corresponding mean AUC (with 95% confidence interval in parentheses). The diagonal dashed line represents the line of no discrimination (AUC = 0.5). **b** AUC vs. observation windows up to 30 minutes per each detection threshold of the motor component score of the Glasgow Coma Scale (GCSm). Points represent observation windows tested and error bars (with the associated shaded region) represent the 95% confidence interval. The horizontal dashed line corresponds to no discrimination (AUC = 0.5).

### Functional outcome at hospital discharge prediction performance

We used clinically evaluated Glasgow Outcome Scale – Extended (GOSE) scores as the primary markers of functional outcomes, both at hospital discharge and at 12 months post discharge. The scores of the 8-point GOSE are outlined in **Table 1**.

We trained and evaluated threshold-level GOSE at hospital discharge prediction models from automated accelerometry-based motion features extracted from the same 19 varying observation windows directly preceding GCSm evaluations (**Fig. 1b**). The median lead window duration (i.e., time between end of observation window and hospital discharge) was 20 days (IQR: 10–33 days). The count distributions of GOSE scores, at discharge, available for each observation window are listed in **Supplementary Table S3 online**. Given the low proportion of patients (1.45%) with good recovery (GOSE > 6) at hospital discharge, we limited our threshold-level analysis to GOSE > 1, GOSE > 2, GOSE > 3, GOSE > 4, and GOSE > 5.

The receiver operating characteristic (ROC) curves of the optimally discriminating models at each GOSE threshold, along with their mean areas under the curves (AUC) and optimal observation windows, are shown in **Figure 3a**. Based on the 95% confidence intervals of mean AUC, significant discrimination (AUC > 0.5, α = 0.05) was achieved by the extracted features only at GOSE > 5. GOSE > 5 prediction models achieve significant discrimination at observation windows of two hours or greater, with a peak mean AUC of 0.82 (95% CI: 0.75–0.90) at an observation window duration of 6 hours (**Fig. 3b**). The mean AUCs, along with 95% confidence intervals, at each tested threshold of GOSE is provided for all 19 tested observation windows in **Supplementary Table S4 online**.

![Fig. 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/05/25/2021.05.19.21257319/F3.medium.gif)

[Fig. 3.](http://medrxiv.org/content/early/2021/05/25/2021.05.19.21257319/F3)

Fig. 3. Discrimination performance of functional outcome at hospital discharge prediction models on validation sets.
**a** Receiver operating characteristic (ROC) curves of models pertaining to the observation windows with the highest achieved area under the ROC curve (AUC) per each tested prediction threshold of the Glasgow Outcome Scale – Extended (GOSE). Shaded areas represent 95% confidence intervals derived using bias-corrected bootstrapping (1,000 resamples) to represent the variation across repeated cross-validation folds (5 repeats of 5 folds) and nine missing value imputations. The values in each box represent the observation window achieving the highest AUC as well as the corresponding mean AUC (with 95% confidence interval in parentheses). The diagonal dashed line represents the line of no discrimination (AUC = 0.5). **b** AUC vs. observation windows up to 6 hours per each tested prediction threshold of the Glasgow Outcome Scale – Extended (GOSE). Points represent observation windows tested and error bars (with the associated shaded region) represent the 95% confidence interval. The horizontal dashed line corresponds to no discrimination (AUC = 0.5).

Binary classification performance metrics of optimally discriminating functional outcome prediction models are provided in **Table 2**. At none of the GOSE thresholds do the models achieve a significantly greater F1 score than the proportion of the positive class or a greater mean accuracy than the proportion of the most represented class. Despite its strong discrimination performance, the GOSE > 5 prediction model achieves near-zero precision and sensitivity. From the precision recall curve for this model (**Supplementary Fig. S3 online**), we observe a mean average precision of 0.08 (95% CI: 0.02–0.18), which, while low, is significantly greater than the proportion of the positive class (0.02). This indicates, that while prediction probabilities for true positive cases are, on average, greater than prediction probabilities for true negative cases, they seldom cross the 0.5 threshold for proper classification (**Supplementary Fig. S3 online**).

### Functional outcome at 12 months post discharge prediction performance

We trained and evaluated threshold-level GOSE at 12 (±1) months post hospital discharge prediction models from automated accelerometry-based motion features extracted from the same 19 varying observation windows directly preceding GCSm evaluations (**Fig. 1b**). The count distributions of GOSE scores, at 12 months, available for each observation window are listed in **Supplementary Table S5 online**. The receiver operating characteristic (ROC) curves of the optimally discriminating models at each GOSE threshold, along with their mean areas under the curves (AUC) and optimal observation windows, are shown in **Supplementary Figure S4 online**. Based on the 95% confidence intervals of mean AUC, significant discrimination (AUC > 0.5, α = 0.05) was not achieved by the extracted features at any of the GOSE thresholds. Mean AUC is largely independent of observation window duration at each of the thresholds (**Supplementary Fig. S4 online**). The mean AUCs, along with 95% confidence intervals, at each threshold of GOSE is provided for all 19 tested observation windows in **Supplementary Table S6 online**. Binary classification performance metrics of optimally discriminating functional outcome prediction, at 12 months post discharge, models are provided in **Table 2**.

### Calibration of motor function detection and functional outcome prediction

The probability calibration curves and associated prediction distributions of the optimally discriminating models at each threshold for GCSm detection and GOSE (at hospital discharge) prediction are provided in **Supplementary Figure S5 online**. We observe that the GCSm > 4 detection model achieves the best graphical model calibration of all those tested (*E**max* = 0.30 [95% CI: 0.08–0.64]). However, when considering the prevalence of predicted probabilities in calibration assessment with the integrated calibration index (ICI)26, we observe that the GOSE > 5 prediction model has the most ideal calibration (ICI = 0.01 [95% CI: 0.00–0.02]). The discrepancy between the weighted and graphical calibration of GOSE > 5 indicates a strong class imbalance, suggesting that more positive cases are necessary to train and recalibrate this model for proper classification. Probability calibration metrics of all optimally discriminating models are provided in **Supplementary Table S7 online**.

### Extracted feature and sensor placement analysis

At the end of our accelerometry processing pipeline (**Fig. 1**), we extracted eight unique feature types (**Table 3**) from each of the six accelerometers placed around SBI patient joints. For each of these 48 feature-sensor combinations, we calculate a relative significance score equivalent to the mean absolute value of the learned coefficients of supervised dimensionality reduction (i.e., the relative importance in explaining the variance in the dataset stratified by the endpoint) weighted by the absolute value of learned logistic regression coefficients (see **Methods**).

View this table:
[Table 3.](http://medrxiv.org/content/early/2021/05/25/2021.05.19.21257319/T3)

Table 3. 
Overview of extracted motion feature types.

We consider the optimally discriminating configurations of the two most promising model types as representatives for motor function detection and functional outcome prediction respectively: (a) GCSm > 4 with a 6-hour observation window and (b) GOSE (at hospital discharge) > 5 with a 6-hour observation window. The feature significance scores of these two model types are visualized as heatmaps in **Fig. 4a** and **Fig. 4b** respectively.

![Fig. 4.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/05/25/2021.05.19.21257319/F4.medium.gif)

[Fig. 4.](http://medrxiv.org/content/early/2021/05/25/2021.05.19.21257319/F4)

Fig. 4. Feature significance matrices of optimally discriminating motor function detection and functional outcome prediction models.
Significance scores are calculated by weighting linear optimal low-rank projection (LOL) coefficients of sensor-feature type combinations by the logistic regression coefficients of the corresponding LOL component. The feature significance matrix in (**a**) corresponds to the optimally discriminating model configuration (6-hour observation window) for detection of GCSm > 4 (**Fig. 2a**) while the matrix in (**b**) corresponds to the optimally discriminating model configuration (6-hour observation window) for prediction of GOSE > 5 at hospital discharge (**Fig. 3a**). Mean significance scores are listed as well as 95% confidence intervals bootstrapped from 1,000 resamples to represent the variation across repeated cross-validation folds (5 repeats of 5 folds) and nine missing value imputations. Sensor placement acronyms correspond to joints shown in **Fig. 1** and feature type acronyms are decoded in **Table 3**.

For both motor function detection and functional outcome prediction, there is more variation in significance scores across feature types than across sensor placements. For motor function detection, the proportion of dynamic activity (PDA) in the observation window, the frequency-domain entropy (FDE), and the median frequency (MFR) are the three most significant feature types, descending in that order. For functional outcome prediction, the descending order of the three most significant feature types is FDE, MFR, and PDA. PDA is a crude measurement of overall physical activity27, while FDE enables differentiation between activity profiles which have simple acceleration patterns and those with more complex patterns16. From the pair of high-pass-filtered medians (HLF (h)) and low-pass-filtered medians (HLF (l)), HLF (h) has a significantly greater mean significance score than HLF (l) for every sensor placement in both model endpoints based on 95% confidence intervals. This, along with the relative significance of MFR, suggests that finer movements, captured in higher frequencies of accelerometry, can be more clinically significant in discriminating functional motor states and global outcomes from SBI. Moreover, the consistently strong significance of PDA, FDE, and MFR suggests that features of both the time domain (PDA) and the frequency domain (FDE, MFR) in combination may be useful for clinical assessments of functional neurological states.

In detecting motor function, the right wrist (RW) sensor was the most significant placement across the five most significant feature types. The trajectories of mean motion feature values in the six hours preceding GCSm evaluations (**Supplementary Fig. S6 online**) visually demonstrate that features extracted from the wrist-placed sensors better discriminate cases of GCSm 5 and 6 from the rest of the GCSm scores. This follows clinical observations of a greater frequency of conscious movement in hands and wrists of bedridden SBI patients during ICU stay. Moreover, abnormal profiles of flexion and extension, associated with SBI, are most often observed in the wrists, and thus, the wrist-placed sensors may be more sensitive to abnormal patterns of movement, corresponding to lower levels of consciousness, than the elbow- or ankle-placed sensors.

In functional outcome prediction, we observe the greatest significance scores ascribed to wrist-placed sensors (RW and LW) in the most significant frequency+domain features (FDE and MFR), but the ankle-(RA and LA) and elbow-placed sensors have the greatest significance scores in the most significant time-domain features (PDA, SMA, and HLF (h)). Wrist movements are finer than elbow and ankle movements and may be best distinguished in the frequency-domain in relation to global outcomes.

The correlation of each of the extracted motion features across the six sensor placements is visualized in **Supplementary Figure S7 online**, and violin plots of the distributions of motion features, stratified by GCSm, are presented in **Supplementary Figure S8 online**.

### Retrospective case study analysis of motor function detection in practice

Six patients in our study experienced a transition between GCSm > 4 and GCSm ≤ 4 within the GCS observations coinciding with 6-hour observation windows of accelerometry recording. For each of these six patients, we trained GCSm > 4 detection models on the remaining patient set with a shorter (27-minute) and a longer (6-hour) observation window. We return predictions with these models on the six case study patients every ten minutes to retrospectively examine the trajectories of probabilities against the recorded times of neurological transition (**Fig. 5**).

![Fig. 5.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/05/25/2021.05.19.21257319/F5.medium.gif)

[Fig. 5.](http://medrxiv.org/content/early/2021/05/25/2021.05.19.21257319/F5)

Fig. 5. Retrospective case study analysis of accelerometry-based detection of motor function in six patients who experienced relevant transition.
The red and blue lines correspond to the predicted probabilities returned my models trained on all other patients on short (27 minutes) and long (6 hours) observation windows respectively. Shaded areas represent 95% confidence intervals derived using bootstrapping (10,000 resamples) to represent the variation across nine missing value imputations. Upward triangle markers designate GCSm > 4 while downward triangle markers designate GCSm ≤ 4.

In case no. 2, we observed that both model types detect an upward transition in GCSm more than three hours before it was reported in the EHR. Likewise, the 27-min observation window model detected a downward transition in GCSm about an hour before the upcoming evaluation in case no. 4 and about two hours before in case no. 3. In cases no. 3, 4, and 6, we observed that the 6-hour observation window detects the appropriate transition in GCSm, but with a delay of 3–6 hours. In cases no. 1 and 5, in which we observe a shift and resettlement of GCSm within a 3– 5-hour span, the 6-hour model fails to detect the transition while the 27-min model uncertainly oscillates above and below the midline. In general, the shorter observation window model was more dynamic and detected GCSm transitions quicker than the longer observation window model. However, persistent transitions, such as the one observed in case no. 6, were detected with more stability and reliability by the longer observation window model.

## DISCUSSION

### Key findings

We introduce an accelerometry-based based system in critically ill SBI patients that quantitatively captures multisegmental motor patterns correlating with clinical scores of motor responsiveness and functional outcome. The results reveal a significant (AUC = 0.70 [95% CI: 0.53–0.85]), consistent (observation windows: 12 min – 9 hours) association between extracted motion features and the discrimination of SBI patients capable of purposeful movement (GCSm > 4) and those who are not (GCSm ≤ 4) (**Fig. 2a**). A significant discrimination of purposeful movement was achieved with only 12 minutes of accelerometry recording (**Fig. 2b**), and reliable calibration (**Supplementary Fig. S5 online**) and informative classification (**Table 2**) for GCSm > 4 detection suggest that iterations of this system could be clinically useful in automating motor function monitoring. In case studies (**Fig. 5**), we demonstrate that accelerometry-based systems may detect transitions in motor function up to five hours before a clinical evaluation.

The utility of accelerometry-based features for functional outcome prognosis remains ambiguous. While we found no signal between motion features and long-term (12 months post discharge) outcomes (**Supplementary Fig. S4 online**), the models accurately predicted functional status at hospital discharge (AUC = 0.82 [95% CI: 0.75–0.90]) at a cutoff of GOSE > 5 vs GOSE ≤ 5 for favorable vs unfavorable outcome (**Fig. 3a**). Patients with a GOSE of >5 have upper moderate disability or good recovery and are generally able to resume work or previous activities.

However, given the small number of SBI patients with GOSE > 5 at hospital discharge, further validation is necessary to determine the reliability of this result. Conflicting results between different calibration metrics (**Supplementary Table S7 online**) underline the class imbalance problem of GOSE > 5 in our dataset; at the same time, we find the consistent discrimination (**Fig. 3b**) and difference in outcome distribution (**Supplementary Fig. S3 online**) as promising markers for further exploration.

Finally, our analysis of feature significance (**Fig. 4**) reveals that both time-domain and frequency-domain features are important for motor function detection and functional outcome prediction. While sensors placed on the wrist achieved the greatest significance scores overall, particularly for features in the frequency-domain, multisegmental motion capture was validated by comparable significance scores of elbow-and ankle-placed sensors across the feature set.

### Relationship with previous studies and future implications

Results presented here represent, to our knowledge, the first approach to relate motion sensor data to neurological states in SBI patients admitted to the ICU. Healthy activity classification with accelerometry-based features has become widespread, especially with advancements in MEMS technology, machine learning, and data sharing16, 28-30. However, applications to intrahospital care, and in particular intensive care, have been limited23, 31 and have largely taken only simple, threshold-based feature approaches to grossly evaluate motor activity (e.g., actigraphy)32-35. Reported success in these studies has been variable, but none of them have combined the high-resolution time-domain, feature-domain, and wavelet-domain analysis found in more recent healthy activity classification studies. The focus of our approach, on the relationship between motor profiles of SBI patients over extended periods of time and clinically relevant neurological states, is novel. Yet, it builds upon the developments in time-series analysis, dimensionality reduction, and supervised machine learning from activity classification projects as well as the hypotheses of the clinical validity and utility of accelerometry from applied, medical projects.

A continuous high-frequency motion capture system in the intensive care setting produces a high-dimensional dataset that is also valuable for data-driven research projects. Profiles of motor activity in SBI are poorly understood and decoding specific features of motion in the time-, frequency-, and wavelet-domains can open a window on internal neurological states. Accelerometry-based features may elucidate fundamental mechanisms underlying the strong association between physical activity and clinical outcomes, and we aim to collect more data in the NIMS project to enable the research and development of motion as a quantitative marker of functional recovery for SBI.

More generally, the critical care setting is a fertile ground for the development of advanced computational methods and applications of artificial intelligence for monitoring and decision support36. Patients are typically interfacing with physiological monitoring systems that generate a large volume of data whose complexity may overwhelm human interpretation alone but may be ideal for the training of analytical systems37. Since critical care specialists typically must make time-sensitive decisions for multiple patients at the same time38, we expect that a near-real-time computational framework assessing motion features alongside other time-series data continuously could provide valuable decision support. We expect that ongoing and subsequent iterations of this work will enable integration of computational physical activity features into the framework of monitoring and prognostication in the critical care setting.

### Study limitations

We recognize several limitations in this work that need to be addressed. Our statistical analyses and retrospective validation of GCSm detection and GOSE prediction were performed on a limited sample size (*n* = 69 patients) from a single institution and intensive care facility. Further validation will require repeated trials on larger patient populations across multiple centers. There are also improvements to be done to the sensor itself. The planar dimensions of our currently used accelerometer (42 mm × 32 mm) can be reduced further to increase the resolution of localized motion capture. Furthermore, since accelerometry measurements depend on the orientation of the accelerometer with respect to the vertical (gravitational axis), additional modalities of motor output (i.e., gyroscopy and electromyography) could be integrated into the sensor system to inform computational models on the precise arrangement and neural activation of body segments. This would allow us to derive more physiologically relevant features that correspond to validated models of nervous system injury or disease6. We also recommend the development of sensors with higher sampling frequencies (≥40 Hz) to capture extremely fine or fast movements of digits or lower extremities. Additionally, GCSm itself has been criticized for lack of standardization among practitioners39, 40. GCSm scores for this work were extracted automatically from EHR and were measured from multiple practitioners across the Johns Hopkins Hospital Neurosciences Critical Care Unit (NCCU) staff. Moving forward, we aim to supplement clinical validation of the motion features with multifactorial associations with other consciousness, functional, cognitive, psycho-behavioral, symptomatic, and social outcome scales of SBI patients41.

## METHODS

### Study population and experimental protocol

This work was conducted with approval from the Johns Hopkins Medicine Institutional Review Board (IRB00135674) and written informed consent from patients or surrogates. We prospectively enrolled 72 patients admitted to the NCCU who met the following criteria: age ≥ 18 years, SBI defined as an acute brain injury or illness resulting in impaired consciousness, absence of injuries or lesions involving the extremities, and not expected to die or have withdrawal of life-sustaining therapies in the 24 hours following enrolment.

Patients were evaluated daily while in the NCCU, at hospital discharge, and at 12 months post discharge by research team members. All GCS evaluations during each patient’s hospital stay were automatically extracted from the institutional EHR system (Epic Systems, Madison, WI, USA). GOSE scores at hospital discharge were obtained by EHR review of discharge reports for patients who survived during hospital stay (*n* = 53). Patients were contacted by telephone 12 months (±1 month) after hospital discharge, and GOSE scores were obtained using a validated questionnaire42 (*n* = 27); in cases where patients could not be contacted, data was extracted from EHR reports (*n* = 8). Additionally, we identified participants who died between discharge and 12 months post-discharge from national obituary records (*n* = 12). Thus, we arrived at a 12-month post-discharge sample size of *n* = 64.

From the first 3 patients, we collected 10 hours of continuous triaxial accelerometry data, and for the remainder of the patients, we augmented our recording duration to between 24 and 48 hours of accelerometry data.

### Instrumentation for accelerometry capture

Triaxial sensors (SensorTags CC2650, Texas Instruments, Dallas, TX, USA) were attached with transparent film dressing (Tegaderm Diamond Pattern 1686, 3M, Maplewood, MN, USA) bilaterally near the joints (with common orientation) designated in **Fig. 1a**. An additional sensor was placed vertically on the foot of the patient bed to detect patient-independent bed movements. Sensors were equipped with MEMS, variable capacitance tri-axial accelerometers (MPU-9250 MotionTracking Device, TDK InvenSense, San Jose, CA, USA) with sampling frequency (*f**s*) set to 10 Hz, the range of measurable amplitude at ±16 g (±157 m/s2), and sensitivity at ±4,800 least significant bits per g (LSB/g).

The sensors transmitted data via a 2.4-GHz Bluetooth antenna to a portable Linux computer (RPi 3 Model B, Raspberry Pi Foundation, Cambridge, UK) placed in the NCCU room. We would execute a Python script on the computer to collect 3 channels (axes) of accelerometry time series from each of the 7 active accelerometers in parallel. The system would log interruptions on a separate .txt file in the instance of a sensor failure. During each trial, we also recorded a video stream (M1045-LW Network Camera, Axis Communications, Lund, Sweden) of the patient that clearly shows the location of each sensor. In the event of sensor interruptions, irregular movement profiles, or bed-sensor-extracted signal magnitude (SMA) values above 0.135 *g*27, we would check the footage to identify the source of these results.

### Accelerometry processing and motion feature extraction

Each axial component of each sensor was convolved with a 4th-order Butterworth high-pass filter with a critical frequency of *f**c* = 0.2 Hz (**Supplementary Fig. S9 online**) to remove the baseline offset of accelerometry readings (**Fig. 1a**) and generally separate the low frequency effect of static orientation from the high frequency effect of active body movement43.

Filtered time-series were segmented into non-overlapping 5-second windows (∼50 data points per window) for motion feature extraction. We selected the motion features listed in **Table 3**, which performed well in physical activity classification tasks16, to represent three different domains (time frequency, and wavelet). PDA is defined by the proportion of SMA over 0.135 *g* for each sensor in an observation window (**Fig. 1b**). The remaining features are defined by the following formulae for each 5-second window: ![Formula][1]</img>  where:

*   ⍰ *x, y, z* represent the x-, y-and z-axes vectors, respectively, of the filtered accelerometry time series within the given 5 second window and *x**n*, *y**n*, *z**n* represent the *n*th elements of these vectors.

*   ⍰ *N* represents the length of each of the x, y, z vectors.

*   ⍰ * represents the 1-dimensional convolution operator.

*   ⍰ *b*h represents a 1-dimensional, 4 -order high-pass Butterworth filter with *f**c* = 2.5 Hz.

*   ⍰ *b*l represents a 1-dimensional, 4 -order low-pass Butterworth filter with *f**c* = 2.5 Hz.

*   ⍰ *X, Y, Z* represent the discrete Fourier transforms of the *x, y, z* vectors respectively where *X**n*, *Y**n*, *Z**n* represent the *n*th elements of these Fourier transform vectors and *X*f, *Y*f, *Z*f represent the coefficients of the Fourier transforms that correspond to linear frequency *f*.

*   ⍰ ![Graphic][2]</img> represent the vector of lth-level detail coefficients of the 5th-order Daubechies wavelet transform of the *x, y, z* vectors respectively.

Post-capture processing of accelerometry were performed offline using MATLAB (Version 9.8.0, MathWorks, Natick, MA, USA) with the Signal Processing, Wavelet, System Identification, and Symbolic Toolboxes.

### Multiple imputation of missing motion features

Due to insufficient battery on the sensors, bedside interventions, interfering equipment, or patient migrations for surgery, imaging, or interunit transfers, a median 1.56% per sensor of each patient’s intended recording duration was missing in our dataset. Missing motion features were multiply imputed (*m* = 9) with a normal (features were normalized with the Box-Cox transform44) multivariate time-series algorithm from the ‘Amelia II’ package (v1.7.6)45 in R (v4.0.0)46. The algorithm exploits both spatial correlation (motion feature correlation across the sensors of the same participant) and temporal correlation (autocorrelation structures within each sensor’s time series) to stochastically impute missing time series values in multiple, independently trained runs. We formed subsequent statistical analyses on all 9 imputations to account for variation across imputation. This model assumes the data is missing at random (MAR) (i.e., the pattern of missingness is independent of unobserved data47), which we validated by observing the independence of missingness from sensor placement or time of day (**Supplementary Fig. S10 online**). A complete characterization of the missing data of each patient can be found in **Supplementary Table S8 online**.

### Correction of gross external movements

At time points where the bed-placed sensor SMA exceeded 0.135 *g* (a proposed threshold between static and dynamic activity27) and preceded a spike in extremity feature values (1.33% of the time), the bed sensor values of SMA, HLF, BPW, and WVL were subtracted from the extremity values and the bed sensor values of MFR and FDE were added to the extremity values. If a resulting correction value ended up out of a feasible range of static activity for the feature, we replaced the value with a random value, selected uniformly from the static activity range of that feature (**Supplementary Table S9 online**).

### Repeated *k*-fold cross-validation for unbiased model validation

The study population (*n* = 69) was partitioned 25 times with repeated *k*-fold cross-validation (5 repeats, 5 folds) into training sets (∼80%, *n* ≈ 55) and validation sets (∼20%, *n* ≈ 14) for each of the 19 tested observation windows (**Supplementary Table S1 online**) for each of the three tested endpoints (**Fig. 2b**). In splits for motor function detection, patients were stratified by median GCSm over their available observations, while in splits for functional outcome detection, patients were stratified by GOSE scores. One of the nine missing value imputations was drawn with replacement for each partition.

Repeated cross-validation partitions were performed with the ‘caret’ package (v6.0-86)48 in R.

### Motor function detection

We tested 19 unique observation window durations, from 3 minutes to 24 hours, (**Supplementary Table S1 online**) of accelerometry-derived features directly preceding GCSm evaluations (**Fig. 1b**). At each of these evaluation points, motion features were organized into matrices where each column represents a unique combination of motion feature type (8 total), sensor placement (6 total), and, for non-PDA features, time before the evaluation. Columns were normalized based on distributions of each placement-feature type combination (48 in total) in the training set. Normalized matrices underwent supervised dimensionality reduction with linear optimal low-rank projection (LOL)49 learned from the training set. Target dimensionality (*d* ∈ [2,20]) was tested as a model hyperparameter. Low-dimensional vectors of each *d* then underwent element-wise Yeo-Johnson transforms50 for scaled normalization (learned from the training set) and were used to train and validate logistic regression (‘glm’) models with binary endpoints at each GCSm threshold. All of these steps were performed in R.

### Functional outcome prediction

The methodology for functional outcome prediction was identical to that of motor function detection except that GOSE thresholds instead of GCSm thresholds were used as endpoints.

### Assessment of model performance and calibration on validation sets

Both motor function detection and functional outcome prediction models were trained and validated on each of the 25 repeated cross-validation splits for each of the 19 observation windows for each of the 19 unique target dimensionalities (*d*) for each of the endpoint thresholds (5 for GCSm, 5 for GOSE at discharge, 7 for GOSE at 12 months). Models returned binary prediction probabilities as well as a classification based on a probability threshold of 0.5 for each validation set observation.

Based on the validation set predictions, we calculated metrics of binary outcome discrimination performance (**Supplementary Tables S2, S4**, and **S6 online**), classification performance (**Table 2**), and probability calibration26, 51 (**Table 3**). We also visualized ROC curves (**Fig. 2a, 3a**, and **Supplementary Fig. S4 online**), probability calibration curves (**Supplementary Fig. S5 online**), and, in one case, the precision recall curve (**Supplementary Fig. S3 online**) of the optimally discriminating (maximal AUC) models to assess discrimination, calibration, and case detection power respectively. We calculated unbiased mean values and 95% confidence intervals for both metrics and curves with bootstrap bias-corrected cross-validation (BBC-CV) with repeats52 on 1,000 resamples of the patient set across the validation set predictions. In this way, 95% confidence intervals account for the variation across the patient set, across the nine missing value imputations, and across the 25 repeated cross-validation partitions.

### Feature significance scores

The coefficients (i.e., loadings) of the trained LOL projection matrix represent the relative importance of each column in explaining the variance in the dataset stratified by the endpoint49. Thus, we derived a relative importance score of each sensor-feature type combination for both motor function detection and functional outcome prediction by multiplying the mean absolute value of the loadings per each combination and the absolute value of the trained logistic regression coefficient of the corresponding reduced dimension. This would be performed across all 25 partitions of each combination of observation window, threshold, and endpoint. We then calculated 95% confidence intervals on feature significance scores by bootstrapping 1,000 resamples across the 25 repeated cross-validation folds and nine missing value imputations.

## Supporting information

Supplementary Materials [[supplements/257319_file07.pdf]](pending:yes)

## Data Availability

Per our current Johns Hopkins Medicine IRB protocol (IRB00135674), we are not permitted to share the clinical data collected for this study. However, we welcome all forms of collaboration, and urge interested investigators to contact the corresponding author (SB: sb2406@cam.ac.uk) with their institutional affiliation and proposed use of the dataset to submit a new protocol for access. The data may not be used for commercial products or redistributed in any way.

## DATA AVAILABILITY

Per our current Johns Hopkins Medicine IRB protocol (IRB00135674), we are not permitted to share the clinical data collected for this study. However, we welcome all forms of collaboration, and urge interested investigators to contact the corresponding author (SB: sb2406{at}cam.ac.uk) with their institutional affiliation and proposed use of the dataset to submit a new protocol for access. The data may not be used for commercial products or redistributed in any way.

## CODE AVAILABILITY

All code used in the data collection and analyses outlined in this manuscript can be found at the following GitHub repository53: [https://github.com/sbhattacharyay/nims](https://github.com/sbhattacharyay/nims) (DOI: 10.5281/zenodo.4765305).

## AUTHOR CONTRIBUTIONS

S.B. aided in the conceptualization of the study, developed the methodology of the experiments, acquired accelerometry data from patients, acquired funding for the project, performed statistical analyses on the data, visualized the results for publication, and wrote the complete manuscript. J.R. and R.E.C. aided in the conceptualization and data collection of this work and revised the manuscript. M.W., H.B.K., and E.J. aided S.B. in the statistical analysis, processing of data, and visualization of results. P.D. extracted neurological assessment scores from electronic health records. E.C. recruited patients for the study, performed clinical surveys, and collected clinical data from patient records. P.K. aided in the conceptualization of the study and the development of the methodology and established the data acquisition infrastructure. R.D.S. served as the principal investigator, conceptualized the study, aided in the development of the methodology, procured IRB approval for data collection from human subjects, aided in data collection, provided access to clinical resources at the Johns Hopkins Hospital, and revised the manuscript.

## COMPETING INTERESTS STATEMENT

The authors declare that they have no conflicts of interest.

## ACKNOWLEDGEMENTS

We graciously acknowledge the patients, families, NCCU nurses, and physicians who participated in and contributed to this study. S.B. would like to thank Kathleen Mitchell-Fox (Univ. of Cambridge) for reviewing and offering comments on the manuscript. We also wish to specifically thank Aditya Joshi (Rowan Univ.), Sanya Yadav (Univ. of Pittsburgh), Tobias Fauser (Univ. of Arizona), Michiru Fredricks (Johns Hopkins Univ.), Alexander Sigmon (Johns Hopkins Univ.), Shikha Gandhi (Johns Hopkins Univ.), and Joshua Vogelstein (Johns Hopkins Univ.) for their roles in the early development, data curation, and advising of statistical methodologies of the NIMS project.

This work was partially supported by awards from the Johns Hopkins University Office of the Provost and the Hodson Trust, received by S.B. S.B. is currently funded by a Gates Cambridge fellowship.

*   Received May 19, 2021.
*   Revision received May 25, 2021.
*   Accepted May 25, 2021.


*   © 2021, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/)

## REFERENCES

1.  1.Maas, A. I. R. et al. Traumatic brain injury: integrated approaches to improve prevention, clinical care, and research. Lancet Neurol. 16, 987–1048 (2017).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1474-4422(17)30371-X&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=29122524&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 

2.  2.Stevens, R. D. & Sutter, R. Prognosis in Severe Brain Injury. Crit. Care Med. 41 (2013).
    
    

3.  3.Turgeon, A. F. et al. Mortality associated with withdrawal of life-sustaining therapy for patients with severe traumatic brain injury: a Canadian multicentre cohort study. CMAJ 183, 1581–1588 (2011).
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTgzLzE0LzE1ODEiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wNS8yNS8yMDIxLjA1LjE5LjIxMjU3MzE5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

4.  4.Alkhachroum, A., Terilli, K., Megjhani, M. & Park, S. Harnessing Big Data in Neurocritical Care in the Era of Precision Medicine. Curr. Treat. Options Neurol. 22, 15 (2020).
    
    

5.  5.Fidali, B. C., Stevens, R. D. & Claassen, J. Novel approaches to prediction in severe brain injury. Curr. Opin. Neurol. 33 (2020).
    
    

6.  6.Winters, J. M. & Crago, P. E. in Biomechanics and neural control of posture and movement 683 (Springer, New York, 2000).
    
    

7.  7.Shadmehr, R. & Krakauer, J. W. A computational neuroanatomy for motor control. Exp. Brain Res. 185, 359–381 (2008).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00221-008-1280-5&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18251019&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000253128500001&link_type=ISI) 

8.  8.Reinkensmeyer, D. J. et al. Computational neurorehabilitation: modeling plasticity and learning to predict recovery. J. Neuroeng. Rehabil. 13, 42 (2016).
    
    

9.  9.Olkowski, B. F. & Shah, S. O. Early Mobilization in the Neuro-ICU: How Far Can We Go? Neurocrit. Care 27, 141–150 (2017).
    
    

10. 10.Drummond, M. J. et al. Short-term bed rest increases TLR4 and IL-6 expression in skeletal muscle of older adults. Am. J. Physiol. Regul. Integr. Comp. Physiol. 305, 216 (2013).
    
    

11. 11.Parry, S. M. & Puthucheary, Z. A. The impact of extended bed rest on the musculoskeletal system in the critical care environment. Extrem. Physiol. Med. 4, 16-7. eCollection 2015 (2015).
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 

12. 12.Topp, R., Ditmyer, M., King, K., Doherty, K. & Hornyak, J. The effect of bed rest and potential of prehabilitation on patients in the intensive care unit. AACN Clin. Issues 13, 263–276 (2002).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/00044067-200205000-00011&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12011598&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 

13. 13.Bloomfield, S. A. Changes in musculoskeletal structure and function with prolonged bed rest. Med. Sci. Sports Exerc. 29, 197–206 (1997).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/00005768-199702000-00006&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9044223&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1997WH75500006&link_type=ISI) 

14. 14.Fowles, J. R., Sale, D. G. & MacDougall, J. D. Reduced strength after passive stretch of the human plantarflexors. J. Appl. Physiol. (1985) 89, 1179–1188 (2000).
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10956367&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000089200200042&link_type=ISI) 

15. 15.Bahouth, M. N. et al. Safety and Feasibility of a Neuroscience Critical Care Program to Mobilize Patients With Primary Intracerebral Hemorrhage. Arch. Phys. Med. Rehabil. 99, 1220–1225 (2018).
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 

16. 16.Preece, S. J. et al. Activity identification using body-mounted sensors--a review of classification techniques. Physiol. Meas. 30, 1 (2009).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1088/0967-3334/30/1/001&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19039165&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000263031300001&link_type=ISI) 

17. 17.Adrian, M. & Cooper, J. M. in Biomechanics of human movement 572 (Brown & Benchmark, Madison, Wis., 1995).
    
    

18. 18.Mathie, M. J., Coster, A. C., Lovell, N. H. & Celler, B. G. Accelerometry: providing an integrated, practical method for long-term, ambulatory monitoring of human movement. Physiol. Meas. 25, 1 (2004).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1088/0967-3334/25/1/001&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15005300&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000189183300002&link_type=ISI) 

19. 19.Bouten, C. V., Koekkoek, K. T., Verduin, M., Kodde, R. & Janssen, J. D. A triaxial accelerometer and portable data processing unit for the assessment of daily physical activity. IEEE Trans. Biomed. Eng. 44, 136–147 (1997).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1109/10.554760&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9216127&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1997WL05800002&link_type=ISI) 

20. 20.Moe-Nilssen, R. Test-retest reliability of trunk accelerometry during standing and walking. Arch. Phys. Med. Rehabil. 79, 1377–1385 (1998).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0003-9993(98)90231-3&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9821897&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000076891300006&link_type=ISI) 

21. 21.Hansson, G. A., Asterland, P., Holmer, N. G. & Skerfving, S. Validity and reliability of triaxial accelerometers for inclinometry in posture analysis. Med. Biol. Eng. Comput. 39, 405–413 (2001).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/BF02345361&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11523728&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000170550800001&link_type=ISI) 

22. 22.Meijer, G. A., Westerterp, K. R., Verhoeven, F. M., Koper, H. B. & ten Hoor, F. Methods to assess physical activity with special reference to motion sensors and accelerometers. IEEE Trans. Biomed. Eng. 38, 221–229 (1991).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1109/10.133202&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=2066134&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1991FK82000001&link_type=ISI) 

23. 23.Verceles, A. C. & Hager, E. R. Use of Accelerometry to Monitor Physical Activity in Critically Ill Subjects: A Systematic Review. Respir. Care 60, 1330–1336 (2015).
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoicmVzcGNhcmUiO3M6NToicmVzaWQiO3M6OToiNjAvOS8xMzMwIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDUvMjUvMjAyMS4wNS4xOS4yMTI1NzMxOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

24. 24.Teasdale, G. & Jennett, B. Assessment of coma and impaired consciousness. A practical scale. Lancet 2, 81–84 (1974).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(02)93219-8&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=4136544&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1974T535500009&link_type=ISI) 

25. 25.Teasdale, G. et al. The Glasgow Coma Scale at 40 years: standing the test of time. Lancet Neurol. 13, 844–854 (2014).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1474-4422(14)70120-6&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25030516&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 

26. 26.Austin, P. C. & Steyerberg, E. W. The Integrated Calibration Index (ICI) and related metrics for quantifying the calibration of logistic regression models. Stat. Med. 38, 4051–4065 (2019).
    
    

27. 27.Lugade, V., Fortune, E., Morrow, M. & Kaufman, K. Validity of using tri-axial accelerometers to measure human movement—Part I: Posture and movement detection. Med. Eng. Phys. 36, 169–176 (2014).
    
    

28. 28.Jordao, A., Torres, L. A. B. & Schwartz, W. R. Novel approaches to human activity recognition based on accelerometer data. Signal Image Video Process. 12, 1387–1394 (2018).
    
    

29. 29.Ignatov, A. Real-time human activity recognition from accelerometer data using Convolutional Neural Networks. Appl. Soft. Comput. 62, 915–922 (2018).
    
    

30. 30.Migueles, J. H. et al. Accelerometer Data Collection and Processing Criteria to Assess Physical Activity and Other Outcomes: A Systematic Review and Practical Considerations. Sports Med. 47, 1821–1845 (2017).
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 

31. 31.Fazio, S. et al. Quantifying Mobility in the ICU: Comparison of Electronic Health Record Documentation and Accelerometer-Based Sensors to Clinician-Annotated Video. Crit. Care Explor. 2 (2020).
    
    

32. 32.Montoye, A. H. K., Moore, R. W., Bowles, H. R., Korycinski, R. & Pfeiffer, K. A. Reporting accelerometer methods in physical activity intervention studies: a systematic review and recommendations for authors. Br. J. Sports Med. 52, 1507–1516 (2018).
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYmpzcG9ydHMiO3M6NToicmVzaWQiO3M6MTA6IjUyLzIzLzE1MDciO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wNS8yNS8yMDIxLjA1LjE5LjIxMjU3MzE5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

33. 33.Kanai, M. et al. Effect of accelerometer-based feedback on physical activity in hospitalized patients with ischemic stroke: a randomized controlled trial. Clin. Rehabil. 32, 1047–1056 (2018).
    
    

34. 34.Grimes, L., Outtrim, J. G., Griffin, S. J. & Ercole, A. Accelerometery as a measure of modifiable physical activity in high-risk elderly preoperative patients: a prospective observational pilot study. BMJ Open 9, e032346 (2019).
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiOS8xMS9lMDMyMzQ2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDUvMjUvMjAyMS4wNS4xOS4yMTI1NzMxOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

35. 35.Stienen, M. N. et al. Objective activity tracking in spine surgery: a prospective feasibility study with a low-cost consumer grade wearable accelerometer. Sci. Rep. 10, 4939 (2020).
    
    

36. 36.Lovejoy, C. A., Buch, V. & Maruthappu, M. Artificial intelligence in the intensive care unit. Crit. Care 23, 7–9 (2019).
    
    

37. 37.Gholami, B., Haddad, W. M. & Bailey, J. M. AI in the ICU: In the intensive care unit, artificial intelligence can keep watch. IEEE Spectr. 55, 31–35 (2018).
    
    

38. 38.Halpern, N. A., Pastores, S. M., Oropello, J. M. & Kvetan, V. Critical Care Medicine in the United States: Addressing the Intensivist Shortage and Image of the Specialty*. Crit. Care Med. 41 (2013).
    
    

39. 39.Reith, F. C., Brennan, P. M., Maas, A. I. & Teasdale, G. M. Lack of Standardization in the Use of the Glasgow Coma Scale: Results of International Surveys. J. Neurotrauma 33, 89–94 (2016).
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 

40. 40.Reith, F. C., Van den Brande, R., Synnot, A., Gruen, R. & Maas, A. I. The reliability of the Glasgow Coma Scale: a systematic review. Intensive Care Med. 42, 3–15 (2016).
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 

41. 41.Kean, J. & Malec, J. F. Towards a Better Measure of Brain Injury Outcome: New Measures or a New Metric? Arch. Phys. Med. Rehabil. 95, 1225–1228 (2014).
    
    

42. 42.Wilson, J. T., Pettigrew, L. E. & Teasdale, G. M. Structured interviews for the Glasgow Outcome Scale and the extended Glasgow Outcome Scale: guidelines for their use. J. Neurotrauma 15, 573–585 (1998).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1089/neu.1998.15.573&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9726257&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000075436500002&link_type=ISI) 

43. 43.van Hees, V. T. et al. Separating movement and gravity components in an acceleration signal and implications for the assessment of human daily physical activity. PLoS One 8, e61691 (2013).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0061691&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23626718&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 

44. 44.Box, G. E. P. & Cox, D. R. An Analysis of Transformations. J. R. Stat. Soc. Series B Stat. Methodol. 26, 211–243 (1964).
    
    

45. 45.Honaker, J., King, G., Blackwell, M. Amelia II: A Program for Missing Data. J. Stat. Softw. 45 (2011).
    
    

46. 46.R Core Team. R: A Language and Environment for Statistical Computing. 4.0.0(2020).
    
    

47. 47.Rubin, D. B. Inference and missing data. Biometrika 63, 581–592 (1976).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/biomet/63.3.581&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1976CP66700021&link_type=ISI) 

48. 48.Kuhn, M. Building Predictive Models in R Using the caret Package. J. Stat. Softw. 28 (2008).
    
    

49. 49.Vogelstein, J. T. et al. Geometric Dimensionality Reduction for Subsequent Classification. Preprint at [https://arxiv.org/abs/1709.01233](https://arxiv.org/abs/1709.01233) (2017).
    
    

50. 50.Yeo, I. & Johnson, R. A. A new family of power transformations to improve normality or symmetry. Biometrika 87, 954–959 (2000).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/biomet/87.4.954&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000166132800017&link_type=ISI) 

51. 51.Harrell, F. E., Jr.. in Regression Modeling Strategies (Springer International Publishing AG, Cham, 2015).
    
    

52. 52.Tsamardinos, I., Greasidou, E. & Borboudakis, G. Bootstrapping the out-of-sample predictions for efficient and accurate cross-validation. Mach. Learning 107, 1895–1922 (2018).
    
    

53. 53.Bhattacharyay, S., Wang, M. & Joshi, E. sbhattacharyay/nims: Neurological Injury Motion Sensing (NIMS) Project Repository. v1.0.2 (2021).
    
    

54. 54.Mathie, M. J., Coster, A. C., Lovell, N. H. & Celler, B. G. Detection of daily physical activities using a triaxial accelerometer. Med. Biol. Eng. Comput. 41, 296–301 (2003).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/BF02348434&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12803294&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 

55. 55.Fahrenberg, J., Foerster, F., Smeja, M. & Muller, W. Assessment of posture and motion by multichannel piezoresistive accelerometer recordings. Psychophysiology 34, 607–612 (1997).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1469-8986.1997.tb01747.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9299915&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1997XV82900013&link_type=ISI) 

56. 56.Foerster, F. & Fahrenberg, J. Motion pattern and posture: correctly assessed by calibrated accelerometers. Behav. Res. Methods Instrum. Comput. 32, 450–457 (2000).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3758/BF03200815&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11029819&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 

57. 57.Bao, L. & Intille, S. S. Activity recognition from user-annotated acceleration data (International conference on pervasive computing, Springer-Verlag, Berlin, Germany, 2004).
    
    

58. 58.Sugimoto, A., Hara, Y., Findley, T. W. & Yoncmoto, K. A useful method for measuring daily physical activity by a three-direction monitor. Scand. J. Rehabil. Med. 29, 37–42 (1997).
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9084104&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F25%2F2021.05.19.21257319.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1997WN61200006&link_type=ISI) 

59. 59.Wang, N., Ambikairajah, E., Lovell, N. H. & Celler, B. G. Accelerometry Based Classification of Walking Patterns Using Time-frequency Analysis (29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Institute of Electrical and Electronics Engineers, Piscataway, New Jersey, USA, 2007).

 [1]: /embed/graphic-9.gif
 [2]: /embed/inline-graphic-1.gif