Validation of an open-source smartphone step counting algorithm in clinical and non-clinical settings

Introduction

Walking is the most common form of physical activity (1). It is also important to prevent chronic disease and premature mortality (2–4). The recent proliferation and integration of wearable activity trackers into public health and clinical research studies has allowed investigators to identify gait-related biomarkers, such as decreased daily step counts, as risk factors for cardiovascular disease, cancer, stroke, and type 2 diabetes (5–10).

Despite the potential for wearable activity trackers to increase physical activity, improve health, and provide unique behavioral insights, there are several important limitations. First, the adoption of wearables is uneven across the population, and most people stop using wearable activity trackers after 6 months (11–14). Second, commercial devices rarely allow access to their raw (sub-second level) data, or provide open-source algorithms for processing data into clinically meaningful endpoints (15–17). Third, the accuracy of step count estimates are affected by metrological and behavioral factors, such as the location of the wearable on the body and temporal gait speed (18–20).

Smartphones are a promising alternative for collecting objective, scalable, and reproducible data about human behavior (21–24). Although smartphones can overcome many limitations of wearable activity trackers—e.g., through access to raw sensor data (25) and increased adoption among older individuals (26)— the quantification of gait-related biomarkers remains challenging. This is largely because of the variation in the location and orientation of smartphones in relation to the body in real-life conditions, which affects the data collected from smartphones’ inertial sensors (27–29).

To address this problem, we have recently proposed an open-source walking recognition method (29), which can be applied to accelerometer data collected from various locations on the body, making it suitable for smartphones. In this paper, we demonstrate how our method can be used for quantifying steps, and we validate its performance in eight independent datasets. We validate this method in three ways: (i) by comparing step counts estimated from multiple sensors worn simultaneously at various known body locations; (ii) by comparing step counts estimated from a sensor worn at an unspecified body location against visually assessed and manually annotated ground truth (i.e., observed step counts); and (iii) by comparing step counts estimated from a sensor worn at an unspecified body location against estimates provided by an independent commercial activity tracker (Fitbit Charge 2) worn on the wrist. The first (“internal”) and second (“manual”) validations involve healthy subjects whose data were obtained from publicly available datasets collected in controlled, semi-controlled, and free-living conditions, while the third (“wearable”) validation includes data collected by our team from patients with advanced cancer receiving chemotherapy as outpatients in free-living conditions.

Methods

Step counting algorithm

Our method leveraged the observation that regardless of the sensor location, orientation, or subject, during walking activity device’s accelerometer signal oscillates around a local mean with a frequency equal to the performed steps (29). To extract this information, we used the continuous wavelet transform to project the original signal onto the time-frequency space of wavelet coefficients which are maximized when a particular frequency matches the frequency of the observed signal at a given time point (Figure 1). To translate this information into number of steps, we split the projection into non-overlapping one-second windows, and we estimated the temporal step frequency as a frequency with the maximum average wavelet coefficient. The estimated frequency reflects the number of steps a person performs within this time window. Finally, the total number of steps was calculated as a sum of all one-second step counts calculated over the duration of the observed period of walking.

• Download figure
• Open in new tab

Figure 1.

The step counting algorithm. The original signal (a.) is projected onto the time-frequency space (b.) using wavelet transformation, which shows the relative weights of different frequencies over time (brighter color indicates higher weight). This scalogram is then split into non-overlapping one-second windows (c.). The temporal step frequency (cadence) is estimated as a frequency with maximum average wavelet coefficient inside each window (d.). The total number of steps in a signal is calculated as a rounded sum of all one-second counts in that signal (e.).

The step counting method described above is embedded into the walking recognition algorithm published in the public domain (30,31).

Results

Internal validation

The aggregated internal validation dataset consisted of data from healthy 103 participants (66 males, representing 64% of the dataset) between 16 and 62 years of age (25.2±7.1). All datasets, except for PARUSS provided data on participants’ height and weight, which ranged between 151 and 196 cm (173.8±8.5), and 47 and 112 kg (72.2±14.7), respectively. Participants’ body mass index (BMI) ranged between 17 and 35 kg/m² (23.8±4.1).

In this validation, step counts were aggregated separately for each walking bout across different body locations, including the thigh (n=83 bouts), waist (n=102), chest (n=51), and arm (n=25). Cumulatively, we examined 232 sensor body location pairs: thigh vs. waist (n=83), thigh vs. chest (n=32), thigh vs. arm (n=25), waist vs. chest (n=51), waist vs. arm (n=25), and chest vs. arm (n=15).

On average, in the aggregated internal validation dataset participants performed a mean of 751.7±581.2 steps per walking bout. Mean step counts varied by the dataset; e.g., participants’ mean step counts were 501.5±127.2 in DaLiAc, 337.5±14.6 steps in PARUSS, 1007.2±79.6 steps in RealWorld, 14.6±1.7 steps in SFDLA, and 1408.7±561.5 steps in SPADES.

Figure 2a displays the Bland-Altman plots for the aggregated internal validation dataset. Comparisons between individual studies are provided in Supplementary Figure 1a-e. Across the aggregated dataset, the mean bias was equal to -7.2 steps (LoA -47.6, 33.3) or -0.5 %. The largest relative overestimation observed was between the waist and chest in SFDLA, and equaled 1.2 steps (LoA -4.3, 6.8) or 8.5 % of the total steps. The largest underestimation was observed between the thigh and waist in SPADES, and equaled -28.7 steps (LoA -107.1, 49.7) or -2.0 % of the total steps.

• Download figure
• Open in new tab

Figure 2.

Bland-Altman plots with comparison of step counts in three validation approaches: internal (a.), manual (b.), and wearable (c.). The horizontal axis indicates a mean step count from the two body locations (a.), estimated steps and manually counted ground truth (b.), and estimated steps and step counts obtained from Fitbit (c.). The vertical axis indicates a difference between step counts from the two methods. Blue solid lines indicate mean bias while dashed red lines indicate ±95% limits of agreement calculated as ±1.96 of standard deviations of the differences between the two methods.

Discussion

The limitations of wearable devices, particularly commercial activity trackers, have motivated public health researchers to seek alternative means to quantify human activities. While commercial wearable activity trackers are limited by proprietary summaries of the data, often provided at a daily or hourly level, smartphones offer an opportunity to access raw sub-second level data which can be used to generate reproducible and interpretable digital biomarkers, such as step counts, with research software. Moreover, smartphones are typically carried close to the center of the body, which produces less noisy inertial sensor data compared with the locations that many available wearables are worn (i.e., on the wrist), which provides more accurate estimations of whole-body motion (18,19,41–44).

In this paper, we validated the performance of step counting using a previously published open-source walking recognition method for body-worn devices that contain an accelerometer (29). This method leverages the observation that regardless of sensor location on the body, during walking activity the predominant component of the accelerometer signal transformed to the frequency domain, i.e., step frequency, remains the same, enabling the calculation of the number of steps a person performed in a given time fragment. In our previous study, we validated this approach for walking recognition using data from 1240 subjects gathered in 20 publicly available datasets, and demonstrated that our method estimates walking periods with high sensitivity and specificity: the average sensitivity ranged between 0.92 and 0.97 across various body locations, and the average specificity was largely above 0.95 for common daily activities (household activities, using motorized transportation, cycling, running, desk work, sedentary periods, eating, drinking). Importantly, the method’s performance was not sensitive to different demographics and metrological factors for individual subjects or studies, including participants’ ages, sexes, heights, weights, BMIs, sensor body locations, and measurement environments.

In this study, we further extend this work by validating the performance of step counting method using data collected from 255 subjects in eight independent studies with three goals in mind: (1) to assess the concordance of step counts across various body locations, (2) to compare the method’s estimates with observed step counts, and (3) to compare the method’s estimates with indications of commercial activity tracker (Fitbit Charge 2). The first comparison, an internal validation, demonstrated very high agreement between step counts measured from smartphones locate at most of the places where smartphones are typical worn, i.e., thigh, waist, chest, and arm. This result suggests that our method can be used to assess steps without restricting where participants wear their smartphones, which may reduce participant burden during data collection and help improve long-term study adherence.

Our manual validation on uninterrupted walking revealed almost perfect agreement between the step counts estimated with our method and those denoted by a visual observer. In this case, the absolute difference observed between the two measures was consistently below 1 % (Supplementary Figure 1g), which is similar to the results achieved with deep learning methods validated on this dataset in the past (45,46). These results reinforce the utility of using this method in controlled conditions, e.g., to evaluate participants’ functional capacity using a 6-minute walk test, and indicates that the method provides highly accurate estimation of step counts across various sensor locations during regular flat walking.

The mean step counting bias was also low for semi-controlled walking tasks recorded in PedEval, free-living tasks recorded in WalkRec, and for both scenarios within the wearable validation (first day, first day with at least 1000 steps). In these instances, however, the analysis revealed wider limits of agreement, which may result from a more complex structure of the underlying data which involved walking only few steps at a time as well as sudden changes in walking direction and altitude (e.g., stair climbing) (47). As discussed previously (29), in walking signals with such characteristics the step frequency tends to be modulated by it sub- and higher harmonics which might be identified as dominant in the wavelet decomposition outcome, and mislead our method.

Even more challenging data were analyzed in the wearable validation cohort. Here, the data were collected at unspecified locations (including novel locations, e.g., a bag or backpack), and included data representing various activities of daily living, such as grocery shopping, riding in a car, and doing dishes which might artificially inflate the estimated step counts by either method. This is a likely reason why the comparisons had wider discrepancies, even after removing minutes with 0 steps recorded by either device. Nevertheless, the estimated bias remained low, which indicates that our validated method provides reliable step count estimates across populations and conditions.

Our analysis has several limitations that should be addressed in future studies. First, due to the lack of available datasets, our method was not validated in individuals with walking impairments or those requiring walking aids, such as cane or walkers. Similarly, this method has not been validated in children and many elders, although the mean age of participants in our wearable validation set was 61.5 years and 10% were above 74 years of age.

Further research is needed to understand the frequency-domain gait characteristics in the presence of limping, as well as the potential overlap between the step frequency of walking activity in children and that of running activity in adults (48,49). The latter might be particularly important in studies that differentiate steps performed during leisure and exertional activities. Second, the wearable validation was performed with the use of a proprietary activity tracker (Fitbit Charge 2). Although this device has demonstrated reliable step counts during naturalistic gait performed in laboratory conditions (50,51), its accuracy in free-living conditions is inconclusive, and it is presumably dependent on the characteristics of the studied population (20,52,53). Importantly, the selected activity tracker was placed on the wrist, a body location that can be activated by many repetitive movements (e.g., gesticulating) while the rest of the body is still, hence it is more likely to overestimate steps compared to locations closer to the body mass center. To improve comparisons with our method, in wearable validation we removed data instances when either method indicated 0 steps. Finally, the estimation of step counts in free-living studies must account for non-wear time of smartphones (e.g., while the phone is charging or sitting on a table). Unlike many wearables that are attached to the body (e.g., wristbands), smartphones can be easily set aside, sometimes for prolonged periods of time. Such situations introduce considerable discrepancy between the estimated and actual number of steps a person performs during the day and should be reported, ideally with confidence intervals.

In conclusion, we performed a three-way validation of a robust, reproducible, and scalable method for step counting using smartphones and other wearable activity trackers. This validation demonstrates that our approach provides reliable step counts across sensor locations and populations, including healthy adults and those with incurable cancers. The method performed well in multiple environments, including indoors, outdoors, and in day-to-day life across settings. This method is a promising strategy for studying human gait with personal smartphones which does not require active patient participation or the introduction of new devices.

Conflicts of Interest

The authors declare no competing financial or non-financial interests.