Abstract
Stroke is a leading cause of disability worldwide. Following a stroke, high doses of intensive motor rehabilitation provide maximal benefit. Barriers to realizing recommended levels of rehabilitation therapy include cost, physical access, and clinical resource availability. Virtual Reality (VR) offers a potential solution to overcome these challenges and enable home-based, self-directed therapy. Here we present a low-cost system utilizing the off-the-shelf Meta Quest 2 headset running custom software to deliver immersive rehabilitation exercises. The system employs movement amplification to partially reduce motor deficits and enable more engaging task completion. It leverages recent advances in VR technology, including inside-out hand tracking, voice recognition, and an AI-driven virtual coach, to create a more accessible environment for users with no prior VR experience. We present the development of the virtual rehabilitation implementation and an initial feasibility assessment to determine if motor amplification increases the visually induced motion sickness experienced by the user. We find that motor amplification does not result in additional reported motion sickness suggesting that low-cost VR systems may serve as a feasible option to increase individualized and guided post-stroke motor rehabilitation.
I. Introduction
Stroke is a leading cause of disability worldwide, especially with recent improvements to acute treatments that have increased survival rates. In the past 30 years there has been a substantial increase in stroke incidence and disability-adjusted life-years lost, especially in lower and lower-middle-income countries [1]. Motor, sensory, and cognitive deficits contribute to a loss of function that makes daily life challenging, often requiring intensive and expensive assistive care. The motor disability following a stroke is a reinforcing cycle, the more difficult it is to use an affected limb, the less it will be used, and the more quickly remaining function will be lost. Overcoming this process of learned non-use, particularly in the approximately 70% of stroke patients with upper limb motor deficits [2], is a key focus of chronic stroke rehabilitation.
Higher doses of therapy following a stroke – including physical therapy (PT), occupational therapy (OT), and speech and language therapy – lead to better recovery outcomes [3], [4], [5]. Using and moving the limb affected by the stroke is critical to leveraging the brain’s natural neuroplasticity in regaining lost motor and sensory function, and ultimately improving daily life functionality. Many effective treatments capitalize on this principle including constraint-induced movement therapy [6] and emerging robot-assisted systems [7]. Therapy soon after the stroke seems to be the most effective; most motor recovery occurs in the first month following a stroke with nearly all recovery complete within 3 months [2], [8], [9], [10], [11]. However, recent efforts have sought to leverage the inherent neuroplasticity in the adult brain to continue functional recovery in chronic stroke patients beyond this initial critical window [12].
There are numerous impediments to providing high volumes of motor rehabilitation therapy soon after a stroke, which means patients often receive less care than they should. Access to treatment can be limited by cost. A typical constraint-induced stroke rehabilitation protocol spans 10 to 15 days with up to 6 hours of therapy per day [13]. In a public health care setting in Canada in 2011, a 15 session CIMT protocol was estimated to cost $1,857 CAD per patient, compared to $265 CAD per patient for current standard care to treat post-stroke arm dysfunction [14] ($2,489 vs. $355 adjusted to 2024 CAD [15]). Private practitioner-led CIMT can cost significantly more [e.g. $5,000 to $10,000 AUD] and usually qualifies for only limited private health insurance reimbursement [16]. Other emerging rehabilitation technologies such as robot-assisted therapy also require high costs and significant clinician oversight [7].
Physical access to stroke rehabilitation services is another barrier patients face. Long distances to clinics in areas with limited transportation options can be difficult to overcome, especially for patients that often suffer from limited mobility. Barriers to travel to seek rehabilitation services are exacerbated in rural areas [17].
Some of these barriers to accessing treatment can be reduced by prescribing home exercises which have been shown effective in reducing disability, improving quality of life, and increasing patient self-efficacy [18]. However, at-home therapies face their own barriers to maximal effectiveness. Rates of exercise completion are low with about 60% of patients not fully adhering to at-home physiotherapy programs [18], [19], and even compliant patients incorrectly executing 10-20% of exercises after clinician instruction [18], [20]. One Canadian study found only 36% of patients demonstrated a high level of adherence to a home-based exercise program following stroke [21]. Patients often struggle to incorporate exercises into their daily lives [19]. Memory issues, especially in older adults, contribute to noncompliance [22]. Increased clinician-patient interaction improves compliance with home exercise programs [23] yet resource constraints and access barriers limit this option.
Technology-based telemedicine solutions have the potential to enhance clinician-patient interaction, improve compliance, and ultimately improve stroke rehabilitation outcomes. Telehealth stroke care has become increasingly popular, particularly in response to the COVID-19 pandemic [24], and provides the potential to increase therapeutic dosing and access for patients. Accumulating evidence suggests virtual stroke therapy is equivalent to in-person interventions in terms of effectiveness and cost [24], [25]. One study showed that patients are generally satisfied with telerehabilitation services when they are implemented by trained professionals and include some social interaction [26].
Existing virtual reality rehabilitation systems have been shown to effectively support more self-directed stroke rehabilitation [27], [28], [29]. Some rehabilitation games have demonstrated efficacy in increasing stroke patient sensitivity to proprioceptive feedback [30] and decreasing non-use through visual amplification [31]. Patients receiving VR rehabilitation therapy as a supplement [32] or instead of [33] conventional rehabilitation therapy have shown significant improvements on the Box and Block Test and Fugl-Meyer Upper Extremity Scale, respectively. Research has also shown that VR rehabilitation might be ideal when delivering intensive high dose upper-limb therapy to maintain prolonged engagement [7], [8]. These systems do have some limitations, however, including the need for external motion capture systems [27], [29] and constant clinician involvement for operation. These findings highlight the potential VR has to assist in stroke rehabilitation for both patients and therapists.
One key concern with virtual reality-based rehabilitation systems is the potential for induced motion sickness [34]. In the past, VR users have demonstrated sometimes debilitating motion sickness, an effect more pronounced in women [35]. The expectation is that emerging VR technologies may reduce visually-induced motion sickness (e.g. [36]) but this has not been systematically tested with the latest technology. A key step in determining the initial feasibility of a VR-based stroke rehabilitation system is to assess levels of motion sickness in its users, particularly when visual amplification is used.
Here we describe the development of a novel virtual reality stroke rehabilitation system that addresses some of the systemic and individual patient barriers to treatment that persist with existing systems, namely limited portability, high costs, and the need for a dedicated clinician operator. For an initial assessment of feasibility, we tested our novel rehabilitation program that operates on the affordable standalone Meta Quest 2 headset with healthy younger adults to determine if motor amplification leads to increased visually-induced motion sickness in users.
II. Technology Development
We developed a software program in which participants could engage in 12 gamified tasks, three of which were tested in this work. Tasks were designed to either emulate activities of daily living, such as pouring water into a glass, or serve as immersive games in which the hand becomes a controller in a classic game like Pong or Snake. Each task focused on one category of movement: 1) Gross reaching; 2) Reach and grasp; or 3) Precision Grasping.
A. Virtual reality system
The VR setup used a Meta Quest 2 VR Headset connected to a Windows PC via a USB tether cable. The software used to develop this program was Unity3D with the use of the Oculus Integration Kit for VR interactions, including simulating the user’s position in the virtual environment and capturing their inputs to drive actions like picking up an object. The headset can be used as a wireless standalone system to support eventual at home use.
B. Hand tracking and user input
The Meta Quest 2 headset utilizes four IR cameras built into the headset to facilitate positional tracking of the user in their environment, as well as to track hand positions for interactions in the program. This standalone headset enables the potential for future at home use with minimal setup, unlike typical VR headsets that require external IR cameras to be physically mounted in the environment around the user. Users can interact with the system by touching virtual buttons, manipulating objects and using voice commands detected with an onboard microphone.
C. Physics interactions
The virtual environment presented to participants simulates the physics of gravity and normal forces in the real-world using engines built into the Unity environment. Due to the lack of touch feedback with hand tracking, interactions with virtual objects rely on visual changes in VR. When the hand hovers over an object that it can interact with, the hand changes color from gray to yellow. When the first finger and thumb are pinched together while overlapping the object, the hand turns blue indicating that the object is being grasped. Now the object will move as the hand moves and can be moved to a different part of the virtual world. Users are given a demonstrative tutorial at the beginning of their VR experience to learn the nuances of interacting with a virtual environment.
D. Motor amplification
For users with limited range of motion, physical movements can be amplified in the virtual space, effectively increasing the range of motion in VR [31]. The motor amplification algorithm works similarly to how a computer mouse’s sensitivity can be adjusted on a desktop computer. A “rest” or center position is determined during calibration, and the position of the amplified hand relative to that center point is multiplied by an amplification factor. For example, a user sitting with their arm on a chair’s armrest would have their rest position at that point in the world. Lifting the arm up 20 cm from that point, with an amplification amount of 2, would mean that the virtual hand would appear in VR to be 40 cm above the starting rest position. We accomplish this by taking the amplification factor and multiplying it by the position vector (the x,y,z value) of the amplification target. In the Unity3D environment, the x is the horizontal axis relative to the subject, the y is the vertical axis, and z is the depth (or the axis that extends directly out in front of the user).
The amplification factor is determined through a software-guided calibration process which measures the range of motion for both of the upper arm extremities. The limb with more limited range of motion is selected to be amplified to match the side with greater range of motion, and can be selected as the target hand to be used during virtual rehabilitation exercises.
In addition to the positional range of motion, the movement extent for wrist rotation and grasping can also be amplified. The grasping and wrist rotation amplification were not enabled during this study due to the tested tasks not employing those types of movements.
E. Motor smoothing
Noise in the IR camera tracking system due to factors like lighting condition and the system’s confidence in localizing hand positions can cause jitter in a visually amplified hand. In typical unamplified movements this is not an issue as the positional noise is nearly imperceptible, but when amplified it becomes apparent. To resolve this issue, the program uses a spring-damper function to adjust the hand’s position proportional to the amount of amplification applied (i.e., the more a hand is amplified, the stronger the smoothing function becomes).
III. Assessment of Visually-induced motion Sickness
After the initial development of the VR software, we tested a sample stroke VR rehabilitation session with healthy younger adult participants. All study protocols were conducted with the approval and oversight of the Acadia University Research Ethics Board (File #22-07). Sixty participants were recruited from the Acadia University student population using an online study recruitment platform. Students were enrolled in psychology courses and received course credit for their participation. Demographic information was collected from each participant including gender identity [M/F/other], handedness [L/R/other], and age.
A. Study setup
A Meta Quest 2 VR headset connected to a desktop computer, as described in Section II.A., was used to implement the VR exercises (Fig. 1(a)). The experimenter could watch the participant’s view of the virtual environment on a separate screen and provide assistance when needed. The VR exercises lasted approximately 10-15 minutes, which included an initial range of motion calibration. Participants completed a motion sickness survey immediately following their VR session. Other data were collected to generate a normative motor performance dataset and will be reported elsewhere. Motor amplification of hand position was applied to half of the participants; the other half served as a baseline control with no amplification.
REVIVE System overview. (a) User operating the system. (b) Screenshot of the Bubble Pop game. (c) Screenshot of the Pong Paddle game. (d) Screenshot of the Animal Feeding game.
B. VR tasks
All participants first completed a Calibration phase, after which amplification was enabled for the experimental group and then all participants played the Bubble Pop game (Fig. 1(b)) (see below for detailed descriptions of each task). Participants then completed two more tasks, a Pong Paddle game (Fig. 1(c)) and an Animal Feeding game (Fig. 1(d)). The order of the last two tasks was counterbalanced: half of the participants within each group completed the Pong Paddle game first, the other half completed the Animal Feeding task first. Here is a description of each task in the VR session:
Calibration
During this phase, participants reach towards different locations in the room indicated by colored cubes. The software logs the minimum and maximum extent of points reached by the left and right hands to calculate the range of motion for each. In the Amplification group, the hand with the smaller movement extent is selected to be amplified by the proportional difference between the sides. The range of motion is also used to set the boundary area for object interaction in the tasks that follow.
Bubble Pop Task
The user pops as many bubbles that appear around them within one minute. The bubbles appear one at a time and only appear within the participant’s field of view within the headset. During this period the user’s hand positions are recorded every 50th of a second to map their movements in the space.
Pong Paddle Game
While seated at a table, the user controls a paddle by moving their hand forward or backward. The hand with the lower range of motion is assigned to control the paddle, with this hand amplified in the Amplification condition. A small ball bounces across the board which is oriented from left to right. On the side opposite the user’s paddle is a paddle controlled by the computer. A point is earned if the ball passes by the paddle of the opposing player. Participants practice the task for one minute and then complete a two-minute game.
Animal Feeding Task
The participant is seated at the end of a long table with an animal visualized at the far end of the table. The participant pushes forward one of three cans sitting on the table until it crosses a white line to “feed” the animal, with the line’s distance calibrated to the user’s range of motion. Each can is labeled as a food for one of three animals (a dog, cat, and bird) and users must push the can that matches the visualized animal. Once the can has crossed the line, the animal will change its appearance into one of the other available options and play a sound clip (a bark for a dog, a meow for a cat, and a tweet for a bird). Participants practice the task for one minute and then complete a two-minute game.
C. Visually induced motion sickness survey
Following the VR exercises, an abbreviated motion sickness symptom assessment was administered consisting of 5 of 9 items from the Virtual Reality Sickness Questionnaire (VRSQ) [37]: General Discomfort, Fatigue, Eyestrain, Difficulty Focusing, Headache. The mean score across the five items was used to indicate visually induced motion sickness immediately following the VR experience.
D. Analysis plan
To determine if motion sickness levels exceeded minimal levels following the VR experience, a Wilcoxon signed rank test was conducted (on the medians) with the alternative hypothesis that the true location is greater than 1 (median > 1). Wilcoxon rank sum tests were run for comparisons between groups for reported motion sickness levels. Medians (and thus the nonparametric Wilcoxon tests) were selected due to non-normally distributed zero-inflated data. Analysis scripts in the R programming language along with all data are available at OSF [https://osf.io/45kad].
IV. Results
Sixty healthy young adults completed VR rehabilitation exercises, with or without motor amplification. 47 participants identified as women, 10 as men and 3 as non-binary. Participants ranged in age from 18 to 39 years old (mean age = 19.7 years, SD = 3.22 years). Three self-reported as left-handed.
Across all participants in both conditions there was a mean motion sickness score of 0.57 immediately following the VR experience (Fig. 2) (SD = 0.44, median = 0.4). Note that three participants did not complete all of the questions on the motion sickness survey and their data have been omitted from these results. As a motion sickness score below 1 indicates a minor effect (average response below “slightly” in response to individual symptoms), a Wilcoxon signed rank test with continuity correction rejected the alternative hypothesis that u > 1 (V = 95.5, p = 1).
Average motion sickness scores for all participants following the VR session. Dots represent individual participant scores, the box plot indicates the group median.
Amplified group participants, with an average of 4.4% amplification applied, showed slightly higher motion sickness scores (mean = 0.64, median = 0.6) compared to control participants (mean = 0.50, median = 0.4) but a Wilcoxon rank sum test with continuity correction indicated that this was not a statistically significant difference (W = 326, p = .20) (Fig. 3).
Average motion sickness scores by amplification status during the VR session. Dots represent individual participant scores, the box plots indicate the group medians.
V. Discussion
We developed a virtual reality stroke rehabilitation platform using a low-cost commercially available headset, the Meta Quest 2. In the healthy younger adults tested in this work, we observed low levels of reported motion sickness, though slightly higher motion sickness levels were observed in the motor amplification group. Although this was not a statistically significant difference, it could indicate potential issues with the greater amplification factors required in clinical populations. Careful study with participants with reduced ROM requiring increased amplification is necessary to further investigate this.
Seven out of 60 participants did present average motion sickness scores above 1 suggesting that there is room for improvement in the VR system. However, no baseline motion sickness data were collected so it is possible that other factors could have contributed to the feelings of motion sickness data reported. Nevertheless, more recently released low-cost headsets with improved graphics and hand tracking, like the Meta Quest 3, may organically reduce experienced motion sickness. Lessons learned from this study have led to subsequent software improvements including a more standardized testing environment, improved virtual coach instructions and demonstrations, and more stringent task constraints to reduce head movements.
One key benefit of the system presented is its accessibility, which increases its potential for widespread implementation. By using a commercially available low-cost head mounted display (HMD), costs are reduced, thus increasing its availability. Aided in part by a virtual coach guiding participants through the exercises, the system requires limited clinician oversight making it suitable for broad use. User-friendly interaction options make the technology more approachable for novice users, including hand tracking (i.e. the patients do not hold anything) and multiple interaction options (e.g. an option to speak commands to the virtual coach). Assistive motor amplification in gamified tasks enables more consistent task completion and more positive task engagement, supporting those with severe motor deficits.
In this work, the limited motion sickness reported indicates the potential for the use of low-cost off-the-shelf HMDs to deliver stroke rehabilitation programs. We are currently testing the system’s feasibility in both healthy and stroke survivor samples of younger and older adults. With these additional results, we will continue exploring the platform’s potential to provide widespread access to the intensive rehabilitation interventions that are most effective in driving motor recovery following stroke.
Acknowledgment
We thank Sam Churchill and Ansaar Khadaroo for assistance with data collection. We thank Saif Miskin for assistance with figure graphics.
Footnotes
aidanfiskvr{at}gmail.com
summerfox{at}acadiau.ca
jennadawnfloyd{at}gmail.com
This work was supported by ResearchNS under the New Health Investigator Grant program.
It has been restructured to focus on initial feasibility testing of the stroke VR system. The normative dataset component has been removed and will appear in a different manuscript. We have removed the image showing a person.