Navigation

Contents

• Technology Enhanced Stroke Rehabilitation
  • Virtual Reality and Games for Stroke Rehabilitation
  • Match between cognitive and physical effort
  • Virtual Reality Mirror Therapy Stroke Rehabilitation
    • Review of Previous VRMT Research
  • Summary

Technology Enhanced Stroke Rehabilitation

Technology has already been used in stroke rehabilitation. Specifically, VR has been used, and other VRMT systems have been used since 2011. To date, no other VRMT system has implemented the 6 DoF for VRMT as done by MINERVA. This chapter includes specifications of current and future Head-Mounted Displays to give a historical context of the capabilities as this affects design.

Virtual Reality and Games for Stroke Rehabilitation

VR is a flexible technology that supports high-intensity, repetitive training, often found to be motivating, engaging, and enjoyable because VR systems, tailored to a learner’s individual needs introduce meaningful, challenging, and progressive exercises available in a variety of settings ^[115]. “As a therapist, it is these key areas that we need to be aware of when planning our interventions. As therapists, we can change the environment or the task in such a way as to enable our patients to achieve their goals.” ^[133]. As therapists are professionals who assist in rehabilitation process, their input is valuable over the sequence of development and there was communication with multiple professionals for their input on the subject. An important question to consider is if repetition and skill learning is essential for motor learning and function after stroke, and what does VR technology add over the real and above real-world task practice ^[56]. VR is a tool than can be used for upper limb rehabilitation by incorporating Motor Control and Motor Learning Principles ^[115] that learning experience can be specifically designed using motor control and motor learning principles. The adaptability of the computer environment enables the clinician to focus on certain motor control weaknesses and to give valuable feedback that promotes motor learning based on movement organisation principles ^[115]. Summarises the attributes (Table) of Virtual Reality that align with motor learning variables ^[56]. It is up to the system designer to maximise the advantages of which VR is capable. Knowledge of motor control and motor learning principles can be used for the game design.

Motor learning variable	Attribute
1. Observational learning	1.1. Users can view their own image interacting with virtual objects in the VE 1.2. Users can view an avatar mirroring their movements 1.3. Users can view a virtual teacher demonstrating optimal movement patterns 1.4. VEs can facilitate mental practice or motor imagery
2. Practice: amount, task specificity and meaning	2.1. Potential for abundant repetition of practice trials 2.2. Ecologically valid VEs enhance task specificity of practice 2.3. Train movements that are identical to those required in real-life tasks 2.4. Options to individualise to different challenge levels 2.5. Enriched environment 2.6. Goal-oriented tasks 2.7. Familiarity of commercially available VR gaming systems
3. Augmented Feedback	3.1. Precise and consistent 3.2. Auditory, visual, or tactile 3.3. Knowledge of performance 3.4. knowledge of results 3.5. Positive motivational feedback
4. Motivation	4.1 Novelty of VR technology 4.2 Gaming features 4.3 Feedback 4.4 Goal-oriented tasks 4.5 Capacity to individualise treatment options 4.6 Users can select tasks 4.7 Competition against other players

Match between cognitive and physical effort

An intervention using virtual reality and transcranial direct current stimulation (tDCS) to improve upper limb function in chronic stroke survivors with severe hemiparesis found that the use of TMS (TransCranial Magnetic Stimulation) for VR stroke rehab is promising according to a feasibility study [134] but there is not enough information to generalise the findings. Stroke rehabilitation already uses VR as an intervention choice ^{[134] [135] [136] [137] [138] [139] [140] [141]}

and has shown moderate grade beneficial evidence in outcomes for Upper Limb (UL) Function and UL impairment but lacks evidence for Activities of Daily Living (ADL) ^[69]. Laver et al. ^[142] state that “Virtual Reality therapy may not be more effective than conventional therapy, but there is lowquality evidence that Virtual Reality may be utilised to improve outcomes in the absence of other therapy interventions after stroke”. A reason for this is the broad diversity of Virtual Reality Stroke Rehabilitation interventions. A 2017 systematic review of Virtual Reality for stroke rehabilitation included 72 trials that involved 2,470 participants ^[7]. The review question considered was to compare the effects of Virtual Reality against an alternative treatment or no treatment on recovery following stroke, using arm function and other outcomes such as walking speed and independence in managing daily activities. Study sample sizes were generally small, and interventions varied in both the treatment goals and the Virtual Reality devices used. The risk of bias present in many studies was unclear due to poor reporting. Control groups usually received no intervention or therapy based on a standard-care approach. The primary outcome of the study was that results were not statistically significant for upper limb function when comparing Virtual Reality to conventional therapy ^[142]. In an ideal scenario, MINERVA would like to test on a large sample size for statistical significance, however this is an external factor and may be difficult considering the resources required. It is difficult to evaluate much of the Virtual Reality Stroke rehabilitation literature as history is very important as it is a reference to the technological capabilities of the period. VR HMDs have improved significantly over the last decade. The difference between the Oculus DK1 (2012) and Oculus CV1 (2016) is major. Explanation follows later in the document for the history/evolution of technology and its relevance for a technological solution for stroke rehabilitation. Since much of the research on VR systems is developed privately and is not commercially available, they are nonhomogeneous and have a wide variance in how they implement their design. The Canadian survey ^[143] was a large-scale assessment of VR/AVG usage and learning needs in Canada and mentions that, “Perspectives of both experienced and inexperienced users offer important new insights for effective support of VR/AVG use in rehabilitation.” ^[143]. This is useful to get an understanding from the therapist’s point of view, which is necessary as they will be the practitioners who will be using the tool and know-how implementation would work within a therapy session outside of the lab. “This first-hand knowledge of Canadian therapists’ support needs provides the basis for our ongoing research program to develop and to evaluate KT resources that support their needs and best practices in VR/AVG use.” ^[143] The systematic review and meta-analysis exploring virtual environments and commercial games in therapy by Lohse et al. ^[140] mentions that there is limited data to show a positive benefit of VR compared to CT. These results are constrained by the large degree of inter-study variability, but there were no discernible differences between the VE and CG therapy types, and there was no indication that the passage of time after a stroke diminished the effects of VR therapy. It is worth noting the date is 2014, and this is when the Oculus Rift DK2 was released in July of that year. Although the study is just over five years old, the DK2 VR HMD is already obsolete in comparison to today’s technology. Dos Santo et al. (2016) state that VR-based systems promote motor learning Movement Visualisation (MV) ^[144]. A systematic literature review and meta-analysis of upper-limb and cognitive outcomes (2018) states VR offers an opportunity to interact within a structured enriched environment which is scalable. Multi-sensory feedback to enhance skill acquisition. That VR has an advantage over conventional interventions ^[145]. In comparison to traditional rehabilitation literature, much of the VR stroke rehabilitation literature often have low sample sizes, large variance in the system set up and seems to be more focused on the technological aspect (Table).

Name	Year	Size
Combined Cognitive-Motor Rehabilitation in Virtual Reality Improves Motor Outcomes in Chronic Stroke – A Pilot Study [146]	2018	32
Post-stroke unilateral spatial neglect: Virtual reality-based navigation and detection tasks reveal lateralized and non-lateralized deficits in tasks of varying perceptual and cognitive demands [147]	2018	30
Virtual reality to augment robot-assisted gait training in non-ambulatory patients with a subacute stroke: A pilot randomized controlled trial [148]	2018	27
Virtually home: Feasibility study and pilot randomised controlled trial of a virtual reality intervention to support patient discharge after stroke [149]	2018	16
Patients’ and Health Professionals’ Experiences of Using Virtual Reality Technology for Upper Limb Training after Stroke: A Qualitative Sub-study [150]	2018	7
A Mixed Methods Small Pilot Study to Describe the Effects of Upper Limb Training Using a Virtual Reality Gaming System in People with Chronic Stroke [135]	2017	12
Motion Rehab AVE 3D: A VR-based exergame for post-stroke rehabilitation [151]	2017	10
Efficacy and safety of non-immersive virtual reality exercising in stroke rehabilitation (EVREST): a randomised, multicentre, single-blind, controlled trial [152]	2016	141
Is upper limb virtual reality training more intensive than conventional training for patients in the subacute phase after stroke? An analysis of treatment intensity and content [153]	2016	50
Virtual Reality Reflection Therapy Improves Balance and Gait in Patients with Chronic Stroke: Randomized Controlled Trials [96]	2016	30
Feasibility of an individually tailored virtual reality program for improving upper motor functions and activities of daily living in chronic stroke survivors: A case series [154]	2016	15
Self-directed arm therapy at home after stroke with a sensor-based virtual reality training system [155]	2016	11
Maximizing post-stroke upper limb rehabilitation using a novel telerehabilitation interactive virtual reality system in the patient’s home: Study protocol of a randomized clinical trial [156]	2015	52
Effects of game-based virtual reality on health-related quality of life in chronic stroke patients: A randomized, controlled study [157]	2015	35
The use of virtual reality-based therapy to augment poststroke upper limb recovery [158]	2015	1
Improvement in balance using a virtual reality-based stepping exercise: A randomized controlled trial involving individuals with chronic stroke [159]	2014	22
Development of virtual reality proprioceptive rehabilitation system for stroke patients [160]	2014	10
Virtual reality for the rehabilitation of the upper limb motor function after stroke: a prospective controlled trial [161]	2013	376

At Ulster University, there has been recent research involving Virtual Reality stroke rehabilitation ^[124], ^{[162] [163] [164] [165] [166]} much of which is concerned with designing and creating engaging rehabilitation games so that patients remain motivated to adhere to the rehabilitation process. It focuses on a system that is useradaptive so that the task is presented in such a way that the patient is appropriately challenged rather than being annoyed or bored if the difficulty fluctuates too much. Oddly, there is no mention of discipline in their Capability Opportunity Motivation – Behaviour (COM-B) model, while self-discipline is an important element to self-improvement according to James Clear ^[119] . Ulster University and external partners were recently in Phase 3 trials of the MAGIC project ^[167]. Part of the system discussed in this paper helped the MAGIC project progress through Phase 2. The study ^[167] had 121 participants with a dropout rate of 33. Out of the 33 who dropped out, these were caused by: Non-stimulating (n=18), Technical difficulties (n=6), Arm pain (n=2), Other health issues (n=4), and change in circumstance (n=3). Since the total number of dropouts was 33, out of those 33 drop outs 6 of them were due to “Technical difficulties” and 18 of them was due to “Non-Stimulating”, Adding these two reasons accounts for 24 out of the 33 drop outs (72%). This mismatch in expectations raises concerns. “Challenges to date, recognising hand movements accurately” ^[167]. Literature mentions the problems with the Leap Controller, “Due to the highly unstable nature of forearm pronation/supination data, we could not recommend any accurate ranges of motion for this movement.” ^[168].

Virtual Reality Mirror Therapy Stroke Rehabilitation

The invention of a mirror box used for phantom limb pain was created in 1996 ^[4]. There is a difference in the design goals of a VRMT and a non-VRMT stroke rehabilitation system. Non-VRMT (VR stroke rehabilitation) focuses more on increasing engagement by making the exercises fun through VR. (Table 13). VRMT, on the other hand, must prioritise presence to make sure the illusion is maintained for rehabilitation to work as indicated by the Mirror Therapy Protocol ^[35]. Although there is no dichotomy between fun and presence, it is worth noting the different design goals. These aspects dramatically change the selection of which exercises and execution work best, and that is why much of non-VRMT literature may not translate well.

Table 13: Virtual Reality Mirror Therapy Rehabilitation findings. Source Participants Intervention Comparator Primary Sessions outcome Blinding Main Findings Regenbrecht et al., 2011 24 Healthy Augmented Mirror Optical Mirror Box (AMB) Box 1 No The mirror box technique is able to fool or No Blinding confuse individual’s perceptions and beliefs. The AMB produced strong results in this regard. Hoermann et al., 2012 21 Healthy Video-mediated (advanced) augmented reflection Optical Mirror Box 1 No Video-mediated manipulations of hand-position No Blinding reversals produced equal to stronger effects of ownership compared with the mirror reflection Kang et al., 2012 18 Healthy and 18 hemiplegic patients Virtual Mirror Therapy Relaxtion or Real Mirror 1 No Corticospinal excitability was facilitated to a No Blinding greater extent in the virtual mirror paradigm than in the real mirror. 12 Stroke patients with pusher syndrome ComputerGernerated feedback training No The computer-generated visual feedback Simple training more effectively aided recovery from Blinding pusher syndrome and balance (but no significant (assessors) difference was noted between groups for lower No Simple Applying VRRT might be even more beneficial Blinding than conventional rehabilitation program alone (assessors) in improving affected lower limb function 3 Times a week during 3 weeks 5 times a VRRT (Virtual 25 patients with Standard Mirror week In et al., 2016 Reality Reflection chronic stroke Therapy during 4 Therapy) weeks Yang et al., 2014 Mirror Visual Feedback Training

A 2018 review by Darbois et al. ^[169] states 2nd generation MT is of very low quality. Evidence based rationale is missing, and they do not recommend investment by professionals or institutions as it does not seem reasonable to develop new devices given the cost, time and resources required to assess existing devices with well conducted trials ^[169]. Given the circumstances of the time, this judgement seems reasonable. However, because VR is a disruptive technology that is rapidly gaining traction in society, it is anticipated that while the technical specifications of the systems will eventually become outdated, the basic design principles will not. Performance should improve while costs should also decrease. When comparing identical HMDs by year, this trend is already apparent. Current VRMT solutions are limited in their design; all they seem to do is attempt to recreate traditional MT; a poor application of VRMT, which has the potential to do much more. MT and VRMT have their differences and accepting this will allow each tool to function as intended. MINERVA has potential unique advantages over current systems, with the hope that any future system design extension follows a similar format in the fundamental design by potentially using the 6DoF and asymmetric mirroring mode. In this way, the realisation of the best solutions offered to maximise VRMT potential can occur. Table 14 is a comparison of traditional MT and Augmented/Virtual Reality Applications for the treatment of phantom limb pain.

Table 14: Comparison of traditional Mirror Therapy and Augmented/Virtual Reality Applications for the treatment of phantom limb pain. Feature Traditional Mirror Therapy AR/VR applications Unilateral upper and lower limb Uni- and bilateral upper and lower limb amputees amputees No Achievable Visual Realism High Moderate to high Latency of visual feedback None Adaptability of the visual image to Limited Achievable No Achievable High Low to moderate Low Moderate to high Costs Low Moderate to high Adverse events Occasionally Occasionally Patient group Initiation of motion by the residual limb distorted phantom limb Elements of gamification Degree of self-delivery by patients Technical or material requirements In traditional MT, a patient is encouraged to imagine the reflection of their arm is real. In this way, it is natural to consider the application of Mirror Therapy within VR, and compared to a traditional mirror box, a virtual mirror is not bound by the limitations of the real world and that of the patient imagination. It is important to understand what the original mirror box was trying to achieve rather than just copying the solution without knowing the reason for the inclusion of certain design aspects. Original VR box ^[4]:

Our ‘virtual reality box’ was constructed by simply placing a 2” by 2” mirror vertically inside the middle of a cardboard box, so that it was perpendicular to the patient’s chest and its upper end was almost touching the chin. The top and face of the box were removed to afford the patient full view of the reflection of his normal hand in the mirror. For patients with a shoulder level disarticulation, a much taller mirror was used.

Note the position perpendicular to the patient’s chest, and this is the origin of MINERVA’s sagittal mirroring plane. The mirror plane is supposed to be bound to the patient, and the patient should not have to accommodate themselves to the plane. The mirror box considers patient parameters such as height, i.e., if the patient is tall, a larger mirror is required, as it is needed to occlude the other side for the illusion to work well. The usage of “Virtual Reality” is in the original document, which is fitting as the reflections will only show the interior of the box, this virtual space removes other distractions and other objects that could be in view if the box was not there. As such, it helps maintain the illusion that this virtual area belongs to the mirrored limb and is not just a reflection. From a design point of view, current mirror systems developed focus on creating the sagittal plane from the computer system like an arcade cabinet; later chapters discuss this limitation. The reason for the mirror box’s original design lies in its bounding by reality, which is not the case for VRMT. Developing a digital version of the mirror box does not maximise the potential. Several efforts have combined both MT and VR into an intervention system. The idea of VRMT is still relatively novel, and the few existing or previous implementations vary greatly. A study of a VRMT system by Diers et al. ^[171] using HMD reported that VR showed similar effects to the classical mirror box. VR and MT operate based on illusion, the idea of combining both seems a logical fit. In comparison to traditional MT there are very few VR MT studies ^[95]. In principle, Mirror Therapy provides a convincing optical illusion that can be enhanced further with other types of feedback expected to be perceived similar to the embodiment, felt in the previously mentioned rubber hand illusion experiments ^[172]. Virtual reality can create a rich environment and may be possible to provide greater enhanced feedback than Mirror Therapy if it is appropriate to the motor learning goal and implemented correctly ^[115]. Although already used for stroke rehabilitation VRMT is not common, and its implementation differs from the system under development here as they do not use 6DoF or the asymmetric mirroring mode. SchusterAmft et al. ^[173] reported in a study of a VRMT system using a computer screen as the visual output, VR was found to be both feasible and safe. Both patients experienced intense VR training, and showed improvements after a 4 week intervention with 19 sessions. Each session lasted 20 minutes and had 300400 grasping and opening movements for each hand. Certain existing studies report themselves as VR even though they use a traditional 2D screen rather than VR displays which can cause confusion^[174]. VRMT implementation can vary. Patients may participate by viewing themselves on a screen via a webcam, or perhaps by observing themselves performing tasks from a third-person perspective ^[175]. Another study by Schuster-Amft et al. reported using tracked data gloves, with patients seeing their hands on the 2D screen from a first-person perspective ^[173]. Regenbrecht et al. ^[94] view a 2D screen and camera-captured images of the hand are flipped and then displayed on the screen. In contrast Diers et al.’s study ^[171], the user used a 3D VR HMD lying down in an fMRI and concluded that similar effects were observed on the brain using a mirror box compared to VR and proved its suitability for use with stroke patients. The advantages of VRMT are the potential increased sense of embodiment and presence and the possibility to simplify the environment and block out the complexity of the real world. Additionally, the capacity to gamify the environment, the ability to offer rich virtual coaching and feedback, the ability to change how Mirror Therapy is applied, and the ability to provide extra clues to strengthen the Mirror Therapy “trick.”. The original Mirror Therapy box was also known as a Virtual Reality box because of the illusion created. However, Virtual Reality today refers to the technology which allows the illusion of immersion in new ‘real’ environments where much more than ‘mirroring’ can take place. The ‘trick’ to the brain is still the same wherein it believes the injured limb is performing the task and generates messages in that direction, thereby assisting repair ^[4]. Determining the success of this system compared to other interventions is complicated as many studies lack standardisations of results for a fair comparison. They may have sought a similar goal, but their approach has been different, or the specifics are too limited to draw sound conclusions from the results. Also, they may have been designed for a different part of the body, which in principle may compare relatively well, but the techniques and tasks will have differed. Not only does VRMT intend to assist with improving the performance of the impaired limb by providing appropriate rehabilitation tasks that are repeatable, it also addresses the aspect of patient engagement to overcome problems regarding motivation or fatigue for continued attention. Other interventions, such as those provided by traditional therapies, may well benefit from VRMT as an addition to augment the work of therapists, to be regarded as an effective tool while developed with their input.

Review of Previous VRMT Research

Systems from 2011 to 2019 are discussed in chronological sequence to help readers better grasp how the various VRMT systems differ from one another. 2011: Beyond the looking glass: Fooling the brain with the augmented mirror box ^[94].The system seems to be exploring a digital version of MT, using a camera where the user’s real hand is viewed and a computer- generated one is not required. Having the screen high up means the user is not viewing their hand or arm movements. The small play area and being limited to a box restricts the options for games. 2012: A Novel Virtual Reality System Integrating Online Self-Face Viewing and Mirror Visual Feedback for Stroke Rehabilitation: Rationale and Feasibility ^[175]. The usage of a webcam is an intriguing concept, but it’s unclear whether it will work because the avatar on the screen is making a gross motion that would require different sets of muscles to move the upper arm, while the impaired limb is using the controller (hand/fine motor muscle). It is intriguing to look at oneself from the third person. The illusion’s success depends on how the user sees their disabled arm doing the action. The system ^[176]is very similar to the developed system from 2011 ^[94]. These kinds of setups seem to be more orientated toward making a digital version of Mirror Therapy. The system ^[177] uses VR, and hand representation. Haptic feedback for moving the cup. For greater presence, the cup should be the same dimensions as the cup in the Virtual Environment. It would be interesting to see tolerance levels on the path suggested by the system and why artificial forces will restrict manipulating the object. To improve this, using a sensor that can freely slide on the worktop may be more natural. 2014: Mirror therapy system based virtual reality for chronic pain in-home use. ^[174]. 2015: The system ^[171] seems to be attempting to recreate reality in the virtual environment. Gloves are worn for precise movement during intensive virtual reality-based training aimed at improving upper limb motor function in chronic stroke patients^[173]. Similar issues to ^[174]. This system had three different interesting games; a couple seem appropriate while the other did not seem to translate well to motor learning principles. There is no image of the set-up for the study by the Author ^[178]. 2016: System ^[96].appears to be a lower limb digital adaptation of MT. A 2013 DK1 HMD headset was utilised in 2016 research^[179]. which measured presence using EEG. Leap control is being utilised, although it is mounted on the table rather than the head-mounted display. “Data suggests that both VR and particularly MP can enhance the activation of brain patterns present during overt motorexecution. Further, we show changes in the interhemispheric EEG balance, which might play an important role in the promotion of neural activation and neuroplastic changes in stroke patients in a motor-imagery neurofeedback paradigm. In addition, electrophysiological correlates of psychophysiological responses provide us with valuable information about the motor and affective state of the user that has the potential to be used to predict MI-BCI training outcome based on user’s profile. Finally, we propose a BCI paradigm in VR, which gives the possibility of motor priming for patients with a low level of motor control.” 2017: System: Computerised mirror therapy with Augmented Reflection Technology for early stroke rehabilitation: clinical feasibility and integration as an adjunct therapy ^[180] is similar to the digital augmented mirror box solutions. Having the screen in front of the hand doing the movement allows the hand to remain hidden from view, so it will not disturb the illusion. However, having a screen that close may be uncomfortable or difficult to use. A suggested improvement would have to use a larger screen but slightly further away which also has drawbacks too, as it can interfere with the user’s hand movements. If the screen is tilted more horizontally the user may have to look down at the screen which could be an uncomfortable position that might cause neck pain. 2019: Magic Glass used mirror therapy, as does MINERVA, though the implementations are different as it addresses other design goals as the system is a seated 3DoF world fixed, using a Leap Controller mounted on the Oculus Rift CV1 HMD while MINERVA was 6DoF and was designed around hand held controllers and additional trackers. For the catapult game, the user makes a pinch motion to control the two parameters of the game. The same pinch motion was used in a game where a ball was thrown using this gesture which is unnatural as your hands open when throwing. Considering the system is using fine motor movements, it is an odd design choice that Leap Motion was selected, as the technology struggles with certain hand positions. If system use demands the Leap Controller, it requires a work-surface-mount, not a HMD mount (see study: ^[179]). A sitting user’s hand is detected by the game when it is in range. More detail on this issue which can break presence will be discussed later in the document. For fine motor control, a different technology such as a data glove or Valve Index might be more appropriate. It appears the system did not implement the 6DoF mirroring of MINERVA, as it lacks additional sensors and uses a world-fixed 3DoF system.

• n= 121 participants
• Drop out (n=33) 27%
  • Non-stimulating (n=18)
  • Technical difficulties (n=6)
  • Arm pain (n=2)
  • Other health issues (n=4)
  • Change in circumstance (n=3)

54% (n=18, out of 33) of dropouts are attributable to lack of stimulation. The presentation ^[167] states one of the challenges was “recognising hand movements”, yet their published articles made no mention of problems with the Leap Controller ^[124] despite available literature stating Leap Motion Controller’s limitations ^[123], ^[168], ^[182], ^[183]. System: Immersive Virtual Reality Mirror Therapy for Upper Limb Recovery after Stroke ^[184] is very similar to the UU MAGIC Glass, being seated, 3DoF and using the same VR HMD; however, the key difference is in the input controller which uses the Oculus tracked controllers instead of the Leap Motion controller. This simple change makes the system stable. The use of a coach is an extremely useful implementation as the user can visually try and copy what is happening, and this makes it much simpler to explain and accept guidance through the exercise. Grabbing objects in-game used a trigger squeeze. “Segment I: Exercise (5 mins) In this treatment block, patients completed a basic range of motion exercises demonstrated and described by a human avatar. Exercises included shoulder flexion/extension, abduction/adduction, elbow flexion/extension, forearm pronation/supination, wrist flexion/extension, grasp/release, and composite motions such as punching. Patients completed each exercise for ten repetitions before moving onto the next exercise. Segment II: Rock Stacking (5 mins) This treatment block asked patients to pick up rocks of various shapes and sizes positioned on a virtual tabletop, and to stack as many rocks as possible without knocking any over. Segment III: Functional Task (5 mins) In this treatment block, patients engaged in various functional tasks in the context of a dining room. Subjects were asked to stack plates, set up a tea set, right upturned objects such as a wine bottle or goblet, move pieces of fruit from one plate to another, and unset and reset a table. Patients completed tasks in the same sequence each time, moving onto the next after successfully demonstrating the previous task.” This treatment paradigm could be compared with virtual mirror therapy that shows both arms moving symmetrically to determine relative efficacy. Other distortions with potential therapeutic value could include a non-mirrored therapy providing a representation of the paretic arm with smaller movements than are actually performed to explore whether this might encourage the patient to increase their active arc of movement to accomplish an assigned task. With respect to mirror therapy itself, VR allows us to convert simple movements into functional tasks, perhaps providing more meaningful, engaging treatment for patients. Functional tasks also provide additional opportunities for cognitive and perceptual training and may generalize more readily to real-life environments. This implementation of mirror therapy could allow patients with injuries or movement precautions to engage in some form of therapy without risking injury. Although this form of mirror therapy is more expensive than conventional mirror box treatment, it does represent a more accessible and self-directed alternative to existing rehabilitation technologies. Stand-alone VR headsets are now commercially available for as low as US $200 (e.g., Oculus Go), and wider availability and adoption of lower-cost consumer versions of these devices are likely in the near future. ^[184] In summary, immersive VR mirror therapy appears safe and feasible in individuals with chronic hemiparesis after stroke. Further studies are needed to determine optimal parameters for training and to determine efficacy ^[184]. A system made by Sulimanov ^[185] uses the Leap Controller and CV1, just like the UU MAGIC Glass project ^[167]. Sulimanov cited, “Transposing mirrored hand mesh onto the scene to follow its host,” as a difficulty ^[185]. MINERVA and the UU MAGIC Glass variant experienced this problem in development too. Obtaining the data from the Leap Controller for mirroring, which is not obvious to perform, was a significant challenge. The interaction engine that Leap uses cannot simply be mirrored. This system does not have as much potential presence as only the hands are visible but no arms although the forearm can be made visible.

Summary

Historical context is important to give an understanding of the capabilities, even the recent history of the HMD evolution from the prototype Oculus DK1 (2013) to the commercial Oculus Rift (2016) is significant for VR. Older research is still important as the design principles of how it was implemented can be useful. A lot of the VR stroke research suffers from low sample sizes, also how VR systems are implemented varies greatly so homogeneity is low making it difficult to generalise results. Specifically, no systems use the 6 DoF implementation that was used in this system MINERVA.

References (62)

• [4] V. S. Ramachandran and D. Rogers-Ramachandran, “Synaesthesia in Phantom Limbs Induced with Mirrors,” Proc. R. Soc. B Biol. Sci., vol. 263, no. 1369, pp. 377–386, 1996, doi: 10.1098/rspb.1996.0058. ↩
• [56] P. L. Weiss, E. A. Keshner, and M. F. Levin, Virtual Reality for Physical and Motor Rehabilitation (Virtual Reality Technologies for Health and Clinical Applications), 2014 editi. Springer, 2016. ↩
• [94] H. T. Regenbrecht, E. A. Franz, G. McGregor, B. G. Dixon, and S. Hoermann, “Beyond the looking glass: Fooling the brain with the augmented mirror box,” Presence Teleoperators Virtual Environ., vol. 20, no. 6, pp. 559–576, 2011, doi: 10.1162/PRES_a_00082. ↩
• [95] K. Hartman and E. L. Altschuler, “Mirror Therapy for Hemiparesis Following Stroke: A Review,” Curr. Phys. Med. Rehabil. Reports, vol. 4, no. 4, pp. 237–248, 2016, doi: 10.1007/s40141-016-0131-8. ↩
• [96] T. In, K. Lee, and C. Song, “Virtual Reality Reflection Therapy Improves Balance and Gait in Patients with Chronic Stroke: Randomized Controlled Trials,” Med. Sci. Monit., vol. 22, pp. 4046–4053, 2016, doi: 10.12659/MSM.898157. ↩
• [115] M. F. Levin, P. L. Weiss, and E. A. Keshner, “Emergence of Virtual Reality as a Tool for Upper Limb Rehabilitation: Incorporation of Motor Control and Motor Learning Principles,” Phys. Ther., vol. 95, no. 3, pp. 415–425, 2015, doi: 10.2522/ptj.20130579. ↩
• [123] D. Bachmann, F. Weichert, and G. Rinkenauer, “Evaluation of the leap motion controller as a new contact-free pointing device,” Sensors (Switzerland), vol. 15, no. 1, pp. 214–233, 2014, doi: 10.3390/s150100214. ↩
• [124] D. E. Holmes, D. K. Charles, P. J. Morrow, S. I. Mcclean, and S. Mcdonough, “Usability and performance of Leap Motion and Oculus Rift for upper arm virtual reality stroke rehabilitation,” Int. Conf. Disabil. Virtual Real. Assoc. Technol., no. January 2017, pp. 217–226, 2016. ↩
• [133] “Motor Control and Learning.” https://www.physio-pedia.com/Motor_Control_and_Learning (accessed Jun. 28, 2019). ↩
• [134] R. Llorens, J. Latorre, and C. Colomer, “A combined transcranial direct current stimulation and virtual reality-based intervention on upper limb function in chronic stroke survivors with severe hemiparesis A feasibility study,” 2017. ↩
• [135] R. C. Stockley, D. A. O’Connor, P. Smith, S. Moss, L. Allsop, and W. Edge, “A Mixed Methods Small Pilot Study to Describe the Effects of Upper Limb Training Using a Virtual Reality Gaming System in People with Chronic Stroke,” Rehabil. Res. Pract., vol. 2017, pp. 7–9, 2017, doi: 10.1155/2017/9569178. ↩
• [136] D. Lee, M. Lee, K. Lee, and C. Song, “Asymmetric training using virtual reality reflection equipment and the enhancement of upper limb function in stroke patients: A randomized controlled trial,” J. Stroke Cerebrovasc. Dis., vol. 23, no. 6, pp. 1319–1326, 2014, doi: 10.1016/j.jstrokecerebrovasdis.2013.11.006. ↩
• [137] D. Perez-Marcos et al., “Increasing upper limb training intensity in chronic stroke using embodied virtual reality: A pilot study,” J. Neuroeng. Rehabil., vol. 14, no. 1, pp. 1–14, 2017, doi: 10.1186/s12984-017-0328-9. ↩
• [138] R. Spicer, J. Anglin, D. M. Krum, and S. L. Liew, “REINVENT: A low-cost, virtual reality brain-computer interface for severe stroke upper limb motor recovery,” Proc. - IEEE Virtual Real., pp. 385–386, 2017, doi: 10.1109/VR.2017.7892338. ↩
• [139] J. Liu, M. Abd-El-Barr, and J. H. Chi, “Long-term Training with a Brain-Machine Interface-Based Gait Protocol Induces Partial Neurological Recovery in Paraplegic Patients,” Neurosurgery, vol. 79, no. 6, pp. N13–N14, 2016, doi: 10.1227/01.neu.0000508608.92059.64. ↩
• [140] K. R. Lohse, C. G. E. Hilderman, K. L. Cheung, S. Tatla, and H. F. M. Van Der Loos, “Virtual reality therapy for adults post-stroke: A systematic review and meta-analysis exploring virtual environments and commercial games in therapy,” PLoS One, vol. 9, no. 3, 2014, doi: 10.1371/journal.pone.0093318. ↩
• [141] I. Brunner et al., “Virtual reality training for upper extremity in subacute stroke (VIRTUES): Study protocol for a randomized controlled multicenter trial,” BMC Neurol., vol. 14, no. 1, pp. 1–5, 2014, doi: 10.1186/s12883-014-0186-z. ↩
• [142] K. Laver, B. Lange, S. George, J. Deutsch, G. Saposnik, and M. Crotty, “Virtual reality for stroke rehabilitation,” Cochrane Database Syst. Rev., no. 11, p. CD008349, 2017, doi: 10.1002/14651858.CD008349.pub4.www.cochranelibrary.com. ↩
• [143] D. Levac, S. Glegg, H. Colquhoun, P. Miller, and F. Noubary, “Virtual Reality and Active Videogame-Based Practice, Learning Needs, and Preferences: A Cross-Canada Survey of Physical Therapists and Occupational Therapists,” Games Health J., vol. 6, no. 4, pp. 217–228, 2017, doi: 10.1089/g4h.2016.0089. ↩
• [144] L. Ferreira dos Santos, O. Christ, K. Mate, H. Schmidt, J. Krüger, and C. Dohle, “Movement visualisation in virtual reality rehabilitation of the lower limb: A systematic review,” Biomed. Eng. Online, vol. 15, no. 3, pp. 75–88, 2016, doi: 10.1186/s12938-016-0289-4. ↩
• [145] A. Aminov, J. M. Rogers, S. Middleton, K. Caeyenberghs, and P. H. Wilson, “What do randomized controlled trials say about virtual rehabilitation in stroke? A systematic literature review and meta-analysis of upper-limb and cognitive outcomes,” J. Neuroeng. Rehabil., 2018, doi: 10.1186/s12984-018-0370-2. ↩
• [146] A. L. Faria, M. S. Cameirão, J. F. Couras, J. R. O. Aguiar, G. M. Costa, and S. Bermúdez i Badia, “Combined Cognitive- Motor Rehabilitation in Virtual Reality Improves Motor Outcomes in Chronic Stroke – A Pilot Study,” Front. Psychol., vol. 9, no. May, pp. 1–13, 2018, doi: 10.3389/fpsyg.2018.00854. ↩
• [147] T. Ogourtsova, P. S. Archambault, and A. Lamontagne, “Post-stroke unilateral spatial neglect: Virtual reality-based navigation and detection tasks reveal lateralized and non-lateralized deficits in tasks of varying perceptual and cognitive demands,” J. Neuroeng. Rehabil., vol. 15, no. 1, pp. 1–16, 2018, doi: 10.1186/s12984-018-0374-y. ↩
• [148] J. Bergmann, C. Krewer, P. Bauer, A. Koenig, R. Riener, and F. Müller, “Virtual reality to augment robot-assisted gait training in non-ambulatory patients with a subacute stroke: A pilot randomized controlled trial,” Eur. J. Phys. Rehabil. Med., 2018, doi: 10.23736/S1973-9087.17.04735-9. ↩
• [149] K. Threapleton, K. Newberry, G. Sutton, E. Worthington, and A. Drummond, “Virtually home: Feasibility study and pilot randomised controlled trial of a virtual reality intervention to support patient discharge after stroke,” Br. J. Occup. Ther., 2018, doi: 10.1177/0308022617743459. ↩
• [150] H. Pallesen, M. B. Andersen, G. M. Hansen, C. B. Lundquist, and I. Brunner, “Patients’ and Health Professionals’ Experiences of Using Virtual Reality Technology for Upper Limb Training after Stroke: A Qualitative Substudy,” Rehabil. Res. Pract., vol. 2018, pp. 1–11, 2018, doi: 10.1155/2018/4318678. ↩
• [151] M. Trombetta, P. P. Bazzanello Henrique, M. R. Brum, E. L. Colussi, A. C. B. De Marchi, and R. Rieder, “Motion Rehab AVE 3D: A VR-based exergame for post-stroke rehabilitation,” Comput. Methods Programs Biomed., vol. 151, pp. 15–20, 2017, doi: 10.1016/j.cmpb.2017.08.008. ↩
• [152] G. Saposnik et al., “Efficacy and safety of non-immersive virtual reality exercising in stroke rehabilitation (EVREST): a randomised, multicentre, single-blind, controlled trial,” Lancet Neurol., vol. 15, no. 10, pp. 1019–1027, 2016, doi: 10.1016/S1474-4422(16)30121-1. ↩
• [153] I. Brunner et al., “Is upper limb virtual reality training more intensive than conventional training for patients in the subacute phase after stroke? An analysis of treatment intensity and content,” BMC Neurol., vol. 16, no. 1, pp. 1–7, 2016, doi: 10.1186/s12883-016-0740-y. ↩
• [96] T. In, K. Lee, and C. Song, “Virtual Reality Reflection Therapy Improves Balance and Gait in Patients with Chronic Stroke: Randomized Controlled Trials,” Med. Sci. Monit., vol. 22, pp. 4046–4053, 2016, doi: 10.12659/MSM.898157. ↩
• [154] J. Kim, M. Lee, Y. Kim, S. D. Eun, and B. C. Yoon, “Feasibility of an individually tailored virtual reality program for improving upper motor functions and activities of daily living in chronic stroke survivors: A case series,” Eur. J. Integr. Med., vol. 8, no. 5, pp. 731–737, 2016, doi: 10.1016/j.eujim.2016.05.001. ↩
• [155] F. Wittmann et al., “Self-directed arm therapy at home after stroke with a sensor-based virtual reality training system,” J. Neuroeng. Rehabil., vol. 13, no. 1, pp. 1–10, 2016, doi: 10.1186/s12984-016-0182-1. ↩
• [156] D. Kairy et al., “Maximizing post-stroke upper limb rehabilitation using a novel telerehabilitation interactive virtual reality system in the patient’s home: Study protocol of a randomized clinical trial,” Contemp. Clin. Trials, vol. 47, pp. 49–53, 2015, doi: 10.1016/j.cct.2015.12.006. ↩
• [157] J. H. Shin, S. Bog Park, and S. Ho Jang, “Effects of game-based virtual reality on health-related quality of life in chronic stroke patients: A randomized, controlled study,” Comput. Biol. Med., vol. 63, pp. 92–98, 2015, doi: 10.1016/j.compbiomed.2015.03.011. ↩
• [158] G. S. Samuel, M. Choo, W. Y. Chan, S. Kok, and Y. S. Ng, “The use of virtual reality-based therapy to augment poststroke upper limb recovery,” Singapore Med. J., vol. 56, no. 7, pp. e127–e130, 2015, doi: 10.11622/smedj.2015117. ↩
• [159] R. Lloréns, J. A. Gil-Gómez, M. Alcañiz, C. Colomer, and E. Noé, “Improvement in balance using a virtual reality-based stepping exercise: A randomized controlled trial involving individuals with chronic stroke,” Clin. Rehabil., vol. 29, no. 3, pp. 261–268, 2015, doi: 10.1177/0269215514543333. ↩
• [160] S. Cho et al., “Development of virtual reality proprioceptive rehabilitation system for stroke patients,” Comput. Methods Programs Biomed., vol. 113, no. 1, pp. 258–265, 2014, doi: 10.1016/j.cmpb.2013.09.006. ↩
• [161] M. Yates, A. Kelemen, and C. Sik Lanyi, “Virtual reality gaming in the rehabilitation of the upper extremities post-stroke,” Brain Inj., vol. 30, no. 7, pp. 855–863, 2016, doi: 10.3109/02699052.2016.1144146. ↩
• [162] G. Chaponneau, D. E. Holmes, D. K. Charles, S. Mcclean, P. J. Morrow, and S. M. Mcdonough, “Application of invisible playground theory to assistive technology design for motivating exercise within activities of daily living,” 11th Int. Conf. Disabil., p. 7049, 2016. ↩
• [163] D. K. Charles, T. Charles, and M. D. McNeill, “Using player and world representation techniques from computer games to improve student engagement,” Proc. 2009 Conf. Games Virtual Worlds Serious Appl. VS-GAMES 2009, pp. 36–42, 2009, doi: 10.1109/VS-GAMES.2009.27. ↩
• [164] J. Boureaud, D. E. Holmes, D. K. Charles, S. McClean, and P. J. Morrow, “Application of a rehabilitation game model for assistive technology software design,” 11th Int. Conf. Disabil., 2016. ↩
• [165] M. D. J. D. J. McNeill, D. K. K. Charles, J. W. W. Burke, J. H. H. Crosbie, and S. M. M. McDonough, “Evaluating user experiences in rehabilitation games,” in Journal of Assistive Technologies, 2012, vol. 6, no. 3, pp. 173–181, doi: 10.1108/17549451211261290. ↩
• [166] J. W. Burke, M. D. J. McNeill, D. K. . Charles, P. J. Morrow, J. H. Crosbie, and S. M. McDonough, “Designing Engaging, Playable Games for Rehabilitation,” 8th Int. Conf. Disabil. Virtual Real. Assoc. Technol., pp. 195–201, 2010, [Online]. Available: http://www.icdvrat.reading.ac.uk/2010/papers/ICDVRAT2010_S07_N02_Burke_etal.pdf. ↩
• [177] “167 Partner overview,” 2019. http://magic-pcp.eu/wp-content/uploads/2019/12/Dr-Aoife-Stephenson-pdf.pdf. ↩
• [168] A. H. Smeragliuolo, N. J. Hill, L. Disla, and D. Putrino, “Validation of the Leap Motion Controller using markered motion capture technology,” J. Biomech., vol. 49, no. 9, pp. 1742–1750, 2016, doi: 10.1016/j.jbiomech.2016.04.006. ↩
• [169] N. Darbois, A. Guillaud, and N. Pinsault, “Do Robotics and Virtual Reality Add Real Progress to Mirror Therapy Rehabilitation ? A Scoping Review,” vol. 2018, 2018, doi: 10.1155/2018/6412318. ↩
• [170] A. Rothgangel and R. Bekrater-Bodmann, “Mirror therapy versus augmented/virtual reality applications: towards a tailored mechanism-based treatment for phantom limb pain,” Pain Manag., vol. 9, no. 2, pp. 151–159, 2019, doi: 10.2217/pmt-2018-0066. ↩
• [171] M. Diers et al., “Illusion-related brain activations: A new virtual reality mirror box system for use during functional magnetic resonance imaging,” Brain Res., vol. 1594, pp. 173–182, 2015, doi: 10.1016/j.brainres.2014.11.001. ↩
• [172] T. Stanton and C. Berryman, What is the effect of bodily illusions on corticomotoneuronal excitability ? A systematic review. 2018. ↩
• [173] C. Schuster-Amft et al., “Intensive virtual reality-based training for upper limb motor function in chronic stroke: a feasibility study using a single case experimental design and fMRI,” Disabil. Rehabil. Assist. Technol., vol. 10, no. 5, pp. 385– 392, 2015, doi: 10.3109/17483107.2014.908963. ↩
• [174] S. Fukumori, A. Gofuku, K. Isatake, and K. Sato, “Mirror thrapy system based virtual reality for chronic pain in home use,” IECON Proc. (Industrial Electron. Conf., pp. 4034–4039, 2014, doi: 10.1109/IECON.2014.7049106. ↩
• [175] S. Shiri et al., “A Novel Virtual Reality System Integrating Online Self-Face Viewing and Mirror Visual Feedback for Stroke Rehabilitation: Rationale and Feasibility,” Top. Stroke Rehabil., vol. 19, no. 4, pp. 277–286, 2012, doi: 10.1310/tsr1904-277. ↩
• [176] S. Hoermann, E. A. Franz, and H. Regenbrecht, “Referred Sensations Elicited by Video-Mediated Mirroring of Hands,” PLoS One, vol. 7, no. 12, 2012, doi: 10.1371/journal.pone.0050942. ↩
• [177] Y. J. Kang et al., “Upper extremity rehabilitation of stroke: Facilitation of corticospinal excitability using virtual mirror paradigm,” J. Neuroeng. Rehabil., vol. 9, no. 1, p. 1, 2012, doi: 10.1186/1743-0003-9-71. ↩
• [178] Y. R. Yang, Y. H. Chen, H. C. Chang, R. C. Chan, S. H. Wei, and R. Y. Wang, “Effects of interactive visual feedback training on post-stroke pusher syndrome: A pilot randomized controlled study,” Clin. Rehabil., vol. 29, no. 10, pp. 987– 993, 2015, doi: 10.1177/0269215514564898. ↩
• [179] A. Vourvopoulos and S. Bermúdezi Badia, “Motor priming in virtual reality can augment motor-imagery training efficacy in restorative brain-computer interaction: A within-subject analysis,” J. Neuroeng. Rehabil., vol. 13, no. 1, pp. 1–14, 2016, doi: 10.1186/s12984-016-0173-2. ↩
• [180] S. Hoermann et al., “Computerised mirror therapy with Augmented Reflection Technology for early stroke rehabilitation: clinical feasibility and integration as an adjunct therapy,” Disabil. Rehabil., vol. 39, no. 15, pp. 1503–1514, 2017, doi: 10.1080/09638288.2017.1291765. ↩
• [181] J. McKinney, S. Mcdonough, P. J. Morrow, and N. C. Kennedy, “Reflections on the Design and Development of a Virtual Reality Mirror Therapy System for Upper Limb Stroke Rehabilitation,” ICDVRAT2018, 2018. ↩
• [182] C. Falcao, A. C. Lemos, and M. Soares, “Evaluation of Natural User Interface: A Usability Study Based on the Leap Motion Device,” Procedia Manuf., vol. 3, no. Ahfe, pp. 5490–5495, 2015, doi: 10.1016/j.promfg.2015.07.697. ↩
• [183] G. Placidi, L. Cinque, M. Polsinelli, and M. Spezialetti, “Measurements by a LEAP-based virtual glove for the hand rehabilitation,” Sensors (Switzerland), vol. 18, no. 3, pp. 1–13, 2018, doi: 10.3390/s18030834. ↩
• [184] L. M. Weber, D. M. Nilsen, G. Gillen, J. Yoon, and J. Stein, “Immersive Virtual Reality Mirror Therapy for Upper Limb Recovery after Stroke: A Pilot Study,” Am. J. Phys. Med. Rehabil., vol. 98, no. 9, pp. 783–788, 2019, doi: 10.1097/PHM.0000000000001190. ↩
• [185] “Mirror Therapy: Redesigned in VR,” 2019. https://blog.siggraph.org/2020/04/mirror-therapy-redesigned-in-vr.html/. ↩

Previous: Chapter 3. Virtual Reality (VR) and Game Design Factors Next: Chapter 5. System Design and Development

---