Abstract
Modern immersive virtual reality (VR) systems include embedded hand tracking, stand-alone and wireless operation, fast donning and calibration: these features are precious for usability of rehabilitation serious games in the clinical practice, envisaging also home-care applications. Can wearable haptics well integrate with the above features? Different designs result in a trade-off between wearability and richness of feedback. Yet, engagement of the user is also one of the key-features for rehabilitation serious games. We developed two novel fingertip devices aiming the first at lightweight and wearability, the second at rich and powerful cutaneous feedback. We compared the two designs in terms of usability and users’ engagement within a modern rehabilitation system in immersive VR.
This work was supported by Project “TELOS - Tailored neurorehabilitation thErapy via multi-domain data anaLytics and adapative seriOus games for children with cerebral palSy”, funded by Tuscany Region, Italy (CUP J52F20001040002).
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Immersive Virtual Reality (VR) systems, and in particular Head Mounted Displays (HMD), have seen in the past decade an impressive progress in terms of quality of the visual feedback, involving precise head tracking, wider field of view and higher resolution of the rendered image. Even more, in the last couple of years certain off-the-shelf products (i.e. Oculus Quest 2) offer advanced characteristics that considerably improve usability and comfort of the user: they include wireless stand alone operation, simple donning of the system and fast calibration of the workspace, and embedded hand tracking. The latter is relevant for VR applications that include manipulation: the HMD alone becomes now enabled for immersive manipulation task without the need of any additional device. Together with the above features improving overall usability, such recent technology advances might significantly change the use of VR in certain critical applications, such as rehabilitation and serious games for educational and training in general [2, 12, 15]. In neurorehabilitation, VR systems have been proposed and experimented in the shape of serious-games, combining engagement of the user with task oriented motor exercises [5]. Virtualization of the exercise brings the advantage of intrinsic parametrization, adaptability and repeatability of the exercises.
In a recent study, cutaneous feedback devices have been used into a prolonged pilot clinical study, integrated into a VR serious game for children with cerebral palsy [1]. Richness and intensity of the feedback versus lightweight and compactness of the device represents a common trade-off in the design of wearable haptic devices, especially in the design of fingertip haptic devices. In the last decade a lot of innovative fingertip haptic or multimodal devices have been proposed in the scientific literature [11, 14]. Each device is though to render one or more specific feature of the interaction between object and fingerpad such as contact orientation [3, 4, 13], friction action [6, 10], thermal transients [7] or high frequency vibrations due to texturized surface exploration [8, 9]. In most of the cases wearability and user’s comfort have been considered as secondary objectives. In this paper novel fingertip haptic devices, able to provide the user with tactile stimuli in a wide frequency range and specifically designed focusing on the usability, are in terms of comfort, wearability, usability in a modern VR rehabilitation system designed for both high immersivity and involvement of the patient (Fig. 1). The proposed highly wearable thimbles named “LightThimbles” are compared with more powerful fingertip haptic devices named “HapticThimbles” and designed for the rendering of contact no-contact transitions and tactile stimuli in the range of 0–350 Hz. The novelty of this work is given by the approach of understanding the trade-off between comfort and usability, and the richness of haptic feedbacks in a VR application. Specifically this trade-off is crucial when dealing with rehabilitation contexts, in which patients often struggle to achieve even simple hand or fingers movements and the comfort become essential for delivering the therapy. As a preliminary study, devices evaluation was performed with healthy subjects only.
2 Materials and Methods
In this section we describe design of the two haptic devices used in the experiment, the developed serious game in immersvie VR, and the experimental procedure.
2.1 Device 1: The Light Haptic Thimble
A light and thin haptic thimble has been developed to enhance lightweight, compact size and wearability. We focused on rendering high frequency components and fast transients of the haptic feedback, while eliminating static and slow force components. This allows to reduce power requirements and size of the actuators, at the cost of limited rendering capabilities (i.e. no out of contact transition and no static grasping force). A miniaturized custom-made electromagnetic voice coil (outer diameter \(12\,\mathrm {mm}\), output force \(0.4\,\mathrm {N}\)) was implemented due to the high quality and wide frequency response of this type of actuators. The above design choices allowed other advantages improving wearability: the moving part of the actuator (1\(^\circ \) of freedom) is designed to be kept always in contact with the fingerpad through a soft and compliant structure of the whole thimble (no calibration phase is needed). The thimble is fabricated in soft resin (Photocentric UV DLP Flexible) making it adaptable to different finger sizes. Stereo-lithography 3D printing method allows for a high degree of customization. The thimble shape increases flexibility and compliance of the frontal part, supporting the embedded voice coil, and of the two lateral brackets adapting to different finger size. Total mass of the device is just \(7\,\mathrm {g}\) (Fig. 2).
2.2 Device 2: The Haptic Thimble
The HapticThimble has been designed with focus on both the quality and richness of the feedback provided to the user. The device has the same actuated DoF of the LightThimble but the HapticThimble allows a 4 mm stroke of the contact plate performing also the rendering of contact no-contact transitions between fingerpad and virtual object. The moving part is thought to be lightweight and it moves on low friction plastic bushings in order to increase the maximum feedback frequency allowed (i.e. 350 Hz). Moreover, the compact custom designed voice coil placed on top of the device, is able to provide the user also with constant forces. To increase the users comfort the device does not constraints the finger distal joint, in fact it is thought to be fastened on the middle phalanx of the finger, whereas the actuated part, placed on the distal phalanx, is connected to the fastened part by means of a revolute flexure hinge coaxial with the finger joint axis. A small contact area between the actuated part of the device and the fingertip guarantee that the two moves together. Finally, the part of the device covering the user’s finger has been white painted in order to better perform with the vision based markerless tracking system (Fig. 3).
2.3 The Virtual Reality Serious Game: Potions
The virtual reality experience is part of a serious game developed in our laboratory for neurorehabilitation of children with cerebral palsy. It takes place into an alchemist laboratory where the player is asked to prepare a magic potion, mixing several ingredients in a cauldron (Fig. 1). The game requires the player to perform parametrized pick-and-place and prono-supination motor tasks, in order to pour the right ingredients into the cauldron. The game asks for precision in pouring the correct amount of liquid, shown by a green reference line on a large purple progress bar on top of the cauldron. The game loop is structured as follows: a) a red arrow suggests to the player the ingredient that has to be picked up, b) the player grabs the ingredient performing a hand-pinching motion, c) the player brings the ingredient on top of the cauldron, d) starts to pouring it after rotating the wrist over 80\(^\circ \), and e) tries to get the progress bar as close as possible to the shown target, interrupting the flow decreasing the wrist rotation angle towards less than 70\(^\circ \). This loop is repeated for five ingredients, until the recipe is completed.
Regarding the rendered haptic feedback, the grasping action generates a fast transient at the contact threshold, followed by a continuous normal force, as long as the object is grasped. The pouring phase adds a continuous vibration, synthesized from the audio signal of a pouring liquid.
2.4 Experimental Setup and Protocol
Nine subjects (3 females), age \(29.40\,\pm \,3.59\) years, height \(1.74\,\pm \,10.43\) m have been enrolled in this study and signed an informed consent before joining the experiment. The study was approved by the Joint Ethical Committee of the Scuola Superiore Sant’Anna and Scuola Normale Superiore of Pisa, Italy. Each subject experienced three gaming sessions in random order, one for each experimental condition: hands-free (HF), wearing the Light Thimble (HS), and wearing the Haptic Thimble (HB). Each session required the subject to wear the Oculus Quest 2 VR headset, to complete one round of the Potions game, and to answer a questionnaire. Questions were selceted in order to evaluate comfort and ease of interaction, immersion, and role of the visual and haptic feedback. The same questions were proposed after each condition. Q1, regarding previous VR experience, was asked once (Table 1).
Regarding objective results, time duration and the pouring precision were chosen as performance metrics. The time duration included the overall time spent to accomplish each session. The pouring precision is the difference between the reference and the poured amount, averaged over all the five potions used in the session. Data were analyzed using the Kruskal-Wallis non-parametric statistical method with Dunn’s post-hoc test.
3 Results
Results showed noticeable differences among the three conditions, reflecting the expected trends in both in-game recorded data and questionnaires, although not all the differences were statistically significant. For the former case, Time and Pouring Precision performance metrics were analyzed and Kruskal-Wallis test found a statistical difference among the three groups for both metrics (see Fig. 4). For Time, Dunn’s test reported a significant difference between HF and HS groups (p = .0040) and HF and HB groups (p = .0004) but not among HS and HB groups (p = .7984). For Pouring Precision, a significant difference was found among HF-HB groups (p = .0369) but not among HF-HS (p = .1755) and HS-HB groups (p = .7763).
For what concerns questionnaire results, Kruskal-Wallis test did not found any significant difference among conditions. Answwers to the questionnaire are reported in Fig. 5 for each experimental condition. The mean value of the Q1 (VR experience) for the participants was indicating prevalence of naive subjects.
4 Discussion and Conclusions
With respect to the past, modern VR systems noticeably improve wearability and dexterity to the user, adding wireless connectivity and embedded hand tracking. These features are of great advantage in certain applications, such as virtual serious games for neurorehabilitation. In this work we evaluated two different haptic thimble designs (plus the hands-free condition) in terms of both effectiveness of the haptic feedback and perception of the immersive VR equipment and environment. In terms of quantitative measurements, the Time and Precision metrics of the virtual task showed a trend between the three conditions: the hands free condition scored the shortest time to perform the task, while the most intense feedback of the Haptic Thimble performed the best in terms of precision. The finding suggests on one side the importance of a rich haptic feedback to improve precision in manipulation tasks, and on the other side how the addition of wearable devices seemed to slow down task execution. This might be due to a combined effect of the physical presence and perception of the wearable devices and of the possibly deteriorated tracking performance of the VR system. We intend to further investigate this point, by means of redundant tracking systems. The questionnaire suggests additional information, although not statistically significant. Differences between conditions were narrowed by the overall high reported scores Surprisingly, the hands-free condition did not show noticeably higher score in questions regarding comfort, naturalness and ease of the task. However, the virtual experience was relatively short and possibly not sufficient to emphatize different levels of fatigue. The LightThimble reported in general the highest scores related to the overall evaluation of the experience, naturalness of the interaction, and in particular immersivity. The HapticThimble condition was rated the highest score in terms of haptic effectiveness, with lower scores than the LightThimble for coherency of hand tracking, natural interaction and comfort. Concluding, the experimental experience investigated how different designs of wearable devices influence interaction and perception into an immersive virtual experience. Quantitative metrics of task execution showed a clear trend at increasing size of devices and richness of the haptic feedback, resulting in a trade-off between speed and precision. Participants preferences suggested that a highly wearable but limited intensity haptic device was the most effective for the given environment. Other design choices (or hands-free) can be more suitable for different scenarios involving precise manipulative tasks, or, at the opposite, faster movements and dynamic interaction.
References
Bortone, I., et al.: Immersive virtual environments and wearable haptic devices in rehabilitation of children with neuromotor impairments: a single-blind randomized controlled crossover pilot study. J. NeuroEngineering Rehabil. 17(1), 1–14 (2020)
Bortone, I.: Wearable haptics and immersive virtual reality rehabilitation training in children with neuromotor impairments. IEEE Trans. Neural Syst. Rehabil. Eng. 26(7), 1469–1478 (2018)
Chinello, F., Malvezzi, M., Prattichizzo, D., Pacchierotti, C.: A modular wearable finger interface for cutaneous and kinesthetic interaction: control and evaluation. IEEE Trans. Ind. Electron. 67(1), 706–716 (2019)
Chinello, F., Pacchierotti, C., Malvezzi, M., Prattichizzo, D.: A three revolute-revolute-spherical wearable fingertip cutaneous device for stiffness rendering. IEEE Trans. Haptics 11(1), 39–50 (2017)
Deutsch, J., McCoy, S.W.: Virtual reality and serious games in neurorehabilitation of children and adults: prevention, plasticity and participation. In: Pediatric Physical Therapy: The Official Publication of the Section on Pediatrics of the American Physical Therapy Association 29(Suppl 3 IV STEP 2016 CONFERENCE PROCEEDINGS), S23 (2017)
Fani, S., Ciotti, S., Battaglia, E., Moscatelli, A., Bianchi, M.: W-fyd: a wearable fabric-based display for haptic multi-cue delivery and tactile augmented reality. IEEE Trans. Haptics 11(2), 304–316 (2017)
Gabardi, M., Chiaradia, D., Leonardis, D., Solazzi, M., Frisoli, A.: A high performance thermal control for simulation of different materials in a fingertip haptic device. In: Prattichizzo, D., Shinoda, H., Tan, H.Z., Ruffaldi, E., Frisoli, A. (eds.) EuroHaptics 2018. LNCS, vol. 10894, pp. 313–325. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93399-3_28
Gabardi, M., Solazzi, M., Leonardis, D., Frisoli, A.: a new wearable fingertip haptic interface for the rendering of virtual shapes and surface features. In: 2016 IEEE Haptics Symposium (HAPTICS), pp. 140–146. IEEE (2016)
Leonardis, D., Gabardi, M., Solazzi, M., Frisoli, A.: A parallel elastic haptic thimble for wide bandwidth cutaneous feedback. In: Nisky, I., Hartcher-O’Brien, J., Wiertlewski, M., Smeets, J. (eds.) EuroHaptics 2020. LNCS, vol. 12272, pp. 389–397. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58147-3_43
Leonardis, D., Solazzi, M., Bortone, I., Frisoli, A.: A 3-rsr haptic wearable device for rendering fingertip contact forces. IEEE Trans. Haptics 10(3), 305–316 (2016)
Pacchierotti, C., Sinclair, S., Solazzi, M., Frisoli, A., Hayward, V., Prattichizzo, D.: Wearable haptic systems for the fingertip and the hand: taxonomy, review, and perspectives. IEEE Trans. Haptics 10(4), 580–600 (2017)
Simões, M., Bernardes, M., Barros, F., Castelo-Branco, M.: Virtual travel training for autism spectrum disorder: proof-of-concept interventional study. JMIR Serious Game. 6(1), e8428 (2018)
Solazzi, M., Frisoli, A., Bergamasco, M.: Design of a novel finger haptic interface for contact and orientation display. In: 2010 IEEE Haptics Symposium, pp. 129–132. IEEE (2010)
Wang, D., Ohnishi, K., Xu, W.: Multimodal haptic display for virtual reality: a survey. IEEE Trans. Ind. Electronics 67(1), 610–623 (2019)
Wiemeyer, J.: Serious games in neurorehabilitation: a systematic review of recent evidence. In: Proceedings of the 2014 ACM International Workshop on Serious Games, pp. 33–38 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2022 The Author(s)
About this paper
Cite this paper
Camardella, C., Gabardi, M., Frisoli, A., Leonardis, D. (2022). Wearable Haptics in a Modern VR Rehabilitation System: Design Comparison for Usability and Engagement. In: Seifi, H., et al. Haptics: Science, Technology, Applications. EuroHaptics 2022. Lecture Notes in Computer Science, vol 13235. Springer, Cham. https://doi.org/10.1007/978-3-031-06249-0_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-06249-0_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06248-3
Online ISBN: 978-3-031-06249-0
eBook Packages: Computer ScienceComputer Science (R0)