Abstract
This study investigates weight perception in virtual reality without kinesthetic feedback from the real world, by means of an illusory method called pseudo-haptic. This illusory model focuses on the dissociation of visual input and somatosensory feedback and tries to induce the sensation of virtual objects' loads in VR users by manipulating visual input. For that, modifications on the control-display ratio, i.e., between the real and virtual motions of the arm, can be used to produce a visual illusionary effect on the virtual objects' positions as well. Therefore, VR users perceive it as velocity variations in the objects' displacements, helping them achieve a better sensation of virtual weight. A primary contribution of this paper is the development of a novel, holistic assessment methodology that measures the sense of the presence in virtual reality contexts, particularly when participants are lifting virtual objects and experiencing their weight. Our study examined the effect of virtual object weight on the kinematic parameters and velocity profiles of participants' upward arm motions, along with a parallel experiment conducted using real weights. By comparing the lifting of real objects with that of virtual objects, it was possible to gain insight into the variations in kinematic features observed in participants' arm motions. Additionally, subjective measurements, utilizing the Borg CR10 questionnaire, were conducted to assess participants' perceptions of hand fatigue. The analysis of collected data, encompassing both subjective and objective measurements, concluded that participants experienced similar sensations of fatigue and changes in hand kinematics during both virtual object tasks, resulting from pseudo-haptic feedback, and real weight lifting tasks. This consistency in findings underscores the efficacy of pseudo-haptic feedback in simulating realistic weight sensations in virtual environments.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction and related work
By creating a virtual environment (VE) in virtual reality (VR) that is comparable to real environments, researchers can explore ideas that are not easily developed or evaluated in real conditions. Consequently, this field of science has recently caught the attention of many researchers. Numerous VR applications from various fields have spurred concepts like immersion and sense of the presence in virtual environments to become more prevalent. Nonetheless, although VR researchers have made good progress in the field of presence and immersion in virtual reality, there are still open problems and challenges that can currently affect the sense of presence for people immersed in VR.
The challenge we want to address in this paper concerns the feeling of weight in virtual reality, caused by the lack of kinesthetic cues in this environment. In real environments, the integration of visual and real sensory systems helps humans perceive the weight of objects (Ernst and Banks 2002). The real sensory system provides various sources of information, including force feedback, proprioception, and tactile feedback. As a result, real perception is multimodal, encompassing multiple sensory inputs. On the other hand, visual information is unimodal, as it is solely obtained through the retinas of our eyes (Aman et al. 2010).
In virtual environments, visual information is provided entirely through the virtual scene via a head-mounted device (HMD). Despite technical progress in visual simulation in terms of quality, information related to virtual objects like texture, shape, size, and weight is still partially missing. This information is mostly provided by haptic feedback (a combination of force feedback, tactile, and proprioception), but it remains a challenge due to the lack of presence of virtual objects in the real environment. To solve this, different studies in the literature have introduced vibration stimuli approaches, providing a solution in terms of tactile feedback, but not in terms of force feedback, since this relies on the physical direction of the interaction and is hard to compensate (Rietzler et al. 2018b). Therefore, to overcome force feedback issues in VR, other researchers' methods have focused on grounded devices such as phantom devices (Burdea 1999; Massie et al. 1994; Pacchierotti et al. 2017) or portable and wearable real devices like 'Grabity' (Choi et al. 2017). We can refer to Dominjon et al. as examples of grounded devices. Dominjon et al. (2005) manipulated the C/D ratio of a physical ball connected to a PHANToM device to alter the speed of physical ball movement compared to virtual ball movement. Participants perceived a difference in weight when lifting the physical ball while viewing the virtual ball, based on a change in C/D ratio.
Despite the advantages of grounded devices, they require grounding, i.e., attachment to a heavy object or permanent fixture, to produce required forces and withstand reciprocal kickback (Suzuki and Kobayashi 2005; Weiss et al. 2011). Mechanical joints (Araujo et al. 2016), wires (Hirata and Sato 1992; Agronin 1987), and air jet actuators (Suzuki and Kobayashi 2005) are used for grounded force feedback. Although these methods can provide realistic force feedback, we focus on ungrounded force feedback for mobile applications. These devices are costly, and their mobility limitations have been addressed by researchers through form factors allowing more movement freedom, like propellers (Heo et al. 2018), or by altering air drag (Zenner and Krüger 2019). Wearable real devices mainly provide tactile feedback and reduce the range of motion (Nisar et al. 2018; Pacchierotti et al. 2017).
However, it should be noted that perceiving object heaviness relies on both the real sensory system and visual stimuli (Runeson and Frykholm 1981). 'Pseudo-haptic feedback' manipulates the human perception system to induce object heaviness. This approach exploits the visual context dependency in creating real perception. VR users can experience realism through visual stimuli (Lécuyer 2009). Ernst and Banks (2002) showed that vision dominates the haptic sensory system, and haptic receptors can be stimulated by visual stimuli (Ujitoko and Ban 2021). Studies have used pseudo-haptic feedback to simulate virtual object weight for VR users (Jauregui, et al. 2014; Palmerius et al. 2014; Samad, et al. 2019; Yu and Bowman 2020). The 'control/display' (C/D) ratio, the ratio between user input displacement and visual feedback displacement, has been a creative pseudo-haptic method (Argelaguet and Andujar 2013; Lécuyer 2009). Studies Dominjon et al. (2005), Nakakoji et al. (2010, 2011), and Rietzler et al. (2018a) used the C/D ratio in two-dimensional and three-dimensional environments for weight perception (Samad et al. 2019). Pseudo-haptics also explores sensations like touch in virtual environments (Tano et al. 2015). Our study employs pseudo-haptic methodology to create an impression of virtual object weight through visual feedback, aiming to introduce a holistic methodology by bridging objective and subjective metrics for measuring virtual object weight sensation during a lifting task in VR users.
Two methods have been mainly used to evaluate weight perception in VR. The first is subjective measurements, typically based on questionnaires (Witmer and Singer 1998). The drawback is that subjects must verbalize their physical experience. Kim et al. (2022) used traditional methods like the Witmer-Singer presence questionnaire to assess perceived virtual object heaviness. These methods, while accepted, have limitations, such as participants must leave the VR environment and rely on their recollection to respond, which may introduce bias and may not capture their immediate sensations and experiences. Alternatively, it may be beneficial to use subjective measurements relevant to the context of the virtual environment to assess the subjects' feelings and thoughts during the experiment. An example of such a measurement in our experiment, would be fatigue and tiredness experienced during the lifting of an object, which would be a measurement of presence. A subjective measurement of fatigue in participants' hands could be the Borg CR10 questionnaire (Borg 1990), which would align with this study's goal.
The alternative approach for subjective measurements would rely on objective measurement methods, which are based on human physiological data. Previous studies like Kumar et al. (2023) introduced a new methodology based on mechanical rules, and Moosavi et al. (2023) attempted to measure weight perception in VR users using a new objective method based on physical work (physics rule). However, these studies has their own drawbacks, for example Kumar et al. (2023) studied weight perception using the Weber fraction, they used weights up to 0.23 kg, which differ from those typically weights used in daily life interactions.
One of the main objective measurements in VR relies on the use of kinematic features (such as movement duration, Maximum Velocity (MaxV), Time to peak Velocity (TPV) that are independent of the subject's point of view. Additionally, kinematic features, especially MaxV and TPV, along with the velocity profile of hand movement, provide insights into how quickly users try to move objects and when they anticipate the most effort is required, respectively. This could indicate their perceived weight of the object, and thus, their sensory integration within VR. In our research, we align with VR weight perception studies like those by Kim et al. (2022), but take a novel approach by introducing an objective method for measuring the sense of presence in VR, focusing on participant interactions with virtual objects. However, our method, centered on CNS movement planning and kinematic analysis, goes beyond the objective measures used in previous researches, such as the methodologies used by Kumar et al. (2023) or Moosavi et al. (2023).
Inspired by previous studies (Bock 1990; Atkeson and Hollerbach 1985; Gaveau et al. 2014; Hoffman and Strick 1993) that examined the impact of weight on kinematic features in real environments, our study shifts the focus to exploring weight perception in virtual reality (VR). Prior studies demonstrated that if a person moved their hand in one direction, a bell-shaped velocity profile would be observed when plotting the velocity data. This bell-shaped velocity profile is directionally dependent (Papaxanthis et al. 1998a, b). It is asymmetrical in upward movement, meaning the acceleration part of the bell-shaped velocity is shorter than the deceleration part. Conversely, it is symmetrical in downward motion.
According to Bock (1990), increasing load affected both normalized and non-normalized velocity profiles, but they did not observe symmetry in the velocity profile shape. Gaveau et al. (2014) illustrated that with increased load, the TPV value (Time to Peak Velocity/movement time) of the hand movement in the upward direction changed. The equation used to determine time to peak velocity is:
where \({time}_{P{V}_{movement}}\) is the time at which peak velocity occurs, \({time}_{{{\text{offset}}}_{movement}}\) is the time at the end of the movement and \({time}_{{{\text{onset}}}_{movement}}\) is the starting time of the hand movement.
On the other hand, (Atkeson and Hollerbach 1985) showed that the velocity of hand movement increased when the weight decreased.
2 Scientific issue
In this paper, we acknowledge the existing literature on weight perception using pseudo-haptic, C/D ratio, and other approaches. However, our study is not about re-establishing known facts. Rather, it explores the following research questions:
-
Q1 Is it possible to objectively measure and analyze weight perception from human motion?
-
Q2 To what extent do traditional metrics (Borg CR10) correlate with newer metrics like kinematic measurements?
The primary purpose of this study is to introduce an objective method for evaluating arm movements based on kinematic characteristics and velocity profiles. It also includes determining and comparing the effects of different weights on subjects' temporal features. For validation, we compare the velocity profile and kinematic features of subjects' lifting movements in both virtual and real environments.
Through our new evaluation methodology, we aim to provide a comprehensive understanding of weight perception in VR. By juxtaposing real-world, virtual-world, and pseudo-haptic weight perceptions and their corresponding metrics, we aim to present a comprehensive picture of weight perception in VR that is, to our knowledge, unprecedented.
The main contributions of this study include:
-
Objective Analysis of Weight Perception Our study goes beyond existing literature by applying pseudo-haptic feedback innovatively to understand weight perception in VR.
-
Holistic Evaluation Methodology We propose a unique evaluation approach using both real and virtual weights to derive actionable insights on the realism of pseudo-haptic models.
-
Objective Kinematic Metrics in VR Our emphasis on Max V and TPV introduces a fresh dimension to the study of weight perception, capturing nuances beyond traditional metrics.
-
Incorporating Borg CR10 This study doesn't just rely on objective measures; it seeks to bridge the objective and subjective divide by correlating kinematic features with the Borg CR10 questionnaire.
3 Materials and method
3.1 Virtual reality environment and proposed lifting conditions
We developed a virtual environment that is represented as a virtual office, closely resembling the real environment of our experiment, complete with desks, chairs, and a green box (see Fig. 1A). In terms of size, position, and orientation, the white box perfectly matched the platform where subjects performed the experiment in the real environment.
The green box, shown in Fig. 1B, serves as the starting point for our lifting experiment, where subjects are required to lift a virtual bottle. Conversely, a green virtual window, depicted in Fig. 1B, represents the end point of the lifting task. To maintain consistency with the participants' sense of presence, a 3D model of a water bottle was used to represent the objects being lifted. This model corresponds to an actual bottle present in the experiment room.
Then, as our study aimed to compare the effects of pseudo-haptic feedback on VR users' perception of virtual object weight with that of real object lifting, we proposed using the following three lifting conditions:
Condition R Real condition. We developed this condition as our reference for real lifting, i.e., without experiencing any conflict between somatosensory feedback and visual information, but with the perception of weight related to the mass of the real bottle lifted. An exact correspondence between the real bottle and the virtual bottle was ensured through precise tracking of the real one. To avoid any differences due to a possible effect of our VR environment between conditions, the lifting of the real object was also done in VR. In this condition, we used 2 HTC Vive hand trackers, one attached to the surface of the real bottle and the other to the wrist of the subject's non-dominant hand. The hand trackers were activated as soon as the subjects grabbed the bottle, recording the kinematic data of their hand movements.
Condition V VR reference condition. This condition did not include any pseudo-haptic feedback. It was developed so that subjects would perform a lifting task with an HTC Vive hand controller, without experiencing any conflict between their somatosensory feedback and visual information, and without perceiving any weight effect, except for the mass of the VR controller (m = 0.31 kg).
Condition P Pseudo-haptic condition. This condition is characterized by manipulating the Control/Display (C/D) ratio, which indicates the relationship between a participant's physical movement (control) and the movement displayed in a virtual environment (display). According to our scheme, the C/D ratio is inversely proportional to the perceived weight of a virtual object. A lower C/D ratio indicates a heavier virtual object, and vice versa. C/D ratios ranging from 0.5 to 1 were selected for the experiment.
In our previous study Moosavi et al. (2023), we employed a method for creating a sense of feel (or feedback) in virtual reality that mimics the act of lifting objects, based on physics principles. When you lift an object in real life, you need to exert a force at least equal to the object’s weight. In a typical virtual reality setting, users lift VR controllers, not actual objects with varying weights. Therefore, the user’s force is focused on lifting the controller rather than the 'virtual object' they see in the VR environment.
We proposed a ratio function, denoted as k(m), to define the relationship between the force applied to the VR controller and the force required to lift a virtual object, where 'm' represents the mass of the object
The ratio function k(m)is then transformed into ratios of Physical Work, providing a form of feedback, or 'pseudo-haptic' experience, that helps simulate the act of lifting virtual objects of different weights.
We propose to provide pseudo-haptic feedback using this ratio function, where m is the mass of the object; If we apply the distance of the hand controller and distance of the virtual object we will have:
Moreover, in order to fully compensate for the difference in mass between the object and the controller, we need a ratio of work equal to 1:
To accommodate virtual objects that may weight more than ten times the VR controller, we introduced a constant c to the equation, limiting the applied ratio and preventing a disruption in the user's sense of presence in the virtual world.
There is a need for caution in stating that the chosen C/D ratio values are not arbitrary. The constant 'c' was set at 0.5 for our experiment due to its beneficial properties: it limits the visual discrepancy between users’ actual movements and the movements of virtual objects to a factor of 0.5, and it allows for feedback to start for objects weighing more than 0.61g. This setup enables significant evaluation points at 1 and 2 kg, with the feedback ratio decreasing from 1 to 0.5 as the object’s weight increases. These weights are reflective of everyday items, providing a relevant and practical frame of reference for our study.
Considered object masses we proposed to use three different masses: We proposed to use three different masses: 0.61, 1, and 2 kg, for all lifting technique conditions.
3.2 Subjects
A total of 20 right-handed dominant subjects (6 women and 14 men, with a mean age of 26.24 ± 7.98 SD) participated in the experiment. None of the subjects had any neurological, muscular, or cognitive disorders. The subjects' heights ranged from 156 to 185 cm, with a mean height of 173 ± 6.5 cm. Participants from various backgrounds, both within and outside the university, voluntarily agreed to take part in the study without compensation
3.3 Experimental setup and procedure
Subjects stood in front of a real 46 cm-high box, positioned in the experiment room to match a similar virtual box. They were at a distance that allowed them to comfortably and fully extend their arms to pick up, without bending, a real bottle in the real environment or a virtual bottle placed on the box in the virtual environment. Throughout the experiment, subjects were immersed in VR, wearing an HTC Vive Pro headset, a VR HMD equipped with an integrated g-sensor and gyroscope. Its resolution was 1440 × 1600 pixels per eye, with a 110-degree field-of-view, and a data output frequency of 120 Hz. Subjects’ movements were tracked by Vive cameras, monitoring the positions of the HMD, the hand controllers, and an additional hand tracker placed on the wrist of the dominant hand. These tracked elements provided visual feedback to the subjects about their hand position. In the case of lifting virtual bottles, a VR controller was used to capture the subjects’ arm motions, as well as the positions of the virtual bottles. For real bottles, wrist tracker data were used, and bottle displacements were also tracked with a Vive tracker fixed to it. For pseudo-haptic feedback, data from the tracked elements were used to compute modified positions in real-time and to visually provide adapted feedback to the subjects.
The task for the subjects was to lift real or virtual bottles upward, at their own pace, using a single joint arm movement, i.e., using their shoulder while keeping their elbow stationary. In all conditions, subjects were immersed in VR. They had to place the lifted object at a defined height (1 m upward), indicated by a virtual window (see Fig. 1). After a brief training phase, to test our lifting techniques, subjects were required to perform this task in three blocks of lifting trials, each consisting of fifteen trials (n = 45 in total). In each block, a unique lifting technique was employed, and three different object masses (0.6, 1, and 2 kg) were used for the bottles, each repeated five times. A pseudo-randomized order was applied within each block, and a Latin-square order was used to randomize the sequence of blocks between subjects.
3.4 Data collection and measurements
Raw data, including time, hand position, and hand velocity in three-dimensional space, were extracted from the VR hand controller for VR reference and direct weight conditions, and from VR hand trackers for real conditions. By applying a low-pass filter (Butterworth) with a cut-off frequency of 6 Hz to the velocity data, we extracted the smooth velocity profile of hand movement.
Regarding computed measurements, we calculated the following features from the raw data: Movement Duration (MD), Maximum Velocity (MaxV), and Time to Peak Velocity (TPV). Movement duration is defined as the time interval during which the hand velocity reaches 5% of its maximum. Time to peak velocity is the ratio of the acceleration duration to the total duration of the hand movement. Previous studies have shown that this feature is a good index of movement timing and an estimation of planned movement timing, which is drastically direction-dependent (Sciutti et al. 2012; Berret et al. 2008). Additionally, we considered the bell-shaped unimodal velocity profile. According to Abend et al. (1982) and Morasso (1981), when a person or a monkey moves their hand in a straight path, the speed profile of the hand is bell-shaped. However, as noted by Gentili et al. (2007) and Berret et al. (2008), hand velocity profiles differ significantly between upward, downward, and horizontal arm movements. In upward movements, the acceleration duration is shorter than the deceleration duration, whereas in horizontal movements (to the left or right), the acceleration phase is equivalent to the deceleration phase. Conversely, downward movements have longer acceleration phases than deceleration phases.
In our study, due to the upward direction of the lifting movement, we focused on the velocity profile of the subject's hand movement in the upward direction, which has an asymmetric bell-shaped velocity profile when the subject is not holding additional weight in their hands (acceleration is shorter than deceleration)
4 Results
In this section, we present a detailed report of the results obtained from our VR user experiment, analyzing both objective measurements (kinematic features) and subjective measurements [Borg CR10 questionnaire (Borg 1990)] of weight-lifting movement.
For objective measurements, our focus was primarily on movement duration, maximum velocity, and the shape of the velocity profile of the subjects’ hand movement in an upward direction. These were analyzed in relation to two experimental factors: the loaded weights (object masses) and the simulated models (virtual lifting conditions). In real environments, the effects of load on movement kinematics are typically represented by a smaller Max V, a more symmetric velocity profile shape, and an increased TPV. Accordingly, our aim was to determine whether the simulated models introduced in our study could visually induce the perception of loaded movement in VR.
4.1 MD
To compare the kinematics of simulated movement in the pseudo-haptic (P) condition with those in the real (R) and VR reference (V) conditions, we performed a two-way ANOVA on the MD, MaxV, and TPV measurements. Figure 2 displays the mean value and standard deviation of the MD for the weight-lifting task (0.6, 1, and 2 kg masses) under different conditions (real, VR reference, and pseudo-haptic). Furthermore, Table 1 indicates that the MD mean and SD in the VR reference condition remain nearly constant across the different weights (0.6, 1, 2 kg), while MD increases with increased weight in the real and pseudo-haptic conditions.
The results of the two-way ANOVA revealed that there was no significant interaction effect between weight and lifting conditions on MD (F (4, 162) = 2.04, p > 0.05). However, the main effect of weight on MD was statistically significant (F (2, 162) = 3.36, p < 0.05). Post-hoc differences, assessed using Tukey tests, revealed significant MD differences between 0.6 and 2 kg (p < 0.05). As illustrated in Fig. 2, with increasing weight, the movement duration increases in both the pseudo-haptic and real conditions.
On the other hand, the MD in the VR reference condition remained consistent. The two-way ANOVA analysis confirms that the main effect of the condition on MD was significant (F (2, 162) = 14.71, p < 0.05). Figure 2 shows that the movement duration for lifting 2 kg in the pseudo-haptic condition is significantly longer than in the real and VR reference conditions. Tukey's post hoc analysis revealed that the conditional differences between direct weight and VR reference are significant (p = 0.002), as well as between VR reference and real (p = 0.0001). However, Tukey's post hoc analysis did not indicate any significant difference in condition between pseudo-haptic and real (p = 0.16).
4.2 Max V
Following the previous observations, we thoroughly analyzed the maximum velocity (MaxV) for different weight-lifting movements in all conditions. Figure 3 and Table 1 illustrate the mean value and standard deviation (SD) of MaxV for the three lifted weights and the three conditions across all subjects. In the real condition, the MaxV of the lifting movement decreased as the weight increased. In contrast, in the VR reference condition, where MaxV remained consistent throughout the trials, the analysis of MaxV in the pseudo-haptic condition revealed a clear tendency to decrease MaxV when the lifted bottle was heavier. This pattern aligns with the results observed in the real condition. Interestingly, these results suggest that the C/D ratio model, applied in the directed-weight model, altered the MaxV scaling of the lifting movement in a manner consistent with real movements.
However, the two-way ANOVA analysis showed that the interaction effect of condition and weight on MaxV was not significant (F (4, 162) = 0.148, p > 0.05). Additionally, the main effect of weight on MaxV was also not significant (F (2, 162) = 0.86, p > 0.05). Furthermore, there was no statistical difference between the different conditions in terms of MaxV (F (2, 162) = 0.17, p > 0.05), and post hoc analysis did not reveal any difference between the real and pseudo-haptic conditions (p = 0.97).
4.3 Bell-shaped velocity profile
The timing of the movement in weight-lifting activities was evaluated through the ratio of time to peak velocity to total Movement Duration (TPV; see Data and Statistical Analysis for details). Figures 4, 5, and 6 depict the typical shapes of velocity profiles (normalized by duration and Maximum Velocity, MaxV) for the real, VR reference, and pseudo-haptic conditions, respectively. These profiles were produced by a typical subject under experimental conditions while lifting different weights (0.6, 1, and 2 kg).
In the real condition (as shown in Fig. 4), the movement is characterized by an asymmetric velocity profile when lifting lighter weights (e.g., m = 0.6 kg), where the acceleration phase is shorter than the deceleration phase. However, as the weight increases, this asymmetric velocity profile tends to become more symmetric, and the ratio of acceleration duration to total movement duration increases from 0.42 to 0.5. This change indicates that as subjects lift heavier loads, the duration of the acceleration phase increases relative to the total movement duration.
In the VR reference condition (as shown in Fig. 5), the velocity profile for an upward movement with an unloaded arm is observed to be asymmetric. In this condition, there is no evident effect of varying weights on the velocity profile. The velocity profile maintains an asymmetric bell shape, where the acceleration phase is shorter than the deceleration phase.
Finally, in the pseudo-haptic condition (as illustrated in Fig. 6), the velocity profiles indicate that subjects tend to increase the duration of acceleration as the delay between the virtual movement and the actual arm movement increases. Specifically, the acceleration phase of the velocity profile is prolonged with an increase in the virtual weight of the object.
4.4 TPV
Following the observation of a typical velocity profile, we statistically analyzed the TPV values with respect to different weights and conditions, as depicted in Fig. 7. According to this figure, TPV increases with the increase in weight under both real and pseudo-haptic conditions. In the real condition, based on mean TPV values, the acceleration duration ranged from 45% of the total movement duration while lifting a weight of 0.6 kg, to 49% during the lifting of 2 kg. In contrast, in the VR reference condition, TPV does not change with increasing weight. The mean and standard deviation of TPV are shown in Table 1. In the VR reference condition, the TPV value for 0.6, 1, and 2 kg is consistently 0.43, while in the real condition, the values are 0.46, 0.49, and 0.50, respectively.
Similarly, in the pseudo-haptic condition, TPV shows an increasing trend, with values of 0.44, 0.46, and 0.48 for 0.6, 1, and 2 kg, respectively. The result of the two-way ANOVA shows that the interaction effect of weight and condition on TPV was not significant (F (2, 162) =2.17, p > 0.05). Moreover, the ANOVA analysis revealed that TPV was not weight-dependent (F (2, 162) =2.54, p > 0.05). Despite this, there was a significant main effect of the condition on TPV (F (2, 162) =6.00, p<0.05). Tukey post hoc results confirm there is a significant difference between pseudo-haptic and VR reference (p < 0.05) and real and VR reference (p < 0.05). However, Tukey post hoc analysis did not show any statistical differences between real and pseudo-haptic. (p > 0.05). These results confirmed that pseudo-haptic model, can simulate a lifting behavior (pseudo-haptic) that is similar to the real one (real condition), in terms of kinematic features.
4.5 Borg CR10
Figure 8 displays the mean and standard deviation (SD) of responses from 20 participants to the Borg questionnaire under different conditions. Participants in the real and pseudo-haptic conditions reported greater tiredness compared to those in the VR reference condition. The mean and SD of participant tiredness for the real, pseudo-haptic, and VR reference conditions are 3.65 ± 2.66, 3.65 ± 2.41, and 0.95 ± 1.19 respectively. A one-way ANOVA revealed significant differences in fatigue values under different conditions (F (2, 58) = 10.624, p < 0.0001). According to Tukey's post hoc analysis, there was a significant difference in fatigue between the real and VR reference conditions (p < 0.05), as well as between the pseudo-haptic and VR reference conditions (p < 0.05). However, the post hoc analysis did not indicate a significant difference in fatigue between the pseudo-haptic and real conditions (p > 0.05).
5 Discussion
In this study, we employed the effect of pseudo-haptic feedback to induce the sense of virtual object’s weight in VR. By investigating the kinematic patterns of upward single-joint lifting movements, focusing on rotation around the shoulder, we have uncovered significant insights into how the Central Nervous System (CNS) adapts to different weight perceptions in VR. This approach marks a departure from previous studies that relied heavily on subjective measures like questionnaires, offering a more objective perspective on VR user experiences.
Our experimental setup in VR encompassed various conditions, including a reference condition, a pseudo-haptic condition, and a real-weight lifting condition. This approach was designed to frame our research problem, which focused on identifying new objective measurements for assessing the sense of presence in VR contexts, particularly when participants engage in lifting virtual objects. We chose kinematic features as the subject of our study since they allow for the quantitative evaluation of VR users’ lifting behavior in different conditions and in relation with the type of movement performed (Berret et al. 2008). Our study was influenced by key findings in previous research. For instance, Papaxanthis et al. (1998a, b) observed that the CNS considers gravity and movement direction in motor planning. They noted a symmetric velocity profile in downward movements, favoring gravity, and an asymmetric one in upward movements. Similarly, Gaveau et al. (2014) demonstrated that adding loads intensifies the effects of gravity, leading to more symmetric velocity profiles in upward arm movements. While these studies were set in real environments, we were intrigued to examine if these principles would be observable in a VR setting. Our goal was to explore the effects of both real and virtual weights on kinematic hand features in VR, seeking to understand if the CNS could replicate similar effects with virtual weights as it does with real weights in a physical environment.
In our study, the VR reference condition's velocity profile closely resembled non-loaded arm motion in real-world conditions, suggesting the light weight of the VR hand controller (0.31 kg) didn't significantly alter hand movement. This similarity in non-loaded arm motion between VR and real-world settings extends insights from previous research to the realm of VR.
Further, our experiments showed that increasing virtual weight in VR altered the velocity profile from asymmetric to more symmetric, mirroring real-world behaviors under load as observed in Gaveau et al. (2014). This shift, alongside changes in TPV and Max V, indicates that the VR environment can replicate the real-world lifting patterns using pseudo haptic. However, these patterns differed from those in the non-loaded VR reference condition, emphasizing the impact of simulated weight on movement dynamics in VR. Such findings underscore the delicate influence of virtual weights on user behavior and the CNS's adaptability in virtual environments.
Finally, we found Borg questionnaire results that are consistent with the kinematic features: indeed, in both real and pseudo-haptic conditions, participants felt the same level of fatigue. The results indicate that the pseudo-haptic condition not only altered the participants' CNS and changed their motion planning, thus affecting kinematic features, but it also led to trick subject sensations regarding their tiredness.
Our research, aligning with previous studies such as those by Kim et al. (2022) on VR weight perception, significantly advances the field by introducing a new objective method for measuring the sense of presence in VR, particularly when participants interact with virtual objects. By focusing on the analysis of kinematic data in each condition, coupled with an examination of Central Nervous System (CNS) movement planning in both real and VR environments with and without lifting real weights, our study has extended our understanding of objectively measuring the sense of presence beyond the scope of previous approaches. This includes studies conducted by Moosavi et al. (2023), Kumar et al. (2023), and Tano et al. (2015).
By examining the effects of both real and virtual weights on kinematic hand features in VR, our study revealed that the pseudo-haptic approach could replicate real-world lifting patterns, as evidenced by changes in velocity profile and kinematic features under varying weights. These results highlight the CNS's adaptability and the fine influence of virtual weights on user behavior, demonstrating the effectiveness of pseudo-haptic feedback in creating realistic weight perceptions in VR. This new knowledge is important for enhancing the sense of presence and interaction in virtual settings, offering valuable insights into the potential applications of VR in various fields.
6 Conclusions and future work
In summary, our findings provided a new evaluation method for weight perception in virtual reality by means of kinematic features and velocity profile of their hand movement. These findings will help VR developers to quantitatively measure their model and compare it with real world experiments. In addition, it confirms the result of previous studies that, by using concept of pseudo-haptic, it can be possible to simulate virtual object loads to VR, without needing to large and expensive hardware.
In the future, this methodology could include a variety of motions integral to the virtual reality experience. For example, pseudo-haptic feedback mechanisms could be enhanced for lateral, rotational, and complex multi-directional movements. In order to enhance realism and user immersion, different force distributions and tactile feedback are necessary for each type of motion.
In order to adapt the present approach to these diverse motions, it will be necessary to analyze and understand the physics and biomechanics involved. Additionally, iterative testing and user feedback will be necessary to fine-tune the pseudo-haptic feedback parameters for each motion type, ensuring an intuitive and seamless user experience.
This extended application will enable users to interact with various virtual objects and spaces with a richer and more engaging sensory experience by extending the range and fidelity of interactions within virtual environments. We intend to explore and investigate this extension and investigation as part of our ongoing work in virtual reality and pseudo-haptic feedback development.
By highlighting velocity profile and TPV changes in VR, developers can modify sensory feedback to achieve desired perceptions. As an example, if the weight of a virtual object is consistently underestimated, the visual or haptic cues can be adjusted accordingly. Current methods provide us with an overview, but proposed metrics allow us to focus on more specific and effective interventions.
Our efforts could have a lasting impact in the near future, notably through collaborations with VR developers and industry professionals for dedicated purposes, where transmitting the feeling of weight in virtual and safe environments would be beneficial, e.g., digital twins of industry, surgeries etc. This could be particularly beneficial in applications like VR-based physical therapy, or ergonomic design and analysis (Haj Mahmoud et al. 2021), where understanding the nuances of user motion is critical, or in VR training simulations where the accurate perception of weight is crucial. In collaboration with such organisations, we could refine and adapt our evaluation method, as well as incorporate it into existing VR development pipelines for developers, and conduct real-world usability and fatigue evaluation studies.
Data availability
The data that support the findings of this study are available from the corresponding author, Mahdiyeh Sadat Moosavi, upon reasonable request.
References
Abend W, Bizzi E, Morasso P (1982) Human arm trajectory formation. Brain: a J Neurol 105:331–348
Agronin ML (1987) The design of a nine-string six-degree-of-freedom force-feedback joystick for telemanipulation. In: Proceedings of the workshop on space telerobotics, vol 2
Aman J, Lu C-H, Konczak J (2010) The integration of vision and haptic sensing: a computational & neural perspective. Integration 96:98
Araujo B et al (2016) Snake charmer: physically enabling virtual objects. s.l., s.n., 218–226
Argelaguet F, Andujar C (2013) A survey of 3D object selection techniques for virtual environments. Comput Graph 37:121–136
Atkeson CG, Hollerbach JM (1985) Kinematic features of unrestrained vertical arm movements. J Neurosci 5:2318–2330
Berret B et al (2008) The inactivation principle: mathematical solutions minimizing the absolute work and biological implications for the planning of arm movements. PLoS Comput Biol 4:e1000194
Bock O (1990) Load compensation in human goal-directed arm movements. Behav Brain Res 41:167–177
Borg G (1990) Psychophysical scaling with applications in physical work and the perception of exertion. Scand J Work Environ Health 16(Suppl 1):55–58. https://doi.org/10.5271/sjweh.1815
Burdea GC (1999) Haptic feedback for virtual reality. s.l., s.n., 17–29
Choi I et al (2017) Grabity: a wearable haptic interface for simulating weight and grasping in virtual reality. s.l., s.n., 119–130
Dominjon L et al (2005) Influence of control/display ratio on the perception of mass of manipulated objects in virtual environments. s.l., s.n., 19–25
Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415:429–433
Gaveau J et al (2014) Energy-related optimal control accounts for gravitational load: comparing shoulder, elbow, and wrist rotations. J Neurophysiol 111:4–16
Gentili R, Cahouet V, Papaxanthis C (2007) Motor planning of arm movements is direction-dependent in the gravity field. Neuroscience 145:20–32
Haj Mahmoud O et al (2021) A neural networks approach to determine factors associated with self-reported discomfort in picking tasks. Human Factors, pp 00187208211047640
Heo S, Chung C, Lee G, Wigdor D (2018) Thor's hammer: an ungrounded force feedback device utilizing propeller-induced propulsive force. s.l., s.n., p 1–11
Hirata Y and Sato M (1992) 3-dimensional interface device for virtual work space. s.l., s.n., pp 889–896
Hoffman DS, Strick PL (1993) Step-tracking movements of the wrist. III. Influence of changes in load on patterns of muscle activity. J Neurosci 13:5212–5227
Jauregui DAG et al (2014) Toward" pseudo-haptic avatars": modifying the visual animation of self-avatar can simulate the perception of weight lifting. IEEE Trans Visual Comput Graphics 20:654–661
Kim J, Kim S, Lee J (2022) The effect of multisensory pseudo-haptic feedback on perception of virtual weight. IEEE Access 10:5129–5140
Kumar N, Gourishetti R, Vasudevan MK, Manivannan M (2023) Mechanics of pseudo-haptics in virtual reality: weight perception. s.l., s.n., pp 1093–1104
Lécuyer A (2009) Simulating haptic feedback using vision: a survey of research and applications of pseudo-haptic feedback. Presence: Teleoper Virt Environ 18:39–53
Massie TH, Salisbury JK et al. (1994) The phantom haptic interface: a device for probing virtual objects. s.l., s.n., pp 295–300
Moosavi MS et al (2023) Weight perception analysis using pseudo-haptic feedback based on physical work evaluation. Front Virt Real 4:973083
Morasso P (1981) Spatial control of arm movements. Exp Brain Res 42:223–227
Nakakoji K, Yamamoto Y and Koike Y (2010) Toward principles for visual interaction design for communicating weight by using pseudo-haptic feedback. s.l., s.n., pp 1–6
Nakakoji K, Yamamoto Y, Matsubara N and Koike Y (2011) Tciex: an environment for designing and experiencing a variety of visuo-haptic sensory conflicts. s.l., s.n., pp 23–26
Nisar S et al (2018) Effects of different hand-grounding locations on haptic performance with a wearable kinesthetic haptic device. IEEE Robot Autom Lett 4:351–358
Pacchierotti C et al (2017) Wearable haptic systems for the fingertip and the hand: taxonomy, review, and perspectives. IEEE Trans Haptics 10:580–600
Palmerius KL, Johansson D, Höst G and Schönborn K (2014) An Analysis of the Influence of a Pseudo-haptic Cue on the Haptic Perception of Weight. s.l., s.n., pp 117–125
Papaxanthis C, Pozzo T, Popov KE, McIntyre J (1998a) Hand trajectories of vertical arm movements in one-G and zero-G environments evidence for a central representation of gravitational force: evidence for a central representation of gravitational force. Exp Brain Res 120:496–502
Papaxanthis C, Pozzo T, Stapley P (1998b) Effects of movement direction upon kinematic characteristics of vertical arm pointing movements in man. Neurosci Lett 253:103–106
Rietzler M, Geiselhart F, Frommel J, Rukzio E (2018a) Conveying the perception of kinesthetic feedback in virtual reality using state-of-the-art hardware. s.l., s.n., pp 1–13
Rietzler M, Geiselhart F, Gugenheimer J, Rukzio E (2018b) Breaking the tracking: enabling weight perception using perceivable tracking offsets. s.l., s.n., pp 1–12
Runeson S, Frykholm G (1981) Visual perception of lifted weight. J Exp Psychol Hum Percept Perform 7:733
Samad M et al (2019) Pseudo-haptic weight: Changing the perceived weight of virtual objects by manipulating control-display ratio. s.l., s.n., pp 1–13
Sciutti A et al (2012) Visual gravity influences arm movement planning. J Neurophysiol 107:3433–3445
Sciutti A et al (2010) Predicted sensory feedback derived from motor commands does not improve haptic sensitivity. Exp Brain Res 200:259–267
Suzuki Y, Kobayashi M (2005) Air jet driven force feedback in virtual reality. IEEE Comput Graph Appl 25:44–47
Tano S et al (2015) Quantitative analysis of pseudo-haptics based on three types of hand form and two phases of perception. s.l., s.n., pp 1–6
Ujitoko Y, Ban Y (2021) Survey of pseudo-haptics: haptic feedback design and application proposals. IEEE Trans Haptics 14:699–711
Weiss M, Wacharamanotham C, Voelker S and Borchers J (2011) FingerFlux: near-surface haptic feedback on tabletops. s.l., s.n., pp 615–620
Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a presence questionnaire. Presence 7:225–240
Yu R, Bowman DA (2020) Pseudo-haptic display of mass and mass distribution during object rotation in virtual reality. IEEE Trans Visual Comput Graphics 26:2094–2103
Zenner A and Krüger A (2019) Drag: on: a virtual reality controller providing haptic feedback based on drag and weight shift. s.l., s.n., p 1–12
Acknowledgements
This work was supported by French government funding managed by the National Research Agency under the Investments for the Future program (PIA) grant ANR-21- ESRE-0030 (CONTINUUM). The authors want to thank Jean-Luc Martinez who proposed to use the C/D ratio as a weight metaphor and Jérémy Plouzeau who made the first implementation of this metaphor.
Author information
Authors and Affiliations
Contributions
This study was conceptualized by Mahdiyeh Sadat Moosavi, Pierre Raimbaud , Christophe Guillet, and Frédéric Mérienne. Mahdiyeh Sadat Moosavi, Pierre Raimbaud , Christophe Guillet, and Frédéric Mérienne contributed to the development of the study's protocols and conditions. An application for virtual reality and a system for recording data were developed by Pierre Raimbaud. Mahdiyeh Sadat Moosavi organized the database and performed the data analysis and statistical analysis. The first draft of the manuscript was written by Mahdiyeh Sadat Moosavi, and Pierre Raimbaud. Some parts of the manuscript were written by Christophe Guillet, and Frédéric Mérienne.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Moosavi, M.S., Raimbaud, P., Guillet, C. et al. Enhancing weight perception in virtual reality: an analysis of kinematic features. Virtual Reality 28, 72 (2024). https://doi.org/10.1007/s10055-024-00948-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10055-024-00948-7