Abstract
We study whether the vibrotactile and thermal senses are independent in terms of information transmission. In Exp. 1, we estimated the respective information transmission capacities of vibrotactile and thermal stimuli. In Exp. 2, we measured the information transfer (IT) of vibrotactile-thermal multimodal stimuli. We compare the IT values obtained in the two experiments and demonstrate that approximately 90% of the information encoded through the two sensory modalities is preserved when they are combined. This result can contribute to the design of multimodal haptic stimuli for various user-interactive purposes.
This study was supported by a grant 21ZS1200, Fundamental Technology Research for Human-Centric Autonomous Intelligent Systems, from the Electronics and Telecommunications Research Institute (ETRI) of Korea.
J. Park and J. Kim—Equally contributed to this work.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
The essential role of tactile devices is to deliver information, and the information is often abstract and categorical [2, 6]. Research has sought effective methods in improving the amount of information that can be transferred by tactile communication. In this paper, we report a preliminary study that quantifies the extent to which the information transmission capability is preserved when two different types of tactile stimuli, vibrotactile and thermal, are factorially combined into multimodal stimuli.
A large number of previous studies measured the information transfer (IT) for vibrotactile stimuli. When a single vibration actuator is used, the highest IT value reported is 3.06 bits [17]. When multiple vibration actuators are used, the highest IT value reported is 7.02 bits [7]. In comparison, the IT values reported for thermal stimuli are rare. The highest IT value reported for the identification of thermal patterns is 2.13 bits [11]. Furthermore, there have been attempts to test the IT for multimodal stimuli. Shim et al. [9] measured the IT values for the simultaneous and sequential patterns of vibration and wind. The IT values were 1.70 bits (simultaneous) and 3.29 bits (sequential), where the IT values for individual vibration and wind were 1.76 bits and 1.23 bits, respectively. A recent comprehensive review on the IT of tactile displays is available in [13].
Presenting vibrotactile and thermal stimuli simultaneously is a promising approach to enriching tactile interaction [14, 16]. If the two tactile channels are sufficiently independent, vibrotactile-thermal stimuli may offer a substantially greater information transmission capacity (ITC). This research hypothesis was investigated by two perceptual experiments. In Exp. 1, we measured the individual ITCs of the vibrotactile and thermal channels. Only frequency was varied for the vibrotactile stimuli, and the number of stimuli was increased from 3 to 7. For the thermal stimuli, temperature change direction and rate were controlled. The number of stimuli was between 5 and 9. In Exp. 2, we factorially combined the optimum number of vibrotactile and thermal stimuli to make multimodal stimuli and obtained their estimated IT values (\(IT_{est}\)). Then, we examined whether the unimodal \(IT_{est}\)s and the multimodal \(IT_{est}\) satisfy the additivity, which means the complete perceptual independence of the two channels.
Our study can contribute to 1) the decision of optimal vibrotactile or thermal stimuli for user interaction purposes, 2) the design of vibrotactile-thermal stimuli that are highly recognizable for information and communication purposes.
2 Exp. 1: Unimodal Stimuli
We estimated the ITCs for both vibrotactile and thermal stimuli. The experiments reported in this paper were approved by the Institutional Review Board at POSTECH (PIRB-2021-E054).
2.1 Methods
Apparatus. We implemented a device for vibrotactile and thermal stimulation (Fig. 1). Its vibrotactile module consisted of a mini-shaker (Bruel & Kjær, 4810) and a power amplifier (Bruel & Kjær, 2718). The thermal module included a Peltier module (20 \(\times \) 20 \(\times \) 3.1 mm; MULTICOMP, MCPE1-07106NC-S), three DC motor drivers, a thermistor (SEMITEC, 223Fu3122-07U015), a microcontroller (Arduino Due), and a water-cooler. The temperature change rate of the Peltier module was controlled within ±4 ℃/s (0.1 ℃ average error) by PID control. The thermal module was put on the vibrotactile module using an acrylic connector. To block vibration propagation from the water-cooler, a silicon damper attenuating noise over 95% was placed between the tactile module and the cooler.
Experimental Conditions. Figure 2 depicts the time profiles of vibrotactile and thermal stimuli. We designed three conditions for vibrotactile stimuli with the number of stimuli \(N=\) 3, 5, and 7 (see Table 1). The frequencies of the vibrotactile stimuli were varied from 40 400 Hz in equidistant intervals on a logarithmic scale, stimulating RA 1 and PC channels [1]. The adjacent frequency differences were larger than the vibrotactile Just Noticeable Difference (JND) (approximately 18% [8]). The intensity of each stimulus was scaled to be 33 dB from the detection threshold on the finger at its frequency [3]. All of the vibration stimuli had the same duration of 1.5 s.
We also designed three conditions for thermal stimuli. Their baseline temperature was 32 ℃. Considering the heat pain threshold, we varied the temperature change rate within ±4 ℃/s [15]. Given the number of stimuli \(N=\) 5, 7, and 9, the change rates were evenly spaced in a linear scale, as shown in Table 1. The temperature changes from the baseline were higher than the difference thresholds for both warm and cool stimuli (+0.23 and −0.14 ℃ at 33 ℃ [12]).
The thermal stimuli had three phases. In the beginning, the temperature changed to the target temperature at a given change rate over 1.5Â s. In the second phase, the temperature remained for 0.5Â s for stability. Then, the temperature returned slowly to the baseline temperature over 8Â s.
Participants. The experiment had a between-subjects design. Each experimental condition was tested with seven participants each (see Table 1 for their demographics). No participants participated in the conditions of the same stimulus type. Before the experiment, the participants were informed of the experiment’s goals and procedure via a written document, and then they signed a consent form. They were paid at the rate of KRW 10,000 (\(\approx \) USD 9) per hour.
Procedure. Participants put their right index fingertip on the Peltier module. Note that vibration stimuli were presented through the Peltier module (Fig. 1). They manipulated a experiment program using their left hand. They wore noise-canceling headphones which played white noise to block environmental sounds.
The thermal conditions had additional constraints: 1) The room temperature was controlled to 25 ℃; 2) The Peltier module maintained its temperature to 32 ℃ when there was no stimulus; 3) Participants were asked to put their fingertips on the Peltier module for 2 s before stimulus onset to make their contact skin temperature to 32 ℃; and 4) They were also asked to take off their fingers from the Peltier module after receiving 1.5 s of thermal stimulus.
To complete each experimental condition, participants went through five sessions: training 1, practice 1, training 2, practice 2, and main session. In the training session, participants experienced the haptic stimuli assigned to the condition and wrote down their identification criteria on a sheet of paper for at least 5 min. In the practice session, participants perceived each stimulus and were asked to find the correct stimulus. Correct answer feedback was visually provided on the experiment program. Participants could freely revise their criteria notes. These training and practice sessions were repeated once again to ascertain and deepen their identification criteria. In the main session, participants’ task was the same as that of the practice session, but there was no correct answer feedback. Participants could feel the stimulus as many times as they wanted.
Table 1 shows the numbers of repetition made for each stimulus. For each experimental condition, the stimuli were randomly distributed within four sub-sessions. A break of approximately 1 min was provided between sub-sessions. The experiment took 30 to 120 min, depending on the condition.
Data Analysis. For each experimental result, we computed the confusion matrix from the results of each participant and obtained its IT using the standard formulae in [13]. These individual \(IT_{est}\)s were used to compute the conditions’ mean \(IT_{est}\) and additional statistical tests.
2.2 Results and Discussion
The confusion matrices obtained in the experiment are shown in Fig. 3 for the vibrotactile stimuli and Fig. 4 for the thermal stimuli. The \(IT_{est}\)s are presented in Fig. 5 along with the maximum possible values of IT.
Vibrotactile Stimuli. The \(IT_{est}\)s were 1.51Â bits (SDÂ =Â 0.07Â bits), 1.48 bits (SDÂ =Â 0.40Â bits), and 1.56Â bits (SDÂ =Â 0.23Â bits) for \(N=3\), 5, and 7. We conducted one-way between-subjects ANOVA to analyze the effects of the number of stimuli on \(IT_{est}\). The number of stimuli did not significantly affect \(IT_{est}\) (\(F_{2,18}=0.157,p=0.856\)). Thus, we can conclude that the \(IT_{est}\)s were saturated to 1.52Â bits (the mean of the three \(IT_{est}\)s; 2.9 stimuli).
The vibrotactile stimuli had only one design variable of frequency. The \(IT_{est}\)s are comparable to those reported in the literature in similar settings [13].
Thermal Stimuli. The \(IT_{est}\)s were 1.72Â bits (SDÂ =Â 0.36Â bits), 1.69Â bits (SDÂ =Â 0.26 bits), and 1.75Â bits (SDÂ =Â 0.36Â bits). According to one-way between-subjects ANOVA, there was no significant difference in \(IT_{est}\) between the number of stimuli (\(F_{2,18}=0.068,p=0.935\)). Hence, the \(IT_{est}\)s were saturated to 1.72Â bits (about 3.3 stimuli) regardless of the number of stimuli.
For further analysis, we calculated the percent-correct (PC) scores for different temperature change directions (cooling, no change, and warming). The results were 90.8, 88.5, and 84.3% for \(N=5\), 7, and 9, respectively. Thus, the participants could identify the direction of temperature change relatively well. Furthermore, the participants identified the cool stimuli with higher accuracies than the warm stimuli (see Fig. 4) because cold-sensitive thermoreceptors are distributed more densely than warm receptors in the human skin [5].
3 Exp. 2: Multimodal Stimuli
The aim of this experiment was to estimate the IT of multimodal stimuli combining vibrotactile and thermal stimuli, and then test the additivity law for IT to assess the level of perceptual independence between the two haptic channels.
3.1 Methods
Experimental Conditions. The experimental conditions were designed by combining the vibrotactile and thermal stimuli selected from Exp. 1 (Fig. 2). For the vibrotactile stimuli, \(IT_{est}\)s were saturated to 1.52 bits (2.9 stimuli). Thus, we chose three stimuli (slightly larger than the vibrotactile channel capacity) with frequencies 40, 126, 400  Hz. For the thermal stimuli, \(IT_{est}\)s were saturated to 1.72 bits (3.3 stimuli). We selected four thermal stimuli, also slightly larger than the thermal channel capacity, as −4, −2, 0, and +4 ℃/s (very cold, cold, no change, and warm). We included two cold stimuli because the cold stimuli were identified better than the warm stimuli in Exp. 1. Consequently, we used 12 experimental stimuli (3 vibrotactile and 4 thermal stimuli). The two types of stimuli were presented simultaneously with the same durations (1.5 s).
Participants and Procedure. Seven participants (3 males and 4 females; age \(M=\) 22 years and \(SD=\) 0.58) who did not participate in Exp. 1 took part in this experiment. The participants’ task and experimental procedure were the same as those of Exp. 1. The number of repetitions was 12 for each multimodal stimulus. The experiment took up to 120 min.
3.2 Results and Discussion
The confusion matrix of the multimodal stimuli is shown in Fig. 6 (left). The \(IT_{est}\) was 3.06 bits (SD = 0.24 bits). This value is slightly less than the sum (3.24 bits) of the \(IT_{est}\)s of the vibrotactile and thermal stimuli obtained in Exp. 1.
For more detailed analysis, we show two confusion matrices for each modality Fig. 6 (right). Their \(IT_{est}\)s were 1.37 and 1.62 bits for the vibrotactile and thermal stimuli. These numbers are reduced to 90.0% and 94.2%, respectively, compared to the \(IT_{est}\) of the individual modalities.
Singhal et al. [10] found that the perception of thermal stimuli was affected by concurrent vibrotactile cues in terms of IT. Green [4] also reported a multi-sensory effect between the tactile and thermal stimuli: a cold stimulus impairs the sensitivity of high-frequency vibration and a warm stimulus lowers the vibrotactile sensitivity in a wide frequency range. Shim et al. [9] showed that the simultaneous presentation of wind and vibration stimuli remarkably reduce the accuracy and the IT of each unimodal stimuli compared to the sequentially combined ones. Despite such potential crossmodal interferences, the \(IT_{est}\) of the multimodal stimuli achieved over 90% of the information transmission performance of the individual stimuli. Therefore, the two modalities may not be completely perceptually independent, but combining them increases the identification ability of tactile stimuli to a large extent.
4 Conclusions
In this paper, we evaluated the IT of vibrotactile and thermal stimuli in two cases. The first case was when the two stimuli were provided separately. The \(IT_{est}\)s were saturated to 1.52Â bits (perfect recognition of 2.9 stimuli) and 1.72Â bits (perfect recognition of 3.3 stimuli) for the vibrotactile and thermal stimuli. The second case was when the two types of stimuli were provided simultaneously to the same body location. The \(IT_{est}\) was 3.06Â bits (perfect recognition of 8.3 stimuli), which is approximately 90% of the sum of the individual modality \(IT_{est}\)s (3.24Â bits). This result indicates that the degree of crossmodal interference between vibrotactile and thermal senses is approximately 10% in terms of IT, suggesting a high potential of combined vibrotactile and thermal stimuli for effective information delivery.
This study was preliminary in nature. Its positive results encourage us to explore this research space with improved depth and breadth. Our results can also be applied to contexts demanding high information transmission like gaming controllers and in-car infotainment systems. Our future work may consider introducing more tactile modalities, e.g., impact, and comparing the IT of their combination.
References
Bolanowski, S.J., Gescheider, G.A., Verrillo, R.T., Checkosky, C.M.: Four channels mediate the mechanical aspects of touch. J. Acoust. Soc. Am. 84(5), 1680–1694 (1988)
Brown, L., Brewster, S., Purchase, H.: A first investigation into the effectiveness of tactons. In: Proceedings of World Haptics Conference, pp. 167–176 (2005)
Goble, A.K., Collins, A.A., Cholewiak, R.W.: Vibrotactile threshold in young and old observers: the effects of spatial summation and the presence of a rigid surround. J. Acoust. Soc. Am. 99(4), 2256–2269 (1996)
Green, B.G.: The effect of skin temperature on vibrotactile sensitivity. Percept. Psychophys. 21(3), 243–248 (1977). https://doi.org/10.3758/BF03214234
Hensel, H.: Cutaneous thermoreceptors. In: Iggo, A. (ed.) Somatosensory System, vol. 2, pp. 79–110. Springer, Cham (1973). https://doi.org/10.1007/978-3-642-65438-1_4
MacLean, K.E.: Foundations of transparency in tactile information design. IEEE Trans. Haptics 1(2), 84–95 (2008)
Park, G., Cha, H., Choi, S.: Haptic enchanters: attachable and detachable vibrotactile modules and their advantages. IEEE Trans. Haptics 12(1), 43–55 (2019)
Pongrac, H.: Vibrotactile perception: examining the coding of vibrations and the just noticeable difference under various conditions. Multimedia Syst. 13(4), 297–307 (2008). https://doi.org/10.1007/s00530-007-0105-x
Shim, Y.A., Lee, J., Lee, G.: Exploring multimodal watch-back tactile display using wind and vibration, pp. 1–12. ACM, New York (2018)
Singhal, A., Jones, L.A.: Perceptual interactions in thermo-tactile displays. In: Proceedings of World Haptics Conference, pp. 90–95 (2017)
Singhal, A., Jones, L.A.: Creating thermal icons—a model-based approach. ACM Trans. Appl. Percept. 15(2), 1–22 (2018)
Stevens, J.C., Choo, K.K.: Temperature sensitivity of the body surface over the life span. Somatosens. Mot. Res. 15(1), 13–28 (1998)
Tan, H.Z., Choi, S., Lau, F.W.Y., Abnousi, F.: Methodology for maximizing information transmission of haptic devices: a survey. Proc. IEEE 108(6), 945–965 (2020)
Wilson, G., Brewster, S.A.: Multi-moji: combining thermal, vibrotactile, and visual stimuli to expand the affective range of feedback. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 1743–1755 (2017)
Yarnitsky, D., Ochoa, J.L.: Studies of heat pain sensation in man: perception thresholds, rate of stimulus rise and reaction time. Pain 40(1), 85–91 (1990)
Yoo, Y., Lee, H., Choi, H., Choi, S.: Emotional responses of vibrotactile-thermal stimuli: effects of constant-temperature thermal stimuli. In: Proceedings of International Conference on Affective Computing and Intelligent Interaction, pp. 273–278 (2017)
Yoo, Y., Regimbal, J., Cooperstock, J.R.: Identification and information transfer of multidimensional tactons presented by a single vibrotactile actuator. In: Proceedings of World Haptics Conference, pp. 7–12 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2022 The Author(s)
About this paper
Cite this paper
Park, J., Kim, J., Park, C., Oh, S., Park, J., Choi, S. (2022). A Preliminary Study on the Perceptual Independence Between Vibrotactile and Thermal Senses. In: Seifi, H., et al. Haptics: Science, Technology, Applications. EuroHaptics 2022. Lecture Notes in Computer Science, vol 13235. Springer, Cham. https://doi.org/10.1007/978-3-031-06249-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-06249-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06248-3
Online ISBN: 978-3-031-06249-0
eBook Packages: Computer ScienceComputer Science (R0)