Abstract
Design and modification of human-robot collaboration workspace requires analysis of the safety of systems. Generally, the safety analysis process of a system commences with conducting a risk assessment. There exists a number international standards for design robotics work cells and collaborative shared workspaces. These guidelines expound on principles and measures to identify hazards and reduce risks. Measures of risk reductions include eliminating hazards by design, safeguarding, and providing supplementary protective measures such as user training. This study analyzed the technical feasibility and industrial readiness of Virtual Reality (VR) technology for safety training in manufacturing sector. The test case of a VR-based safety training application is defined in the human-robot collaboration pilot-line of diesel engines. The Analytic Hierarchy Process method was utilized for conducting a quantitative analysis of the survey with ten experts. The participants performed the importance rating with respect to two hierarchy level criteria. Regarding the evaluation of safety training methods in a human-robot collaboration environment, two alternatives of traditional and Virtual Reality -based training are compared. The results indicates that the VR-based training is valued over the traditional method, with a scored proportion of approximately 65 percent over 35 percent.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Today, the industry is tackling with changing cutomer needs that leads to the increased need of mass-customisation and personification, and to the reconfiguration of production lines. In order to implement product customization and reconfigurability, there is a need to incorporate human dexterity and intelligence with machine accuracy and repeatability by developing human-robot collaboration (HRC) concepts. In order to keep the operator safe and overall system productive, there is a need to tackle safety risks for the operator. However, the continuous change, risks mitigation and training is expensive and time consuming.
Among the emerging technologies, Virtual Reality (VR) became prevalent in the gaming industry and the social community (E.g., the advent of metaverse). Principally, Reality-Virtuality Continuum consists of two environments category, whereas one defined as pure real environment and another as virtual environment. VR provides solely of virtual objects for user where it utilizes computer graphic simulation for creating objects where it could be seen with monitor-based displays, partially or fully immersive head-mounted display [1]. As these categories is expanding and more hardware and implementation are introduced, the term Extended Reality(XR) is defined to consist of all these hardware in Reality-Virtuality continuum. While the ICT sector is advancing in the use of XR, the manufacturing sector is hesitant due to technical maturity and cost of use.
This paper proceeds to study the technical feasibility of VR technology as a tool in the safety training process and assess industry readiness level with a HRC use case.
2 Related Work
The safety of an operator in close proximity of robot system is a challenge in industry, regarding interfaces, interaction, trust, and technical validation of these systems, several comprehensive studies were explored in [2,3,4,5]. Virtual reality has been used in assembly applications in manufacturing and consists of different categories such as simulation design, as discussed in the literature [6, 7]. To examine different possible interactions, a study was conducted by [12] to compare mouse-based, phantom haptic, two configurations with the markless motion capture method, and video-based training. The result demonstrated no notable differences between these five groups’ experiments. Another finding was that haptic feedback in case of the collision did not improve learning transfer. In parallel, Augmented Reality has been largely investigated in the same categories. Multiple researchers studied applications where the goal was to provide augmented information on 2D surfaces with projector-based and camera systems. These systems monitor the operator’s movements and dynamically project working areas on a table [8,9,10]. Various studies were conducted to contribute to training applications for providing information, instruction, and visual presentation. In the following sub-sections, literature on VR in training, and safety training is presented.
2.1 Virtual Reality Training
Due to the demand of the manufacturing industry, integrating digital manufacturing and technology is growing rapidly. The users are needed to be trained for changing environments while providing relevant information for the tasks at hand. As virtual reality can provide a safe environment for training purposes various studies have been carried out. Gorecky [11] targeted the two main challenges: training system design,,and training content generation from data management that is required for virtual training. They proposed a methodology consisting of VISTRA’s architect, knowledge platform, training simulator, and knowledge sharing center for the content generation of the virtual training system based on semantic technologies.
Training users in online mode (factory floor) is time-consuming and difficult. Therefore, game engines such as UNITY are playing a big role in the advancement of the training system. Pérez [12] implemented a training system, where lab environments such as 3D scanners were mapped into the real environment and simulated the kinematic and dynamic of the robot and VR could provide a promising solution for companies regarding costs, standardization, usability, and training time. In [13] study is conducted where the operator worked in close proximity of an industrial robot where VR was used to increase the immersiveness. From the first study, the survey result depicted that assembly tasks with their hands (VR controller) without the presence mass of components improved the feeling of presence and immersion. Later on Mastas et al. [14] conducted the experiment consisting of two techniques: passive, which aimed to prepare proactive human behavior by utilizing cognitive help via VR, and active approach where the intention was to examine robot adaptive behavior in case of deceleration of trajectories and modification of them.
2.2 Virtual Reality Safety Training
Virtual reality safety training has been studied in fewer industry sectors; however, they had common reasoning behind, reducing injuries concerning user safety while confronting hazardous events. The same concept of training, whereas operators are trained to act when facing hazardous event, and how the operator could react on such circumstances. Buttussi [15] conducted a study in the aviation safety domain for the door opening process. The result demonstrated the significant usability of VR headsets over printed materials. Additionally, the study displayed that VR headset applications led to more engagement and satisfaction compared to other methods. Moreover, a similar result was carried out in the study of effectiveness compared between traditional and VR training approaches in a generic environment of power plants regarding theoretical and practical learning purposes [16].
Among research communities in collaboration with companies, safety training was examined in the construction and mining industry, where operator’s safety could have been violated more frequently. Zheng [17] developed a use case that demonstrates the effectiveness of safety training with respect to memorizing critical points when facing hazardous situations in a construction site of urban cities. In addition, improving training contents of safety for construction professionals and workers are studied in [18] and performed empirically experiment that result in user’s identification of hazardous events. Additionally, recent research was carried out on developing a framework to study immersive learning in the safety training process of mine rescuers [19]. Their model proposed multiple criteria such as trainee experience with gaming and technology, in addition to the learning experience with real case scenarios, utilizing VR features, and understandability of VR usability. The study demonstrated the positive impact of such criteria in immersive VR-based training. Tichon in [20] performed comprehensive literature based on the effectiveness of virtual reality safety training in the mining industry regarding different skills such as problem-solving, decision making, cognition.
3 Research Methods, Material and Approach
The research methods in this study consist of the definition of a case study, and assessment of the approach with AHP method. The aim of the study is to include the acknowledgement of safety risks and guidelines for operator, and how to avoid or react in case of hazardous events. The VR training will guide and explain to the user what events can trigger the safety system and lead to stopping a robot. The overall goal is to reduce the production stops that origin from the user errors.
The test case is defined in this study relates to the HRC pilot line for assembly of a diesel engine with the industrial robot ABB IRB 4600. In this regard, a VR-based safety training is presented in our research laboratories to investigate the possibility of reducing user mistakes and hazardous events while facing robots in close proximity. The concept of this training was introduced in [21], and 2D snapshot of environment is depicted in Fig. 1.
In this study, a VR HMD with high resolution selected as user required to experience fully immersed environment with realistic detail of each component of laboratory. The 3D models were rendered carefully to represent replica of real components. For an instance, the engine’s CAD models were 3D scanned, and robot models were used from simulation software library with detail rendering. The users can interact with VR HMD controllers to interact with virtual components for movement and assembly, where virtual components contain game engine Physics features. In addition, it is required to walk in the full-scale simulation area to perceive safety distances and working areas.
The hardware selected for this application are HTC VIVE Pro headset with resolution of 2880 x 1600 pixels, refresh rate of 90 Hz and field of view of 110 degrees, and gaming desktop with 32 GB of RAM, NVIDIA Geforce GTX 1060 graphic card. The reason for selecting wireless HMD is that the virtual environment representation is on 1:1 scale of laboratory to improve feel of presence of an operator. HTC VIVE pro controllers have haptic feedback feature, but this feature has not integrated to the system as psychological effect of that in training system requires investigations. The UNITY game engine v 2020.3 is utilized to create visual elements such as working area borders and User Interface such as instruction for acknowledging of different safety measures. For creating more realistic laboratory presentation, manufacturing simulation’s software Visual Components v4.4 and Blender adds-on for animation rendering of robot’s movements were used.
4 Assessment Approach
Investigating technology feasibility and integration of technical development for manufacturing follows the path between the research community and SMEs and finally adaptation of it in the industry sector. Regarding this, technical development should go through a process to get mature enough to be accepted in the industry. Tirmizi in [22] provided an assessment method to analyze use cases in the development process and steps they have taken toward maturity. Pöysäri [23] proposed a feasibility analysis concept of reconfigurable pilot lines. In this study, KPIs in the aforementioned studies were employed, which are important and relatable for the VR training concept and conduct a technology feasibility assessment. An assessment of this multi-criteria was carried out through the AHP method introduced by [24]. Performing this analysis could be beneficial in the decision-making of adaptation VR-based safety training based on these criterions.
At first, the definition of the overall goal is to evaluate safety training in a HRC environment. Two alternatives of training was chosen: traditional training (i.e., paper-based, video instruction, and instructor-based training) and VR based training using fully immersive VR HMD devices. Following the hierarchy Fig. 2, at the second level of criteria, twelve KPI’s are considered, which is respect to first level criteria, use case, and technical criteria. First, use case criteria focus on managerial level, and there are common factors that companies consider when using different solutions. These criteria are as follows: hardware cost, development cost, implementation cost, ergonomic, and lifecycle. Second, technical criteria focus on technical aspects consist of seven KPI’s where there are studied in [22, 23], and the definition originated in the context of reconfigurability in [25]. The definition of KPIs is included in Table 1.
A questionnaire with 10 participants from university researchers who are experts in industrial robotics and manufacturing has been conducted. Researchers were informed about the definition of VR application through the text and later by a demo video of application. This step taken toward clarification of application opportunities and demonstrated possible level of realistic representation of laboratory in VR HMD.
5 Results
The local weights of safety training represented in Table 2 including two criteria levels. The table provides the interpretation of the critical features that contributed to the final decision. The results indicate that technical criteria, with a rate of approximately 60 percent, are significantly influential. Whereas use case criteria account for the remaining proportion with approximately 40 percent. The second level criteria are devoted to the fundamental components constructing each use case and technical criteria. Regarding use case criteria, the development cost is the most crucial KPI at first rank, and Ergonomic takes the second rank, with 21.36% of the local wight. In the case of technical criteria, customization alongside modularity was at the highest rate. It is worth mentioning that customization, modularity, and availability account for more than 55% of the importance of local weight.
Figure 3 provides an overview of the participants’ response distribution to the various KPIs. The hardware cost appears to be right-skewed, with three participants dedicating a weight of more than 0.2. This signifies the weight of hardware cost is relatively spread out with a median of about 0.1. However, 75% of the hardware cost survey results fall under 0.2, and the standard deviation is 0.1. As Table 2 depicted, the respondents allocated greater importance to development costs. This is also evident in the boxplot, the highest median of the use case criteria is associated with development cost, where 75% of responses are in the range of 0.18 to 0.42. Moreover, the implementation cost and lifecycle weights seem to experience less variability. Ultimately, the ergonomic criteria, the second local influential weight, has the median of 0.25 and upper quartile of 0.27.
In technical criteria, the given weights of integrability, customization, support, and diagnosability are to some degree concentrated. The median of integrability, support, and diagnosability is comparably close to one another, 0.1, 0.8, 0.8, respectively. Customization criteria appear left-skewed, representing that the majority of participants have a consensus of great importance towards this measure. Such trend can be seen by the standard deviation of 0.05 and close proximately of the median of 0.22 and maximum value of 0.29. Additionally, the median of modularity and availability is about 0.18. This highlights that the experts have a moderately indistinguishable position concerning both criteria. However, on average, partakers tend to favor modularity compared to availability. The criteria of support follow a normal distribution. Finally, the specialists in the field of factory layout design yield outliers of diagnosability and integrability.
During the survey, participants were asked to provide an importance scale from 0 to 9 in pairwise comparison for both training methods regarding second level criterion. Afterward, the average local weight compared to each second level criteria, and the proportional ranks are depicted in Fig. 4. It can be seen that development cost and ergonomics are instrumental in the VR. Correspondingly, the proportion of hardware cost and lifecycle in the VR outweighs the traditional method. Nevertheless, partakers believe that implementation cost is more important in the traditional method, and the valuation of implementation costs for both training methods is insignificant. Furthermore, scalability and modularity are critical in the VR. Experts asserted that diagnosability and support are comparatively vital in the case of VR. Moreover, customization and integrability in the VR method are third and fourth. It is worth mentioning that eleven out of twelve local priorities were valued more in the VR-based training method.
Ultimately, we conclude our assessment results by presenting global priority sensitivity. Generally, the VR training is appreciated over the traditional method with 65.22%. Seven out of ten experts tend to stress their interest in the VR-based method. One participant ranked both traditional and VR training methods nearly equivalent.
6 Conclusions
This study introduced a novel application of VR for safety training in human-robot collaboration, assessment method and indicative results of its technical feasibility, and user acceptance. This study is founded on the notion that in HRC, VR are a potentially innovative type of safety training. The results suggest that VR is a viable solution to design and train operators regarding required safety concerns in collaborative workcells. Technical criteria plays a vital role in investigating solutions with potential adaptation for industrial use. However, the survey results indicate the importance of availability, modularity, and customization of VR-based safety training solutions.
Similar to any study, this research has several limitations. The assessment is done by experts in human-robot collaboration involved in small-scale use case projects. Due to the Covid-19 pandemic, receiving survey results from the respondents was challenging, which limited our assessment to 10 experts. The larger studies are planned to be launched with different users such as managers, machinery safety experts, and shop-floor technicians can synergies different perspectives on the question.
References
Milgram, P., Takemura, H., Utsumi, A., Kishino, F.: Augmented reality: a class of displays on the reality-virtuality continuum. 2351, 282–292 (1995)
Villani, V., Pini, F., Leali, F., Secchi, C.: Survey on human-robot collaboration in industrial settings: Safety, intuitive interfaces and applications. Mechatronics 55, 248–266 (2018). https://doi.org/10.1016/j.mechatronics.2018.02.009
Maurtua, I., Ibarguren, A., Kildal, J., Susperregi, L., Sierra, B.: Human-robot collaboration in industrial applications: safety, interaction and trust. Int. J. Adv. Robot. Syst. 14, 1 (2017). https://doi.org/10.1177/1729881417716010
Arents, J., Abolins, V., Judvaitis, J., Vismanis, O., Oraby, A., Ozols, K.: Human–robot collaboration trends and safety aspects: a systematic review. J. Sens. Actuator Networks 10(3), 48 (2021). https://doi.org/10.3390/JSAN10030048
Valori, M., et al.: Validating Safety in Human–Robot Collaboration: Standards and New Perspectives. https://doi.org/10.3390/ROBOTICS10020065
Berg, L.P., Vance, J.M.: Industry use of virtual reality in product design and manufacturing: a survey. Virtual Reality 21(1), 1–17 (2016). https://doi.org/10.1007/s10055-016-0293-9
Seth, A., Vance, J.M., Oliver, J.H.: Virtual reality for assembly methods prototyping: a review. Virtual Real. 15(1), 520 (2011). https://doi.org/10.1007/S10055-009-0153-Y
Vogel, C., Walter, C., Elkmann, N.: Safeguarding and supporting future human-robot cooperative manufacturing processes by a projection- and camera-based technology. Procedia Manuf. 11, 39–46 (2017). https://doi.org/10.1016/j.promfg.2017.07.127
Hietanen, A., Halme, R.-J., Latokartano, J., Pieters, R., Lanz, M., Kämäräinen, J.-K.: Depth-sensor-projector safety model for human-robot collaboration
Vogel, C., Poggendorf, M., Walter, C., Elkmann, N.: Towards safe physical human-robot collaboration: a projection-based safety system. In: IEEE International Conference Intelligent. Robots System, pp. 3355–3360 (2011). https://doi.org/10.1109/IROS.2011.6048158
Gorecky, D., Khamis, M., Mura, K.: Introduction and Establishment of Virtual Training in the Factory of the Future. 30, 182–190 (2015)
Pérez, L., Diez, E., Usamentiaga, R., GarcÃa, D.F.: Industrial robot control and operator training using virtual reality interfaces. Comput. Ind. 109, 114–120 (2019). https://doi.org/10.1016/J.COMPIND.2019.05.001
Matsas, E., Batras, D., Vosniakos, G.C.: Beware of the robot: a highly interactive and immersive virtual reality training application in robotic manufacturing systems. IFIP Adv. Inf. Commun. Technol. 397, 606–613 (2012)
Matsas, E., Vosniakos, G.C., Batras, D.: Prototyping proactive and adaptive techniques for human-robot collaboration in manufacturing using virtual reality. Robot. Comput. Integr. Manuf. 50, 168–180 (2018). https://doi.org/10.1016/j.rcim.2017.09.005
Buttussi, F., Chittaro, L.: A comparison of procedural safety training in three conditions: virtual reality headset, smartphone, and printed materials. IEEE Trans. Learn. Technol. 14, 1–15 (2021). https://doi.org/10.1109/TLT.2020.3033766
Avveduto, G., Tanca, C., Lorenzini, C., Tecchia, F., Carrozzino, M., Bergamasco, M.: Safety training using virtual reality: a comparative approach. In: De Paolis, L.T., Bourdot, P., Mongelli, A. (eds.) AVR 2017. LNCS, vol. 10324, pp. 148–163. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60922-5_11
Xu, Z., Zheng, N.: Incorporating Virtual Reality Technology in Safety Training Solution for Construction Site of Urban Cities. Sustain 13(1), 243 (2020)
Jeelani, I., Han, K., Albert, A.: Development of virtual reality and stereo-panoramic environments for construction safety training. Eng. Constr. Archit. Manag. 27, 1853–1876 (2020). https://doi.org/10.1108/ECAM-07-2019-0391
Pedram, S., Palmisano, S., Skarbez, R., Perez, P., Farrelly, M.: Investigating the process of mine rescuers’ safety training with immersive virtual reality: a structural equation modelling approach. Comput. Educ. (2020). https://doi.org/10.1016/J.COMPEDU.2020.103891
Tichon, J., Burgess-Limerick, R.: A review of Virtual Reality as a medium for safety related training in Mining. J. Heal. Saf. Res. Pract. 3, 33–40 (2011)
Dianatfar, M., Latokartano, J., Lanz, M.: Concept for virtual safety training system for human-robot collaboration. In: Procedia Manufacturing (2020)
Tirmizi, A., et al.: Technical Maturity for Industrial Deployment of Robot Demonstrators. pp. 310–317 (2022). https://doi.org/10.1109/ICAR53236.2021.9659436
Pöysäri, S., Siivonen, J., Lanz, M.: Dimensions for reconfiguration decision-making and concept for feasibility analysis of reconfigurable pilot lines in industry, research and education
Saaty, T.L.: How to make a decision: the analytic hierarchy process. Eur. J. Oper. Res. 48, 9–26 (1990). https://doi.org/10.1016/0377-2217(90)90057-I
Wang, G.X., Huang, S.H., Yan, Y., Du, J.J.: Reconfiguration schemes evaluation based on preference ranking of key characteristics of reconfigurable manufacturing systems. Int. J. Advanced Manufacturing Technol. 89(5–8), 2231–2249 (2016). https://doi.org/10.1007/s00170-016-9243-7
Acknowledgements
This project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No. 825196.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2023 The Author(s)
About this paper
Cite this paper
Dianatfar, M., Heshmatisafa, S., Latokartano, J., Lanz, M. (2023). Feasibility Analysis of Safety Training in Human-Robot Collaboration Scenario: Virtual Reality Use Case. In: Kim, KY., Monplaisir, L., Rickli, J. (eds) Flexible Automation and Intelligent Manufacturing: The Human-Data-Technology Nexus . FAIM 2022. Lecture Notes in Mechanical Engineering. Springer, Cham. https://doi.org/10.1007/978-3-031-18326-3_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-18326-3_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18325-6
Online ISBN: 978-3-031-18326-3
eBook Packages: EngineeringEngineering (R0)