Keywords

1 Introduction

With the development of technologies such as 5G communication, big data, and artificial intelligence, the automation level of ports has been rapidly improved [1]. Automated remote operation port loading and unloading equipment has been widely applied to port enterprises at home and abroad. Given this new type of automated remote control mode, the existing operating driver training is faced with high safety risks, difficult training, and high learning cost, so this paper proposes an intelligent simulation system architecture for automated quay crane training based on embedded digital twin technology.

The concept of a digital twin includes both physical structures in physical space and digital models in cyberspace, aiming to realize real-time interaction between physical entities and virtual models [2]. At present, digital twin technology has been widely used in automated container terminals. For example, there are applications in port structure and auxiliary monitoring systems [3], port crane operation status monitoring [4], intelligent scheduling of port logistics [5], and the construction of ship digital twins [6].

In recent years, scholars at home and abroad have also done a lot of research on virtual crane simulators. For example, the immersive virtual reality simulator proposed by Patrao and Menezes [7]. Juang et al. [8] developed a crane simulator called SimCrane 3D. Noda et al. [9] studied a training system that enables operators to safely and effectively master the operation of electric bridge cranes while suppressing load sway. In addition, there are bridge crane simulator systems [10, 11], both of which are effective in training crane drivers.

However, the virtual crane simulator and the real crane are independent of each other, so the operation of the real crane cannot be taken into account in time when the virtual crane simulator is used for training. However, the intelligent simulation system architecture of automatic quay crane training based on embedded digital twin technology proposed in this paper to use digital twin technology to establish a digital twin model for the structure and function simulation of the real automatic quay crane remote control system. And a digital virtual simulation System with the functions of the Chassis Parking System and Ship Profile Scanning System (SPSS) is designed. Moreover, the system can be directly embedded into the real quay crane remote control operation platform without changing the original automatic quay crane control system hardware, and realize the real-time switch between the virtual quay crane remote control operation simulation and the real quay crane remote control operation. It can not only realize the training function of the opposite quay crane driver but also take into account the real gantry crane operation in time, to achieve the effect of idle training and busy operation.

2 Implementation

2.1 Automated Quay Crane Remote Control System

The overall architecture of the system is shown in Fig. 1. When the driver remotely controls the actual quay crane, the on-board PLC of the shore container needs to transmit the sensor signals from the field operations, various mechanical actions, field control signals, and other signals to the PLC of the remote control room operator console through the control bus. Then, the received signal data is analyzed and processed by the PLC and presented to the CMS system. Simultaneously, the CMS system reads and analyzes the task instructions, and returns them to the PLC for the next specific operation of the quay crane.

Fig. 1
A flowchart outlines the training and operational process for real and virtual quay cranes. It details the training content, effects of training, components of the cranes, and the workstations involved in their operation.

The overall architecture diagram

Full size image

The remote console is the use terminal of the remote control system and the visualization device that interacts with the driver. As shown in Fig. 2, it is composed of 6 Closed-Circuit Television (CCTV) screens, the remote port machinery monitoring system, the driver's console and the GP touch screen. According to the CCTV screen and the remote port machinery monitoring system, the driver in front of the remote operation platform sets the work task information of the quay crane with the button on the operation platform. GP touch screen is used to display PLC operation status, such as unlocking, locking, spreader size, car speed, position, etc. In addition, the driver can also switch the video screen through the GP touch screen, turn on the projector light, adjust the camera shooting parameters, etc.

Fig. 2
A photograph of a room with multiple monitors displaying live feeds of a construction site. The desk is equipped with joysticks and other control interfaces. There are diagrams or plans on one monitor, and graphical data or interface on another.

Remote operator console

Full size image

2.2 Intelligent Simulation Sensing System

The automated quay crane remote control system has a ship profile scanning system, chassis parking system, optical character recognition, and other sub-systems instead of on-site manual operation of the link. To achieve the same effect as the real quay crane remote control control system, our automated quay crane remote control operation of the intelligent simulation system is also equipped with a large number of virtual sensor modules. For example, the realization of the SPSS function is through the system's virtual laser range finder constantly launching rays on the ship shape scanning, when the rays hit containers or obstacles, real-time detection of distance information, and then the ship shape data obtained will be transmitted to the PLC for interaction, the spreader to limit the speed of deceleration of the action, the realization of the quay crane hoisting, and the direction of the trolley spreader collision protection and intelligent deceleration function. Figure 3 shows the schematic diagram of the 2D LiDAR installation position in the virtual quay crane simulation system.

Fig. 3
A photograph of a 2-D LIDAR sensor installed on a complex metal structure, possibly part of a large machine or building. The sensor is highlighted and marked.

Installation of virtual LiDAR

Full size image

The realization of the container truck self-positioning function in this system mainly relies on the built-in collision detection technology of Unity3D to set up the collision body on each lane and the corresponding collector truck, as shown in Fig. 4, and the collision official body in the actual system is transparent, which adjusts the position of container truck by detecting the collision between the container truck and the front and rear limits and the lane limits and sends the position of container truck to the PLC when the container truck reaches the specified position. Send the position of the container truck to the PLC, in addition, according to the collision detection information of arriving at the front and rear limits, call the function of the container truck to move forward or backward, so that the container truck arrives at the specified position, to realize the virtual simulation of the container truck positioning system.

Fig. 4
A simulation window has an overhead view of a lane marked with lines on a floor, with a large machine positioned within the lane. The lane has the rear and front limits marked at the top and bottom, respectively. A vertical bar on the right is labeled lane limit.

Schematic diagram of virtual set card perception

Full size image

2.3 Control System

In order to embed this system into the real quay crane remote control console without changing the hardware of the existing automated quay crane control system, so as to realize the one-key switching function between the real quay crane remote control operating system and the virtual quay crane remote control operating system, the data transmission between the automated quay crane remote control operation intelligent simulation system and the remote control room operating console PLC needs to be considered when carrying out the hardware design. In the virtual quay crane remote control operating system, the system integrates a large number of virtual sensors, which not only highly reproduces the real quay crane in terms of visual and physical effects, but also reproduces the actual operating conditions. The specific process is to realize bi-directional communication with the PLC of the remote control room operator console through OPC UA communication protocol, i.e., the system can not only receive the mechanism drive signal from the PLC of the remote control room operator console, but also transmit the signal from the virtual sensors to the PLC of the remote control room operator console.

Through the above operation, the remote control room console PLC can not only obtain the relevant data information of the real shoreline remote control operation but also read the data information of the virtual shoreline remote control operation and carry out the same processing as the real shoreline remote control operation, so that it can realize the function of switching between the real shoreline remote control operation and the virtual shoreline remote control operation with one key on the same console.

3 Experiment

Based on the overall architecture diagram in Fig. 1, it is evident that the driver can familiarize themselves with the operational procedures of the console through the intelligent simulation system. This allows them to safely and efficiently acquire the necessary skills for controlling the remote quay crane. These skills encompass basic operations such as loading and unloading ships and trucks, utilizing the spreader guide effectively, as well as managing car acceleration and deceleration. Alongside these foundational skills, the system also trains drivers in specialized tasks like lifting heavy parts and handling dangerous goods or special containers under challenging conditions such as rain or wind. Furthermore, the system records historical training data which facilitates the assessment of the driver's performance. This enables us to evaluate the standardization of their operation, optimize their working methods, enhance their efficiency, and assess their readiness to operate real remote control quay cranes.

As shown in Fig. 5, this paper also conducted functional tests of the system in six aspects, namely basic function, communication function, operational task module, operation perception, performance and stability, and one-click switching function, at a marina, as shown in Table 1.

Fig. 5
A photograph of a person operating the joysticks connected to the 6 monitors mounted on stands in front of them. A movable monitor is on the person's left and the person wears a headset.

System functional test

Full size image
Table 1 System function test status
Full size table

The above test results show that the system can not only realize the training function of the quay crane driver but also realize the real-time switching function between the virtual quay crane remote control operation simulation and the real quay crane remote control operation after being directly embedded in the real quay crane remote control console. Thus, the effect of training in idle time and operation in busy time can be achieved.

4 Conclusion

This paper proposes an intelligent simulation system architecture based on embedded digital twin technology for automated quay crane training, which has been practically applied in a terminal, and the results show that the system architecture can realize the real automated quay crane operation and operation simulation with one-click switching function by directly embedding it into the real remote control console without modifying the hardware of the existing automated quay crane control system.

It can realize the training of novice drivers and the daily training of other drivers, and it can not delay the normal operation of the automated remote control quay crane, and the transformation cost is low and the safety risk is low. However, due to time constraints, certain aspects of the system have not been fully investigated. To further enhance the virtual quay crane simulation system, future plans involve incorporating an automatic evaluation module to provide feedback on driver training. Additionally, a study on the training effectiveness of novice drivers in relation to spreader swing amplitude and time in both virtual and real quay cranes is also planned.