Abstract
Addressing the current issues of low accuracy in container positioning and posture recognition, as well as long response times during the port automation loading and unloading process, this paper designs a rapid container target recognition and measurement device and method for automated loading and unloading, thereby optimizing the acquisition of key parameters in automated loading and unloading operations. This method combines advanced convolutional neural networks and traditional image processing algorithms to achieve precise detection and tracking of container corner fittings. Furthermore, this paper proposes a high-speed response method for small target measurement, which integrates minimized deep learning network technology and fuzzy image morphology matching algorithms to enhance the accuracy and stability of corner fitting detection. Through experimental verification, this method effectively improves the speed of single detection and reduces the localization error of small targets.
You have full access to this open access chapter, Download conference paper PDF
Keywords
1 Introduction
In modern port automation systems, gantry rail-mounted container cranes play a critical role, responsible for efficiently and swiftly transferring containers between transportation equipment and container storage areas. Container hoisting operations primarily involve two key alignment steps, as shown in Figure (see Fig. 1).
The increasing demand for automation technology in container docks stems from the inefficiencies of manual alignment operations, characterized by low precision and heightened error probability due to the significant distance between the driver's cabin and hoisting equipment. Current technologies predominantly utilize laser radar for container positioning, identifying and measuring geometric shapes. Despite its resistance to adverse weather and lighting, the technology is marred by high costs and limited accuracy.
Machine vision and its associated algorithms have significantly advanced dock container automation, facilitating tasks such as container identification and safety inspections [1]. However, the existing research, including the stereo vision-based positioning method proposed by Yoon et al., encounters limitations in accuracy, with errors sometimes reaching up to 60 mm, primarily due to the baseline length between cameras and confined space on lifting equipment [2]. In real dock settings, image processing algorithms grapple with environmental factors affecting image clarity, such as container color variations and surface contamination. Current methods, although capable of locating containers within cabins, necessitate substantial computation time, extending single image processing to 0.6 s [3]. Moreover, existing vision tracking methods, despite their speed, fail to meet the stringent accuracy requirements of container hoisting operations due to simplistic algorithm structures [4].
Addressing these challenges, machine learning emerges as a promising avenue. Recent advancements in deep learning have revolutionized industrial detection and recognition. Studies by Qi and others [5], Kazmi and others [6], and Liu and Wang [7] underscore the potential of deep learning in enhancing product quality and manufacturing efficiency. New methodologies, such as those proposed by Zhang and others [8], He and Liu [9], and Yu and others [10], are paving the way for more precise and efficient container positioning technologies, heralding a new era in port automation systems.
The aforementioned methods primarily target object recognition, detection, and measurement in the industrial sector, mainly applied within factory indoor environments utilizing specialized cameras and computing equipment. However, container automation loading and unloading necessitate adapting to all-weather outdoor scenarios, and due to limited installation space within port equipment, high-computational power devices cannot be employed. Therefore, implementing an efficient all-weather lightweight recognition network under limited computational power conditions presents a current challenge. Given that recognition and measurement methods based on deep learning still exhibit slow response speeds and poor accuracy in container target detection, this paper proposes a fast container target identification and measurement device and method for automated loading and unloading. By compressing and optimizing traditional deep learning networks and integrating container appearance features, the device and method described in this paper achieve higher detection efficiency compared to conventional methods.
2 Vision-Based Measuring System
In this study, a vision-based measurement system structure is proposed, as shown in Figure (see Fig. 2). The primary objective is to accurately determine the pose information of the container. The first step of the system involves detecting the container corners in the image, followed by utilizing this data to calculate the exact position of the container. Upon completion of all steps, the positional data will be transmitted to the ACCS (Automated Crane Control System) to facilitate precise control of the hoisting actions.
2.1 Image Capture Device
The system proposed in this paper relies on multiple cameras to capture images of the top of the container, as illustrated in the figure (see Fig. 3) depicting the device installation method.
2.2 Improved SSD Image Processing Section
An image processing section based on the improved SSD, a model grounded on Convolutional Neural Networks (CNN), is utilized. Figure (see Fig. 4) provides a detailed display of the basic model based on SSD-300 (with an input image size of 300 × 300) and its structural components. To further optimize the SSD model to adapt to the target detection of container corners, the following two main improvements were made to the model based on SSD-300:
Backbone Layer Update: The DSSD (Deconvolutional Single Shot MultiBox Detector) is adopted as the improved SSD detector, which enhances the representation ability of shallow features and increases the recognition rate of small targets by replacing VGG-16 with the updated ResNet-50 as the backbone layer. The deeper ResNet can retain more feature information, thereby enhancing the robustness to small targets.
Feature Map Layer Adjustment: The feature map layer of the basic SSD model was optimized. Since the original high-level feature map layer is relatively insensitive to the recognition of small targets, the Conv10_2 and Conv11_2 layers were removed to accelerate detection speed, while adding a feature map layer with higher resolution to enhance the recognition ability for small targets at container corners.
2.3 High-Precision Calculation Method for Container Pose Parameters
To precisely adjust the control strategies of lifting equipment, the pose parameters of the container are subdivided into two core parts: firstly, the offset vector of the container's geometric center on the horizontal plane, and secondly, the deviation angle of the container on the horizontal plane relative to its geometric center. Based on this, further consideration was given to the issue of measurement data fusion in the dual-camera system. Specifically, when calculating the pose parameters of the container, the average of the measurement results obtained from two independent cameras will be used to derive more accurate overall pose information.
The displacement vector of the container position is denoted as \(l=(\Delta x,\Delta y)\), which characterizes the offset difference on the plane between the current detected position of the container and its theoretical position under standard working conditions. As shown in figure (see Fig. 5), to more comprehensively utilize the information obtained from the dual cameras, a method was designed to calculate the displacement vector based on the Euclidean distance between the spatial coordinates of the four corners of the detected container and their calibrated positions under standard working conditions, as detailed in Eq. (1)and (2).
On the other hand, the deviation angle of the container is defined as the counterclockwise central angle of deviation relative to its standard working state (i.e., parallel to the observation plane). As revealed in Figure (see Fig. 6).
The calculation of this parameter also integrates the lock hole coordinate information from the images of the front and rear cameras. The angle is determined with reference to the inclination angle of the short side of the container in the detection state relative to the standard state, with specific calculation details shown in equation.
Through this approach, it is anticipated that the real-time pose parameters of the container can be calculated more accurately, thereby providing more accurate and stable control instructions for the hoisting equipment.
3 Experimental Result
The device described in this paper has been deployed in a certain railway automated container yard. Experiments are conducted based on this platform, where the image data used in the experiments is captured by high-resolution dome cameras installed at designated positions, with hardware parameters as shown in the Table 1. Detailed configurations of the operating platform relied upon by the measurement algorithm can be seen in Table 2, with its hardware performance matching the capabilities of industrial computers commonly used in the current field of industrial control.
3.1 Verification of the Improved SSD Image Processing Section
The optimized SSD algorithm was trained based on 8700 images of the upper surfaces of containers, which were captured during the actual operation process of container lifting equipment. These images encompass the upper surface images of containers under different environmental conditions of day and night lighting, with each image encompassing visual data of about 2 to 6 container corner areas.
The performance evaluation of the improved SSD detector is divided into two stages. The first stage involves evaluation by comparing the detection performance of the SSD model before and after modification. The experimental group for this stage includes our optimized version of the improved SSD-300 model proposed in this paper and the original SSD-300 model. 500 sample pictures taken at different operating times are used to evaluate the detection performance of these two networks, with multiple container corner features annotated in each image. As shown in Table 3, detection performance parameters are calculated through Average Precision (AP). The experiments proved that our optimized SSD network proposed in this study surpasses the original version in terms of detection accuracy and speed, reducing computation time by 5.35 ms and increasing AP by 3.45%. The corner area detection results of the optimized SSD algorithm are shown in the figure (Fig. 7).
The second stage involves localization error analysis, where the deviation between the detection results and image calibration results is statistically analyzed, and its distribution fitting curve is calculated using normal distribution statistical analysis. The maximum error values at 95% and 90% confidence intervals are taken as the maximum error of the calibration results, and the actual error values are calculated, with results shown in Table 4.
The standard dimensions of the container corner lock holes are 124 mm × 63.5 mm. The measurement method implemented in this study has an error distribution of 21.3 mm × 15.9 mm at a 95% confidence level. The final measurement accuracy of this method can satisfy the practical application requirements for container pose measurement tasks.
4 Conclusion
This paper proposes a fast container pose measurement device and method for automated loading and unloading. By compressing and optimizing traditional deep learning networks and integrating container appearance features, precise detection and tracking of container corners are achieved. This system reduces the single detection time by 5.35 ms, with a high detection rate of up to 90%, capable of achieving a positioning error between 14.3 and 19.6 mm at a frame rate of 10 fps. This research paves the way for further advancements in port automation, potentially fostering more efficient, safer, and cost-effective operations through the integration of sophisticated detection and tracking technologies in container handling processes.
References
Mi C, Huang Y, Fu C, Zhang Z, Postolache O (2021) Vision-based measurement: actualities and developing trends in automated container terminals. IEEE Instrum Meas Mag 24(4):65–76
Yoon H, Hwang Y, Cha E (2010) Real-time container position estimation method using stereo vision for container auto-landing system. In: ICCAS Gyeonggi-do, pp 872–876.https://doi.org/10.1109/ICCAS.2010.5670111
Dai M, Liu Q, Wang J (2019) An auxiliary container loading location algorithm based on computer vision. In: 2019 34rd youth academic annual conference of Chinese association of automation (YAC), Jinzhou, China, pp 280–284. https://doi.org/10.1109/YAC.2019.8787723
Lourakis M, Pateraki M (2021) Markerless visual tracking of a container crane spreader. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 2579–2586
Qi S, Yang J, Zhong Z (2020) A review on industrial surface defect detection based on deep learning technology. In: Proceedings of the 2020 3rd international conference on machine learning and machine intelligence
Kazmi W, Nabney I, Vogiatzis G, Rose P, Codd A (2020) An efficient industrial system for vehicle tyre (tire) detection and text recognition using deep learning. IEEE Trans Intell Transp Syst
Liu F, Wang Z-Y (2020) Automatic “ground truth” annotation and industrial workpiece dataset generation for deep learning. Int J Autom Comput
Zhang C, Yu J, Wang S (2020) Fault detection and recognition of multivariate process based on feature learning of one-dimensional convolutional neural network and stacked denoised autoencoder. Int J Prod Res
He Z, Liu Q (2020) Deep regression neural network for industrial surface defect detection. IEEE Access
Yu J, Liu X, Ye L (2021) Convolutional long short-term memory autoencoder-based feature learning for fault detection in industrial processes. IEEE Trans Instrum Meas
Acknowledgements
This research was supported by the Science and Technology Commission of Shanghai Municipality (No. 122ZR1427700 and No. 223692106900)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2024 The Author(s)
About this paper
Cite this paper
Zhang, Y., Mi, C. (2024). A Fast Vision-Based Algorithm for Automated Container Pose Measurement System. In: Halgamuge, S.K., Zhang, H., Zhao, D., Bian, Y. (eds) The 8th International Conference on Advances in Construction Machinery and Vehicle Engineering. ICACMVE 2023. Lecture Notes in Mechanical Engineering. Springer, Singapore. https://doi.org/10.1007/978-981-97-1876-4_64
Download citation
DOI: https://doi.org/10.1007/978-981-97-1876-4_64
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-1875-7
Online ISBN: 978-981-97-1876-4
eBook Packages: EngineeringEngineering (R0)