Keywords

1 Introduction

When the sensors are arranged on the finite points of the structure to obtain the dynamic response information, the structural damage identification is carried out on the assumption that the structural model degree of freedom is consistent with the observation degree of freedom. However, in practical engineering, the observation data is often incomplete due to the influence of various conditions. In order to solve this problem, more sensors are needed, but the actual number of sensors is limited. For structural damage identification, if the monitoring data is incomplete, the useful information obtained is insufficient, making the damage identification problem only be solved under insufficient known information. Therefore, the damage identification method faces the problem of difficulty in damage identification due to insufficient test data.

In recent years, with the development of preinstalled cameras, unmanned aerial vehicles (UAV), wearable virtual reality devices and other hardware devices, damage detection and recognition technology based on machine vision have been applied to actual structures. Image processing technology has been widely used in the detection of local cracks in structural components, such as concrete cracks [1], concrete spalling [2], pavement cracks [3], underground concrete pipe cracks [4], asphalt pavement potholes [5] and so on. Prasanna [6] et al. researched the automatic detection of concrete bridge cracks. First, use the increasing structural elements to carry out alternating open-close filtering on the image, smooth the image and remove noise. Then the multi-scale morphological edge detector was used to accurately extract the edge of bridge cracks and track and locate the development of cracks. Sarvestani [7] et al. developed a vision-based robotic image acquisition device and proposed a more advanced automated visual surveillance system. The system adopts a remote control robot to acquire the image, and identifies the size of the acquired crack through digital image processing software, so that the detection process is more rapid, safe, reliable and low cost. Dyke [8] et al. proposed a vision-based bridge crack detection technology through automatic target detection and grouping processing. Yang [9] and Nagarajaiah [10] et al. used video to detect local structural damage in real-time by combining the low-rank sparse representation method.

These methods have no essential difference from the conventional sensor monitoring methods, and they are aimed at the local area of the structure, so the problem of damage identification caused by incomplete test data has not been fundamentally solved. With the development of digital image processing technology and image acquisition hardware, the holographic monitoring of bridge structures by the camera has become the future development direction of bridge structural safety monitoring. The ability of the image to describe the structure shape can effectively alleviate the problem of structural damage identification caused by incomplete test data. The surface of the bridge structure has many natural texture features. The feature extraction method can obtain the natural texture feature points on the surface of the bridge. Furthermore, analyze the mathematical model of the feature points’ relative position change before and after deformation, propose the displacement field calculation theory of the feature points, and establish a full-field displacement monitoring method for the structure under natural texture conditions. The research results can conduct non-contact deformation monitoring on the bridge structure, significantly improving the completeness of the bridge structure monitoring data.

2 Method of Feature Point Tracking

The surface of the bridge structure is uneven, and some surface defects that are difficult to avoid, such as small holes, voids, pits, cracks, etc., constitute the natural texture of the structure surface. Can identify these natural texture features and track the positions of these natural textures before and after deformation. The displacement calculation theory of natural texture feature points can be analyzed, and the displacement of natural textures can be collected to form the displacement field of the structure surface.

In the case of small deformation, the natural texture of the structure surface does not change significantly. The image scale space extreme point detection method can extract the structure surface’s natural texture features. Image scale-invariant feature transform algorithm (SIFT) [11, 12] can extract natural texture’s extreme points in the scale space of the image through the method of image’s gray difference, called feature points. The obtained feature points have scale, position, and orientation invariant features.

Use the camera calibration board to test the feasibility of natural texture feature extraction on the structure surface by SIFT algorithm. Place the calibration board in two different positions, and take images. Figure 1 shows the extracted feature points of the two images.

Fig. 1
2 photos of the results on the calibration board using the stereo camera system. a. Dots are plotted on the board and the things around it. b. Dots are plotted in the center, corners, and along the diagonal of each cell.

Feature point extraction of two images

Full size image

Match the feature points extracted in Fig. 1 as shown in Fig. 2. As the number of feature points extracted in Fig. 1 is up to 2117, to intuitively display the matching results of feature points, Fig. 2.only shows the matching effects of the first 40 feature points.

Fig. 2
A photograph of the results on the calibration board using the stereo camera system. Lines between the boards link the numbers marked on the cells.

Registration of feature points

Full size image

3 Image Acquisition Test of Bridge Structure

Figure 3 shows the structure and actual object of the steel truss-concrete composite beam specimen used in this test.

Fig. 3
An illustration and a photograph of a steel truss concrete composite beam. The beam has a series of vertical frames and diagonal frames in between 2 adjacent vertical frames.

Physical Drawing of Steel Truss-Concrete Composite Beam (mm)

Full size image

3.1 Method of Test Data Collection

The selection of the monitoring camera is based on the analysis of measurement accuracy, field angle, focal length and other aspects, and Fuji GFX 100 ordinary civilian camera is selected. According to the actual shooting distance to choose the appropriate focal length of the lens, this paper chooses Fuji GF 32-64/4 RLMWR lens. See Table 1 for the technical parameters of the camera and lens.

Table 1 Parameters of Camera and Lens
Full size table

The camera was positioned 5 m from the center of the test beam. In the test, dial indicators are used to obtain the measured value of structural deformation. A total of 13 dial indicators are arranged at the bottom nodes of the test beam test. Figure 4 shows the camera placement position and percentile table arrangement are shown in Fig. 4.

Fig. 4.
2 photographs. a. A steel truss concrete composite beam and a measuring camera. b. Cables and dials under the beam.

Measuring Camera and Dial Indicator Arrangement

Full size image

3.2 Loading of Test Beam

The test beam was loaded at the midspan in 100kN increments from 0 to 400kN. After each loading is completed, the load is held for two minutes to capture images and collecte the dial indicator data. Figure 5 shows the loading scheme.

Fig. 5
An illustration and a line graph. a. A steel truss concrete composite beam, with the load point marked in the center. b. F in kilonewtons versus t in minutes. A and b for the load-holding stage and data acquisition are marked on each step of an increasing line, along with c for the unloading stage.

Test beam’s loading scheme

Full size image

3.3 Acquiring the Image of Test Beam

As an example of 100kN working condition, the obtained test beam image is shown in Fig. 6(a). SIFT is used to extract the feature points of the images of the test beams under various working conditions. Figure 6(b) shows the feature extraction results of the test beams under 100kN working condition.

Fig. 6
2 photographs of a steel truss concrete composite beam, and the outlines of the extracted features along the beam.

Feature extraction results of the test beam under 100kN

Full size image

The main edge contains the shape information of the structure. It is the main area of the feature points distribution of the structure image, which constitutes an important carrier for extracting the holographic deformation information of the structure. Simplify the feature extraction results of each working condition, delete unnecessary environmental feature points, and highlight the main features of the structure, as shown in Fig. 7.

Fig. 7
A model based on the extracted features of a truss concrete composite beam.

Distribution of test beam’s main body under 100kN

Full size image

4 Method of Extracting Full-Field Displacement Vector of Structural Surface Based on Feature Point Tracking

4.1 Structural Full-Field Displacement Vector Calculation

Feature point matching establishes the relationship between the natural texture of the structure before and after deformation, but the displacement is still uncertain. A fixed point can be arranged on the image measurement plane to constrain the position of each feature point, and then the displacement of the feature points before and after deformation can be calculated. Propose a mathematical model for the relative position change of feature points, as shown in Fig. 8.

Fig. 8
A diagram of position change of feature points. The deformation of the beam is illustrated, and the locations of the feature point before and after the deformation with reference to a fixed point are marked.

Mathematical model of feature points’ position change

Full size image

In Fig. 8, the red square denotes a fixed point. Theoretically, the fixed point O can use any feature point of the chessboard. Select the feature point at the center of the checkerboard as the fixed point. The full-field displacement vector algorithm of the structure is as follows:

  1. (1)

    As shown in Fig. 8, displacement vector \(\overrightarrow{{A}_{i}{A}_{i}^{^{\prime}}}\) is composed of the starting point \({A}_{i}\) before deformation and the ending point \({A}_{i}^{^{\prime}}\) after deformation. This displacement vector is the basic element of the structure’s full-field displacement vector, as represented by the blue vector in Fig. 8, which \({M}_{i}\) denotes.

  2. (2)

    \({M}_{i}\) is generated according to the following algorithm: Given the bridge images \(\left({i}_{1},{i}_{2}\right)\) before and after deformation, first, the feature points are extracted from the images, expressed as \({A}_{i}\) and \({A}_{i}^{^{\prime}}\), and the initial matching \(C\), \(C=\left\{\left({A}_{i},{A}_{i}^{^{\prime}}\right):i=\mathrm{1,2},\dots n\right\}\) are obtained, where \({A}_{i}\in {i}_{1},{A}_{i}^{^{\prime}}\in {i}_{2}\).

  3. (3)

    Feature points \({A}_{i}\), \({A}_{i}^{^{\prime}}\) are projected to the fixed point O to obtain the feature point coordinates \({A}_{i}\left({x}_{i},{y}_{i}\right),{A}_{i}^{^{\prime}}\left({x}_{i}^{^{\prime}},{y}_{i}^{^{\prime}}\right),\) extract the coordinate values \(\left({x}_{i},{y}_{i}\right),({x}_{i}^{^{\prime}},{y}_{i}^{^{\prime}})\) between the feature points \({A}_{i}\), \({A}_{i}^{^{\prime}}\), and fixed point O, and obtain \(\overrightarrow{{M}_{i}}=\left[{(x}_{i}^{^{\prime}}-{x}_{i}),\left({y}_{i}+{y}_{i}^{^{\prime}}\right)\right]\). \({M}_{i}\) is a displacement vector with two parameters: length \({L}_{i}\) and angle \({\theta }_{i}\). Length \({L}_{i}\) and angle \({\theta }_{i}\) can be obtained from the known feature point coordinates \({A}_{i}\left({x}_{i},{y}_{i}\right),{A}_{i}^{^{\prime}}\left({x}_{i}^{^{\prime}},{y}_{i}^{^{\prime}}\right)\).

    $${L}_{i}=\sqrt{{\left({x}_{i}^{^{\prime}}-{x}_{i}\right)}^{2}+{\left({y}_{i}^{^{\prime}}+{y}_{i}\right)}^{2}}$$
    (1)
    $${\theta }_{i}={\mathrm{tan}}^{-1}\frac{\left|{x}_{i}^{^{\prime}}-{x}_{i}\right|}{\left|{y}_{i}^{^{\prime}}+{y}_{i}\right|}$$
    (2)
  4. (4)

    Set \(M=\left\{{M}_{i}:i=\mathrm{1,2},\dots ,n\right\}\) is composed of all \({M}_{i}\), and it is the initial calculation result of the structure’s full-field displacement. The full-field deformation vector of the structure’s surface can be extracted by solving the vector set M.

4.2 Full-Field Displacement Monitoring Results of Test Beam

According to the calculation method of the structural full-field displacement vector, we obtain the calculation result of the test beam and the chromatographic assignment is carried out according to the length of the displacement vector, as shown in Fig. 9.

Fig. 9
4 gradient maps of the truss concrete composite beam at 100, 200, 300, and 400 kilonewtons plot beam height versus beam length. The highest value for the displacement vector is around the load point and the values decrease toward either end. A gradient scale is on the right.

Full-field displacement monitoring results of the test beam

Full size image

The full-field displacement vector of the structure surface in Fig. 9 shows the holographic deformation characteristics of the structure, greatly expands the deformation monitoring data, further expands the structural deformation monitoring from single-point measurement to full-field displacement measurement.

4.3 Verification of Structural Displacement Vector

The full-field vector length of the structure surface near the dial indicator under each working condition is extracted and compared with the dial indicator. The verification results are shown in Table 2 (only 400kN is displayed).

Table 2 Comparison of Deformation Measurement Accuracy under 400kN
Full size table

The results show that the maximum absolute error of the lower edge vector is 0.24 mm and the maximum relative error is less than 5%.

5 Conclusion

Propose digital image processing technology to extract the natural texture feature points on the structural surface. Analyzed the feature points’ displacement field calculation theory before and after deformation and proposed a structural full-field displacement monitoring method. Obtained structural full-field displacement monitoring results under the natural texture condition for the first time.

  1. (1)

    The structural surface has rich natural texture features. The natural texture features of the structure surface can be extracted by the image scale-invariant feature transform (SIFT) algorithm to form the point source data of the structural full-field displacement monitoring.

  2. (2)

    Arranging a fixed point can constrain the position of feature points on the structural surface before and after deformation. The full-field displacement vector of the structural surface can be obtained by calculating the relative position relationship between the feature points and the fixed point before and after deformation.

  3. (3)

    The three-dimensional laser scanning verifies the accuracy of the structure’s full-field displacement vector. The verification results show that the maximum absolute error of the full-field vector length is 0.24 mm, and the maximum relative error is less than 5%, indicating that the extracted displacement vector can accurately reflect the full-field displacement characteristics of the structure.