Towards Sustainable Digitization

Abstract

Complexity and data quality of a project are strongly linked together. The higher the need for better data quality is, the more complex projects become. In digitization, various complexity factors have an impact on the quality, such as the object properties or environmental factors. However, it also largely depends on the employed technology. Digitization instruments are designed for specific purposes and applications and have defined specifications and limitations. Some devices perform better with one material or another, and can capture invisible detail or geometry at large distance.

The aim of a holistic documentation often implies to take into account significant amount of additional data and of different nature. Equipping devices with multiple sensors, allows capturing more information and detail at the same time, which means a massive increase of data - intensifying the challenge. Thus, it is mandatory to be aware of the strengths and limitations to decide which technology, or a combination thereof, fits best for each case.

Smart technological ideas and sensor design can mitigate the complexity. At the example of terrestrial 3D laserscanners by Zoller + Fröhlich, this article provides insight to manufacturer’s strategies about ensuring maximum survey quality and proposing efficient solutions for complexity reduction in survey projects. The technological answers range from sensorial adaptions to sophisticated workflow approaches and the incremental use of intelligent algorithms to an outlook on the necessary focus for the future. The approaches are illustrated with examples from emblematic use cases of outstanding tangible Heritage digitization in Europe.

1 Introduction

Cultural heritage projects are manifold and each object or site is unique. Hence, each digitization project is challenging in its own way. The approaches to document moveable and non-movable objects differ and there is no single technology that fits best for all documentation projects. The decision about the instruments to use is primarily based on the required level of detail. It directly affects the choice of the appropriate deployed capturing technology. Thus, it also influences the resulting data quality and the complexity of the documentation process. The higher the required detail, the more the capturing data and the more time it will take, which increases the project costs.

The example in Fig. 1 shows an excavated mosaic pavement. The digitization costs would vary significantly between documenting just the rough structure and color and recording each individual tile in detail.

Fig. 1.

Digitization of a mosaic pavement with an example of the detail provided by the point cloud (right). House of Eustolios at Kourion (Cyprus).

The smallest item size of the documentation defines the level of detail to be adapted which includes also an indication about the targeted level of data quality. Hence, the appropriate 3D survey technology can be chosen.

In general, there are three different categories for contactless 3D documentation technologies:

• Active systems, such as lidar, sonar or radar, emit a signal, which is reflected by the object and returned to the receiver, where the signal is analyzed and converted into 3D data. In the following, the focus will be on optical methods.

• Passive instruments, typically cameras, capture the present state of an object without emitting a signal. In terms of cameras, pictures of an object are captured and used to establish 3D coordinates with photogrammetry principles.

• Hybrids combine the strategies and project a light pattern onto the surface, which is photographed by a camera. The 3D shape is derived from the observed distortions.

2 What Makes Heritage Documentation so Complex? Key Limiting Factors in Surveys with TLS

2.1 Technological Aspects

Each technology has its strengths but also limitations and it is crucial that the user is aware of these to choose the best solution for each project. No single technology may fit perfectly to all needs. Often, the solution to a holistic documentation is a combination. This chapter gives an overview on some of the main parameters, which decide about data quality with active optical systems.

Resolution and Distance to the Object.

No matter which optical measurement technology (active, passive or hybrid systems) the principle is based on a receiving sensor for signal analysis and optical components. Fundamentally, digitization is a discrete method, which results in a digital approximation of a surface. The resolution is a fixed sampling raster without automatic adaption to the needs of the current object. The higher the sampling amount – thus, the resolution - the more accurately its digital twin will represent the object. As a rule of thumb, the minimum resolution needs to be at least twice as fine as the smallest required detail. Otherwise, the detail may not be readable from the spatial survey data, as can be seen in Fig. 2.

Fig. 2.

Principle of minimum resolution for sensing small surface features.

Resolution is defined by an angular increment and changes with range. As depicted in Fig. 3, the point spacing increases linearly with the distance from the sensor. For instance, a sampling resolution of 5 mm at 10 m distance means a point spacing on a frontal surface of 50 mm at 100 m and of 0.5 mm at 1 m.

Fig. 3.

Principle of linear increase of the point spacing by distance from the sensor.

Focal Point.

From photography, it is known that the focus needs to be adjusted for to capture a sharp image of object across various distances. Usually, with laserscanning devices the focal point is static. It is optimized for the entire range of the instrument and cannot be adjusted by the user (Fig. 4).

Fig. 4.

Relation between spot size of the laser beam and point spacing

It is an important fact to be aware of though, since it is not beneficial to increase the resolution of the sampling raster much beyond the actual footprint size of the laser beam on the object. The maximum factor depends on the system used, however, as orientation, it should not exceed 1.5x. Therefore, details of high importance should be captured from closer setup positions rather than being surveyed across longer distances. However, the instrument should not be too close either, as the focus is usually not closely in front of the instrument and often noise increases on short ranges as well.

Angle of Incidence.

As shown in Fig. 5, the resolution also changes with the angle of incidence of the active optical signal onto the object’s surface. The flatter the angle of incidence, the coarser the resolution becomes. It is preferable to have a frontal view onto the object - and thus a steep angle - in order to achieve higher resolution of details. This might be easy for a wall, but cultural heritage objects – except paintings - are rarely flat. It is also worthwhile noting, that with a flat angle of incidence, less light is being reflected back to the instrument, and thus noise in the data can increase.

Fig. 5.

Principle of incidence angle variation of the laser beam

Field of View.

Each optical survey sensor has a defined field of view. All single recordings need alignment in a common reference system in order to describe the 3D shape of an object correctly.

Fig. 6.

Color mapped scan (extract) and example of single imagery tiles from the integrated camera sensor with overlaps (right). Example: Panagia Church of Asinou (Cyprus)Footnote

Courtesy by Cyprus University of Technology.

.

To be able to align all recordings to each other – a process that is referred to as registration - the individual parts need to be captured in a way that they share common areas with another part (Fig. 6). An ideal overlap area duplicates the amount of recorded area by 20–40%, depending on the situation and the technology employed. Thus, the complexity increases with a smaller field of view.

In addition, the accuracy of the registration process influences the data quality. After each pairwise alignment of images or scans a residual error remains, which accumulates with the amount of alignments and thus, can influence the quality of the result. Hence, a larger field of view of the capturing sensor with more geometrical information in one shot allows reducing the errors of the registration process.

Range.

As discussed above, the data quality usually decreases with further distance to the instrument, especially regarding the resolution and focal point. In addition, over longer ranges, less emitted light returns to the instrument, which increases the range noise (error in depth) in the data in an almost linear relation, similar to angular errors. While decision makers often focus on the maximum range an instrument can cover, the importance of the minimum range is often underestimated. In narrow and tiny spaces, such as corridors and technical rooms, a larger minimum range can leave larger blank areas in the data, see Fig. 7. These holes require filling with data from additional setups.

Fig. 7.

Data loss (magenta) by min. Range at 30 cm (left) and 60 cm (right) inside a tight space.

Accuracies.

Each instrument is subject to certain accuracies, which manufacturers of professional equipment should specify in an appropriate data sheet. High-level devices allow verifying at any time in their life cycle, whether the accuracies stated in the data sheet are valid and offer the possibility of calibration by the manufacturer, or alternative countermeasures.

Measuring Time.

Another fundamental factor for data quality is the measuring time, or data rate as it strongly affects the range noise in the data.

For passive optical sensors, such as cameras, mainly the exposure time and sensor sensitivity determines how dark and noisy the image is.

For active laser scanners, the data rate (amount of measurements per second) has a similar impact. A higher rate leaves less measuring time per pixel, which increasingly causes more range noise. Therefore, a lower data rate leads to less range noise and subsequently to higher quality of the data (Fig. 8).

In this context, it is important to be aware that the resolution setting of laser scanners entail a higher data rate setting in the background, which will increase the range noise, typically by factor 1.4x. Only a few instruments allow adjusting the data rate via a quality setting to compensate that effect. However, this of course implies a longer scanning time. Therefore, with respect to an optimum of accuracy, data volume and scanning time, the resolution should not exceed the real needs.

Fig. 8.

Noise at data rates: fast (left) - slow (right). Panagia Church of Asinou (Cyprus)¹.

2.2 Challenging Object Properties

The complexity and data quality depend also on the object properties.

Shape.

The shape of the object is a key factor for the setup configuration. Comparing to a flat shape captured frontally from very few positions, a staggered shape with concavities and convexities requires multiple setups from different perspective and vantage points in order to cover the shaded sides. Each scan or photo contributes and provides data to fill in gap-areas, shadowed from previous viewpoints (Fig. 9).

Fig. 9.

Shadow effect on the survey data by staggered surfaces (tympanum and jamb figures at Cologne Cathedral, west portal) and with filled gapsFootnote

Courtesy by 3DOM: Fresenius Univ. Köln, Heriot-Watt Univ. Edinburgh, Dombauh. Köln.

.

Depending on the accessibility and the movability of the object, as well as the limitations of the used technology, some cavities might not be documented at all or require the usage of different technology.

Dimensions.

The physical dimensions of the object have a direct impact on the complexity due to the need of more scan data, like additional small pieces in a big puzzle (Fig. 10). The position and orientation of all these tiles need to be established and each one needs to share sufficient data with others in order to fit together. Each alignment introduces a residual error, which then may accumulate over the entire project.

Fig. 10.

Cologne Cathedral: point cloud of the north elevation with the urban context. Scanner setup positions on the roof (right). Geometrical identities due to large-scale symmetries require a sophisticated project structure for the correct assembling of scans.²

Whilst the multiplication of capturing positions at ground level mostly faces organizational questions, the multiplication on vertical levels quickly leads to physical limitations and cutbacks in coverage and data quality.

Fig. 11.

Alignment of three 3D scans. Result (left) and single scans (right).

Symmetry.

Especially on a large scale - geometric symmetry challenges algorithms and users, when trying to align the pieces together (see Fig. 11) and the likelihood of a false alignment increases. Thus, more time is needed for the alignment and makes a thorough check of the data even more important.

Material.

The data quality largely depends on object materials. Thus, it is important to be aware of its properties and effects on technology when choosing the appropriate digitization system. In this context, the reflectivity of an object is the most crucial factor. The more reflective, the more light returns to the instrument, and within limits, the data will be less noisy. On the other side, less light is returned by dark or distant surfaces, which leads to an increase in noise.

Further, the surface finishing is important, whether matt or glossy. The latter can cause strong reflections, which may even blind the sensor. This effect is well visible on polished metal. Fully reflecting materials, such as mirrors, lead to range measurements of reflected objects falsely allocated on the vector of the sensing laser beam.

Fig. 12.

Laser light diffusion on marble [7]

Laser measurements on translucent objects, such as marble, often suffer of higher range noise and incorrect measurements, as the light penetrates the surface to a certain extent and is reflected from various depth levels of the object (Fig. 12). Translucent materials, such as glass, lead to wrong range measurements, as the refraction properties of the material are unknown and cannot automatically be compensated for. In addition, the state of aggregation matters. Water, rain or fog might have further limiting effects on the data capture. Especially water or liquids in general cannot be digitized with common technology and may cause artefacts.

Object Conditions.

When looking at the material it is important to look also at the condition of the object surface. Vegetation, such as moss, or vandalism may also affect the quality of the survey data (Fig. 13).

Fig. 13.

Factors of survey data alteration: vandalism and surface flora at Cologne Cathedral².

2.3 Challenging Circumstances and Environment

Accessibility.

The ideal vantage points for the survey are sometimes out of reach or may lead to challenging logistics (Fig. 14). Often, the practical aspects of the survey equipment - such as weight, size and autonomy – become key factors for overcoming these limitations. The accessibility is directly linked to the physical properties of the object and its direct surroundings. Vertical extensions and narrow passages lead to steep angles of incidence by the sensor signal while digitizing the surfaces.

Fig. 14.

Exposed logistics on the exterior of a tall gothic spire (Cologne Cathedral)².

Ambient Conditions: Moving Objects, Illumination.

Moving Objects potentially cause complications regarding the processing steps of the survey data (alignment, coloring) and temporarily obstruct the sensor’s view towards the object, which results in more or less large data gaps. Especially on touristic sites, it can become a relevant limitation. However, often these sites cannot be closed to a large extend to public visitors during the survey. Then, other strategies become necessary, for example collecting additional data from similar vantage points, which may increase the project time and complexity. In addition, animals may obstruct the view, especially when present in large numbers, such as pigeons (Fig. 15).

Fig. 15.

Hostile climatic conditions and wildlife presence during survey of historic British bases in AntarcticaFootnote

Courtesy by British Antarctic Survey and the United Kingdom Antarctic Heritage Trust.

.

Adverse environmental conditions can be hostile and limit the access to the site or part thereof, such as temperatures, radioactivity or explosive dust and gases, or just weather factors, such as wind, fog, rain, snow, or ice.

Finally, the illumination has a significant impact on the data quality. While high-end laser scanners are not limited by sun or ambient light, it is still critical for additional sensors of the system, such as RGB-photography (shadows, color temperature etc.). For instance, light conditions may vary during the fieldwork time and induce color variations in the final data.

3 Technological Strategies for Managing Complexity and Data Quality in High Profile Projects

3.1 Software and Workflow Strategies

The strategies for managing complex projects largely depend on the employed technology. A general survey of the topic would be beyond the scope of this paper. In the following, the authors will focus on strategies for 3D laserscanner devices and data.

Complex projects usually imply more data volume to be managed. In terms of 3D laser scans, the data needs to be puzzled together with sufficient overlap. The alignment process of the 3D puzzle tiles is commonly known as registration. For this purpose, dedicated field solutions have appeared on the market, which automatically download the scanner’s data and register all scans in parallel to scanning [3, 4]. It allows verifying the completeness and data quality of the project immediately in the field that no data is missing and sufficient detail is captured. A field software helps managing the project complexity by providing tools:

• To automatically align all data and detecting issues during field phase

• To ensure the completeness of data;

• To confirm the level of detail;

• To check the quality of data;

• To plan remotely the data capturing phase.

Field Registration.

Similar to solving a puzzle, the alignment of scans is not a trivial task even for powerful computers. In order to simplify the registration process, additional markers or physical objects can be placed in the scene to be recorded by the laserscanner and located in the data by software. Although markers, i.e. targets, are largely considered highly accurate for the alignment of survey data, in the reality of cultural heritage survey the usage of such targets is often restricted. In this context, the physical size of targets is relevant and in some cases, the placement may cause damage to the surface, e.g. by using adhesive tape to attach these to walls or objects. Also, the use of survey targets at large scale requires an additional planning step in order to achieve an ideal distribution. Thus, it is a significant time factor.

The so-called Cloud-to-Cloud registration is an established alternative to substitute or integrate with targets. The Iterative Closest Point (ICP) algorithm [2, 5] identifies closest point neighbors between scans and minimizes the distance between these. It can be considered a statistical fine-tuning of the alignment and delivers reliable results especially when the scans are already pre-orientated, typically within meters.

Another method is to identify certain features in the 3D scans. The so-called Plane-to-Plane registration [8] offers an alternative target-less approach by detecting abstracted planes that allows initial alignment and an accurate reliable registration.

Even though automatic processes exist, a registration process is not fail proof and often requires manual intervention for completion. In complex projects, the manual alignment can be cumbersome and time-intensive. The scan alignment methods without targets require 30–50% overlap between scans and it is difficult for an operator to keep track of all covered surfaces. Thus, depending on the size and complexity of a site, surveyors often tend to scan more than necessary for avoiding registration issues back in the office. Addressing these issues, Z + F presented a field registration workflow in 2015 [4], which, has been widely accepted as a standard principle for 3D laserscanning by today. The position and orientation of the single scans is estimated by additional sensor information, such as video, inertial systems (IMU) or GNSS. The system combines this information with target-less ICP registration on site. In this manner, the operator monitors the registration quality continuously and the redundancy is lowered towards a minimum level without risk.

The current systems are often subject to technical limitations and thus, for example, are used only outdoors (GNSS based) or indoors (videometry based). Others allow a combination thereof. Zoller + Fröhlich proposes the so-called “Blue Workflow” [4]: the scan positioning system (a combination of GNSS, compass, barometer and IMU sensors), which tracks the displacement of the device between consecutive scan positions, works in- and outdoors. The software Z + F LaserControl® Scout runs in parallel on a WiFi connected tablet PC, automatically downloading and registering (cloud-to-cloud) the incoming pre-orientated data. Feedback about sufficient overlap and the registration outcome is provided immediately. The registration can also be combined with targets, if required. Hence, the operator proceeds with a minimum number of scans without inadequate high redundancy of covered area or potential registration issues. Since the tablet PC allows parallel processing, the operator efficiently uses idle time until the completion of the next scan. The algorithm may not be able to register all scans automatically due to limited overlap, symmetry or noise by moving objects. Therefore, the solution integrates a toolset for immediate manual adjustment. In addition, the software allows to examine and to assess the data directly on site. The user is able to verify and ensure the required point coverage on the focused objects, as well as the data quality of the measured points.

Such on-site processing tools allow the user to decrease the complexity of a project and ensure the best data quality possible, as well as the overall project success.

Annotations, Metadata and Project Planning.

Annotation capabilities of the employed software are of great benefit, as starting with the first visit on site by the decision-making staff, these tools can log field notes and meta data (as text, photography, audio, video or any format) to create detailed tasks for the field team and include referenced photographs about key setups [6]. Further, these tools allow planning scan positions in advance in order to prevent unexpected and significant deviations of the main project objectives, which result in increasing costs.

Data Management.

Opposed to industrial applications - where scan data typically is replaced with best-fit primitives - every data point may matter for documenting cultural heritage where irregular, non-planar surfaces prevail. Surfaces of ancient structures hardly fit to geometric primitives without significant loss of information. Filling larger gaps in the data is not trivial but always work intensive. Therefore, it is good practice to multiply scan positions and capture with lower resolution from various different angles than to scan with higher resolution from less positions. The coverage increases and an overall high point density is achieved by overlapping multiple scans. Generally, depending on the complexity and the required data quality, the aim to achieve a maximum coverage tends to increase the size of the dataset and the complexity for the registration. Further, difficult scan positions require an immediate visual validation option of the registration result before moving to the next position.

The software Z + F LaserControl® works on a scan-by-scan logic for processing or analysis [4]. Thus, the solution is scalable to process extensive projects on a tablet PC.

3.2 Hardware Approaches

Today there are laserscanner hardware solutions available, which combine the spatial data with data from additional sensors in order to mitigate the complexity for the use of different documentation technologies. These data are complementary and the processing is highly automated.

Handheld Integration.

A static 3D laserscanner is typically able to capture panoramic full-dome scans, repeated from various vantage points in order to reach a high coverage. At a certain ratio between redundant and uncovered surfaces, the efficiency of additional static full-dome scans may decrease. Then the complimentary use of handheld scanning devices may be a method to quickly fill-in small remaining gaps.

Fig. 16.

The Z + F scanner can serve as a data hub for handheld devices (here: DotProduct) sending the data to the field software Z + F LaserControl® Scout for registration [3]

The handheld data integration requires sufficient overlap with the panoramic scans. A dedicated workflow from Z + F synchronizes via WiFi F between both devices (Fig. 16). The data of the handheld scanner is directly registered and verified its completeness.

Infrared Camera.

Infrared cameras allow reading the surface temperature. For conservational analysis, it can provide indirect information about the material condition, such as thermal bridges, structural discontinuities, electrical faults or voids. In some cases, the infrared data can even unveil information from below the object’s surface.

Combining spatial and infrared data manually requires the identification of common points (homologies) in both data sets. This had been a quite challenging task before the advent of automatic solutions due to the typically great difference in resolution between both systems.

Fig. 17.

Mapped point cloud with infrared image unveiling hidden technical installations. Add-on IR-camera on a 3D laserscanner (right) [3]

Zoller + Fröhlich developed a dedicated calibrated infrared system, which senses thermal information and spatially allocates each value automatically (Fig. 17). Hence, thermal analysis is carried out directly on the 3D geometry, as opposed to 2D images [3].

High Dynamic Range (HDR) Technology.

In cultural heritage applications, the interest beyond the pure geometry includes capturing high quality RGB information in order to map scan data with photographic information. The combination with image data is used for surface analysis of materials and assessment of degradation [3]. Further, museum and immersive experiences require photorealistic color of the data.

Fig. 18.

HDR photography. Mapped result (left) and raw images with different exposures².

In this regard, it is challenging to ensure a constant color quality as the lighting conditions change in the course of a project. In addition, photos can differ in appearance, depending on where the internal sensors focused on. The image can result under- or overexposed, if the focus or light measuring area differs within the same image frame. HDR photography is a standard technique for panoramic photography. Outdoors, the scene is usually well illuminated but shaded objects are often too dark. Standard photography can hardly find one suitable exposure setting for all areas in an image without over- or underexposed parts. Instead of just one image at one exposure setting, HDR merges a set of images with different exposure times into one single image (Fig. 18). Thus, this technology is often integrated in laserscanning devices.

SmartLight.

Color scanning in dark environments is a challenge. Due to massive presence of visitors at daytime, some applications require to execute the capturing work at night hours. Then, illumination of the environments most likely is required, which can be a challenge due to cables, shadows and even multi-shadows different light sources. Therefore, Z + F sensors operate with a HDR enabled camera system and provide integrated spotlights to illuminate the scene [3, 4]. These lights are mounted radially around the camera optics allowing shadow-free imagery even in near proximity, which is then mapped onto the point cloud.

Photogrammetry.

Although deriving from different technologies, the output of active (e.g. laserscanners) and passive (e.g. separate, non-integrated cameras) systems can be successfully combined. For this purpose, the camera photos are used to create a 3D model with algorithms of photogrammetry, using the laserscanning data as a reference or skeleton model. Photos from an external sensor are usually taken at the same instance with similar lighting conditions and ensure best RGB color results. On the opposite site, laserscans provide highly accurate geometric detail. It should be noted that highly accurate data can be acquired locally on close-range with photogrammetry alone. However, for large sites it is much more complex to achieve the same accuracies as with laserscanners. Combining both technologies, the advantages of a combination ensure highest data quality in terms of color and geometry [1].

Special Equipment.

When the physical size of the object exceeds common dimensions, then the vantage points for data capturing need to be chosen also in elevation.

Fig. 19.

Shaft tripod with laserscanner lowered through a vault keystone and special support bracket for scans exterior to the spire (right) at Cologne Cathedral².

Special fixtures and support structures may be necessary to reach these positions. There are different solutions available from technology suppliers, such as lever arms or inverted tripods (Fig. 19). The latter can be used to orient a scanner upright or upside down. Especially when lowered down to privileged positions, such as below the key stones of tall cathedral vaults, it allows to enrich the data significantly at a high quality level.

4 Conclusions and Outlook

As shown above, technology is key for an efficient documentation of cultural heritage and can minimize a project’s complexity while ensuring highest data quality. Also, technological supplements or sensor fusion, as shown above, can significantly improve data quality.

Although different solutions are available, 3D digitization is still a time intensive process. Thus, there is a trend towards much faster mobile systems, which acquire 3D data while the sensors are moved through the scene. However, the systems cannot yet provide the same data quality as of static panoramic laserscans. However, mobile systems will play a significant role in the future. They also collect a much higher data volume to be processed in less time in combination with higher resolution. Thus, it is mandatory to explore new processing and storage possibilities. Computer clouds can help in this regards, but a dedicated cultural heritage solution is still missing.

Apart from data capturing, there is still intensive research and development necessary to segment and classify the data automatically. At present, the data is visualized as a large collection of 3D points or surfaces consisting of triangles. However, the elements do not provide information about the object they belong to. Manual augmentation of this information is a tedious process and thus automation, maybe with the help of artificial intelligence, is certainly one of the future key research topics.