Visualization

Abstract

3D computer-based visualization refers to all those methodologies adopted to produce, represent, describe, transmit, and present graphically/visually digital 3D models in a way that is perceivable by the human eye. Visualization is one of the core aspects of 3D reconstruction because it is the most effective medium to synthesize complex data in a visual way and makes the results more accessible and comprehensible not only to professionals but also to laypersons.

Guiding questions

• What is digital visualization?

• What are the media methods and techniques?

• How can it be used as a scientific tool?

• How can it be used to communicate information?

• What are the potentials/challenges of digital and physical visualization?

• What technologies present and interact with digital 3D reconstructions?

Basic terms

• Texturing

• Photo-realistic and abstract shading

• False colors

• Perspective and axonometric projection

• Interactive, linear, and static visualization

• Rapid prototyping

• Extended reality.

7.1 From Digits to Visuals

Anything in modern computers is stored as sequences of ones and zeroes, however, binary code is not practical to be read by humans, so any data of such type needs to be converted in some other language and displayed on an interface that is easier for humans to read. Digital 3D models are no exception: to be able to interact with, experience, and present them, their binary description needs to be processed and output visually, e.g., as RGB (red, green, blue) values on a display. The visualization is such an important aspect of the 3D model that someone might say that 3D modeling, and 3D visualization are entangled concepts and one has no reason to exist without the other. In 3D reconstruction of cultural heritage, it would probably not even be possible to generate the 3D model in the first place without constant visual feedback on a display. The most popular interfaces used to view digital 3D models are 2D displays (computer, smartphone, TV screen, projector, etc.). But in the past few decades, many other technologies are beginning to be considered valid alternatives: VR, AR, 3D displays, holograms, and so on.

The use of different interfaces and our needs can radically change the way we perceive, present, experience, and interact with 3D model. Three main ways to present the 3D models virtually are listed below:

• Static presentation (single image/static rendering).

• Linear presentation (scripted video/precomputed animation).

• Interactive presentation (real-time exploration/computer games).

In static presentation, the 3D model is projected in a 2D medium and captured as in a photograph. These types of pictures are fast to make through modern rendering engines, but they are limited in terms of interaction/navigation freedom within the 3D space, because the points of view from which the viewer can experience the 3D model are predetermined.

In linear presentation, the 3D model is experienced through a pre-arranged tour in the form of an animated video. Interaction/navigation within the 3D space is limited also in this case because the observer cannot move freely. However, the level of engagement/comprehension of the 3D shapes can be higher than in static presentation since the depth perception is enhanced through motion.

In interactive presentation, the model can be explored in a self-guided way through a specifically designed digital 3D environment such as a gaming platform or a web app. This type of presentation is for sure the most versatile one because the models can be perceived from every possible point of view. Users need a basic understanding of how to move in the digital space, usually with gamepads, touch screens, or mice and keyboards. Visualizing high-resolution models interactively is a much more hardware-demanding task than static and linear presentations, which can run on any device with a display. 360° panoramas can be considered a hybrid type because they are made with a static 360° image projected onto a sphere (or a cube), but they can be experienced through VR headsets or navigated with mice and keyboards. The limit of the navigation in this kind of panorama is that the point of view is always in the center of the sphere.

Visualization can also be useful as an analysis tool to highlight certain formal or superficial aspects. According to Ware, visualization can support research and understanding in five ways [1, cited from 2, p. V]:

• It may facilitate the cognition of large amounts of data.

• It can promote the perception of unanticipated emergent properties.

• It sometimes highlights problems in data quality.

• It clarifies the relationships between large- and small-scale features.

• It helps in the formulation of hypotheses.

7.2 Digital 2D-3D Visualization

Digital 2D-3D visualization is the process of creating graphics and renderings by combining 2D and 3D modeling and rendering software. It is used in many industries and applications ranging from architecture, film and games industries, engineering and manufacturing, to advertising and fashion. Until the advent and subsequent widespread use of digital 3D modeling and visualization systems, knowledge relating to the 3D world was studied, transmitted, and analyzed in physical forms through the projection of the 3D objects onto 2D media (paper, wooden tables, cloth, etc.) or through the creation of 3D physical mock-ups (in wood, cardboard, clay, or other materials), usually made by hand by artists, architects, or engineers.

The creation of 3D physical maquettes (${\bf{ \to }}$ Basics and Definitions) was time-consuming work that usually required a precise and long phase of planning, and each minimal change to the design required non-trivial effort to update the model. The most popular way to study, analyze, and present any design was through 2D projections, usually made by and for specialists who knew the representation codes (format/type of lines, color, standard direction of view, etc.) and were able to mentally process that information, and mentally compose the 3D object. Digital 2D-3D visualization has overcome some of the limitations that have characterized analog physical representation. Nevertheless, there are positive and negative aspects of this new technology. In Table 7.1, we compare the pros and cons.

Table 7.1 Pros and cons of digital 2D-3D visualization

7.3 Aspects of Digital 3D Visualization

Four aspects contribute to the visualization of 3D models:

• Formal/geometrical aspects (e.g., digital 3D object, segmentation).

• Shading aspects (e.g., materials, light, rendering algorithms).

• Representation aspects and methods (e.g., point of view, camera, projection methods).

• Media and interfaces (e.g., display, monitor, image, 3D prototype).

Formal/geometrical aspects concern how the 3D model/scene is made ($\to$ 3D modeling): the spatial relations between the parts; the level of detail; the mathematical description of surfaces; the scale; the segmentation.

Shading aspects include the surface appearance (textures, material properties, photorealistic vs. abstract shading); the use of light/shadows (position and intensity of lights); the rendering algorithm (biased or unbiased).

Representation aspects and methods concern the ways of projecting 3D objects into a 2D plane (e.g., perspective, axonometric, and double orthogonal projection, etc.), this includes camera settings, points of view, anamorphisms, and solid perspective.

Media and interfaces are the technological devices used to visualize 3D models: 2D displays (pixel monitors, projectors, etc.); 3D displays (holographic displays, holograms, 3D stereo displays, etc.); VR/AR headsets; physical 2D drawings, prints, 3D rapid prototyping (3D printing, laser cutting, CNC milling, etc.).

Further reading: Degrees of freedom in 3D visualization

What are the visual properties of 3D/4D visualizations?

The level of detail (LoD) needs to be appropriate to the purpose [3]. While detailed visualizations of historical reconstructions are advantageous for imaginability [4], they may distract scholars from their research questions [5] or lead to cognitive overload [6].

Visual style is often discussed in terms of fit to scholarly recommendations and distraction of viewers [7]. Since early 3D/4D reconstructions strived for more immersive and realistic visualization [8], a current scope comprises a large variety of photorealistic and non-photorealistic styles [9,10,11].

Different degrees of certainty [12,13,14] have led to a multitude of visual strategies for heritage content [15]. Current approaches can be roughly categorized into enrichment of representations by explanatory elements [16] and adaptation of representation quality, e.g., LoD or visual styling [14, 3, 17,18,19,20,21].

Scaling has been frequently assessed as an important parameter for perceiving architecture [3, 22].

Visual acuity is the ability to distinguish details: in contrast to scale, the main influencing factor is the distance to a virtual or physical object [23,24,25,26].

Perspective depiction and perception of architecture include the viewer effect of different fields of view [27, 28, 26]; e.g., the top-down view is the predominant mode for investigating cityscapes [29].

Lighting is relevant to imagine historical architecture [30]. Specific workflows and approaches (based on visual comparisons) are proposed for virtual visualization [31,32,33,34,35,36].

Color is highly relevant for the perception of historical objects and ranges from realistic coloring to scales coding parameters or supporting visual distinction of model parts [37, 38]. Accurate reproduction of colors is challenging in both modeling [39, 40] and reproduction by digital devices [41].

In a research context visualization methods and technologies have a crucial role because they graphically synthesize aspects of the research, from critical reasoning via generating and sharing ideas to dissemination. Visualization choices might enhance the communication of the study or strongly mislead or distort the interpretation of the results.

For example, it was long known that Greek temples were brightly colored, however, the public misconception is still that these temples were made with raw white marble. This misconception is mainly because the ruins that survived the centuries lost their colors and even when these structures were graphically reconstructed through operations of digital anastylosis, they were often represented with a white color in catalogs, magazines, pictures, and museum exhibitions.

The same risk looms over the visualization of any digital 3D reconstruction, thus those professionals aiming to produce scientifically accurate reconstructions have the responsibility not only to produce an accurate model, while documenting every step of the process, but also to communicate it clearly, limiting as much as possible any misconception. Shading and representation play the most important role in this regard.

7.4 Formal/Geometrical Aspects

Quite obviously, among all the elements necessary to produce any rendering of a 3D scene, the 3D geometrical model is the unavoidable starting point from which each other aspect derives. Without a 3D shape, there would be no medium to apply materials and textures into, target to point the camera toward, or obstacle with which the rendering algorithm could calculate the light bounces. All the aspects related to the generation of 3D geometry are extensively described in this chapter ($\to$ 3D Modelling).

7.5 Shading Aspects

In painting and drawing, the word “shading” is usually used to define the process of darkening or coloring an illustration. It is fundamental to give an accurate perception of volume, sense of perspective, and material credibility to the objects. Painters who master the shading technique have deep knowledge of how our physical world works. For example, they know where a sphere is darker or lighter according to the direction of light, that the part of the sphere not exposed directly to the light source can be slightly lighted thanks to the light bouncing away from a near object, that the reflection of the light pointed on the sphere is sharp if the sphere is made with glossy material and smooth if the sphere has a rough material. Shading depends on the shape, the light, and the materials.

To be able to produce the same sphere properly shaded digitally, all this knowledge is not crucial anymore, because the proper calculation of the behavior of light bouncing on the surfaces is all delegated to the render engine through its sophisticated algorithm. A nonexpert, who does not know the laws of descriptive geometry well, may not notice errors in the model or in the automatic calculation of light. The two main types of render engines on the market nowadays are biased (approximated) and unbiased, (accurate). The most popular render engines in architecture and gaming have a certain level of bias because they produce images faster than unbiased engines, as even with a small amount of approximation/bias, they can be still very close to producing physically plausible results. A good render artist can compensate for the biases by artistically manipulating the shading of the scene (adding hidden lights or manually changing the color channel of the surfaces of the objects to simulate penumbrae color bleeding, bouncing light, etc.), so it is still possible to get very close to physically plausible results even with biased engines.

Both biased and unbiased render engines support the applications of synthetically generated procedural textures and reality-based textures. Since synthetically generated procedural textures are computer-calculated images as e.g. fractal patterns; reality-based textures are based on images of material surfaces. This, paired with the ability of the render engines to produce images with a physically plausible light distribution are the ingredients of the recipe for photorealistic rendering. Render engines can also produce non-photorealistic (NPR) images, by applying rules which are not physically based on the distribution of light and the definition of the properties of the materials.

7.5.1 Photorealistic Shading

A completely textured scene with photorealistic textures (Figs. 7.1 and 7.2) and a physically plausible distribution of light is the aspiration of most reconstructions in entertainment and exhibition contexts [44, 45]. However, photorealism is sometimes mistrusted in the academic and scientific context of 3D historical reconstructions [11], not only because the radiosity of the scene can be approximated by itself due to biases of the render engines, but most importantly because it is very hard to retrieve enough information about materials for every object in the scene by direct inference from historical sources. When information is lacking, subjective interpretations has to compensate, which is one of the main causes of misinterpretations.

Fig. 7.1

(Image: Foschi)

Reconstruction of the water jets of Neptune fountain in Bologna. Photogrammetry + laser scanning + liquid simulations [42]

Fig. 7.2

(Image: Foschi)

Reconstruction of the market in Piazza delle Erbe in Verona during the Middle Age. Project presented at Expo 2015 [43]

Photorealism (physical plausibility) and theatricalization, are often priorities for entertainment applications (documentaries, games, movies, museums, etc.). To achieve that, it is necessary to precisely define the properties of every material applied to all the surfaces so the light can properly bounce on them. This requires a level of knowledge that is usually very hard (if not impossible) to retrieve by direct sources and thus it is usually heavily subject to personal interpretations, which is not desirable if the aim is to produce a scientific reconstruction with minimal subjective additions. Nevertheless, photorealistic visualizations can drive academic research forward as they can provide insights otherwise hardly deducible from an abstract visualization.

Thus, can photorealistic texturing ever be scientifically accurate in the context of digital 3D historical reconstruction? The reproduced result does not necessarily need to match the original artifact 100%, as despite the efforts and the abundance of sources, the reproduction will always be an approximation of the original. Scientific reproducibility in this context means that the process is documented so that any other researcher who follows the same process based on the same sources would end up with the same result. Given this definition, we can certainly assert that, yes, photorealistic texturing can be scientifically acceptable as far as uncertainties and subjective conjectures are clearly identified and documented.

Even if all the conjectures are perfectly documented, and the process complies with the requirements of the scientific method, the risk of misinterpretation by a casual viewer facing a photorealistic rendering, heavily based on conjectures, is still high. If there are no visual clues that some part of the scene or texture might have been completely conjectured, this might lead the viewer to think that the scene was exactly like it is pictured. To avoid ambiguities and misconceptions, in academic context photorealistic texturing should be used with caution and only when there is a robust documental basis. The surface appearance uncertainties could be communicated visually, by pairing the photorealistic view with another false color view indicating which areas have which level of uncertainty ($\to$ Documentation).

Further reading: Scale of uncertainty

The scale of uncertainty is a common tool often proposed in the context of 3D reconstruction of unbuilt or lost architecture (some authors use the terms reliability, plausibility, etc. [19]). These scales help to make each hypothetical reconstruction transparent and easier to understand and evaluate. A color is assigned to each level of the scale, and each level has a description usually related to the type and quality of the sources. The colors are then applied to each element of the 3D reconstructed model. This abstract shading gives information about the level of uncertainty of each element and the overall model at a single glance.

Several scales of uncertainties were developed over the years [19]. The scale presented in Tables 7.2 and 7.3 was developed to minimize ambiguities and overlapping between the different levels: it is based on the presence/absence of preserved/damaged sources and their authors [37].

Table 7.2 Level of uncertainty scale [37]

Table 7.3 Level of uncertainty scale, reliability [37]

This particular scale not only gives clues about the type of sources and their author (primary sources by the same author, secondary sources by different authors, etc.), but also about their quality (damaged/undamaged, readable/unreadable, consistent/inconsistent, etc.), and about the level of reliability of the results of the reconstruction based on these sources (reliable/unreliable, conjecture/abstention). Each level of the scale was assigned to a recognizable/nominable color and number. These numbers are a possible alternative to colors in monochrome and visually accessible publications, and can be used to calculate the average global uncertainty of the reconstruction (Fig. 7.3). This specific scale was designed with 7 steps (+ one step for abstention) that can be reduced to 5 and 3 steps without losing comparability (Fig. 7.4).

Fig. 7.3

Left: false-color scale to show uncertainties; Right: calculation of the average uncertainty, Andrea Palladio, Villa Pisani, Bagnolo [37]

Fig. 7.4

(Image: Foschi)

Granularity of the scale of uncertainty [37]

Can photorealistic texturing ever be free of subjective interpretations? Theoretically yes, if it is based on strong evidence. In modern render engines, diffuse color (albedo) of the surface is not the only property that is required to define a proper material; other properties are needed, such as roughness, glossiness, bump, IOR, and so on. Even if we find sources where the materials are precisely described textually and graphically with colored drawings, it is still formally impossible to retrieve only from textual and graphical sources all the properties that modern 3D render engines require. Not to mention that the color of the drawings is influenced by the intrinsic color of the paper and they suffer discoloration or color shifting over time, so even colored documental sources are not 100% reliable.

The only plausible way to be able to identify with maximum objectivity the texture and material properties would be by having a physical sample of the original material. In architectural hypothetical reconstruction, it is very rare to have such a clear and complete source.

It is true that in photorealistic renderings the level of uncertainty is usually higher compared to abstract rendering, but it is also important to not demonize it. Sometimes, e.g. to enable visual assessment, definition of photorealistic surface appearance becomes a crucial aspect of the scope of the reconstruction. In these cases, it becomes even more important to document clearly the process of texturing as much as it is for the modeling.

7.6 Abstract Shading

When the sources about surface appearance are lacking, the most popular alternative to photorealistic shading is the application of a neutral mono-material, usually grey or white, because it is a fast solution and is considered more objective. However, this choice might cause misconceptions similar to those described above for Greek temples and does not add any useful information. The white mono-material is part of the more general method of abstract shading, also called non-photorealistic (NPR) shading, the counterpart of photorealistic (PR) shading. Abstract shading conveys additional information through colors and textures.

One of the most popular ways to convey additional information through abstract shading is to apply false colors to the elements of the model, organized in scales where each step is described through textual legends (Fig. 7.5). False colors can be used to express many different concepts: restorers use red and yellow to indicate which part of the building was reconstructed or demolished; archaeologists use colors to define the different palimpsests and ages; lighting designers use scales of color to identify which parts of the model receive more light radiation; fluid or gas speed in 3D simulations can be described with a false color scale. In digital reconstruction, one of the most popular uses of false color scales is to visually communicate eventual variants, the level of uncertainty or the type of sources used [37] ($\to$ Documentation).

Fig. 7.5

(Image: Apollonio)

Andrea Palladio, Villa Saraceno. False color scale to show uncertainties [46]

Another possible alternative abstract texturing technique is the projection of the graphical sources on the 3D model (Figs. 7.6, 7.7 and 7.8) [37]. This method is aimed to enrich the sources by adding the third dimension, or vice versa to enrich the model through the projection of its 2D graphical sources onto the 3D surfaces of the model, and it does not need any legend to be explicit. A similar shading technique which tries to mimic the graphic style of the sources, but without projecting the sources directly onto the model as textures, is described by Daniela Sirbu [47]: the textures and lights are redesigned entirely by hand trying to reproduce the same lighting, surface appearance, and atmosphere of the reference source. This technique might be preferable to projection when the reference sources are particularly damaged or lacking, or low resolution.

Fig. 7.6

(Image: Foschi)

Projection of the original drawing on the 3D model by Mauro Guidi, Sepolcro antico di figura quadrata con portico di due gradini, Cesena Nuova, Atlante 41, Carta 48. Top: 3D CDM; Bottom: original drawing [37]

Fig. 7.7

(Image: Foschi)

Projection of the original source on the 3D model. Claude Nicolas Ledoux, Atelier des gardes de la forêt, Cité idéale de Chaux, Tome 1, Pl. 102. Top: 3D CMD; Bottom: original drawing [37]

Fig. 7.8

(Image: Architectura Virtualis, Darmstadt 2018)

Projection of the original sources on the 3D model, Art gallery castle Schwerin

Borderline techniques lie between photorealistic and abstract. For example, the black and white view can be close to the photorealistic rendition because it can use textures from photos and the distribution of light can still be physically plausible, but undeniably not realistic in a strict sense because the information about the colors was removed. This solution decreases the level of uncertainty when color information is lacking. Rather than converting the renderings from colored to black and white as the last step, it is also possible to convert each texture to black and white before rendering; with this solution, however, the surfaces would eventually inherit a bit of color from colored light sources.

Other abstract shading techniques (Fig. 7.9) used for various scopes are listed below:

Fig. 7.9

(Image: Foschi)

Some abstract shading techniques

• Wireframe

• Ghosted

• X-ray

• Cartoon/Comix

• Silhouette

• Hand drawn

• Flat colors

• Technical drawing

• Ambient occlusion

• Water colored

• Inverted colors

• Black and white.

All these techniques, and many others not listed here, are very different from each other, but share the fact that they do not try to mimic reality. Some of them have only ornamental and aesthetic uses, while others can be useful for enhancing the readability of the images, bringing attention to particular details, or showing hidden details. For example, through wireframe/ghosted/X-ray shading it is possible to look inside the models without disassembling them, or through flat color shading, it is possible to see the albedo of the surfaces without the effect of lights and shadows.

Abstract shading is sometimes used only to characterize some elements in the scene, while others are photo-realistic textures. This solution can be useful to highlight explicit elements with known surface appearance (Figs. 7.10, 7.11 and 7.12). Sometimes abstract shading can also be used as an additional shading style, that accompanies photorealistic views, to highlight elements in the scene (Fig. 7.13).

Fig. 7.10

(Image: Architectura Virtualis Darmstadt, 2019)

Cartoon style + photorealistic textures to indicate different surface appearance uncertainty, Carolingian Church Saint Gall

Fig. 7.11

(Image: Architectura Virtualis Darmstadt 2019)

Reconstruction in ghosted style to show the photo-realistically textured plan underneath, Plan of Saint Gall

Fig. 7.12

Photorealistic texturing + gray monomaterials to indicate surface appearance uncertainty. Reconstruction of the exhibition organized by Antonio Canova around 1818, Church of Santo Spirito, Bologna [48]

Fig. 7.13

(Images: Stadt Köln, Dezernat Kunst und Kultur, VII/3—Archäologische Zone/Jüdisches Museum, MiQua. LVR-Jüdisches Museum im Archäologischen Quartier Köln and Technischen Universität Darmstadt, Fachgebiet Digitales Gestalten, 2019)

Left: bird’s eye view of a cartoon-style shaded reconstruction; Right: the same reconstruction framed at human height with photorealistic shading, the medieval Jewish quarter in Cologne

7.7 Representation Aspects and Methods

Nowadays the traditional methods of representation are used mostly to visualize 3D models. The visualization of 3D models uses various projections. Most relevant are:

• Double orthogonal projections (Fig. 7.14)

  Fig. 7.14

  

  (Image: Foschi and Fallavollita 2024)

  Double orthogonal projection: Villa Pisani Riba by Palladio

• Axonometric projection (Fig. 7.15)

  Fig. 7.15

  

  (Image: Foschi 2024)

  Oblique axonometric projection: Villa Pisani Riba by Palladio

• Perspective projection (Fig. 7.16)

  Fig. 7.16

  

  (Image: Foschi 2024)

  Perspective projection: Villa Pisani Riba by Palladio

• Topographic terrain projection (Fig. 7.17).

  Fig. 7.17

  

  (Image: Foschi 2024)

  Terrain topographic projection: Contour lines

These methods start from the classical projection from a center and section (with the picture plane). We can identify two general cases: conical projection generating perspectives, and parallel projections (where the projection center is a direction) that generate axonometric projections and double projections. If the projection direction is perpendicular to the picture plane, we can obtain an orthogonal axonometric projection or a double orthogonal projection (plan and elevation, or section). If the projection direction is angled relative to the picture plane, we will obtain an oblique axonometric projection.

Double orthogonal projections are still used as a starting point to draw the 2D blueprints used as references for 3D models: plans, sections, and elevations. After building the 3D model, it is possible to extract an orthogonal view, an axonometry, or a perspective automatically. These methods of representation/visualization can be differentiated according to their use/scope. Double projections are used to geometrically and metrically describe the project; for example, the plan is used to control the positioning and size of spaces, walls, and furniture. Sections and elevations in general are used to design and communicate height. The axonometric view is used to study the relations between volumes and spaces. Lastly, the perspective view is used to formally control the perception of space: if used correctly it is the only method capable of representing the space viewed from the inside, from the point of view of an imaginary human visitor (placed in a measurable point in the digital space). In contrast, double projections and axonometric projections always have a point of view outside the designed space (since their point of view is infinitely far).

There are three types of orthogonal axonometry: trimetric, dimetric, and isometric. This distinction addresses the three ways to view the three orthogonal Cartesian axes X, Y, and Z when viewed with a certain angle. The three orthogonal axes shrinking ratio and orientation are the main focuses of axonometric projection; even its etymology encapsulates these concepts. In digital representation, generating an orthogonal axonometric view is automated by the software, in fact, almost every 3D modeler or visualization software integrates the axonometric view as one of the standard visualization modes. Traditionally, the distinction of these three main types of axonometric projections was important to differentiate which construction to use (usually performed with a ruler and compass), but in computer programs, it is not important anymore because the visualization is automatic from any angle and generally, we can choose the best axonometric view based on other criteria, such as the best angle of view.

In some rendering applications, axonometric views are created upon the activation of the specific viewport option or through virtual cameras placed inside the scene. However, some cameras, such as physical cameras, might not have any option to make orthogonal views, because to be able to generate parallel projections they should be placed infinitely far from the target, and this would not be acceptable for a simulation of a physical camera. A workaround that would return a good approximation of an axonometric projection would be adopting a very narrow field of view or long focal length (500/1000mm or more) and moving the camera very far away in the direction of the axis of view [49]. With this expedient, the final projected view would be very similar to an axonometric projection because the perspective foreshortening would be minimized. Even a professional with a keen eye would not be able to notice the difference without measuring some distances with highly accurate tools. This method is also useful to obtain approximations of obliquus axonometric projections (such as the military axonometry where the plan is in true form) which are much rarer than standard visualization modes in 3D computer applications.

For perspective views, it is important to know some principles of descriptive geometry that help to frame the scene. The first issue is the definition of the viewpoint. As mentioned, perspective is the only method capable of framing the space from the inside. This important quality can be exploited to see the space at human height. Even if the virtual cameras have many options that can be set, the perspective projection only depends on the viewpoint location. The focal length and the field of view of the virtual cameras depend on each other and only determine the amount of scene that is captured inside the frame. Shorter focal lengths and wider field of view capture more space (and vice versa) but changing these values will not change the perspective projection as far as the point of view is not moved. Given that, there are a few good practices that one can follow to capture good architectural views:

• Place the viewpoint at realistic heights (e.g., for views at a human height the average height would be 1.5/1.7 m).

• Keep the projections of the vertical lines of the 3D object parallel to the sides of the frame (to do so, it is sufficient to keep the camera projection plane perfectly vertical, or the camera and its target at the same height). This is not valid for bird’s eye views or shots that try to emphasize the height of a building.

• Choose the appropriate picture ratio and field of view of the virtual camera according to what we want to include in the shot and according to the formal aspects of the content of the shot.

Virtual cameras are often similar to real cameras; thus, we refer to 35mm cameras. Focal lengths of 70/100mm are comparable to telephoto lenses, 35/50 mm are considered standard focal lengths, 18/28 mm are considered wide-angle lenses, and 18mm or less are considered fisheye lenses. There is no univocal rule to choose the best focal length in every situation because every architecture is different and can be appreciated with different lenses. However, it is preferable to avoid very short focal lengths for framing historical architecture because apparent deformations (more visible along the perimeter of the picture) would not allow the viewers to appreciate the spaces properly and would distort the perception of proportions and shapes [27]. Contemporary architecture might benefit from the use of wide-angle lenses because the extreme and unconventional spaces and shapes would be emphasized by these apparent perspective deformations.

Apparent deformations in the field of visualization can be exploited to make anamorphic illusions. An iconic case in the field of architecture is the Church of Sant’Ignazio di Loyola in Campo Marzio, Rome. This church is well known for the paintings by Andrea Pozzo (1685) where, if standing from a specific point, the visitors can perceive the illusion of an additional architectural order that breaks through the vault. All the paintings performed with the technique known as trompe-l’œil are anamorphic illusions.

Other perspective illusions in the field of architecture can be obtained through the theory of solid perspective. In the solid perspective, the perspective space is not confined anymore in the picture plane, as in the previous examples, but it is expanded into another 3D space [49]. Some architects in the past used this technique to produce impressive perspective illusions, such as Andrea Palladio in the Teatro Olimpico, Vicenza (1580) or Francesco Borromini in the Galleria Spada, Rome (1652).

Another practical use of solid perspective in digital reconstruction can be seen in the study of the error committed by Leonardo da Vinci in his preparatory drawing of the Adoration of the Magi now housed at the Louvre, Paris [50]. Thanks to digital tools it is now possible to study the transformation of a shape from the real space to the perspective space, and vice versa, dynamically [51].

7.8 Media and Interfaces

The most popular media and interfaces to view any digital content comprise any type of 2D display such as the ones mounted on smartphones, TVs, laptops, desktop PCs, and so on. Old technologies that are still available but have rapidly fallen into disuse, such as cathode-based displays, have given way to the newer LCD and LED displays. These are mostly 2D but can also be 3D, namely, they can reproduce two images at the same time that can be synchronized to give the illusion of depth through the stereoscopic view. 3D standard displays usually need secondary optical devices as e.g. polarizing filter glasses, Other techniques as parallax displays give the perception of depth without the need for secondary optical devices. Approaches like head tracking or eye tracking try to improve the visualization depending on the viewer's position of depth perception or the level of interactivity at the cost of limiting the number of viewers that can use the display at the same time. Beside those fixed screen devices there are several technologies using head mounted displays, which require user devices such as headsets or holographic glasses. Other recent strands using fixed screens are holographic visualizations, including volumetric displays of heritage objects [52]. Deviceless approaches include rapid prototyping of manufactured models or printed images, which can be observed by viewers without specific devices.

7.8.1 Images and Films

Visual media and perception are the main sources of information, communication, and research on cultural heritage [53,54,55] and images are still the most frequent output of 3D modeling projects [56, 57]. Images are used in films and animations [58] or as still images. Formats and purposes differ greatly between the main target groups: the general public and domain experts [59]. Visualizations equally serve to represent relations, processes, and a constructive structure [60]. All these types of outputs are usually presented through 2D, or 3D displays shared digitally (online or offline) or printed on a physical support (e.g., paper, cardboard, wood).

7.8.2 Extended Reality

In contrast to asynchronous media like images and films, extended realities (XR) provide a real-time, high level of engagement visualization, which is not constrained to a single point of view and allows variable levels of interaction between the users and the virtual content [61]. A wide scope of interactive technologies [62] lies on the continuum between real and virtual [63] (Fig. 7.18). This ranges from real environments, augmented realities (AR), and augmented virtuality (AV)—together called mixed reality (MR)—to virtual reality (VR), or fully computer-generated visual representations. According to Russo [64], MR can be additive by adding information that does not exist, or subtractive by hiding or deleting parts of the real world. Layouts for user interaction are highly dynamic, perspective-dependent, and require a high degree of temporal coherence [65, 66]. So far, the representation of time-independent data has been investigated most and developed into interaction patterns [67, 68]. Overviews about XR applications in cultural heritage have been presented in various publications with regard to museums [69, 70] and virtual tourism [71].

Fig. 7.18

(Images: Left: Sauer 2019, Middle: Niebling 2019, Right: Münster 2019)

Qualities of extended reality visualizations

7.8.3 Rapid Prototyping

Rapid prototyping can be a useful visualization medium because it allows direct interaction with the model without needing prior knowledge of digital tools, so it is a more democratic way of presenting 3D objects. Furthermore, compared to a model visualized on a 3D screen, it helps to better understand the relation between 3D volumes and how light interacts with them. Lastly, it is less susceptible to digital deprecation, which might make older digital 3D models to not be accessible in future. However, 3D physical maquettes are susceptible to wearing and can break; are longer to produce, refine, and modify; and they are not suitable for joint remote working or inspection at different scales.

A wide range of rapid prototyping techniques is used to create physical representations of tangible heritage [72] ($\to$ Fig. 7.19). This comprises additive manufacturing techniques that add layers of material to a 3D structure and subtractive techniques shaping a 3D structure by removing material [73,74,75, 76]. The used technology highly depends on the material—e.g., thermic processes such as FDM require materials with a low melting point, while sintering or milling approaches require specific physical properties of the material [77]. Another major influencing factor is the scale of reproduction, ranging from 1:1 reproduction at various scales to miniaturizations (e.g., city models) or maximizations of very small objects. There are three highly relevant technologies for additive manufacturing. (1) stereolithography, which builds objects in layers by tracing a laser beam on the surface of a vat of liquid photopolymer and hardening by UV or thermal processing. (2) laser sintering is based on powders (e.g., metals, polymers, composite materials) applied in layers and selectively fused or melted at the surface by laser bonding to the layer below; after removing unprocessed powder, a 3D structure remains. (3) Fused filament fabrication is application of thermoplastic material in layers, which are heated and deposited layer by layer [78]. Subtractive approaches comprise computer-controlled milling tools (CNC machinery), which mill 3D structures out of solid blocks of material [79] or laser engraving, which uses a cross laser beam to selectively overheat spots within solid glass and cause micro-cracks with differing refraction properties [80].

Fig. 7.19

(Images: Left/Middle: Architectura Virtualis GmbH 2016, Right: Münster 2009)

Rapid prototyping

7.8.4 Interaction and Motivational Design

The visual appearance, user presentation, and interaction with 3D content are marked by the level of interactivity and pose various design variables. Specifically for educational settings, motivational design is of relevance for engaging with users and enabling learning.

For interaction design, the interaction is classified by the engagement taxonomy developed by Grissom et al., which differentiates between six degrees of interactivity for visual output [80]. According to Tullis and Albert, user experience “refers to all aspects of someone’s interaction with a product, application, or system” [81, p. 15].

For digital interface design, many interaction parameters are technically determined—e.g., by the capabilities of VR glasses. Especially, XR applications go well beyond passive information presentation and enable multiple freedom degrees for interaction design, as well as user requirements such as avoiding motion-sickness due to continuous movement [82, 83]. With regards to acceptance, the perceived usefulness of applications plays a major role, which in turn depends on optimal stimuli of representation and interaction [84]. Layouts for user interaction are highly dynamic and perspective-dependent and require a high degree of temporal coherence [65, 66]. Currently, mainly the representation of time-independent data has been investigated and developed into interaction patterns [67, 68], since validated strategies for time-dependent 4D data presentation are still missing.

Especially in cultural institutions and at sites, digital 3D applications are embedded in physical spaces. There are various recommendations for the design of linked virtual and physical spaces, e.g., in museums [85, 86] or at heritage sites via location-based 3D applications [87, 88]. Some recent evolvements comprised multi-user story-based virtual experiences in physical museums, enriching museum visits with augmented experiences [89].

Motivational design involves gamified and playful approaches. According to Deterding et al., gamification implicates the use of “elements of games that do not give rise to entire games” [90, p. 2]. Playful design—in contrast to gamification—contains no rules or specific goals, and serious games are defined as full-fledged games for non-entertainment purposes [90]. All three types are intensively used with 3D contexts in heritage, particularly in educational settings. Examples are:

• Playful design: Examples are Minecraft-like creation games [91] or massive open online environments (MUVEs) as 3D social interaction spaces [92].

• Gamification: Applications using gamified elements and strategies to enhance interaction with heritage content [93].

• Serious games: Games either developed for didactic purposes or derived from entertainment games [94, 95].

Storytelling stands for the use of narratives and fictional or non-fictional stories to present a subject [96]. Psychological [97,98,99] and educational studies [100] have demonstrated that narratives can have both a positive motivational effect by making a subject more alive and engaging to the audience, and a positive learning effect by reducing cognitive load. Storytelling is widely used to digitally present heritage content [101], and particularly for digital 3D models in humanities in various educational settings [102,103,104,105,106]. Among the multitude of freedom degrees for design (e.g., by fictionality, media, storytelling modes, or poly-vocationality) [101], general types of scenarios from an educational perspective are:

• Expositional: Story elements are used in a pre-scripted way to explain specific 3D objects. Examples include the presentation of underwater archaeology through stories told by an avatar [102].

• Explorative: Open-world games or open-ended platforms encourage dialogic encounters where the user has various choices to interact with a dynamic supply of narrative contents. Examples include the discovery mode in the game Assassins Creed [107] or dynamically scripted heritage experiences [108].

• Constructive or connective approaches: Users can co-create stories and contribute story content and share with others. An example is the Jena4D project where users contribute location-based personal stories and photographs in a 4D environment [109].

Summary

This chapter deals with the topic of visualizing architectures that have never been built or have been lost, including formal, geometrical, shading, and representation aspects, media and interfaces, interaction and motivational design. All the concepts are treated in terms of the digital reconstruction of cultural heritage.

Standards and guidelines

• Beacham, R.; Denard, H.; Niccolucci, F. An Introduction to the London Charter. In Papers from the Joint Event CIPA/VAST/EG/EuroMed Event, Ioannides, M., Arnold, D., Niccolucci, F., Mania, K., Eds.; 2006; pp. 263–269.

• Denard, H. (2009) The London Charter. For the Computer-Based Visualization of Cultural Heritage, Version 2.1. (https://www.londoncharter.org, accessed on 1.2.2023).

• Principles of Seville. International Principles of Virtual Archaeology. Ratified by the 19th ICOMOS General Assembly in New Delhi, December 2017 (http://sevilleprinciples.com, accessed on 1.2.2023).

Projects

• The Computer-based Visualization of Architectural Cultural Heritage (CoVHer) project is a European project focused on fixing shared standards at the European level for the field of digital 3D reconstruction for heritage, and on the dissemination at the high education level for students and scholars.

• Time Machine is aiming to join Europe’s rich past with up-to-date digital technologies and infrastructures, creating a collective digital information system mapping the European economic, social, cultural, and geographical evolution across times.

• The Inception European project realizes innovation in 3D modeling of cultural heritage through an inclusive approach for time-dynamic 3D reconstruction of artifacts, built and social environments. It enriches the European identity through an understanding of how European cultural heritage continuously evolves over long periods of time.

Key literature

• Migliari R, Fasolo, M. (2022) Prospettiva Teoria. Applicazioni Grafiche e Digitali. Hoepli Editore, ISBN 9788836008841 [49].

• Akenine-Moller, T., Haines, E., Hoffman, N., Pesce, A., Iwanicki, M. and Hillaire, S. (2018) Real-Time Rendering, 4th Edition. ISBN-13: 978–1138627000, ISBN-10: 1138627003 [110].

• Pharr, M., Jakob, W. and Humphreys, G. (2016) Physically Based Rendering: From Theory to Implementation, 3rd Edition. ISBN: 9780128006450 [111].