Abstract
Taihu stone is an important landscape element in Chinese Private Garden on Southern Yangtze river, which is known for its profound cultural and aesthetic significance. In this paper, we intend to introduce the 3D spacial character of Taihu stone into architectural forms and spaces using machine learning, trying to explore the possibilities of AI-assisted 3D form finding and design latent space exploration. Existing spacial transformation of Taihu stone is mainly demonstrated by continuous section slicing, which cannot directly analyze and generate 3D space, thus cannot learn the most characteristic internal space of Taihu stone. This paper used the latest 3D point cloud probabilistic diffusion model to achieve 3D form generation and interpolation of Taihu stone and architectural massing through latent space exploration. Experiments show that a sufficiently trained diffusion model can generate 3D point clouds of Taihu stone and building massing, as well as generate interpolations between them. The latent vector can be manipulated to generate outputs that are more oriented towards the Taihu stone or the building massing, to meet the different needs of designers. Generated point clouds can be reconstructed into triangle meshes or voxelized, as a morphological prototype for further design implementation. Generated forms are capable to provide inspiration and reference for the designers to create free forms, showing the potential of the diffusion model to assist architecture design in conceptual phases.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
The pursuit of innovative forms is a constant topic in architectural design. With the development of deep learning in recent years, form-finding based on the case study and data research brought designers more innovative techniques. Seeking the combination of two elements to develop innovative forms is a common technique in architectural design. For example, exploring the expression of exotic styles in local environments, and discovering new application scenarios for traditional forms and elements in modern contexts. Charles Jencks believes that postmodernism has a dual nature, that is, modern architecture is often combined with something else (usually traditional architectural methods). The solution for the postmodernist movement is to create an architecture that is based on both new technology and old paradigm, as well as being professional and popular [1]. This is usually a challenge for architects. Algorithmic assistance makes design results unpredictable and has great advantages in assisting designers in the early stages of creation and shape finding, but it also requires a new workflow to support this shift in mindset [2]. The machine works in a unique way that can be combined with cultural imagery to produce creative morphologies. A machine can quickly acquire and learn data across historical and geographical dimensions, assisting human architects to understand cultural imagery. In addition, because machine learning is not influenced by emotions or personal preferences, it can also give integrated solutions to different cultural imaginaries and provide a different perspective for human designers, even incorporating different cultural imagery or historical contexts. Campo et al. have combined the characteristics of outstanding examples from architectural history with specific design contexts to generate creative images and inspire the design of 3D architectural spaces [3].
1.1 Traditional Cultural Imagery Transformation in Modern Chinese Architecture
This paper takes Modern Chinese Architecture as an case study. The concept of Modern Chinese Architecture is among the most popular topics that Chinese architects have been exploring and practicing in recent years. The core philosophy of Modern Chinese Architecture is the inheritance of Chinese elements and the pursuit of innovation. The meanings of Modern Chinese Architecture are evolving as time changes, with the connotation of Chinese elements gradually changing from the initial traditional architectural elements such as large roofs to a broader one. Some culturally rich landscapes, components, appliances, etc. have been abstracted and transformed into modern architectural volumes and urban environments.
Architects like Wangshu and Lixinggang are trying to introduce the spatial characteristics of Taihu stone into the architectural space to create innovative forms within the philosophy of Modern Chinese Architecture. Taihu stone is an important traditional cultural imagery in the private gardens in China (Fig. 1), with rich cultural connotations (Fig. 2). The morphological characteristics of Taihu stone are exceptionally complex and varied, formed by years of erosion and carving by water or acidic soil in nature, retaining only the hard part of the limestone texture [5]. Its porous and intricate forms coincide with principles of modern architecture in transparency and flowing spaces. For example, in Taihu house designed by Wangshu, the highly abstract geometric shapes of Taihu stone are used as the prototype of the architecture, which reproduced and reinterpreted the spatial characteristics of Taihu stone in the architectural space (Fig. 3).
1.2 Diffusion Probabilistic Models
The cutting-edge diffusion probabilistic model is introduced into the architectural spatial translation of Taihu stone in this article. The diffusion probabilistic model is a machine learning model inspired by the simulation of the reversed diffusion process developed by Ho et al. in 2015, which can generate high-resolution pictures from random noise [7]. A standard diffusion model is composed of two main process domains: forward diffusion and reverse diffusion. During the forward diffusion period, the original dataset is contaminated with gradually introduced noise until the image becomes completely random noise. In the reverse process, the data are recovered from Gaussian noise by gradually removing the predicted noise at each time step using a series of Markov chains. The diffusion model is more diverse in generating results than the generative model GAN, which has been developed maturely in recent years, and the process of training and generation is extremely tractable and flexible [8].
This paper intends to use the latest 3D point cloud diffusion model to conduct experiments on 3D shape finding and shape interpolation generation for a complex and porous 3D morphology. Taihu stone is taken as the case in this paper. We try to make a practical transformation of the generated morphology to form a 3D shape-finding workflow at the early stage of design.
2 Related Works: Form Finding and AI Creativity
Generating design alternatives through shape interpolations is a common tool to boost AI-assisted form creation. As early as 1988, Hong et al. used the faceted representation method to achieve interpolation between two morphologies, and the generated results can be used for the simulation of biological evolutionary processes, animation, and portrait robots [9]. Metrics such as the connection between cubical contents (V) and surface quadrature (A) and the relationship between V/A are examined. An interactive platform is constructed to achieve a user-friendly evolutionary design workflow for designers [10]. Recently, machine learning tools such as CNN and GAN-assisted design workflows have made great progress in 2D generations. Corresponding 3D generation is inspired and reconstructed on basis of 2D planar. In deep learning, interpolation generation is often combined with the high-dimensional vectors obtained during training to generate images with semantic meanings. Chen et al. investigated the potential of different representation learning-related techniques in the latent space semantic representation in GAN generation models based on data from the SUN database. The experiment achieved the generation of more diverse and tractable design alternatives [11]. Zhang et al. developed a technique to implement the transformation between 2D pixels and 3D voxels of continuous sections from architectural volumes to build a 2D pixel to 3D voxel workflow [12]. With the workflow, the author improved the ability of machine learning to provide designers with several intermediate solutions between two design styles. However, the generation of low to high dimensions is restricted by manipulation techniques and the designer's own thinking stereotypes. The generated shapes will be confined to predefined rules, and the potential of AI in aiding innovative design will be difficult to explore.
The complex morphology of Taihu stone has also been explored by scholars as a prototype for promoting innovation in complex forms. Feng et al. simulated and optimized the curved surface topology of Taihu stone using CFD and BESO, resulting in a complex porous form [13]. Similarly, Ye et al. developed a computational algorithm for generating tafoni (porous rock morphology similar to Taihu stone), using evolutionary algorithms and 2.5D descriptive algorithms [14]. Furthermore, they explored the practical value of such complex porous forms in architectural design. The aforementioned studies are based on rule-based reconstruction of Taihu stone morphology, which is a cumbersome process and may result in limited outcomes. Notably, Liu et al. employed deep learning techniques to extract and grasp the spatial characteristics of Taihu stone based on labeled cases [15], and the 3D morphologies are reconstructed by continuous sections. However, the experiment did not propose specific methods or references for the translation of the morphological elements of Taihu stone into architectural space, and there is also limitation of dimensions.
It is evident from the related studies that most of the current research in the field of computer-aided form finding and creativity is focused on generating images, with limited emphasis on directly generating three-dimensional forms. Additionally, research on the translation of Taihu stone morphology in architectural space has mostly focused on the study of the complex form of Taihu stone itself, without providing abstract methods that can be directly applied to architectural form. Thus, this paper aims to explore the spatial characteristics of Taihu stone from a 3D perspective using the diffusion model latent space, in order to provide alternative solutions for form finding in three-dimensional architectural space.
3 Method
This paper intends to build a 3D form-finding workflow (Fig. 4) based on diffusion probabilistic models for designers to provide them with morphological alternatives between imagery and target massing blocks. The whole workflow is composed of the development of the front-end and back-end. The back-end work mainly includes training database construction, diffusion model training, and interpolation model construction. The back-end development environment and pre-trained models are packaged as the basis for the front-end development. The front end consists mainly of a user-friendly interface incorporating a GUI, and several 3D reconstruction methods that easy for architects to further develop the architectural spaces. The front end enables the latent code generated by the algorithm not only for the design space but also as a modeling tool that supports multiple operations to generate 3D point clouds.
3.1 The Back-End
Data Preparation
This paper aims to propose a paradigm for deep learning to help designers in 3D shape finding, and Taihu stone is selected as a case for the rich cultural connotations it contains. The 3D model dataset of Taihu stone in the experiment is constructed based on the natural generation process of Taihu stone (Fig. 5). The matrix of Taihu stone is limestone, and its complex and porous form was formed through thousands of years of weathering and water erosion. Therefore, in this paper, we simulate the process of generation of Taihu rocks with rhino combined with grasshopper: firstly, 50 samples of high-quality Taihu rocks with different morphological characteristics that meet the aesthetics of traditional Chinese literati are collected as references; the contours of the rocks are generated with Voronoi algorithm; we used surface subdivision on the rock contour to simulate the natural weathering process; finally, the porous morphologies formed by water erosion is simulated with ant colony optimization. To generate the transition form between the building massings and the target Taihu stone, 50 randomly generated building massings are added to the training dataset.
Before training, the data is required to be processed into 3D point cloud format first. The triangular mesh model of the Taihu stone and the randomizer block is sampled into a point cloud format consisting of 8192 points and normalized to a spatial coordinate system between -2 and 2, which facilitates the extraction of features by the deep learning model.
Model Training
The model used in this paper is a 3D point cloud diffusion model developed by Luo et al. based on the standard diffusion mode [16]. Luo et al. added a PointNet structured point cloud auto-encoder to the standard model so that the model can generate point cloud data by denoising process.
The training process contains two stages: encoding and decoding. Firstly, the processed 3D point cloud data is input to the PointNet auto-encoder, and the input point cloud data is encoded into a 512-dimensional latent space vector Z. Z is added as a parameter to each step of the noise-adding process in the forward diffusion process, and the target of the reverse diffusion process is to predict the noise in the forward diffusion. During the decoding process, the noise addition of the forward process and the denoising of the reverse process are cycled continuously, until the predicted noise generated by the reverse diffusion process is highly fitted to the real noise. The fully trained decoder generates 3D point cloud data that highly reproduces the features of the target dataset (Fig. 6).
Shape Interpolation Algorithm
The shape interpolation generation is mainly achieved by manipulating the 512-dimensional latent vector z generated from the input samples. Inputting the two target morphologies into a fully trained PointNet auto-encoder can generate latent codes z1 and z2 corresponding to the two morphologies. The line-space algorithm can derive interpolation between the two latent space vectors. A sufficiently trained decoder can decode the high-dimensional interpolation vectors into a new 3D form to generate a transition form between the two target forms (Fig. 7).
In this paper, the fully trained model can generate intermediate shapes of Taihu stone and building volumes. The experimental results show that the model can generate 3D shapes with a mixture of different categories, rather than simply finding the average value of the coordinates of two target point clouds (Fig. 8). The number of interpolations is an adjustable input parameter of the testing model, which is set to facilitate the users to adjust the features of the generated morphology closer to Taihu stone or cubic massings in real-time.
3.2 The Front-End
Leveraging on the previous work, the resulting pre-trained model and generation environment can be packaged as the basis for front-end development. The latent space parameters are allowed to be edited by users (Fig. 9), making the latent space of the diffusion model not only an exploratory space that can be used for design research but also can be transformed into a practical 3D shape-finding and modeling tool for designers. Further reconstruction techniques include the down-sampling of point clouds and reconstructed surfaces via open3D to further optimize morphology and abstracting features for architectural design (Fig. 10).
Mesh Reconstruction
To enable observation and manipulation by designers, the front end must first reconstruct the 3D point cloud generated by the model into a mesh. This paper uses the surface mesh reconstruction method provided by the open3D platform, and further optimizes the reconstructed mesh for surface refinement in Rhino. Through further subdivision and smoothing of the reconstructed surface, a smooth free-form surface shape is obtained, which can serve as a prototype for free-form architectural forms or skins.
Voxel Down-Sampling
Another approach is to downsample the generated point cloud, abstracting the main features of the generated shape, and then voxelizing it into a more universal and easy-to-operate cuboid form. By adjusting the size of the voxels on the X, Y, and Z axes, the size of the reconstructed voxel units can be controlled. Users can adjust the size and proportion of the voxels according to their requirements for the shape. The generated cuboid form can be edited globally and partially as a prototype for architectural design.
4 Discussions
This article is based on the cutting-edge 3D point cloud diffusion model and combines various methods for point cloud 3D reconstruction, to establish a designer-friendly 3D shape-finding workflow. The main goal of the workflow is to assist designers in generating alternative design proposals between two target forms. The generated proposals preserve the selected style and site volume while also possessing a certain level of creativity.
The main methodology of this paper is to establish an artificial intelligence-assisted design exploration pipeline based on the diffusion model, to discover the lateral thinking and innovative design capabilities of machine learning based solely on 3D datasets. The pre-trained PointNet auto-encoder in the diffusion model encodes a high-dimensional latent code Z during the generation process. By interpolating Z and inputting it into the pre-trained 3D diffusion decoder, the intermediate shapes between two input forms can be obtained. These intermediate shapes possess elements of both forms and have the potential to further develop into building forms. In the front-end part of this paper, two methods: surface reconstruction and voxelization downsampling are proposed to further abstract the generated forms into building forms, demonstrating the potential of the diffusion model in assisting architects in complex form design.
The Practical Value of Taihu Stone Shape Transformation
In addition, this paper intends to explore the possibility of artificial intelligence-assisted incorporation of traditional cultural intentions into architectural space and to explore the depth of artificial intelligence can explore in fields of architectural design cognition.
In architectural design research and practice, seeking the combination of two elements to inspire new forms of design is a commonly used approach. The significance of transforming and combining traditional elements such as Taihu stone with modern architecture lies in creating unique architectural forms by reinterpreting traditional elements and integrating them with modern architecture, making buildings more artistic and culturally meaningful. At the same time, the implementation of traditional elements can also raise awareness of historical culture, promoting cultural inheritance and development. In addition, the combination of traditional elements with modern architecture can also expand the ideas and methods of architectural design, bringing more possibilities and innovations to architectural design. Building on the research and practice of predecessors such as Wang Shu and Li Xinggang, this paper studies the transformation and application of Taihu stone formative elements in modern architectural massing, exploring the directions of Taihu stone form translation in practical applications. The form of the “Taihu house” is restored in the reconstruction process, providing a machine-learning interpretation of modern Chinese architecture.
References
Schumacher, T.L., Jencks, C.: The language of post-modern architecture. JAE [Internet] 32(3), 30 (1979). https://doi.org/10.2307/1424566
Hovestadt, L.: Beyond the Grid–Architecture and Information Technology: Applications of a Digital Architectonic. 978th-3rd-0346-0096–5th ed. Birkhäuser (2012)
del Campo, M., Carlson, A., Manninger, S.: Towards hallucinating machines—designing with computational vision. Int. J. Archit. Comput. 19(1), 88–103 (2020). https://doi.org/10.1177/1478077120963366
Chen, J.: Taihu Stone and the Spirit of Chinese Culture (2021)
Taihu, The Famous Chinese Lake Tai Stone [Internet]. VSANA. https://www.vsana.org/newpage0b2a1951
Zhang, M., Baek, J.: Appropriation of Taihu stone and its formal evolution in Wang Shu’s architecture. J. Asian Archit. Build. Eng. 1–17 (2022). https://doi.org/10.1080/13467581.2022.2145211
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Adv. Neural. Inf. Process. Syst. 33, 6840–6851 (2020)
Yang, L., Zhang, Z., Song, Y., et al.: Diffusion models: a comprehensive survey of methods and applications (2022). arXiv:2209.00796
Hong, T.M., Thalmann, N.M., Thalmann, D.: A general algorithm for 3D shape interpolation in a facet-based representation. Proc. Graph. Interface. 2(88), 229–235 (1988)
Dürr, C.: Morphogenesis–evolution of shape; imagination amplifier version 1.0 (2004)
Chen, J., Stouffs, R.: From exploration to interpretation—adopting deep representation learning models to latent space interpretation of architectural design alternatives. In: Proceedings of the 26th Conference on Computer Aided Architectural Design Research in Asia (CAADRIA) [Internet] (2021). https://doi.org/10.52842/conf.caadria.2021.1.131
Zhang, H., Huang, Y.: Machine learning aided 2D-3D architectural form finding at high resolution. In: Proceedings of the 2020 DigitalFUTURES, pp. 159–68 (2021). https://doi.org/10.1007/978-981-33-4400-6_15
Feng, Z., Gu, P., Zheng, M., Yan, X., Bao, D.W.: Environmental data-driven performance-based topological optimisation for morphology evolution of artificial Taihu Stone. In: Proceedings of the 2021 DigitalFUTURES [Internet], pp. 117–28 (2021). https://doi.org/10.1007/978-981-16-5983-6_11
Ye, W., Chen, S., Zhao, X., Xu, W.: Porous space—biomimetic of tafoni in computational design. Archit. Intell. 1(1) (2022). https://doi.org/10.1007/s44223-022-00019-4
Deng, Q., Li, X., Liu, Y.: Using Pix2Pix to achieve the spatial refinement and transformation of Taihu Stone. Hybrid Intell. [Internet] 359–70 (2023). https://doi.org/10.1007/978-981-19-8637-6_31
Luo, S., Hu, W.: Diffusion probabilistic models for 3D point cloud generation. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) [Internet] (2021). https://doi.org/10.1109/cvpr46437.2021.00286
Acknowledgments
This research is supported by National Natural Science Foundation of China (No. 51978268, No. 51978269).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2024 The Author(s)
About this paper
Cite this paper
Liu, Y., Li, H., Deng, Q., Hu, K. (2024). Diffusion Probabilistic Model Assisted 3D Form Finding and Design Latent Space Exploration: A Case Study for Taihu Stone Spacial Transformation. In: Yan, C., Chai, H., Sun, T., Yuan, P.F. (eds) Phygital Intelligence. CDRF 2023. Computational Design and Robotic Fabrication. Springer, Singapore. https://doi.org/10.1007/978-981-99-8405-3_2
Download citation
DOI: https://doi.org/10.1007/978-981-99-8405-3_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8404-6
Online ISBN: 978-981-99-8405-3
eBook Packages: EngineeringEngineering (R0)