Abstract
Part quality manufactured by the laser powder bed fusion process is significantly affected by porosity. Existing works of process–property relationships for porosity prediction require many experiments or computationally expensive simulations without considering environmental variations. While efforts that adopt real-time monitoring sensors can only detect porosity after its occurrence rather than predicting it ahead of time. In this study, a novel porosity detection-prediction framework is proposed based on deep learning that predicts porosity in the next layer based on thermal signatures of the previous layers. The proposed framework is validated in terms of its ability to accurately predict lack of fusion porosity using computerized tomography (CT) scans, which achieves a F1-score of 0.75. The framework presented in this work can be effectively applied to quality control in additive manufacturing. As a function of the predicted porosity positions, laser process parameters in the next layer can be adjusted to avoid more part porosity in the future or the existing porosity could be filled. If the predicted part porosity is not acceptable regardless of laser parameters, the building process can be stopped to minimize the loss.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
The increasing availability of data from the first three paradigms of science (experiments, theory, and simulations), along with advances in artificial intelligence and machine learning (AI/ML) techniques has offered unprecedented opportunities for data-driven science and discovery, which is the fourth paradigm of science (Agrawal & Choudhary, 2016). Within the field of AI/ML, deep learning (DL) (LeCun et al., 2015) has emerged as a transformative technology in the last few years, and has been extensively used in materials science (Agrawal & Choudhary, 2019; Ge et al., 2020; Gupta et al., 2021, 2022; Jha et al., 2018, 2019, 2021, 2022; Li et al., 2019; Mater & Coote, 2019; Yang et al., 2018, 2019, 2020; Ziletti et al., 2018) to enhance materials property prediction, discovery, and design, as well as in additive manufacturing (AM) (Francis & Bian, 2019; Ghungrad et al., 2021; Imani et al., 2019; Li et al., 2020; Mozaffar et al., 2018; Qin et al., 2020; Scime & Beuth, 2018; Xie et al., 2021; Zhang et al., 2019). Laser Powder Bed Fusion (L-PBF) is a popular AM technique, widely researched in both academia and industries ranging from aerospace to healthcare. However, pores in an AM part lead to stress concentration contributing to the formation and propagation of cracks that are detrimental to mechanical properties, such as tensile strength and fatigue life (Sabzi et al., 2020). Porosity formation mechanism is still difficult to describe, although it is popularly investigated in not only one but also hybrid additive manufacturing process (Webster et al., 2021).
In-situ process monitoring of the melt-pool, laser track, or build layer during the AM process is critical in detecting potential porosity and controlling part quality. Commonly measured signatures are thermal (Bakish, 1983; Nath & Mahadevan, 2021) or acoustic features (Eschner et al., 2020), and optical images (Bugatti & Colosimo, 2021; Li et al., 2021; McCann et al., 2021) before or after powder recoating or powder fusion (Grasso & Colosimo, 2017). However, In-situ process monitoring, and porosity detection in current literature (Paulson et al., 2020; Scime et al., 2020; Zhang et al., 2020) can only detect porosity that has already occurred but cannot predict potential porosity which might occur in the future, a much-desired ability to guide real-time process parameter optimization for better part quality. One possible direction is to understand the process–structure–property relationships by exploring the effect of different processing parameters (i.e., laser power, scanning speed, hatch spacing, etc.) on the microstructure, mechanical properties, or porosity of the L-PBFed parts (Gawade et al., 2022; Liu & Zhang, 2021; Nath & Mahadevan, 2021; Tang et al., 2017). However, simulation methods require prior knowledge and experience from expertise when designing parameters according to different materials, set-ups, geometries, etc.
Given the limitations of the previous studies mentioned above and the idea that one can adjust laser process parameters in the next layer if the potential of porosity formation can be predicted, it is critical to build a porosity prediction system that can predict potential porosity occurrence. At the very least, if such a predictive model existed, the current build could be stopped before unallowable part defects occur using in-situ monitored signals to avoid more serious part porosity and save on machine time.
Inspired by advanced applications of deep learning methods in material science and additive manufacturing, a deep learning framework is proposed in this paper to predict potential porosity formation using in-situ monitored signatures. To the best of our knowledge, this is the first work that attempts porosity prediction before it happens based on in-situ thermal signatures using a deep learning approach. The proposed detection-prediction framework is composed of two modules, a detection and a prediction module. The detection module detects porosity based on thermal signatures while the prediction module predicts porosity in the next layer according to previous layers. Each module can be applied separately or in unison for different purposes.
The related works are reviewed in section “Related works”. The methods used in this work are introduced in section “Methods”, including the description of the dataset in section “Dataset preparation” and the proposed framework in section “Deep learning methods”. The experiments and results are presented in section “Results and discussion”. Finally, conclusions of this work and future work are discussed in section “Conclusions and future work”.
Related works
Simulation methods
Tang et al. (2017) proposed a geometry-based simulation method to predict the volume fraction of the unmelted material (i.e., lack of fusion porosity) in parts built by L-PBF by calculating the overlap between melt pools. It was assumed that melt pools are in the shape of dual half-ellipses on the cross-section along with the scanning and building directions. The inputs into the simulation model were melt-pool cross-sectional area, hatch spacing, and layer thickness. Melt-pool areas were estimated with the analytical Rosenthal equation in which material properties at room temperature need to be carefully collated. However, this simulation model does not incorporate any environmental variables (i.e., environmental temperature, humidity, airflow, etc.) that might lead to a discrepancy in the melt-pool sizes. Nath and Mahadevan (2021) updated the physics-based model at each layer using in situ monitoring data for porosity prediction at future layers. However, lots of prior knowledge is required to build the simulation model, such as heat transfer mechanism in different materials.
Artificial intelligence methods
Deep learning has become the AI/ML technique of choice in the last few years, due to its groundbreaking success in numerous real-world applications (Aghazadeh & Gharehchopogh, 2018; Asghari et al., 2021; Collobert & Weston, 2008; Khataei Maragheh et al., 2022; Krizhevsky et al., 2017; Lin et al., 2019; Niu et al., 2019). Four machine learning methods were adopted by Paulson et al. (2020) to model the relationships between thermal histories and surface porosity formation in Ti6Al4V during single line scans using L-PBF. Each run was equally separated into eight segments based on time and each run had eight data points corresponding to the eight-time segments. For each data point, the input is the thermal history (black-body radiance measured using infrared imaging) at the middle of each time segment and the output is the most serious porosity formation situation of any time segment during the whole run. In the study by Liu and Zhang (2021), physical parameters (i.e., laser energy density and laser radiation pressure) were first extracted from the original machine setting parameters (i.e., laser powder, laser angle, etc.) and used as inputs for six machine learning methods to predict maximum, mean, and median pore diameters as well as median pore spacing. The physics-based model was able to explain pore generation mechanisms and was validated on an L-PBF machine (i.e., Concept Laser M2 machine) with a porosity prediction error of 10–26%. Other significant physical effects influencing pore formation were not considered in their models, such as material energy absorption and heat transfer between material and air.
In the study by McGowan et al. (2022), experimental and simulated laser metal deposition data are applied to train a convolutional neural network (CNN)-based model with physics-informed loss functions for porosity prediction. The results show that the physics-informed model works better than the model without any prior physics knowledge. A CNN-based model named ResNet18, proposed by Zhang et al. (2020), achieved an accuracy higher than 99 % in lack of fusion and keyhole defect detection in L-PBFed single Ti6Al4V beads. Each sample was scanned for 32 s with a width of 200 mm. The inputs to the image classification model were 2D spectral images transformed from audio signals. In the study by Scime et al. (2020), in-situ optical images of the build plate and powder bed at each layer were captured and labeled pixel by pixel. Their results demonstrated that machine learning-based models (i.e., Dynamic Segmentation CNN, etc.) achieved real-time detection and classification of 12 types of defects (such as porosity, super-elevation, soot, etc.) in six different PBF machines. However, for the porosity detection problem, the model was validated only in one PBF machine with a true positive rate of only 78.4%. Although the study by Zhang et al. (2019) presented that their proposed CNN models achieved local volume porosity prediction, their models actually detected pore size using in-process cross-section images or tomography scans after the pores happened instead of predicting them ahead.
Methods
Dataset preparation
The proposed porosity detection-prediction framework will be used to examine the lack of fusion porosity. To obtain the dataset, a part named lack of fusion (LOF) with only lack of fusion porosity was manufactured using an L-PBF machine EOS M290 and Ti6Al4V powder with a size distribution of 23–50 µm. The part had 594 layers with a layer thickness of 30 µm, including 501 part layers, from the 85th to the 585th layer, while the layers below are for building the part supports and the layers above are for part annotations. The part, shown in Fig. 1c, was built in its axial direction as denoted by the red arrow. As shown in Fig. 1a and b, the imparted porosity regions are designed to be spherical, and their diameters decrease from 1.10 to 0.10 mm along the building direction. On a given building level, six porosity regions with the same diameter were designed and distributed in a hexagonal pattern. Fig. 1b shows the top view of parts with the designed six anomalies with a diameter of 1.10 mm. Layer ranges of each designed porosity region are listed in Table 2. For example, a porosity region with a diameter of 1.10 mm is designed to range from the 133th to the 170th layer. When the laser comes to the designed porous regions, less energy is applied to achieve a lack of fusion porosity as shown in Table 1. Optimal processing parameters obtained in preliminary experiments were applied to build the non-defective regions (Table 2).
The built part was tested using computed tomography (CT) and a Standard Tessellation Language (STL) file was created that indicates lack of fusion anomalies in three dimensions, as shown in Fig. 1d. The CT scan data was subsequently transformed into layer-wise labeled images indicating the location of the porosity in each layer of the part LOF. Each pixel in a labeled image has a value. As for a certain pixel in one part layer, if porosity appears at that pixel’s location according to the CT scan data, the label value of the pixel is set to 1, otherwise 0. These labeled images constitute the porosity ground truth for the proposed porosity detection-prediction framework. The supervised learning algorithms are applied in this study.
During the manufacturing process, six thermal features of the melt pool scanned by the moving laser were in-situ monitored by a Co-Axial Planck Thermometry system provided by Sigma Labs Inc. Planck Thermometry is based on Planck’s law. The six thermal features provided by the system are Thermal Energy Planck (TEP), TEP_sigma, Thermal Energy Density (TED), TED_sigma, counts, and TEP_sums, which will be explained as follows. TEP represents the temperature of the melt pool by taking the ratio of two radiation values at two wavelengths. TED is the integrated radiation taken in the whole near-infrared region. Thus, TEP is representative of the temperature of the energy deposited region while TED is representative of the input process parameters and material response. Counts indicate measurement times for each deposited region. The TEP_sums value is the sum of all TEP measurements of the corresponding deposited region. Thermal signatures of all part points are measured right after laser scanning and stitched as images, so that there are six images at each part layer. In other words, each pixel in each layer has six values that correspond to six thermal signatures.
All in all, each layer has six images which represent the six thermal signatures (i.e., TEP, TEP sigma, TED, TED sigma, counts, and TEP sums) and one image that represents the porosity labels. The size of all the images is 160 pixels \(\times \) 160 pixels with one pixel in the imaging system representing 100 µm in the part system. Fig. 2a–f shows six thermal signature images in colormaps and Fig. 2g shows the labeled image of the 185th layer. The values of these six thermal features vary from 10 to \(10^5\), thus, the value of each feature is normalized to 0–1 using the mean and standard deviation correspondingly, which is also applied in the study by Sorkhabi et al. (2020). All images of part LOF built using the EOS M290 machine are collected as a dataset named EOS-LOF (EOS denotes the machine name and LOF denotes the porosity type) with a dimension of 501 layers \(\times \) 7 images \(\times \) 160 pixels \(\times \) 160 pixels.
Sigma Labs Inc. has claimed that the six thermal features described above can indicate four types of defects, including gas porosity, lack of fusion, keyhole, and tungsten inclusion (Betts & Anderson, 2020). In their statement, the average value of all pixels in the window (i.e., 5 pixels \(\times \) 5 pixels) centered at a certain pixel is considered as the feature value of that pixel. All six feature values at the current layer and the previous layer (total of 12 values) were input into ExtraTree Regressor models to predict the probability of each pixel being defective. Each model was trained only for detecting one type of defect. The probability is in the range from 0.0 to 1.0. But in their work, no threshold was applied to its result to actually classify each pixel into porous or non-porous categories, which is done in our study by identifying an appropriate threshold. Secondly, their model can only identify already existing defects rather than predicting potential defects in the future. Thirdly, their way of dividing the dataset into training and testing sets is different (randomly at pixel-level) than the way used in our study (temporally at layer-level, which is a more realistic and stringent evaluation of the predictive capability of the model).
Deep learning methods
A detection-prediction framework based on deep learning is proposed in this paper, which aimed at predicting porosity in the next layer based on the thermal signatures of previous layers is shown in Fig. 3. The framework includes two modules, a detection and a prediction module to be described in detail in section “Detection module” and “Prediction module”, respectively. The inputs are the thermal signatures of layers \(\{l_{i-k}, ..., l_{i-1}\}\), the outputs are the porosity labels in layer \(l_{i}\). k indicates the number of layers in history whose information is used for the integrated framework to predict the next layer’s porosity occurrence. H indicates the number of layers in history whose information is used for the porosity detection of one layer. j presents the number of layers whose porosity labeled images are used to predict the next layer’s porosity labels. Here, \(k=H+j\).
Detection module
Recently, convolutional neural networks (CNN) (LeCun et al., 1998) have led to great successes in a broad range of image-based applications, e.g., image segmentation (Schulz & Behnke, 2012), image classification (Sermanet & LeCun, 2011) and face recognition (Adjabi et al., 2020). For this module, we develop a CNN-based method to detect porosity according to thermal signatures in a pixel-wise fashion. The challenges of the porosity detection problem are limited and imbalanced data. Especially, for this dataset, some layers lack porosity. Thus, it is challenging to use semantic segmentation methods on this dataset. We consider the porosity detection problem as a regression problem, and take each pixel as a data point to predict the probability of this pixel being porous using CNN-based methods. If the probability of a pixel is larger than a threshold, the pixel is classified as porous.
A CNN model is usually comprised of three basic components: convolution layers, pooling layers, and fully connected layers. These components can be repeated multiple times in a CNN model. The convolution layer forms the core layer of CNN models, and its goal is to objectively extract important features from the input images. This is accomplished using a set of convolution kernels (Krizhevsky et al., 2012) in each convolution layer. 2-D kernels are usually used for image classification.
Considering not only the six thermal signatures of the target pixel, but also the information of surrounding neighboring pixels and the history of these pixels might be important to achieve good detection accuracy. The information of surrounding neighboring pixels and the history of these pixels in previous layers are composed of C channels of 2-D images for each data point as the input of a CNN model. For every target pixel in layer \(l_i\), we impose a \(D \times D\) square region in this layer, where the central pixel is the target pixel and other pixels are the surrounding neighbors of the target pixel. Each pixel has six thermal signatures, so we obtain six \(D \times D\) images in layer \(l_i\). The square regions at the same position in H previous layers include the historical information of these pixels. Similarly, we obtain six \(D \times D\) images for each previous layer. Hence, \(C=6 \times (H+1)\), and the input for each data point is \(C \times D \times D\) images. Figure 4 shows an example input, where \(D=5\). The output is the probability of being porous at the current layer at the location corresponding to the target pixel.
A pooling layer is usually employed after one or several stacked convolution layers. The purpose of the pooling layer is to reduce the dimensionality of feature maps, which are the outputs of the convolution layers. The outputs obtained after employing stacks of convolution and pooling layers are flattened to a vector. This vector is then fed into a fully connected layer. The CNN architecture used in this work is shown in Fig. 5, and is designed by stacking several convolutional layers and pooling layers, and one fully connected layer at the end. In the convolutional layer, the kernel size is \(5\times 5\), \(stride=1\), \(padding=2\). In the pooling layer, the kernel size is \(5\times 5\), \(stride=2\). The first and second convolutional layers have 16 and 32 kernels, respectively. ReLU is used as the activation function for all convolutional (Conv) layers and the sigmoid function is used after the fully connected (FC) layer.
Prediction module
The objective of the prediction module is to predict porosity labels in the next layer according to the porosity labels of previous layers. Because both the current and history information about surrounding neighbors can impact the pixel-wise porosity in future layers, one needs to consider the correlation between local neighbors when forecasting. In other words, both spatial and temporal context information needs to be considered for prediction. Therefore, this problem can be formulated as a spatiotemporal sequence forecasting problem with the sequence of porosity labeled images in previous layers as input and the porosity labeled image in the next layer as output. When predicting the porosity labels in layer \(l_i\), the labeled images of previous layers \(\{l_{i-j}, ..., l_{i-1}\}\) are used to directly predict the labeled image in layer \(l_i\). The size of all the images is 160 \(\times \) 160.
In the study by Shi et al. (2015), a convolutional LSTM (ConvLSTM) network was proposed for a spatiotemporal sequence forecasting problem. In ConvLSTM, the inputs, cell outputs and states are all 3-D tensors whose last two dimensions are spatial dimensions (rows and columns). ConvLSTM includes two structures, the convolutional and the LSTM structure. The convolutional structures are in both the input-to-state and state-to-state transitions, that are capable of capturing local spatial features (e.g., correlation between neighbors). The LSTM can capture temporal autocorrelation in the data. Therefore, the ConvLSTM network has desirable properties for spatiotemporal sequence forecasting problems.
For the present problem, we develop an encoder–decoder model to predict porosity labeled images. Figure 6 shows the architecture of our model. The encoder–decoder model has two networks, an encoder and a decoder network. The encoder network compresses the whole input sequence into a hidden state tensor, where three ConvLSTM layers are stacked with 32 hidden states each. The decoder network unfolds this hidden state by three ConvLSTM layers with 32 hidden states and concatenates all the states. This is followed by one 3-D CNN layer to produce the final prediction. All the input-to-state and state-to-state kernels are of size 3 \(\times \) 3. After doing some hyper-parameter tuning, we use these hyper-parameters in our experiments. The model simultaneously reconstructs the input sequence images and predicts the future image. Both the input and output of the model are a sequence of images. The input consists of porosity labeled images in layers \(\{l_{i-j}, ..., l_{i-1}\}\). The output data consists of porosity labeled images in layers \(\{l_{i-j+1}, ..., l_{i}\}\). Here j represents the number of history layers that are used to predict layer \(l_{i}\). The model is thus expected to predict the porosity labeled image of layer \(l_{i}\) according to porosity labeled images of previous j layers.
Results and discussion
We split the dataset EOS-LOF into training and testing sets based on layer number in the ratio of 7:3 for all evaluations of different models. The training set is composed of the first 350 layers (85–434th) while the testing set is composed of the remaining 151 layers (435–585th), as shown in Table 3. Each pixel is taken as a data point. Several metrics to evaluate the effectiveness of our framework will be introduced in section “Evaluation metrics”. In section “Experiments”, the detection and the prediction module are first individually evaluated followed by the combined porosity detection-prediction framework on the testing set. We conducted all modeling experiments using PyTorch with an NVIDIA Tesla V100 PCIe 16 GB GPU. The code is available at https://github.com/MaoYuwei/AM-porosity-prediction.
Evaluation metrics
In this work, we use precision, recall, and the F1-score for the minority class, i.e., porous class to evaluate the models. Based on the confusion matrix, model results are divided into four categories: true positive (TP), false positive (FP), false negative (FN), and true negative (TN). As for the porous class, true positives are those porous part pixels that are correctly predicted as porous. False positives are those non-porous part pixels that are wrongly predicted as porous. False negatives are those porous part pixels that are predicted wrongly as non-porous. True negatives are those non-porous part pixels that are correctly predicted as non-porous.
Precision is calculated as \(TP/(TP+FP)\), while recall is calculated as \(TP/(TP+FN)\). F1-score is the harmonic mean of precision and recall, calculated as \( 2 \times precision \times recall/(precision+recall)\). The Recall is the proportion of actual porous part pixels that are correctly predicted porous, and precision is the proportion of predicted porous part pixels that are actually porous. From the perspective of engineering, both recall and precision are equally important, so the F1-score is utilized to comprehensively evaluate the prediction performance of the proposed method using a single metric, which by definition is good only when both precision and recall are good.
Experiments
Detection module
In this study, we choose a region \(15 \times 15\) in size, which means \(D=15\). The mean square error loss function is used with an Adam optimizer (Kingma & Ba, 2014). The learning rate is 0.00001. The batch size is 10,000. The number of epochs is 300.
The training set is further randomly divided into a training set and a validation set in the ratio of 7:3 for the detection module. The validation set is used for exploring the parameter H and the threshold. For each pixel, if the predicted probability is higher than the threshold, the pixel is predicted as porous.
We explore the history of up to three previous layers for detection, i.e., \(H =\, \)0, 1, 2, or 3, respectively. The receiver operating characteristics area under the curve (ROC AUC) and the precision recall area under the curve (PR AUC) can be used to evaluate classification performance without considering the threshold, which is used to select the best H values. The ROC AUC and PR AUC for the porous class are shown in Table 4, showing that utilizing information about the layers’ history facilitates better performance. The ROC AUC using different H values are similar and PR AUC is the best when \(H=1\), suggesting that using information from just one previous layer is sufficient for porosity detection. Therefore, we only use one previous layer, i.e., \(H=1\), in the later experiments.
One of the main challenges of this detection problem is that only a small number of pixels are porous. The ratio of the number of non-porous pixels to that of porous ones in the dataset EOS-LOF is about 200:1, as shown in Table 3. Thus, porosity detection and prediction is an imbalanced data problem and the imbalance ratio (200:1) is very high. In binary classification problems, the probability threshold is generally chosen as 0.5. But a smaller threshold may be better for this imbalanced data problem. We thus explore a set of different thresholds on the validation set, as shown in Table 5. All the precision, recall, and F1-score are close to 1.00 for the non-porous class, so we do not show them on this table and the following tables. The threshold is set as 0.4 in the following experiments, because the results for the porous class are the best as shown in Table 5. In real applications, the threshold can be adjusted in accordance with the given tasks (depending on whether recall or precision is more important). When it is more desirable not to miss porous pixels, the recall is more important and a smaller threshold can be used. When the prediction of porous pixels is required to be with high confidence, the precision is more important and a larger threshold can be used.
The proposed CNN model is compared to several machine learning (ML) algorithms well-known for classification problems, such as the Extremely Randomized Trees [or Extra Trees (ET)] (Geurts et al., 2006), Logistic Regression (LR), and the K-Nearest Neighbor algorithm (KNN) (Cover & Hart, 1967). In this study, all ML models are implemented within the Python scikit-learn package. The ML models are used here with hyper-parameter tuning using an extensive grid search for each of these models as shown in Appendix Table 9. The training set and testing set of ML models are the same as the proposed CNN model. And all models are using the same dimension of features, i.e., six thermal signatures of neighbors at the current and previous layers. So the input dimension for each data point is \(15\times 15\times 2\times 6 = 2700\), the output is 0 (non-porous) or 1 (porous).
Table 6 compares the results of these models for porous class. One can see that the proposed CNN model is better than the other ML models in detecting where the porosity occurs with higher precision, recall, and F1-score. The proposed CNN model can not only learn the information of surrounding neighboring pixels but also the relative position information of these pixels.
Prediction module
To assess the predictive ability of the proposed framework, j in Fig. 3 is set to 5, which means that porosity labeled images of the previous five layers are used to predict the next layer. A mean square error loss function is used with the Adam optimizer and a learning rate of 0.001. The batch size is 8. The number of epochs is 300. To evaluate the performance of this module in a stand-alone manner (i.e., without coupling it with the detection module), the raw testing set is used where the actual porosity labeled images are used as input to predict the output images with each pixel assigned the predicted probability of being porous, which are subsequently compared with the actual porosity labeled images for evaluation. We again use 0.4 as the probability threshold to convert the predicted probabilities into porous/non-porous predictions.
Some ML models are used as baselines, which take each pixel as a data point and the label values of the previous five layers are features. Therefore, the dimension of features for a data point is five. The output is 0 (non-porous) or 1 (porous). A CNN model is also used as a baseline using labeled images of the previous five layers as the input to predict the next layer. The dimensions of input and output are \(160 \times 160 \times 5\) and \(160 \times 160\), respectively. The other parameters remain the same as for the ConvLSTM model. Table 7 shows the results of the proposed ConvLSTM model and some baselines. One can see that the results of the ConvLSTM model are better than other models, which indicates that the proposed ConvLSTM encoder–decoder model can accurately predict porosity labeled images in the next layer based on the actual porosity labeled images of previous layers.
Detection-prediction framework
The two detection and prediction models proposed in sections “Detection module” and “Prediction module” are combined into an integrated framework to predict porosity labeled images in the next layer according to only the thermal signatures of previous layers, i.e., without knowing the actual porosity labeled images in previous layers.
During a real manufacturing process, thermal signatures can be immediately obtained after processing each layer and used to label the images using the above-described detection model according to their thermal signatures. After obtaining and labeling the images of the previous layers, the prediction module can be used to predict the porosity labeled image of the next layer, but current porosity detection methods only detect the existing porosity of manufactured layers. The predicted porosity labels of the next layer can thus be used to decide whether or not to continue the manufacturing process depending on the severity of porosity. If the manufacturing process is continued, thermal signatures of the new layers can be collected for predicting the porosity labels of the next layer, and this can be continued until the end of the manufacturing process.
To evaluate the whole framework the process was simulated to the testing set. For each layer \(l_{i}\), the input of the integrated framework is the set of thermal signatures of the previous \(\{l_{i-k}, ..., l_{i-1}\}\) layers and the output is the predicted porosity label of layer \(l_{i}\) as shown in Fig. 3. The ground truth is the actual porosity labeled image. Here \(H =1\) as mentioned in section “Detection module”, which means one previous layer’s information is used in the detection module. Here \(j=5\) as mentioned in section “Prediction module”, which means five previous layers are used to predict the next layer in the prediction module. Thus, \(k=H+j\), i.e., k=6. Specifically, to predict porosity labels in layer \(l_{i}\), the set of thermal signatures of previous layers \(\{l_{i-6}, ..., l_{i-1}\}\) is input to the trained detection model, outputting a set of detected porosity labeled images of \(\{l_{i-5}, ..., l_{i-1}\}\) layers which becomes the input to the trained prediction model.
Some ML models were trained as baselines, which use six thermal signatures of the neighbors and the previous six layers as features, i.e., the feature dimension is \(15\times 15\times 6\times 6=8100\). Table 8 shows the results of the integrated detection-prediction framework and the ML models on the testing set. The results of the proposed framework are significantly better than those of the ML models, which shows that the proposed approach can predict porosity with reasonable accuracy, which may be useful to help prevent more severe part porosity.
The layer range (i.e., from the 444th to the 461th layer) of the porosity region with a diameter of 0.50 mm are taken as examples for detailed results analysis and common conclusion. The same phenomenon is also seen in other layer ranges (i.e., 339–363th, etc.). Figure 7 shows some example predictions and the corresponding ground truth. More pairs of prediction and ground truth are provided in Appendix Fig. 8. The 445th and 459th layers are the beginning and the end layers in the analyzed layer range in the manufactured part LOF. While the 452th layer is in the middle of the layer range.
First, it can be observed that the proposed deep learning framework has a remarkable prediction accuracy during the period of porosity formation. As shown in the middle row of Fig. 7c and d, at the 452th layer, the results in the middle of the layer range are quite accurate. Second, we see that the proposed deep learning framework might struggle in making accurate predictions when the porosity formation starts or ends. As shown in Fig. 7a and b, at the 445th layer, there are six regions of lack of fusion porosity distributed hexagonally, but only one is predicted. Similar results are at the beginning of layer range 444–461th, several porous pixels are predicted as not porous. As shown in Fig. 7e and f, at the 459th layer, the performance is better in predicting porosity than at the beginning of the layer range (tiny porosity is predicted) but still worse than that in the middle layers.
We believe that the proposed method performs nicely during the porosity formation, because the porosity in the current layer and the previous layers is similar and the porosity is relatively obvious. But at the beginning or end of the porosity formation, large change and tiny size make the prediction much harder.
To analyze the practical application, the trained models can be generalized to predict porosity for new prints from the k + 1 layer (since the input of the proposed framework is k layers’ thermal signatures) to the last layer. Note that in order to make the first prediction from the trained model on a new print, we will need the first k layers manufactured (e.g., k = 6 in this work), but that is much smaller than the number of layers used in training (e.g., 350 in this work). Further, if more layers happen to be available for new prints, the trained model on old prints could be used as pre-trained models which can be finetuned using thermal signatures collected while building other prints.
Conclusions and future work
Current works can only detect already existing porosity based on in-situ monitored signals. To solve this problem, a detection-prediction framework is proposed based on deep learning methods for porosity prediction using in-situ monitored thermal signatures. The approach predicts lack of fusion in the next layer based on the thermal signatures of the previous layers. The precision, recall, and F1-score of the integrated framework for the porous class are 0.72, 0.78 and 0.75. The framework includes a detection and a prediction module, each of which can be applied separately for different purposes. For instance, the detection module can be adopted to localize the lack of fusion porosity at each layer of parts built via L-PBF. The prediction module can also be built upon to predict the thermal signature, melt pool geometries, properties, etc.
The main limitation of the proposed method is its limited generalization ability. Since the system learns from the dataset manufactured in the EOS M290 machine, it is unclear how it would perform for other L-PBF machines, like the DMG-MORI machines, etc. This work is validated in predicting lack of fusion porosity, but its performance is not clear for other types of porosity, like gas porosity. Moreover, the performance is also unclear for other types of materials, like In718. And predicting porosity in parts with other unintentional and complex geometries, like lattice structure, is much more challenging. It is unclear how the proposed framework will work in more complex situations. The second limitation is that the proposed method might struggle in predicting tiny porosity and when the porosity formation starts or ends. Size limitation is due to limited dimension accuracy of Co-Axial Planck Thermometry system (each pixel represents 100 µm) and the prediction accuracy of the proposed model (F1-score of 0.75).
The most significant step in the future is to build more parts of different geometries (e.g., cube, globe, etc.) with undesigned and tiny porosity using different L-PBF machines (e.g., DMG-MORI, EOS machine, etc) and collect thermal features during the manufacturing process for finetuning the proposed modules. We could further improve the proposed method based on the new larger dataset and improve generalizability and customize loss functions/models for the beginning and end layers. We could also further analyze the misclassification pixels of both the detection module and prediction module during processing.
References
Adjabi, I., Ouahabi, A., Benzaoui, A., & Taleb-Ahmed, A. (2020). Past, present, and future of face recognition: A review. Electronics, 9(8), 1188.
Aghazadeh, M., & Gharehchopogh, F. S. (2018). A new hybrid model of multi-layer perceptron artificial neural network and genetic algorithms in web design management based on CMS. Journal of AI and Data Mining, 6(2), 409–415.
Agrawal, A., & Choudhary, A. (2016). Perspective: Materials informatics and big data: Realization of the “fourth paradigm’’ of science in materials science. APL Materials, 4(053208), 1–10.
Agrawal, A., & Choudhary, A. (2019). Deep materials informatics: Applications of deep learning in materials science. MRS Communications, 9(3), 779–792.
Asghari, K., Masdari, M., Soleimanian Gharehchopogh, F., & Saneifard, R. (2021). A fixed structure learning automata-based optimization algorithm for structure learning of Bayesian networks. Expert Systems, 38(7), e12734.
Bakish, R. (1983). Proceedings of the conference on electron beam melting and refining—state of the art 1983.
Betts, S., & Anderson, K. C. (2020). Printrite3D® machine learning case study.
Bugatti, M., & Colosimo, B. M. (2021). Towards real-time in-situ monitoring of hot-spot defects in L-PBF: A new classification-based method for fast video-imaging data analysis. Journal of Intelligent Manufacturing, 33(1), 293–309.
Collobert, R., & Weston, J. (2008). A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning (pp. 160–167).
Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21–27.
Eschner, N., Weiser, L., Häfner, B., & Lanza, G. (2020). Classification of specimen density in laser powder bed fusion (L-PBF) using in-process structure-borne acoustic process emissions. Additive Manufacturing, 34, 101324.
Francis, J., & Bian, L. (2019). Deep learning for distortion prediction in laser-based additive manufacturing using big data. Manufacturing Letters, 20, 10–14.
Gawade, V., Singh, V., & Guo, W. (2022). Leveraging simulated and empirical data-driven insight to supervised-learning for porosity prediction in laser metal deposition. Journal of Manufacturing Systems, 62, 875–885.
Ge, M., Su, F., Zhao, Z., & Su, D. (2020). Deep learning analysis on microscopic imaging in materials science. Materials Today Nano, 11, 100087.
Geurts, P., Ernst, D., & Wehenkel, L. (2006). Extremely randomized trees. Machine Learning, 63(1), 3–42.
Ghungrad, S., Gould, B., Soltanalian, M., Wolff, S. J., & Haghighi, A. (2021). Model-based deep learning for additive manufacturing: New frontiers and applications. Manufacturing Letters, 29, 94–98.
Grasso, M., & Colosimo, B. M. (2017). Process defects and in situ monitoring methods in metal powder bed fusion: A review. Measurement Science and Technology, 28(4), 044005.
Gupta, V., Choudhary, K., Tavazza, F., Campbell, C., Liao, W., Choudhary, A., & Agrawal, A. (2021). Cross-property deep transfer learning framework for enhanced predictive analytics on small materials data. Nature Communications, 12(1), 1–10.
Gupta, V., Liao, W., Choudhary, A., & Agrawal, A. (2022). BRNet: Branched residual network for fast and accurate predictive modeling of materials properties. In Proceedings of the 2022 SIAM international conference on data mining (SDM) (pp. 343–351). SIAM.
Imani, F., Chen, R., Diewald, E., Reutzel, E., & Yang, H. (2019). Deep learning of variant geometry in layerwise imaging profiles for additive manufacturing quality control. Journal of Manufacturing Science and Engineering, 141(11), 111001.
Jha, D., Choudhary, K., Tavazza, F., Liao, W. K., Choudhary, A., Campbell, C., & Agrawal, A. (2019). Enhancing materials property prediction by leveraging computational and experimental data using deep transfer learning. Nature Communications, 10, 5316.
Jha, D., Gupta, V., Liao, W., Choudhary, A., & Agrawal, A. (2022). Moving closer to experimental level materials property prediction using AI. Scientific Reports, 12(1), 1–9.
Jha, D., Gupta, V., Ward, L., Yang, Z., Wolverton, C., Foster, I., et al. (2021). Enabling deeper learning on big data for materials informatics applications. Scientific Reports, 11, 4244.
Jha, D., Ward, L., Paul, A., Liao, W. K., Choudhary, A., Wolverton, C., & Agrawal, A. (2018). Elemnet: Deep learning the chemistry of materials from only elemental composition. Scientific Reports, 8, 17593.
Khataei Maragheh, H., Gharehchopogh, F. S., Majidzadeh, K., & Sangar, A. B. (2022). A new hybrid based on long short-term memory network with spotted hyena optimization algorithm for multi-label text classification. Mathematics, 10(3), 488.
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv Preprint arXiv:1412.6980
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 1097–1105.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90.
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.
LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.
Li, J., Zhou, Q., Huang, X., Li, M., & Cao, L. (2021). In situ quality inspection with layer-wise visual images based on deep transfer learning during selective laser melting. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-021-01829-5.
Li, X., Dan, Y., Dong, R., Cao, Z., Niu, C., Song, Y., et al. (2019). Computational screening of new perovskite materials using transfer learning and deep learning. Applied Sciences, 9(24), 5510.
Li, X., Jia, X., Yang, Q., & Lee, J. (2020). Quality analysis in metal additive manufacturing with deep learning. Journal of Intelligent Manufacturing, 31(8), 2003–2017.
Lin, H., Li, B., Wang, X., Shu, Y., & Niu, S. (2019). Automated defect inspection of led chip using deep convolutional neural network. Journal of Intelligent Manufacturing, 30(6), 2525–2534.
Liu, R., & Zhang, X. (2021). A physics-informed machine learning model for porosity analysis in laser powder bed fusion additive manufacturing. The International Journal of Advanced Manufacturing Technology, 113, 1943–1958.
Mater, A. C., & Coote, M. L. (2019). Deep learning in chemistry. Journal of Chemical Information and Modeling, 59(6), 2545–2559.
McCann, R., Obeidi, M. A., Hughes, C., McCarthy, É., Egan, D. S., Vijayaraghavan, R. K., et al. (2021). In-situ sensing, process monitoring and machine control in laser powder bed fusion: A review. Additive Manufacturing, 45, 102058.
McGowan, E., Gawade, V., & Guo, W. (2022). A physics-informed convolutional neural network with custom loss functions for porosity prediction in laser metal deposition. Sensors, 22(2), 494.
Mozaffar, M., Paul, A., Al-Bahrani, R., Wolff, S., Choudhary, A., Agrawal, A., et al. (2018). Data-driven prediction of the high-dimensional thermal history in directed energy deposition processes via recurrent neural networks. Manufacturing Letters, 18, 35–39.
Nath, P., & Mahadevan, S. (2021). Probabilistic predictive control of porosity in laser powder bed fusion. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-021-01836-6.
Niu, S., Lin, H., Niu, T., Li, B., & Wang, X. (2019). Defect-GAN: Weakly-supervised defect detection using generative adversarial network. In 2019 IEEE 15th international conference on automation science and engineering (CASE) (pp. 127–132).
Paulson, N. H., Gould, B., Wolff, S. J., Stan, M., & Greco, A. C. (2020). Correlations between thermal history and keyhole porosity in laser powder bed fusion. Additive Manufacturing, 34, 101213.
Qin, J., Liu, Y., Grosvenor, R., Lacan, F., & Jiang, Z. (2020). Deep learning-driven particle swarm optimisation for additive manufacturing energy optimisation. Journal of Cleaner Production, 245, 118702.
Sabzi, H. E., Maeng, S., Liang, X., Simonelli, M., Aboulkhair, N. T., & Rivera-Díaz del Castillo, P. E. J. (2020). Controlling crack formation and porosity in laser powder bed fusion: Alloy design and process optimisation. Additive Manufacturing, 34, 101360.
Schulz, H., & Behnke, S. (2012). Learning object-class segmentation with convolutional neural networks. In ESANN (pp. 151–156).
Scime, L., & Beuth, J. (2018). A multi-scale convolutional neural network for autonomous anomaly detection and classification in a laser powder bed fusion additive manufacturing process. Additive Manufacturing, 24, 273–286.
Scime, L., Siddel, D., Baird, S., & Paquit, V. (2020). Layer-wise anomaly detection and classification for powder bed additive manufacturing processes: A machine-agnostic algorithm for real-time pixel-wise semantic segmentation. Additive Manufacturing, 36, 101453.
Sermanet, P., & LeCun, Y. (2011). Traffic sign recognition with multi-scale convolutional networks. In The 2011 international joint conference on neural networks (pp. 2809–2813). IEEE.
Shi, X., Chen, Z., Wang, H., Yeung, D. Y., Wong, W. K., & Woo, W. C. (2015). Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Advances in neural information processing systems (pp. 802–810).
Sorkhabi, L. B., Gharehchopogh, F. S., & Shahamfar, J. (2020). A systematic approach for pre-processing electronic health records for mining: Case study of heart disease. International Journal of Data Mining and Bioinformatics, 24(2), 97–120.
Tang, M., Pistorius, P. C., & Beuth, J. L. (2017). Prediction of lack-of-fusion porosity for powder bed fusion. Additive Manufacturing, 14, 39–48.
Webster, S., Lin, H., Carter, F. M., III., Ehmann, K., & Cao, J. (2021). Physical mechanisms in hybrid additive manufacturing: A process design framework. Journal of Materials Processing Technology, 291, 117048.
Xie, X., Bennett, J., Saha, S., Lu, Y., Cao, J., Liu, W. K., & Gan, Z. (2021). Mechanistic data-driven prediction of as-built mechanical properties in metal additive manufacturing. Computational Materials, 7, 86.
Yang, Z., Li, X., Catherine Brinson, L., Choudhary, A. N., Chen, W., & Agrawal, A. (2018). Microstructural materials design via deep adversarial learning methodology. Journal of Mechanical Design, 140(11), 111416.
Yang, Z., Papanikolaou, S., Reid, A. C., Liao, W. K., Choudhary, A. N., Campbell, C., & Agrawal, A. (2020). Learning to predict crystal plasticity at the nanoscale: Deep residual networks and size effects in uniaxial compression discrete dislocation simulations. Scientific Reports, 10, 8262.
Yang, Z., Yabansu, Y. C., Jha, D., Liao, W., Choudhary, A. N., Kalidindi, S. R., & Agrawal, A. (2019). Establishing structure–property localization linkages for elastic deformation of three-dimensional high contrast composites using deep learning approaches. Acta Materialia, 166, 335–345.
Zhang, W., Abranovic, B., Hanson-Regalado, J., Koz, C., Duvvuri, B., Shimada, K., Beuth, J., & Kara, L. B. (2020). Flaw detection in metal additive manufacturing using deep learned acoustic features. In Workshop of conference on neural information processing systems (NeurIPS).
Zhang, B., Liu, S., & Shin, Y. C. (2019). In-process monitoring of porosity during laser additive manufacturing process. Additive Manufacturing, 28, 497–505.
Ziletti, A., Kumar, D., Scheffler, M., & Ghiringhelli, L. M. (2018). Insightful classification of crystal structures using deep learning. Nature Communications, 9(1), 1–10.
Acknowledgements
The authors thank Prof. Jian Cao for helpful discussions and feedback on the manuscript. This work was primarily supported by the following financial assistance award 70NANB19H005 from U.S. Department of Commerce, National Institute of Standards and Technology as part of the Center for Hierarchical Materials Design (CHiMaD). Partial support from Predictive Science and Engineering Design Cluster (PS &ED, Northwestern University); DOE awards DE-SC0019358, DE-SC0021399; and NSF award CMMI-2053929 is also acknowledged.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Mao, Y., Lin, H., Yu, C.X. et al. A deep learning framework for layer-wise porosity prediction in metal powder bed fusion using thermal signatures. J Intell Manuf 34, 315–329 (2023). https://doi.org/10.1007/s10845-022-02039-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-022-02039-3