Abstract
Increasing climate change concerns call for the manufacturing sector to decarbonize its process by introducing a mitigation strategy. Energy efficiency concepts within the manufacturing process value chain are proportional to the emission reductions, prompting decision makers to require predictive tools to execute decarbonization solutions. Accurate forecasting requires techniques with a strong capability for predicting automotive component manufacturing energy consumption and carbon emission data. In this paper we introduce a hybrid autoregressive moving average (ARIMA)-long short-term memory network (LSTM) model for energy consumption forecasting and prediction of carbon emission within the manufacturing facility using the 4IR concept. The method could capture linear features (ARIMA) and LSTM captures the long dependencies in the data from the nonlinear time series data patterns, Root means square error (RMSE) is used for data analysis comparing the performance of ARIMA which is 448.89 as a single model with ARIMA-LSTM hybrid model as actual (trained) and predicted (test) 59.52 and 58.41 respectively. The results depicted RMSE values of ARIMA-LSTM being extremely smaller than ARIMA, which proves that hybrid ARIMA-LSTM is more suitable for prediction than ARIMA.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Automotive manufacturing industries are faced with new challenges in technology adoption, environmental degradation from a significant proportion of carbon emission, supply change for the digital economy, and multi-faceted sustainability drives [1]. Energy efficient system within complex automotive manufacturing has great potential for energy consumption reduction, relative to size, season, and types of manufactured components. Decision makers require fourth industrial revolution (4IR) application techniques to ensure that the industry’s operational energy use is efficiently managed and decarbonized. Understanding the meaning of ‘decarbonization’ is important as it is a term that is used for ‘reduction or total removal of carbon dioxide (CO2) emissions [2].
Accurate forecasting is a challenging process and requires statistical or machine learning techniques with a strong capability for predicting energy consumption and carbon emission, which is part of decarbonization process planning. Time series forecasting is known for a collection of past observation data of the same variables, which can be analyzed to develop a model for future prediction [3]. The widely applied prediction model for time series (TS) data stream is a statistical technique; autoregressive integrated moving average (ARIMA), Box Jenkins methodology proposed in 1970 [4, 5]. ARIMA has been applied to any form of process challenges within the different industries to assist decision makers to plan future-based predictions on trusted applications [6,7,8,9]. ARIMA ais quite flexible in that they can models several different types of time series data, i.e. pure autoregressive (AR), pure moving average (MA) and combined AR and MA (ARMA) series, with major limitation as pre-assumed linear form of the model [3].
Machine Learning (ML) as a subset of artificial intelligence (AI) is considered for the non-linear application data pattern. ML algorithms are used for data-driven fault prediction technology [10] and consist of the process of building an inductive model that learns from a limited amount of data without specialist intervention [11]. There are various ML techniques such as convolution neural network (CNN), gated recurrent network (GCN), artificial neural network (ANN), recurrent neural network (RNN), and long short-term memory (LSTM). Recently, LSTM has transformed from a modified RNN architecture introduced in 1997 [12] and has attracted attention for its capability to capture non-linear trends and dependencies [5]. LSTM-based deep learning methods have achieved great success in artificial intelligence fields involving large datasets [13, 14].
In this paper, we propose a hybrid time series forecasting approach using ARIMA and LSTM models for an automotive component manufacturing company data using the 4IR concept, based on the following motivation: in practice, it is challenging to determine whether the study is generated from a linear or non-linear process and time series are rarely linear or nonlinear in the data patterns. Comparison of the result of the ARIMA model and ARIMA-LSTM model using predictive evaluation indicators (PMI); mean absolute error (MAE), root mean square error (RMSE), and mean percentage error (MPE) are evaluated to calculate prediction accuracy.
2 Related Work on ARIMA-LSTM
The hybrid models are becoming popular in decision-making. This is due to the combination of linear and nonlinear aspects of the data pattern, which further increases the accuracy of the predictions in an application. In recent years, it has become evident that hybrid methods yielded better results compared to a single method. A summary of studies that applied hybrid model for accurate prediction: Soy Temür et al. [15] proposed a hybrid model which consists of a combination of the linear model (ARIMA), nonlinear model (LSTM), and hybrid (LSTM and ARIMA) model to improve system performance compared to a single model. A prediction method (GA‐CNN‐LSTM) combines a convolutional neural network (CNN) and a long‐short‐term memory network (LSTM) and is optimized by a genetic algorithm (GA) [16]. Authors [17] proposed a new hybrid model using long short-term memory (LSTM), a recurrent neural network (RNN) technique, and autoregressive integrated moving average (ARIMA), as time series forecasting technique to capture live stock market data, the method performed very well compared to clairvoyant forecasting library. The work of [18] provided Indonesian governments with an accurate prediction of future exports, a hybrid model that integrates ARIMA and LSTM models based on their specialties, where LSTM was applied to the non-linear component of the data, and ARIMA was applied to the linear component of the data. The results showed that the hybrid (LSTM-ARIMA) model achieved the lowest error metrics among all the tested models. ARIMA and LSTM techniques were used to establish rolling forecast models, which greatly improve the accuracy and efficiency of demand, and inventory forecasting, while the authors proposed ARIMA and LSTM as superior to the manufacturer’s empirical model prediction results [19].
The proposed hybrid-based model on deep learning methods integrates ARIMA and LSTM model to improve the accuracy of short-term drought prediction [20] and the results state that the ARIMA-LSTM model has the highest prediction accuracy [21]. In the paper titled, we present a novel hybrid ARIMA-LSTM model for automotive component manufacturing company production data forecasting considering manual operations to establish the advantages of linearity and nonlinearity, which exhibited better results than the individual models.
3 Methodology
3.1 Data Set and Processing
The study uses tier 2 automotive company-generated electricity data retrieved from National Cleaner Production Centre (NCPC) as secondary data obtained through its energy management training program. The carbon emission equivalent was derived using compared country-specific energy guidelines according to Intergovernmental Panel on Climate Change (IPCC) 2019 emission factors for coal mining [22, 23].
3.2 Auto-regressive Integrated Moving Average (ARIMA)
ARIMA model are a transformational statistical method that supports seasonality in data prediction [24], which have gained popularity among researchers due to their vast applications in manufacturing [25]. The models are known for their notable forecasting OF time series data accuracy and flexibility in different applications [26]. ARIMA models use regression equation to determine how variables respond to stochastic dissimilarity [24]. The independent variables are dependent on the lagged value of the previous values of the forecast as proposed by Box and Jenkins in the 1970s [4]. The equation is given as:
\({y}^{^{\prime}}\left(t\right)\) = differenced series, (p = order of lag depicted as autoregression, q is the order of error lag (moving average) and ε \(\left(t-1\right)\) is residuals of past observation, \({\varphi }_{i}\) is the coefficient of the first AR term, \({\theta }_{i}\) is the coefficient of the first MA term.
3.3 Long Short-Term Memory (LSTM)
Recurrent neural networks are difficult to train, as they often suffer from the exploding/vanishing gradient problem [27]. To overcome this shortcoming when learning long-term dependencies, the LSTM architecture [12] was introduced. The LSTM architecture consists of a set of recurrently connected sub-networks, known as memory blocks, the idea behind the memory block is to maintain its state over time and regulate the information flow through non-linear gating units [28]. Figure 1 is the N architecture of LSTM adopted from the work of [15]. Figure 2 depicted LSTM Structure.
3.4 Hybrid ARIMA-LSTM
ARIMA filters linear trends in the TS data and the residual values are passed to the LSTM model for training and residuals predict for the upcoming year. The LSTM has longer memory and works well for the non-stationary section of the data. LSTM works well for the non-stationary portion of the data with a relatively longer memory. LSTM’s capability of capturing nonlinear patterns in TS data is one of the method’s main advantages, as an attempt to overcome the challenges of obtaining an accurate forecasting model considering the intrinsic characteristics of the demand time series (being nonlinear and non-stationary).
Xt represents the input data at t time step and the output of the previous unit, ht is the hidden units’ output, while ht-1 is their previous output. The new memory in Eq. 3 is the LSTM unit calculated from Eq. 2
3.5 Predictive Evaluation Indicators Validation
The time series prediction performance assessment was validated to evaluate the accuracy of trained models and identified efficient models between ARIMA and ARIMA – LSTM hybrid models. The root mean square error (RMSE) is a means of measuring the error in predicting quantitative data, it is a normalized distance between the observed values and the predicted values. The predictive evaluation indicators are engaged heuristically to decrease the non-absolute size of the error iteration from one step to the next.
where yt is the actual value, ft is the predicted value; k is the sample size square error. In this paper, the proposed hybrid formulation is the residuals from the rolling LSTM model analysis for further optimization, given its non-linear form.
4 Results
ARIMA and ARIMA-LSTM models were used to forecast the electricity generated from coal and carbon dioxide emissions. Figure 3 is the initial visual observation of tier 2 automotive component manufacturing electricity generation. RMSE is used as an indicator for analysis comparing ARIMA as a single model and the performance of the hybrid model ARIMA-LSTM. The RMSE for ARIMA is 448.89 from the initial visual observed data in Fig. 4 and the ARIMA-LSTM actual and predicted are 59.52 and 58.41 as presented in Fig. 5. The results of ARIMA and ARIMA-LSTM predictions are depicted in Fig. 6. The RMSE values of ARIMA-LSTM are smaller than ARIMA, which proves that the hybrid ARIMA-LSTM is more suitable for prediction than the single model that is ARIMA.
5 Conclusion
The long short-term memory network (LSTM) model was used to forecast energy consumption and carbon emission within the manufacturing facility using the 4IR concept. The method captures linear and nonlinear patterns in time series data, ARIMA captures the linear features and LSTM captures the long dependencies in the data. The obtained result with hybrid models were individually compared, it was observed that they could reduce the general variance or error, even if they are unrelated. Due to this reason, hybrid models are recognized as the most successful models for forecasting tasks [15]. This information will support the decision makers for energy management and decarbonization planning. The objective to construct two models was to test which model will best fit for prediction, using the configuration that gives the lowest root mean square error.
References
Katchasuwanmanee, K.: Bateman, R., Cheng, K.: An Integrated approach to energy efficiency in automotive manufacturing systems: quantitative analysis and optimisation. Product. Manufact. Res. 5(1), 90–98 (2017)
Romano, A., Yang, Z.: Decarbonisation of shipping: a state of the art survey for 2000–2020. Ocean Coast. Manag. 214, 105936 (2021)
Zhang, P., Zhang, G.P.: Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50, 159–175 (2003)
Box, G.E.P., Jenkins, G.M., Reinsel, G.C., Ljung, G.M.: Time series analysis forecasting and control.– Fifth edition (2016)
Fan, D., Sun, H., Jun, Y., Zhang, K., Yan, X., Sun, Z.: Well production forecasting based on ARIMA-LSTM model considering manual operations. Energy, 220, 119708 (2021)
Sen, P., Roy, M., Pal, P.: Application of ARIMA for forecasting energy consumption and GHG emission: a case study of an Indian pig iron manufacturing organization. Energy, 116, 1031–1038 (2016)
Haneen, A., Mohammed, N., Rawajfih, A.Y., Bareeq, A.A., Abeer, A.A., Fawaz, S.: On the accuracy of ARIMA based prediction of COVID-19 spread. Results Phys. 27, 104509 (2021)
Phan, T.T.H., Nguyen, X.H.: Combining statistical machine learning models with ARIMA for water level forecasting: the case of the red river. Adv. Water Resour. 142 (2020)
Jia, L., Zou, Y., He, K., Zhu, B.: Carbon futures price forecasting based with ARIMA-CNN-LSTM model. Procedia Comput. Sci. 162, 33–38 (2019)
Xu, H., Ruizhe, M., Li, Y., Zongmin, M.: Two-stage prediction of machinery fault trend based on deep learning for time series analysis. Digit. Sig. Process. 117, 103150 (2021)
Stetco, A., et al.: Machine learning methods for wind turbine condition monitoring: a review. Renew. Energy, 133, 620–635 (2019)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–80 (1997)
Zhang, N., Shen, S., Zhou, Y.: Application of LSTM approach for modelling stress–strain behaviour of soil. Appl. Soft Comput. 100, 106959 (2021)
Sak, H., Senior, A., Beaufays, F.: Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, pp. 338–342 (2014)
Soy Temür, A., Yıldız, Ş.: Comparison of forecasting performance of ARIMA LSTM and HYBRID models for the sales volume budget of a manufacturing enterprise. Istanbul Bus. Res. 50(1), 15–46 (2021)
Lu, W., Rui, H., Liang, C., Jiang, L., Zhao, S., Li, K.: A method based on GA-CNN-LSTM for daily tourist flow prediction at scenic spots. Entropy, 22, 261 (2020)
Sakshi, K.V.A.: An ARIMA- LSTM hybrid model for stock market prediction using live data. J. Eng. Sci. Technol. Rev. 13(4), 117–123 (2020)
Dave, E., Leonardo, A., Jeanive, M., Hanafiah, N.: Forecasting Indonesia exports using a hybrid model ARIMA-LSTM. Procedia Comput. Sci. 179, 480–487 (2021)
Wang, C.C., Chien, C.H., Trappey, A.J.C.: On the application of ARIMA and LSTM to predict order demand based on short lead time and on-time delivery requirements. Processes, 9(7), 1157 (2021)
IPCC, Global warming of 1.5°C, I.P.o.C. Change., Editor, p. 616 (2019)
Dea, GHG National Inventory Report South Africa, D.o.E. Affairs, Editor, Pretoria (2015)
Chou, J.S., Tran, D.S.: Forecasting energy consumption time series using machine learning techniques based on usage patterns of residential householders. Energy 165, 709–726 (2018)
Hakeem, U.R., Raza, R., Mohsin, N.J., Muhammad, A.Z.C.: Forecasting CO2 emissions from energy, manufacturing and transport sectors in Pakistan: statistical vs. Mach. Learn. Meth. (2018)
Zhang, P.: Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50, 159–175 (2003)
Hochreiter, S., Bengio, Y., Frasconi, P., Schmidhuber, J.: Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. a field guide to dynamical recurrent neural networks. IEEE IEEE Trans. Neural Netw. 7(6), 1329–1338 (2001)
Temür, A.S., Akgün, M., Temür, G.: Predicting housing sales in Turkey using arima, LSTM and hybrid models. J. Bus. Econ. Manage. 20(5), 920–938 (2019)
Khashei, M., Reza, H.S., Bijari, M.: A new hybrid artificial neural networks and fuzzy regression model for time series forecasting. Fuzzy Sets Syst. 159(7), 769–786 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2023 The Author(s)
About this paper
Cite this paper
Adenuga, O.T., Mpofu, K., Modise, R.K. (2023). Application of ARIMA-LSTM for Manufacturing Decarbonization Using 4IR Concepts. In: Kim, KY., Monplaisir, L., Rickli, J. (eds) Flexible Automation and Intelligent Manufacturing: The Human-Data-Technology Nexus . FAIM 2022. Lecture Notes in Mechanical Engineering. Springer, Cham. https://doi.org/10.1007/978-3-031-18326-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-18326-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18325-6
Online ISBN: 978-3-031-18326-3
eBook Packages: EngineeringEngineering (R0)