Attention-based long short-term memory fully convolutional network for chemical process fault diagnosis

doi:10.1016/j.cjche.2022.06.029

Abstract

Abstract: A correct and timely fault diagnosis is important for improving the safety and reliability of chemical processes. With the advancement of big data technology, data-driven fault diagnosis methods are being extensively used and still have considerable potential. In recent years, methods based on deep neural networks have made significant breakthroughs, and fault diagnosis methods for industrial processes based on deep learning have attracted considerable research attention. Therefore, we propose a fusion deep-learning algorithm based on a fully convolutional neural network (FCN) to extract features and build models to correctly diagnose all types of faults. We use long short-term memory (LSTM) units to expand our proposed FCN so that our proposed deep learning model can better extract the time-domain features of chemical process data. We also introduce the attention mechanism into the model, aimed at highlighting the importance of features, which is significant for the fault diagnosis of chemical processes with many features. When applied to the benchmark Tennessee Eastman process, our proposed model exhibits impressive performance, demonstrating the effectiveness of the attention-based LSTM FCN in chemical process fault diagnosis.

Key words: Safety, Fault diagnosis, Process systems, Long short-term memory, Attention mechanism, Neural networks

摘要： A correct and timely fault diagnosis is important for improving the safety and reliability of chemical processes. With the advancement of big data technology, data-driven fault diagnosis methods are being extensively used and still have considerable potential. In recent years, methods based on deep neural networks have made significant breakthroughs, and fault diagnosis methods for industrial processes based on deep learning have attracted considerable research attention. Therefore, we propose a fusion deep-learning algorithm based on a fully convolutional neural network (FCN) to extract features and build models to correctly diagnose all types of faults. We use long short-term memory (LSTM) units to expand our proposed FCN so that our proposed deep learning model can better extract the time-domain features of chemical process data. We also introduce the attention mechanism into the model, aimed at highlighting the importance of features, which is significant for the fault diagnosis of chemical processes with many features. When applied to the benchmark Tennessee Eastman process, our proposed model exhibits impressive performance, demonstrating the effectiveness of the attention-based LSTM FCN in chemical process fault diagnosis.

关键词: Safety, Fault diagnosis, Process systems, Long short-term memory, Attention mechanism, Neural networks

Shanwei Xiong, Li Zhou, Yiyang Dai, Xu Ji. Attention-based long short-term memory fully convolutional network for chemical process fault diagnosis[J]. Chinese Journal of Chemical Engineering, 2023, 56(4): 1-14.

Shanwei Xiong, Li Zhou, Yiyang Dai, Xu Ji. Attention-based long short-term memory fully convolutional network for chemical process fault diagnosis[J]. 中国化学工程学报, 2023, 56(4): 1-14.

References

[1] S. Yin, S.X. Ding, A. Haghani, H.Y. Hao, P. Zhang, A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process, J. Process. Control. 22 (9) (2012) 1567–1581.
[2] V. Venkatasubramanian, R. Rengaswamy, K.W. Yin, S.N. Kavuri, A review of process fault detection and diagnosis: Part I: Quantitative model-based methods, Computers & Chemical Engineering. 27 (3) (2003) 293-311.
[3] M. Alauddin, F. Khan, S. Imtiaz, S. Ahmed, A bibliometric review and analysis of data-driven fault detection and diagnosis methods for process systems, Industrial & Engineering Chemistry Research. 57 (32) (2018) 10719-10735.
[4] C.A.K. Gordon, B. Burnak, M. Onel, E.N. Pistikopoulos, Data-driven prescriptive maintenance: failure prediction using ensemble support vector classification for optimal process and maintenance scheduling, Ind. Eng. Chem. Res. 59 (44) (2020) 19607–19622.
[5] S. Yin, X.P. Zhu, O. Kaynak, Improved PLS focused on key-performance-indicator-related fault diagnosis, IEEE Trans. Ind. Electron. 62 (3) (2015) 1651–1658.
[6] A.M. Martinez, A.C. Kak, PCA versus LDA, IEEE Transactions on Pattern Analysis and Machine Intelligence. 23 (2) (2001) 228-233.
[7] L. Yan, X. Peng, C.D. Tong, L.J. Luo, A multigroup fault detection and diagnosis scheme for multivariate systems, Industrial & Engineering Chemistry Research. 59 (47) (2020) 20767-20778.
[8] W.K. Sun, A.R.C. Paiva, P. Xu, A. Sundaram, R.D. Braatz, Fault detection and identification using Bayesian recurrent neural networks, Comput. Chem. Eng. 141 (2020) 106991.
[9] S.Y. Zhang, K.X. Bi, T. Qiu, Bidirectional recurrent neural network-based chemical process fault diagnosis, Ind. Eng. Chem. Res. 59 (2) (2020) 824–834.
[10] J. Hong, J. Yoon, Multivariate time-series classification of sleep patterns using a hybrid deep learning architecture, 2017 IEEE 19th International Conference on e-Health Networking, Applications and Services (Healthcom). Dalian, China. IEEE, 1–6.
[11] Y.M. Han, S.H. Zhang, Z.Q. Geng, Multi-Frequency Decomposition with Fully Convolutional Neural Network for Time Series Classification 2018 24th International Conference on Pattern Recognition (ICPR). August 20-24, 2018. Beijing. IEEE, (2018) 284-289
[12] J. Donahue, L.A. Hendricks, M. Rohrbach, S. Venugopalan, S. Guadarrama, K. Saenko, T. Darrell, Long-term recurrent convolutional networks for visual recognition and description, IEEE Trans. Pattern Anal. Mach. Intell. 39 (4) (2017) 677–691.
[13] X. Cheng, P.H. Han, G.Y. Li, S.Y. Chen, H.X. Zhang, A Novel Channel and Temporal-Wise Attention in Convolutional Networks for Multivariate Time Series Classification, IEEE Access. 8 (2020) 212247-212257.
[14] Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature, 521 (7553) (2015) 436.
[15] S.Q. Ren, K.M. He, R. Girshick, J. Sun, Faster R-CNN: towards real-time object detection with region proposal networks, IEEE Trans. Pattern Anal. Mach. Intell. 39 (6) (2017) 1137–1149.
[16] O. Vinyals, A. Toshev, S. Bengio, D. Erhan, Show and tell: A neural image caption generator, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA. (2015) 3156-3164.
[17] Z.C. Li, L. Tian, Q.C. Jiang, X.F. Yan, Fault diagnostic method based on deep learning and multimodel feature fusion for complex industrial processes, Ind. Eng. Chem. Res. 59 (40) (2020) 18061–18069.
[18] Z.P. Zhang, J.S. Zhao, A deep belief network based fault diagnosis model for complex chemical processes, Comput. Chem. Eng. 107 (2017) 395–407.
[19] S.D. Zheng, J.S. Zhao, A new unsupervised data mining method based on the stacked autoencoder for chemical process fault diagnosis, Comput. Chem. Eng. 135 (2020) 106755.
[20] H. Wu, J.S. Zhao, Fault detection and diagnosis based on transfer learning for multimode chemical processes, Comput. Chem. Eng. 135 (2020) 106731.
[21] H.T. Zhao, S.Y. Sun, B. Jin, Sequential fault diagnosis based on LSTM neural network, IEEE Access. 6 (2018) 12929–12939.
[22] A. Graves, A.R. Mohamed, G. Hinton, Speech recognition with deep recurrent neural networks, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, BC, Canada. IEEE, 6645–6649.
[23] W.J. Zaremba, I. Sutskever, O. Vinyals, Recurrent Neural Network Regularization, arXiv preprint arXiv. 1409.2329 (2014)
[24] S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Computation, 9 (8) (1997) 1735-1780.
[25] S.Y. Zhang, T. Qiu, Semi-supervised LSTM ladder autoencoder for chemical process fault diagnosis and localization, Chem. Eng. Sci. 251 (2022) 117467.
[26] G.S. Chadha, A. Panambilly, A. Schwung, S.X. Ding, Bidirectional deep recurrent neural networks for process fault classification, ISA Trans. 106 (2020) 330–342.
[27] V. Mnih, N. Heess, A. Graves, K. Kavukcuoglu, Recurrent models of visual attention, Advances in Neural Information Processing Systems. 27 (2014) 2204-2212.
[28] J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA. IEEE, 2015.
[29] Y. Lecun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE. 86 (11) (1998) 2278–2324.
[30] S.Y. Zhang, T. Qiu, A dynamic-inner convolutional autoencoder for process monitoring, Comput. Chem. Eng. 158 (2022) 107654.
[31] A. Krizhevsky, I. Sutskever, G.E. Hinton, ImageNet classification with deep convolutional neural networks, Commun. ACM. 60 (6) (2017) 84–90.
[32] S. Ioffe, C. Szegedy, Batch normalization: accelerating deep network training by reducing internal covariate shift, JMLR. 37 (2015) 448-456.
[33] M. Lin , Q. Chen , S.C. Yan , Network in network , OALib Journal arXiv: ( 2013 ) 1321.4400v3 .
[34] F. Karim, S. Majumdar, H. Darabi, S. Harford, Multivariate LSTM-FCNs for time series classification, Neural Netw. 116 (2019) 237–245.
[35] X. Glorot, A. Bordes, Y.S. Bengio, Deep Sparse Rectifier Neural Networks, Journal of Machine Learning Research. 15 (2010) 315-323.
[36] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, Dropout: A Simple Way to Prevent Neural Networks from Overfitting, Journal of Machine Learning Research. 15 (1) (2014) 1929-1958.
[37] J.J. Downs, E.F. Vogel, A plant-wide industrial process control problem, Comput. Chem. Eng. 17 (3) (1993) 245–255.
[38] H. Wu, J.S. Zhao, Deep convolutional neural network model based chemical process fault diagnosis, Comput. Chem. Eng. 115 (2018) 185–197.
[39] F. Karim, S. Majumdar, H. Darabi, Insights into LSTM fully convolutional networks for time series classification, IEEE Access. 7 (2019) 67718–67725.