SmdaNet: A hierarchical hard sample mining and domain adaptation neural network for fault diagnosis in industrial process

doi:10.1016/j.cjche.2025.05.003

Abstract

Abstract: Fault diagnosis in industrial process is essential for ensuring production safety and efficiency. However, existing methods exhibit limited capability in recognizing hard samples and struggle to maintain consistency in feature distributions across domains, resulting in suboptimal performance and robustness. Therefore, this paper proposes a fault diagnosis neural network for hard sample mining and domain adaptive (SmdaNet). First, the method uses deep belief networks (DBN) to build a diagnostic model. Hard samples are mined based on the loss values, dividing the data set into hard and easy samples. Second, elastic weight consolidation (EWC) is used to train the model on hard samples, effectively preventing information forgetting. Finally, the feature space domain adaptation is introduced to optimize the feature space by minimizing the Kullback–Leibler divergence of the feature distributions. Experimental results show that the proposed SmdaNet method outperforms existing approaches in terms of classification accuracy, robustness and interpretability on the penicillin simulation and Tennessee Eastman process datasets.

Key words: Industrial process, Bioprocess, Fault diagnosis, Neural networks, Fermentation

摘要： Fault diagnosis in industrial process is essential for ensuring production safety and efficiency. However, existing methods exhibit limited capability in recognizing hard samples and struggle to maintain consistency in feature distributions across domains, resulting in suboptimal performance and robustness. Therefore, this paper proposes a fault diagnosis neural network for hard sample mining and domain adaptive (SmdaNet). First, the method uses deep belief networks (DBN) to build a diagnostic model. Hard samples are mined based on the loss values, dividing the data set into hard and easy samples. Second, elastic weight consolidation (EWC) is used to train the model on hard samples, effectively preventing information forgetting. Finally, the feature space domain adaptation is introduced to optimize the feature space by minimizing the Kullback–Leibler divergence of the feature distributions. Experimental results show that the proposed SmdaNet method outperforms existing approaches in terms of classification accuracy, robustness and interpretability on the penicillin simulation and Tennessee Eastman process datasets.

关键词: Industrial process, Bioprocess, Fault diagnosis, Neural networks, Fermentation

Zhenhua Yu, Zongyu Yao, Weijun Wang, Qingchao Jiang, Zhixing Cao. SmdaNet: A hierarchical hard sample mining and domain adaptation neural network for fault diagnosis in industrial process[J]. Chinese Journal of Chemical Engineering, 2025, 84(8): 146-157.

References

[1] S.W. Xiong, L. Zhou, Y.Y. Dai, X. Ji, Attention-based long short-term memory fully convolutional network for chemical process fault diagnosis, Chin. J. Chem. Eng. 56 (2023) 1-14.
[2] J.J. Luo, Z.H. Jin, H.P. Jin, Q. Li, X. Ji, Y.Y. Dai, Causal temporal graph attention network for fault diagnosis of chemical processes, Chin. J. Chem. Eng. 70 (2024) 20-32.
[3] C.T. Wang, H.B. Shi, B. Song, Y. Tao, Hierarchical multihead self-attention for time-series-based fault diagnosis, Chin. J. Chem. Eng. 70 (2024) 104-117.
[4] C.B. Low, D.W. Wang, S. Arogeti, J.B. Zhang, Causality assignment and model approximation for hybrid bond graph: fault diagnosis perspectives, IEEE Trans. Autom. Sci. Eng. 7 (3) (2010) 570-580.
[5] L.U. Iurinic, A.R. Herrera-Orozco, R.G. Ferraz, A.S. Bretas, Distribution systems high-impedance fault location: a parameter estimation approach, IEEE Trans. Power Deliv. 31 (4) (2016) 1806-1814.
[6] A. Izadian, P. Khayyer, Application of Kalman filters in model-based fault diagnosis of a DC-DC boost converter, IECON 2010 - 36th Annual Conference on IEEE Industrial Electronics Society. November 7-10, 2010, Glendale, AZ, USA. IEEE, (2010) 369-372.
[7] A. Ben Youssef, S.K. El Khil, I. Slama-Belkhodja, State observer-based sensor fault detection and isolation, and fault tolerant control of a single-phase PWM rectifier for electric railway traction, IEEE Trans. Power Electron. 28 (12) (2013) 5842-5853.
[8] S. Byun, M. Papaelias, F.P.G. Marquez, D. Lee, Fault-tree-analysis-based health monitoring for autonomous underwater vehicle, J. Mar. Sci. Eng. 10 (12) (2022) 1855.
[9] Y. Wilhelm, P. Reimann, W. Gauchel, B. Mitschang, Overview on hybrid approaches to fault detection and diagnosis: Combining data-driven, physics-based and knowledge-based models, Procedia CIRP 99 (2021) 278-283.
[10] Application of the Digraph Method in System Fault Diagnostics.
[11] M. Misra, H.H. Yue, S.J. Qin, C. Ling, Multivariate process monitoring and fault diagnosis by multi-scale PCA, Comput. Chem. Eng. 26 (9) (2002) 1281-1293.
[12] G. Li, S.Z. Qin, Y.D. Ji, D.H. Zhou, Total PLS based contribution plots for fault diagnosis, Acta Autom. Sin. 35 (6) (2009) 759-765.
[13] Q.C. Jiang, X.F. Yan, J. Li, PCA-ICA integrated with Bayesian method for non-Gaussian fault diagnosis, Ind. Eng. Chem. Res. 55 (17) (2016) 4979-4986.
[14] S.W. Choi, C. Lee, J.M. Lee, J.H. Park, I.B. Lee, Fault detection and identification of nonlinear processes based on kernel PCA, Chemom. Intell. Lab. Syst. 75 (1) (2005) 55-67.
[15] J.M. Lee, S. Joe Qin, I.B. Lee, Fault detection of non-linear processes using kernel independent component analysis, Can. J. Chem. Eng. 85 (4) (2007) 526-536.
[16] S. Yin, X.P. Zhu, O. Kaynak, Improved PLS focused on key-performance-indicator-related fault diagnosis, IEEE Trans. Ind. Electron. 62 (3) (2015) 1651-1658.
[17] K. Zhong, M. Han, T. Qiu, B. Han, Fault diagnosis of complex processes using sparse kernel local fisher discriminant analysis, IEEE Trans. Neural Netw. Learn. Syst. 31 (5) (2020) 1581-1591.
[18] A. Hajnayeb, A. Ghasemloonia, S.E. Khadem, M.H. Moradi, Application and comparison of an ANN-based feature selection method and the genetic algorithm in gearbox fault diagnosis, Expert Syst. Appl. 38 (8) (2011) 10205-10209.
[19] R.V. Sanchez, P. Lucero, R.E. Vasquez, M. Cerrada, J.C. Macancela, D. Cabrera, Feature ranking for multi-fault diagnosis of rotating machinery by using random forest and KNN, J. Intell. Fuzzy Syst. 34 (6) (2018) 3463-3473.
[20] N. Saravanan, V.N.S.K. Siddabattuni, K.I. Ramachandran, Fault diagnosis of spur bevel gear box using artificial neural network (ANN), and proximal support vector machine (PSVM), Appl. Soft Comput. 10 (1) (2010) 344-360.
[21] Y.T. Dong, H.K. Jiang, Z.H. Wu, Q. Yang, Y.P. Liu, Digital twin-assisted multiscale residual-self-attention feature fusion network for hypersonic flight vehicle fault diagnosis, Reliab. Eng. Syst. Saf. 235 (2023) 109253.
[22] M.Z. Mu, H.K. Jiang, X. Wang, Y.T. Dong, A task-oriented theil index-based meta-learning network with gradient calibration strategy for rotating machinery fault diagnosis with limited samples, Adv. Eng. Inform. 62 (2024) 102870.
[23] X. Wang, H.K. Jiang, M.Z. Mu, Y.T. Dong, A dynamic collaborative adversarial domain adaptation network for unsupervised rotating machinery fault diagnosis, Reliab. Eng. Syst. Saf. 255 (2025) 110662.
[24] S.S. Zhong, S. Fu, L. Lin, A novel gas turbine fault diagnosis method based on transfer learning with CNN, Measurement 137 (2019) 435-453.
[25] B. Zhao, X.M. Zhang, H. Li, Z.B. Yang, Intelligent fault diagnosis of rolling bearings based on normalized CNN considering data imbalance and variable working conditions, Knowl. Based Syst. 199 (2020) 105971.
[26] W. Zhang, G.L. Peng, C.H. Li, Y.H. Chen, Z.J. Zhang, A new deep learning model for fault diagnosis with good anti-noise and domain adaptation ability on raw vibration signals, Sensors 17 (2) (2017) 425.
[27] H.T. Zhao, S.Y. Sun, B. Jin, Sequential fault diagnosis based on LSTM neural network, IEEE Access 6 (2018) 12929-12939.
[28] C. Zhang, Q.X. Zhu, Y.L. He, Y. Zhang, M.Q. Zhang, Y. Xu, Deep graph convolutional neural network for fault diagnosis of complex industrial processes, 2023 2nd International Conference on Artificial Intelligence, Human-Computer Interaction and Robotics (AIHCIR). December 8-10, 2023, Tianjin, China. IEEE, (2023) 581-585.
[29] K. Zhou, E. Diehl, J. Tang, Deep convolutional generative adversarial network with semi-supervised learning enabled physics elucidation for extended gear fault diagnosis under data limitations, Mech. Syst. Signal Process. 185 (2023) 109772.
[30] G. San Martin, E.L. Droguett, Temporal variational auto-encoders for semi-supervised remaining useful life and fault diagnosis, IEEE Access 10 (2022) 55112-55125.
[31] Z.H. Wang, T. Sun, X.C. Tian, Fault Diagnosis of Rolling Bearing Based on SDAE and PSO-DBN In: 2019 Chinese Control and Decision Conference (CCDC). Nanchang, China. IEEE, 2019.
[32] W. Deng, H.L. Liu, J.J. Xu, H.M. Zhao, Y.J. Song, An improved quantum-inspired differential evolution algorithm for deep belief network, IEEE Trans. Instrum. Meas. 69 (10) (2020) 7319-7327.
[33] Z.Y. Chen, W.H. Li, Multisensor feature fusion for bearing fault diagnosis using sparse autoencoder and deep belief network, IEEE Trans. Instrum. Meas. 66 (7) (2017) 1693-1702.
[34] H. Oh, J.H. Jung, B.C. Jeon, B.D. Youn, Scalable and unsupervised feature engineering using vibration-imaging and deep learning for rotor system diagnosis, IEEE Trans. Ind. Electron. 65 (4) (2018) 3539-3549.
[35] J.H. Yan, Y.Y. Hu, C.Z. Guo, Rotor unbalance fault diagnosis using DBN based on multi-source heterogeneous information fusion, Procedia Manuf. 35 (2019) 1184-1189.
[36] G.X. Niu, X. Wang, M. Golda, S. Mastro, B. Zhang, An optimized adaptive PReLU-DBN for rolling element bearing fault diagnosis, Neurocomputing 445 (2021) 26-34.
[37] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A.A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, R. Hadsell, Overcoming catastrophic forgetting in neural networks, Proc. Natl. Acad. Sci. USA 114 (13) (2017) 3521-3526.
[38] G. Birol, U. Cenk, C. Ali, A modular simulation package for fed-batch fermentation: penicillin production, Comput. Chem. Eng. 26 (11) (2002) 1553-1565.
[39] Z.X. Song, J. Li, Variable selection with false discovery rate control in deep neural networks, Nat. Mach. Intell. 3 (2021) 426-433.
[40] Y.L. Wang, Z.F. Pan, X.F. Yuan, C.H. Yang, W.H. Gui, A novel deep learning based fault diagnosis approach for chemical process with extended deep belief network, ISA Trans. 96 (2020) 457-467.