Application of convolutional neural networks to large-scale naphtha pyrolysis kinetic modeling

doi:10.1016/j.cjche.2018.09.021

Abstract

Abstract: System design and optimization problems require large-scale chemical kinetic models. Pure kinetic models of naphtha pyrolysis need to solve a complete set of stiff ODEs and is therefore too computational expensive. On the other hand, artificial neural networks that completely neglect the topology of the reaction networks often have poor generalization. In this paper, a framework is proposed for learning local representations from largescale chemical reaction networks. At first, the features of naphtha pyrolysis reactions are extracted by applying complex network characterization methods. The selected features are then used as inputs in convolutional architectures. Different CNN models are established and compared to optimize the neural network structure. After the pre-training and fine-tuning step, the ultimate CNN model reduces the computational cost of the previous kinetic model by over 300 times and predicts the yields of main products with the average error of less than 3%. The obtained results demonstrate the high efficiency of the proposed framework.

Key words: Convolutional neural network, Network motif, Naphtha pyrolysis, Kinetic modeling

摘要： System design and optimization problems require large-scale chemical kinetic models. Pure kinetic models of naphtha pyrolysis need to solve a complete set of stiff ODEs and is therefore too computational expensive. On the other hand, artificial neural networks that completely neglect the topology of the reaction networks often have poor generalization. In this paper, a framework is proposed for learning local representations from largescale chemical reaction networks. At first, the features of naphtha pyrolysis reactions are extracted by applying complex network characterization methods. The selected features are then used as inputs in convolutional architectures. Different CNN models are established and compared to optimize the neural network structure. After the pre-training and fine-tuning step, the ultimate CNN model reduces the computational cost of the previous kinetic model by over 300 times and predicts the yields of main products with the average error of less than 3%. The obtained results demonstrate the high efficiency of the proposed framework.

关键词: Convolutional neural network, Network motif, Naphtha pyrolysis, Kinetic modeling

Feng Hua, Zhou Fang, Tong Qiu. Application of convolutional neural networks to large-scale naphtha pyrolysis kinetic modeling[J]. Chin.J.Chem.Eng., 2018, 26(12): 2562-2572.

Feng Hua, Zhou Fang, Tong Qiu. Application of convolutional neural networks to large-scale naphtha pyrolysis kinetic modeling[J]. Chinese Journal of Chemical Engineering, 2018, 26(12): 2562-2572.

References

[1] P. Kumar, D. Kunzru, Modeling of naphtha pyrolysis, Ind. Eng. Chem. Process. Des. Dev. 24(3) (1985) 774-782.

[2] M. Van Goethem, F. Kleinendorst, N. van Velzen, M. Dente, E. Ranzi, Equation based spyro model and optimiser for the modelling of the steam cracking process, Escape-12 Supplementary Proceedings 2002, pp. 26-29.

[3] E. Ranzi, A. Frassoldati, G. Silvia, T. Faravelli, Wide-range kinetic modeling study of the pyrolysis, partial oxidation, and combustion of heavy n-alkanes, Ind. Eng. Chem. Res. 44(14) (2005) 5170-5183.

[4] Z. Fang, T. Qiu, W. Zhou, Coupled simulation of recirculation zonal firebox model and detailed kinetic reactor model in an industrial ethylene cracking furnace, Chin. J. Chem. Eng. 25(8) (2017) 1091-1100.

[5] P. Erdi, J. Toth, Mathematical Models of Chemical Reactions:Theory and Applications of Deterministic and Stochastic Models, Manchester University Press, 1989.

[6] C.W. Gear, The automatic integration of ordinary differential equations, Commun. ACM 14(3) (1971) 176-179.

[7] U.S. Bhalla, Understanding complex signaling networks through models and metaphors, Prog. Biophys. Mol. Biol. (ISSN:0079-6107) 81(1) (2003) 45-65.

[8] G.P. Wagner, M. Pavlicev, J.M. Cheverud, The road to modularity, Nat. Rev. Genet. 8(12) (2007) 921-931.

[9] B.R. Hough, D.A.C. Beck, D.T. Schwartz, J. Pfaendtner, Application of ma-chine learning to pyrolysis reaction networks:Reducing model solution time to enable process optimization, Comput. Chem. Eng. 104(Supplemen-t C) (2017) 56-63.

[10] M. Zhang, X. Liu, Z. Zhang, A soft sensor for industrial melt index prediction based on evolutionary extreme learning machine, Chin. J. Chem. Eng. 24(8) (2016) 1013-1019.

[11] B. Shi, X. Yang, L. Yan, Optimization of a crude distillation unit using a combination of wavelet neural network and line-up competition algorithm, Chin. J. Chem. Eng. 25(8) (2017) 1013-1021.

[12] Q. Zhu, Y. Jia, D. Peng, Y. Xu, Study and application of fault prediction methods with improved reservoir neural networks, Chin. J. Chem. Eng. 22(7) (2014) 812-819.

[13] Y. Lecun, L. Bottou, Y. Bengio, P. Haffner, Gradient-Based Learning Applied to Document Recognition, IEEE, 19982278-2324.

[14] Y.L. Cun, B. Boser, J.S. Denker, R.E. Howard, W. Habbard, L.D. Jackel, D. Hen-Derson, Handwritten digit recognition with a back-propagation network, Advances in Neural Information Processing Systems 1990, pp. 396-404.

[15] Hecht-Nielsen, Theory of the Backpropagation Neural Network, International Joint Conference on Neural Networks, vol. 1, 2002593-605.

[16] A. Wagner, D.A. Fell, The small world inside large metabolic networks, Proc. R. Soc. Lond. B Biol. Sci. 268(1478) (2001) 1803-1810.

[17] L. Zhang, B. Chen, T. Qiu, A multi-scale model of naphtha pyrolysis process, 11th International Symposium on Process Systems Engineering-PSE2012, vol. 15, Elsevier 2012, p. 120.

[18] M. Dente, E. Ranzi, G. Bozzano, S. Pierucci, F.I. Kleinendorst, M.W. van Goethem, Pyrolysis of naphtha feedstocks:automatic generation of detailed kinetics and lumping procedures, Comput. Aided Chem. Eng. 28(2010) 823-828.

[19] M. Bastian, S. Heymann, M. Jacomy, et al., Gephi:an open source software for exploring and manipulating networks, ICWSM 8(2009) 361-362.

[20] M. Niepert, M. Ahmed, K. Kutzkov, Learning convolutional neural networks for graphs, International Conference on Machine Learning 2016, pp. 2014-2023.

[21] R. Kondor, N. Shervashidze, K.M. Borgwardt, The graphlet spectrum, Proceedings of the 26th Annual International Conference on Machine Learning, ACM 2009, pp. 529-536.

[22] N. Shervashidze, S. Vishwanathan, T. Petri, K. Mehlhorn, K. Borgwardt, Efficient graphlet kernels for large graph comparison, Artificial Intelligence and Statistics 2009, pp. 488-495.

[23] S. Wernicke, Efficient detection of network motifs, IEEE ACM Trans. Comput. Biol. Bioinforma. 4(3) (2006) 347-359.

[24] S. Wernicke, F. Rasche, FANMOD:A tool for fast network motif detection, Bioinformatics 22(9) (2006) 1152-1153.

[25] R. Milo, S. Shen-Orr, S. Itzkovitz, N. Kashtan, D. Chklovskii, U. Alon, Network motifs:Simple building blocks of complex networks, Science 298(5594) (2002) 824-827.

[26] S.S. Maslov, U.K. Alon, Correlation profiles and motifs in complex networks, Tech. Rep, Brookhaven National Laboratory (US), 2004.

[27] G.B. Orr, K.-R. Muller, Neural Networks:Tricks of The Trade, Springer, 2003.

[28] X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics 2010, pp. 249-256.

[29] J. Gu, Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, G. Wang, Recent advances in convolutional neural networks, Comput. Sci. (2017), URL http://arxiv.org/abs/1512.07108.

[30] A. Krizhevsky, I. Sutskever, G.E. Hinton, ImageNet classification with deep convolutional neural networks, International Conference on Neural Information Processing Systems 2012, pp. 1097-1105.

[31] V. Nair, G.E. Hinton, Rectified linear units improve restricted Boltzmann machines, International Conference on Machine Learning 2010, pp. 807-814.

[32] G.E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, R.R. Salakhutdinov, Improving neural networks by preventing co-adaptation of feature detectors, Comput. Sci. 3(4) (2012) 212-223.

[33] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhouck-E, A. Rabinovich, Going deeper with convolutions, Computer Vision and Pattern Recognition 2015, pp. 1-9.

[34] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, Computer Vision and Pattern Recognition 2016, pp. 2818-2826.

[35] W. Zhou, T. Qiu, Zone modeling of radiative heat transfer in industrial furnaces using adjusted Monte-Carlo integral method for direct exchange area calculation, Appl. Therm. Eng. 81(2015) 161-167.