Learning control of fermentation process with an improved DHP algorithm

doi:10.1016/j.cjche.2016.06.012

›› 2016, Vol. 24 ›› Issue (10): 1399-1405.DOI: 10.1016/j.cjche.2016.06.012

• Process Systems Engineering and Process Safety • 上一篇下一篇

Learning control of fermentation process with an improved DHP algorithm

Dazi Li, Ningjia Meng, Tianheng Song

Institute of Automation, Beijing University of Chemical Technology, Beijing 100029, China

收稿日期:2015-10-23 修回日期:2016-03-03 出版日期:2016-10-28 发布日期:2016-11-19
通讯作者: Dazi Li,E-mail address:lidz@mail.buct.edu.cn.
基金资助:
Supported by the National Natural Science Foundation of China (61573052).

Learning control of fermentation process with an improved DHP algorithm

Dazi Li, Ningjia Meng, Tianheng Song

Institute of Automation, Beijing University of Chemical Technology, Beijing 100029, China

Received:2015-10-23 Revised:2016-03-03 Online:2016-10-28 Published:2016-11-19
Supported by:
Supported by the National Natural Science Foundation of China (61573052).

摘要/Abstract

摘要： Control of the fed-batch ethanol fermentation processes to produce maximum product ethanol is one of the key issues in the bioreactor system. However, ethanol fermentation processes exhibit complex behavior and nonlinear dynamics with respect to the cell mass, substrate, feed-rate, etc. An improved dual heuristic programming algorithm based on the least squares temporal difference with gradient correction (LSTDC) algorithm (LSTDC-DHP) is proposed to solve the learning control problem of a fed-batch ethanol fermentation process. As a new algorithm of adaptive critic designs, LSTDC-DHP is used to realize online learning control of chemical dynamical plants, where LSTDC is commonly employed to approximate the value functions. Application of the LSTDC-DHP algorithmto ethanol fermentation process can realize efficient online learning control in continuous spaces. Simulation results demonstrate the effectiveness of LSTDC-DHP, and showthat LSTDC-DHP can obtain the near-optimal feed rate trajectory faster than other-based algorithms.

关键词: Dual heuristic programming, Batch process, Ethanol fermentation process, Learning control

Abstract: Control of the fed-batch ethanol fermentation processes to produce maximum product ethanol is one of the key issues in the bioreactor system. However, ethanol fermentation processes exhibit complex behavior and nonlinear dynamics with respect to the cell mass, substrate, feed-rate, etc. An improved dual heuristic programming algorithm based on the least squares temporal difference with gradient correction (LSTDC) algorithm (LSTDC-DHP) is proposed to solve the learning control problem of a fed-batch ethanol fermentation process. As a new algorithm of adaptive critic designs, LSTDC-DHP is used to realize online learning control of chemical dynamical plants, where LSTDC is commonly employed to approximate the value functions. Application of the LSTDC-DHP algorithmto ethanol fermentation process can realize efficient online learning control in continuous spaces. Simulation results demonstrate the effectiveness of LSTDC-DHP, and showthat LSTDC-DHP can obtain the near-optimal feed rate trajectory faster than other-based algorithms.

Key words: Dual heuristic programming, Batch process, Ethanol fermentation process, Learning control

Dazi Li, Ningjia Meng, Tianheng Song. Learning control of fermentation process with an improved DHP algorithm[J]. , 2016, 24(10): 1399-1405.

参考文献

[1] J. Hong, Optimal substrate feeding policy for a fed batch fermentation with substrate and product inhibition kinetics, Biotechnol. Bioeng. 28(9) (1986) 1421-1431.
[2] M.S. Iyer, D.C. Wunsch, Dynamic re-optimization of a fed-batch fermentor using adaptive critic designs, IEEE Trans. Neural Netw. 12(6) (2001) 1433-1444.
[3] U. Yüzgeç, M. Türker, A. Hocalar, On-line evolutionary optimization of an industrial fed-batch yeast fermentation process, ISA Trans. 48(1) (2009) 79-92.
[4] A. Ashoori, B. Moshiri, A. Khaki-Sedigh, M.R. Bakhtiari, Optimal control of a nonlinear fed-batch fermentation process using model predictive approach, J. Process Control 19(7) (2009) 1162-1173.
[5] C.V. Peroni, N.S. Kaisare, J.H. Lee, Optimal control of a fed-batch bioreactor using simulation-based approximate dynamic programming, IEEE Trans. Control Syst. Technol. 13(5) (2005) 786-790.
[6] S. Syafiie, F. Tadeo, M. Villafín, A. Alonso, Learning control for batch thermal sterilization of canned foods, ISA Trans. 50(1) (2011) 82-90.
[7] C. Valencia, G. Espinosa, J. Giralt, F. Giralt, Optimization of invertase production in a fed-batch bioreactor using simulation based dynamic programming coupled with a neural classifier, Comput. Chem. Eng. 31(9) (2007) 1131-1140.
[8] D.Z. Li, L. Qian, Q.B. Jin, T.W. Tan, Reinforcement learning control with adaptive gain for a Saccharomyces cerevisiae fermentation process, Appl. Soft Comput. J. 11(8) (2011) 4488-4495.
[9] D.V. Prokhorov, D.C.Wunsch, Adaptive critic designs, IEEE Trans. Neural Netw. 8(5) (1997) 997-1007.
[10] C.Q. Lian, X. Xu, L. Zuo, Learning control of a bioreactor system using kernel-based heuristic dynamic programming, Proc. World Congr. Intelligent Control Autom. WCICA 2012, pp. 316-321.
[11] B.Wang, D.B. Zhao, C. Alippi, D.R. Liu, Dual heuristic dynamic programming for nonlinear discrete-time uncertain systems with state delay, Neurocomputing 134(2014) 222-229.
[12] C.Q. Lian, X. Xu, L. Zuo, Z.H. Huang, Adaptive critic design with graph Laplacian for online learning control of nonlinear systems, Int. J. Adapt. Control Signal Process. 28(2014) 290-304.
[13] M. Fairbank, E. Alonso, D. Prokhorov, Simple and fast calculation of the second-order gradients for globalized dual heuristic dynamic programming in neural networks, IEEE Trans. Neural Netw. Learn. Syst. 23(10) (2012) 1671-1676.
[14] J. Fu, H.B. He, X.M. Zhou, Adaptive learning and control for MIMO system based on adaptive dynamic programming, IEEE Trans. Neural Netw. 22(7) (2011) 1133-1148.
[15] Z. Ni, H.B. He, J.Y. Wen, Adaptive learning in tracking control based on the dual critic network design, IEEE Trans. Neural Netw. Learn. Syst. 24(6) (2013) 913-928.
[16] F.X. Tan, D.R. Liu, X.P. Guan, Online optimal control for VTOL aircraft system based on DHP algorithm, Proc. of the 33rd Chinese Control Conf., CCC 2014, pp. 2882-2886.
[17] X. Xu, Z.S. Hou, C.Q. Lian, H.B. He, Online learning control using adaptive critic designs with sparse kernel machines, IEEE Trans. Neural Netw. Learn. Syst. 24(5) (2013) 762-775.
[18] X. Xu, C.Q. Lian, L. Zuo, H.B. He, Kernel-based approximate dynamic programming for real-time online learning control:An experimental study, IEEE Trans. Control Syst. Technol. 22(1) (2014) 146-156.
[19] T.H. Song, D.Z. Li, L.L. Cao, K. Hirasawa, Kernel-based least squares temporal difference with gradient correction, IEEE Trans. Neural Netw. Learn. Syst. 27(4) (2016) 771-782.
[20] Z.H. Xiong, J. Zhang, Neural networkmodel-based on-line re-optimisation control of fed-batch processes using a modified iterative dynamic programming algorithm, Chem. Eng. Process. 44(4) (2005) 477-484.
[21] R.S. Sutton, Learning to predict by the method of temporal differences, Mach. Learn. 3(1998) 9-44.
[22] S.J. Bradtke, A.G. Barto, Linear least-squares algorithms for temporal difference learning, Mach. Learn. 22(1-3) (1996) 33-57.
[23] X. Xu, H.G. He, D.W. Hu, Efficient reinforcement learning using recursive leastsquares methods, J. Artif. Intell. Res. 16(2002) 259-292.
[24] S. Bhatnagar, D. Precup, D. Silver, Convergent temporal-difference learning with arbitrary smooth function approximation, Adv. Neural Inf. Process. Syst.-Proc. Conf 2009, pp. 1204-1212.
[25] R.S. Sutton, H.R. Maei, D. Precup, Fast gradient-descent methods for temporaldifference learning with linear function approximation, Proc. Int. Conf. Mach. Learn., ICML 2009, pp. 993-1000.
[26] M. Geist, O. Pietquin, Algorithmic survey of parametric value function approximation, IEEE Trans. Neural Netw. Learn. Syst. 24(6) (2013) 845-867.
[27] H.R. Maei, C. Szepesvári, S. Bhatnagar, R.S. Sutton, Toward off-policy learning control with function approximation, Proc. Int. Conf. Mach. Learn., ICML 2010, pp. 719-726.

Learning control of fermentation process with an improved DHP algorithm

Learning control of fermentation process with an improved DHP algorithm

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	Racheal Aigbe, Doga Kavaz. Unravel the potential of zinc oxide nanoparticle-carbonized sawdust matrix for removal of lead (II) ions from aqueous solution[J]. 中国化学工程学报, 2021, 29(1): 92-102.
[2]	Aihong Li, Changzhan Liu, Zhiyong Liu. A heuristic design method for batch water-using networks of multiple contaminants with regeneration unit[J]. 中国化学工程学报, 2019, 27(5): 1103-1112.
[3]	Li Jia, Wendan Tan. Just-in-time learning based integrated MPC-ILC control for batch processes[J]. Chinese Journal of Chemical Engineering, 2018, 26(8): 1713-1720.
[4]	Yongyong Hui, Xiaoqiang Zhao. Batch process monitoring based on WGNPE-GSVDD related and independent variables[J]. Chinese Journal of Chemical Engineering, 2018, 26(12): 2549-2561.
[5]	Xuan Yang, Xiao'e Ruan. Robustness of reinforced gradient-type iterative learning control for batch processes with Gaussian noise[J]. Chinese Journal of Chemical Engineering, 2016, 24(5): 623-629.
[6]	Saima Noor, Shamsul Qamar. The space time CE/SE method for solving one-dimensional batch crystallization model with fines dissolution[J]. , 2015, 23(2): 337-341.
[7]	Li Jia, Kai Yuan. Auxiliary error and probability density function based neuro-fuzzymodel and its application in batch processes[J]. Chinese Journal of Chemical Engineering, 2015, 23(12): 2013-2019.
[8]	Chen Chen, Zhihua Xiong, Yisheng Zhong. Design and Analysis of Integrated Predictive Iterative Learning Control for Batch Process Based on Two-dimensional System Theory[J]. , 2014, 22(7): 762-768.
[9]	杨国军, 李秀喜, 钱宇. A Real-time Updated Model Predictive Control Strategy for Batch Processes Based on State Estimation[J]. Chinese Journal of Chemical Engineering, 2014, 22(3): 318-329.
[10]	王立敏, 陈曦, 高福荣. An LMI Method to Robust Iterative Learning Fault-tolerant Guaranteed Cost Control for Batch Processes[J]. Chinese Journal of Chemical Engineering, 2013, 21(4): 401-411.
[11]	池荣虎, 张德霞, 刘喜梅, 侯忠生, 金尚泰. An Anticipatory Terminal Iterative Learning Control Approach with Applications to Constrained Batch Processes[J]. Chinese Journal of Chemical Engineering, 2013, 21(3): 271-275.
[12]	郭辉, 李宏光. On-line Batch Process Monitoring with Improved Multi-way Independent Component Analysis[J]. Chinese Journal of Chemical Engineering, 2013, 21(3): 263-270.
[13]	张玉良, 张贝克, 马昕, 曹柳林, 吴重光. Consequence Identification for Maloperation in Batch Process[J]. Chinese Journal of Chemical Engineering, 2013, 21(12): 1347-1359.
[14]	赵露平, 赵春晖, 高福荣. Phase Transition Analysis Based Quality Prediction for Multi-phase Batch Processes^*[J]. Chinese Journal of Chemical Engineering, 2012, 20(6): 1191-1197.
[15]	董伟威, 姚远, 高福荣. Phase Analysis and Identification Method for Multiphase Batch Processes with Partitioning Multi-way Principal Component Analysis (MPCA) Model^*[J]. Chinese Journal of Chemical Engineering, 2012, 20(6): 1121-1127.