A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling

Chin.J.Chem.Eng. ›› 2012, Vol. 20 ›› Issue (4): 715-722.

• PROCESS SYSTEMS ENGINEERING • Previous Articles Next Articles

A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling

WANG Jianlin, FENG Xuying, YU Tao

College of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100029, China

Received:2011-03-12 Revised:2011-06-08 Online:2012-09-15 Published:2012-08-28
Supported by:
Supported by the National Natural Science Foundation of China (20476007,20676013)

A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling

王建林, 冯絮影, 于涛

College of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100029, China

通讯作者: WANG Jianlin, E-mail:wangjl@mail.buct.edu.cn
基金资助:
Supported by the National Natural Science Foundation of China (20476007,20676013)

Abstract

Abstract: Support vector machine(SVM) has shown great potential in pattern recognition and regressive estimation.Due to the industrial development demands,such as the fermentation process modeling,improving the training performance on increasingly large sample sets is an important problem.However,solving a large optimization problem is computationally intensive and memory intensive.In this paper,a geometric interpretation of SVM regression(SVR) is derived,and μ-SVM is extended for both L₁-norm and L₂-norm penalty SVR.Further,Gilbert algorithm,a well-known geometric algorithm,is modified to solve SVR problems.Theoretical analysis indicates that the presented SVR training geometric algorithms have the same convergence and almost identical cost of computation as their corresponding algorithms for SVM classification.Experimental results show that the geometric methods are more efficient than conventional methods using quadratic programming and require much less memory.

Key words: support vector machine, pattern recognition, regressive estimation, geometric algorithms

摘要： Support vector machine(SVM) has shown great potential in pattern recognition and regressive estimation.Due to the industrial development demands,such as the fermentation process modeling,improving the training performance on increasingly large sample sets is an important problem.However,solving a large optimization problem is computationally intensive and memory intensive.In this paper,a geometric interpretation of SVM regression(SVR) is derived,and μ-SVM is extended for both L₁-norm and L₂-norm penalty SVR.Further,Gilbert algorithm,a well-known geometric algorithm,is modified to solve SVR problems.Theoretical analysis indicates that the presented SVR training geometric algorithms have the same convergence and almost identical cost of computation as their corresponding algorithms for SVM classification.Experimental results show that the geometric methods are more efficient than conventional methods using quadratic programming and require much less memory.

关键词: support vector machine, pattern recognition, regressive estimation, geometric algorithms

WANG Jianlin, FENG Xuying, YU Tao. A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling[J]. Chin.J.Chem.Eng., 2012, 20(4): 715-722.

王建林, 冯絮影, 于涛. A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling[J]. Chinese Journal of Chemical Engineering, 2012, 20(4): 715-722.

References

1 Chen,L.Z.,Nguang,S.K.,Chen,X.D.,Li,X.M.,"Modeling and optimization of fed-batch fermentation processes using dynamic neural networks and genetic algorithms",Biochem.Eng.J.,22(1),51-61 (2004).
2 Trelea,I.C.,Titica,M.,Landaud,S.,Latrille,E.,Corrieu,G.,Cheruy, A.,"Predictive modeling of brewing fermentation:from knowledge-based to black-box models",Math.Comput.Simulat.,56,405-424(2001).
3 Vapnik,V.N.,Statistical Learning Theory,Wiley,New York(1998).
4 Vapnik,V.N.,The Nature of Statistical Learning Theory, Springer-Verlag,New York(2000).
5 Osuna,E.,Freund R.,Girosi,F.,"Improved training algorithm for support vector machines",In:Proceedings of IEEE NNSP'97,Principe,J.,Gile,L.,Morgan,N.,Wilson,E.,eds.,IEEE Press,New York,USA,276-285(1997).
6 Platt,J.C.,"Fast training of support vector machines using sequential minimal optimization",In:Advances in Kernel Methods-Support Vector Learning,Schölkopf,B.,Burges,C.J.C.,Smola,A.J.,eds., MIT Press,Cambridge,MA,USA,185-208(1999).
7 Keerthi,S.S.,Shevade,S.K.,Bhattacharyya,C.,"Improvements to Platt's SMO algorithm for SVM classifier design",Neural Comput.,13(3),637-649(2001).
8 Bennett,K.P.,Bredensteiner,E.J.,"Duality and geometry in SVM classifiers",In:Proceedings of the 17th International Conference on Machine Learning,Langley,P.,ed.,Morgan Kaufmann Publishers, San Francisco,CA,USA,57-64(2000).
9 Crisp,D.J.,Burges,C.J.C.,"A geometric interpretation of v-SVM classifiers",In:Advances in Neural Information Processing Systems 12,Solla,S.A.,Leen,T.K.,Müller,K.R.,eds.,MIT Press,Cambridge,MA,USA,244-251(2000).
10 Keerthi,S.S.,Shevade,S.K.,Bhattacharyya,C.,Murthy,K.R.K.,"A fast iterative nearest point algorithm for support vector machine classifier design",IEEE Trans Neural Networks,11(1),124-136 (2000).
11 Cortes,C.,Vapnik,V.,"Support vector networks",Mach.Learn.,36,1991-1996(2003).
12 Schölkopf,B.,Smola,A.J.,Williamson,R.C.,Bartlett,P.L.,"New support vector algorithms",Neural Comput.,12,1207-1245(2000).
13 Zhu,J.,Rosset,S.,Hastie,T.,Tibshirani,R.,"1-norm support vector machines",In:Advances in Neural Information Processing Systems 16,Thrun,S.,Saul,L.,Schòlkopf,B.,eds.,Neural Information Processing Systems Foundation(2003).
14 Tao,Q.,Wu,G.W.,Wang,J.,"A generalized SK algorithm for learning v-SVM classifiers",Pattern Recogn.Lett.,25(10),1165-1171 (2004).
15 Tao,Q.,Wu,G.W.,Wang,J.,"A general soft method for learning SVM classifiers with L1-norm penalty",Pattern Recogn.,41,939-948(2008).
16 Bi,J.B.,Bennett,K.P.,"A geometric approach to support vector regression",Neurocomputing,55,79-108(2003).
17 Gilbert,E.G.,"An iterative procedure for computing the minimum of a quadratic form on a convex set",SIAM J.Control,4(1),61-79 (1996).
18 Schlesinger,M.I.,Hlaváč,V.,"Ten lectures on statistical and structural pattern recognition",Kluwer Academic Publishers,Dordrecht (2002).
19 Franc,V.,Hlaváč,V.,"An iterative algorithm learning the maximal margin classifier",Pattern Recogn.,36,1985-1996(2003).
20 Mitchell,B.F.,Dem'yanov,V.F.,Malozemov,V.N.,"Finding the point of a polyhedron closet to the origin",SIAM J.Control,12,19-26(1974).

[1]	Kexin Bi, Tong Qiu. An intelligent SVM modeling process for crude oil properties prediction based on a hybrid GA-PSO method [J]. Chinese Journal of Chemical Engineering, 2019, 27(8): 1888-1894.
[2]	Mohammad Mesbah, Ebrahim Soroush, Mashallah Rezakazemi. Development of a least squares support vector machine model for prediction of natural gas hydrate formation temperature [J]. , 2017, 25(9): 1238-1248.
[3]	Kai Feng, Jiangang Lu, Jinshui Chen. Nonlinearmodel predictive control based on support vectormachine and genetic algorithm [J]. Chin.J.Chem.Eng., 2015, 23(12): 2048-2052.
[4]	Jian Tang, Tianyou Chai, Zhuo Liu, Wen Yu. Selective ensemble modeling based on nonlinear frequency spectral feature extraction for predicting load parameter in ball mills [J]. Chin.J.Chem.Eng., 2015, 23(12): 2020-2028.
[5]	Juan Chen, Xiaoyi Huang, Yanlei Qi, Xin Qi, Qing Guo. Process Optimization of Ultrasonic Extraction of Puerarin Based on Support Vector Machine [J]. Chin.J.Chem.Eng., 2014, 22(7): 735-741.
[6]	FENG Xuying, YU Tao, WANG Jianlin. NonliNonlinear GPC with In-place Trained RLS-SVM Model for DOC Control in a Fed-batch Bioreactor^* [J]. Chin.J.Chem.Eng., 2012, 20(5): 988-994.
[7]	HUA Chenquan, WANG Changming, GENG Yanfeng, SHI Tianming. Noninvasive Flow Regime Identification for Wet Gas Flow Based on Flow-induced Vibration [J]. , 2010, 18(5): 795-803.
[8]	WANG Jianlin, ZHAO Liqiang, YU Tao. On-line Estimation in Fed-batch Fermentation Process Using State Space Model and Unscented Kalman Filter [J]. , 2010, 18(2): 258-264.
[9]	GAO Xuejin, WANG Pu, QI Yongsheng, ZHANG Yating, ZHANG Huiqing, YAN Aijun. An Optimal Control Strategy Combining SVM with RGA for Improving Fermentation Titer [J]. , 2010, 18(1): 95-101.
[10]	LI Lijuan, SU Hongye, CHU Jian. Modeling of Isomerization of C₈ Aromatics by Online Least Squares Support Vector Machine [J]. , 2009, 17(3): 437-444.
[11]	ZHOU Yunlong, CHEN Fei, SUN Bin. Identification Method of Gas-Liquid Two-phase Flow Regime Based on Image Multi-feature Fusion and Support Vector Machine [J]. , 2008, 16(6): 932-840.
[12]	FU Yongfeng, SU Hongye, ZHANG Ying, CHU Jian. Adaptive Soft-sensor Modeling Algorithm Based on FCMISVM and Its Application in PX Adsorption Separation Process [J]. , 2008, 16(5): 746-751.
[13]	BAO Zhejing, PI Daoying, SUN Youxian. Nonlinear model predictive control based on support vector machine with multi-kernel [J]. , 2007, 15(5): 691-697.
[14]	MAOYong,XIAZheng,YINZheng,SUNYouxian,WANZheng. Fault diagnosis based on fuzzy support vector machine with parameter tuning and feature selection [J]. , 2007, 15(2): 233-239.
[15]	LIYunfeng,WANGZhifeng, YUANJingqi. On-line fault detection using SVM-based dynamic MPLS for batch processes [J]. , 2006, 14(6): 754-758.

A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling

A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments