A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling

Chinese Journal of Chemical Engineering ›› 2012, Vol. 20 ›› Issue (4): 715-722.

A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling

王建林, 冯絮影, 于涛

College of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100029, China

收稿日期:2011-03-12 修回日期:2011-06-08 出版日期:2012-08-28 发布日期:2012-09-15
通讯作者: WANG Jianlin, E-mail:wangjl@mail.buct.edu.cn
基金资助:
Supported by the National Natural Science Foundation of China (20476007,20676013)

A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling

WANG Jianlin, FENG Xuying, YU Tao

College of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100029, China

Received:2011-03-12 Revised:2011-06-08 Online:2012-08-28 Published:2012-09-15
Supported by:
Supported by the National Natural Science Foundation of China (20476007,20676013)

摘要/Abstract

摘要： Support vector machine(SVM) has shown great potential in pattern recognition and regressive estimation.Due to the industrial development demands,such as the fermentation process modeling,improving the training performance on increasingly large sample sets is an important problem.However,solving a large optimization problem is computationally intensive and memory intensive.In this paper,a geometric interpretation of SVM regression(SVR) is derived,and μ-SVM is extended for both L₁-norm and L₂-norm penalty SVR.Further,Gilbert algorithm,a well-known geometric algorithm,is modified to solve SVR problems.Theoretical analysis indicates that the presented SVR training geometric algorithms have the same convergence and almost identical cost of computation as their corresponding algorithms for SVM classification.Experimental results show that the geometric methods are more efficient than conventional methods using quadratic programming and require much less memory.

关键词: support vector machine, pattern recognition, regressive estimation, geometric algorithms

Abstract: Support vector machine(SVM) has shown great potential in pattern recognition and regressive estimation.Due to the industrial development demands,such as the fermentation process modeling,improving the training performance on increasingly large sample sets is an important problem.However,solving a large optimization problem is computationally intensive and memory intensive.In this paper,a geometric interpretation of SVM regression(SVR) is derived,and μ-SVM is extended for both L₁-norm and L₂-norm penalty SVR.Further,Gilbert algorithm,a well-known geometric algorithm,is modified to solve SVR problems.Theoretical analysis indicates that the presented SVR training geometric algorithms have the same convergence and almost identical cost of computation as their corresponding algorithms for SVM classification.Experimental results show that the geometric methods are more efficient than conventional methods using quadratic programming and require much less memory.

Key words: support vector machine, pattern recognition, regressive estimation, geometric algorithms

王建林, 冯絮影, 于涛. A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling[J]. Chinese Journal of Chemical Engineering, 2012, 20(4): 715-722.

WANG Jianlin, FENG Xuying, YU Tao. A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling[J]. Chin.J.Chem.Eng., 2012, 20(4): 715-722.

参考文献

1 Chen,L.Z.,Nguang,S.K.,Chen,X.D.,Li,X.M.,"Modeling and optimization of fed-batch fermentation processes using dynamic neural networks and genetic algorithms",Biochem.Eng.J.,22(1),51-61 (2004).
2 Trelea,I.C.,Titica,M.,Landaud,S.,Latrille,E.,Corrieu,G.,Cheruy, A.,"Predictive modeling of brewing fermentation:from knowledge-based to black-box models",Math.Comput.Simulat.,56,405-424(2001).
3 Vapnik,V.N.,Statistical Learning Theory,Wiley,New York(1998).
4 Vapnik,V.N.,The Nature of Statistical Learning Theory, Springer-Verlag,New York(2000).
5 Osuna,E.,Freund R.,Girosi,F.,"Improved training algorithm for support vector machines",In:Proceedings of IEEE NNSP'97,Principe,J.,Gile,L.,Morgan,N.,Wilson,E.,eds.,IEEE Press,New York,USA,276-285(1997).
6 Platt,J.C.,"Fast training of support vector machines using sequential minimal optimization",In:Advances in Kernel Methods-Support Vector Learning,Schölkopf,B.,Burges,C.J.C.,Smola,A.J.,eds., MIT Press,Cambridge,MA,USA,185-208(1999).
7 Keerthi,S.S.,Shevade,S.K.,Bhattacharyya,C.,"Improvements to Platt's SMO algorithm for SVM classifier design",Neural Comput.,13(3),637-649(2001).
8 Bennett,K.P.,Bredensteiner,E.J.,"Duality and geometry in SVM classifiers",In:Proceedings of the 17th International Conference on Machine Learning,Langley,P.,ed.,Morgan Kaufmann Publishers, San Francisco,CA,USA,57-64(2000).
9 Crisp,D.J.,Burges,C.J.C.,"A geometric interpretation of v-SVM classifiers",In:Advances in Neural Information Processing Systems 12,Solla,S.A.,Leen,T.K.,Müller,K.R.,eds.,MIT Press,Cambridge,MA,USA,244-251(2000).
10 Keerthi,S.S.,Shevade,S.K.,Bhattacharyya,C.,Murthy,K.R.K.,"A fast iterative nearest point algorithm for support vector machine classifier design",IEEE Trans Neural Networks,11(1),124-136 (2000).
11 Cortes,C.,Vapnik,V.,"Support vector networks",Mach.Learn.,36,1991-1996(2003).
12 Schölkopf,B.,Smola,A.J.,Williamson,R.C.,Bartlett,P.L.,"New support vector algorithms",Neural Comput.,12,1207-1245(2000).
13 Zhu,J.,Rosset,S.,Hastie,T.,Tibshirani,R.,"1-norm support vector machines",In:Advances in Neural Information Processing Systems 16,Thrun,S.,Saul,L.,Schòlkopf,B.,eds.,Neural Information Processing Systems Foundation(2003).
14 Tao,Q.,Wu,G.W.,Wang,J.,"A generalized SK algorithm for learning v-SVM classifiers",Pattern Recogn.Lett.,25(10),1165-1171 (2004).
15 Tao,Q.,Wu,G.W.,Wang,J.,"A general soft method for learning SVM classifiers with L1-norm penalty",Pattern Recogn.,41,939-948(2008).
16 Bi,J.B.,Bennett,K.P.,"A geometric approach to support vector regression",Neurocomputing,55,79-108(2003).
17 Gilbert,E.G.,"An iterative procedure for computing the minimum of a quadratic form on a convex set",SIAM J.Control,4(1),61-79 (1996).
18 Schlesinger,M.I.,Hlaváč,V.,"Ten lectures on statistical and structural pattern recognition",Kluwer Academic Publishers,Dordrecht (2002).
19 Franc,V.,Hlaváč,V.,"An iterative algorithm learning the maximal margin classifier",Pattern Recogn.,36,1985-1996(2003).
20 Mitchell,B.F.,Dem'yanov,V.F.,Malozemov,V.N.,"Finding the point of a polyhedron closet to the origin",SIAM J.Control,12,19-26(1974).

[1]	Kexin Bi, Tong Qiu. An intelligent SVM modeling process for crude oil properties prediction based on a hybrid GA-PSO method[J]. 中国化学工程学报, 2019, 27(8): 1888-1894.
[2]	Mohammad Mesbah, Ebrahim Soroush, Mashallah Rezakazemi. Development of a least squares support vector machine model for prediction of natural gas hydrate formation temperature[J]. , 2017, 25(9): 1238-1248.
[3]	Kai Feng, Jiangang Lu, Jinshui Chen. Nonlinearmodel predictive control based on support vectormachine and genetic algorithm[J]. Chinese Journal of Chemical Engineering, 2015, 23(12): 2048-2052.
[4]	Jian Tang, Tianyou Chai, Zhuo Liu, Wen Yu. Selective ensemble modeling based on nonlinear frequency spectral feature extraction for predicting load parameter in ball mills[J]. Chinese Journal of Chemical Engineering, 2015, 23(12): 2020-2028.
[5]	Juan Chen, Xiaoyi Huang, Yanlei Qi, Xin Qi, Qing Guo. Process Optimization of Ultrasonic Extraction of Puerarin Based on Support Vector Machine[J]. Chinese Journal of Chemical Engineering, 2014, 22(7): 735-741.
[6]	冯絮影, 于涛, 王建林. NonliNonlinear GPC with In-place Trained RLS-SVM Model for DOC Control in a Fed-batch Bioreactor^*[J]. Chinese Journal of Chemical Engineering, 2012, 20(5): 988-994.
[7]	华陈权, 王昌明, 耿艳峰, 石天明. Noninvasive Flow Regime Identification for Wet Gas Flow Based on Flow-induced Vibration[J]. , 2010, 18(5): 795-803.
[8]	王建林, 赵利强, 于涛. On-line Estimation in Fed-batch Fermentation Process Using State Space Model and Unscented Kalman Filter[J]. , 2010, 18(2): 258-264.
[9]	高学金, 王普, 齐咏生, 张亚庭, 张会清, 严爱军. An Optimal Control Strategy Combining SVM with RGA for Improving Fermentation Titer[J]. , 2010, 18(1): 95-101.
[10]	李丽娟, 苏宏业, 褚建. Modeling of Isomerization of C₈ Aromatics by Online Least Squares Support Vector Machine[J]. , 2009, 17(3): 437-444.
[11]	周云龙, 陈飞, 孙斌. Identification Method of Gas-Liquid Two-phase Flow Regime Based on Image Multi-feature Fusion and Support Vector Machine[J]. , 2008, 16(6): 932-840.
[12]	傅永峰, 苏宏业, 张英, 褚健. Adaptive Soft-sensor Modeling Algorithm Based on FCMISVM and Its Application in PX Adsorption Separation Process[J]. , 2008, 16(5): 746-751.
[13]	包哲静, 皮道映, 孙优贤. 基于多核支持向量机的非线性模型预测控制[J]. , 2007, 15(5): 691-697.
[14]	毛勇, 夏铮, 尹征, 孙优贤, 万征. 基于结合参数整定和特征选择策略的模糊支持向量机的故障诊断[J]. , 2007, 15(2): 233-239.
[15]	李运锋,汪志锋, 袁景淇. 基于支持向量机MPLS的间歇过程故障诊断方法[J]. , 2006, 14(6): 754-758.

A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling

A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价