收稿日期: 2024-06-04
网络出版日期: 2024-09-23
基金资助
国家自然科学基金资助项目(61966006);贵州省自然科学基金资助项目(ZK[2023]YB449);贵州省教育厅青年科技人才成长项目(黔教合KY字[2020]240);茅台学院高层次人才启动经费项目(mygccrc[2022]105);遵义市科技与大数据局、茅台学院市校科技合作资金项目(遵市科合HZ字(2022)174号,遵市科合HZ字(2024)384号)
Construction of Upper Boundary Model Based on Least Squares Support Vector Regression
Received date: 2024-06-04
Online published: 2024-09-23
Supported by
the National Natural Science Foundation of China(61966006);the Natural Science Foundation of Guizhou Province(ZK[2023]YB449)
目前,基于数据驱动的传统非线性系统建模方法主要着眼于模型拟合和应用,鉴于此,该文针对来自系统的某个重要参数受不确定性影响的最大容忍极限输出,构建基于最小二乘支持向量回归(LSSVR)的上边界模型,深入剖析了上边界模型的精度与稀疏特性之间的平衡关系对上边界模型输出的影响。首先,借助LSSVR的优化问题,将原等式线性约束变成满足上边界模型的不等式约束;接着,为提高模型精度,引入基于上边界模型预测输出与实际输出之间逼近误差的不等式约束;与此同时,借助LSSVR的权值二范数来控制上边界模型结构的复杂度,从而构建出新的目标函数,并与满足上边界模型的不等式约束建立新的优化问题;最后,对所建立的优化问题引入拉格朗日函数并借助Karush-Kuhn-Tucker最优化条件来获取相应的对偶优化问题,并将其转化为标准的二次规划问题来求解上边界模型的参数。由于所构造的新优化问题满足凸性,因此模型系数解是全局最优的。该文还通过实验分析了反映模型精度的最大逼近误差、均方根误差及反映模型稀疏特性的指标,论证了所提方法的有效性和优越性。
关键词: 上边界模型; 对偶优化问题; 全局最优解; 最小二乘支持向量回归; 二次规划
刘小雍 , 曾成斌 , 刘赟 , 何国锋 , 闫庚龙 . 基于最小二乘支持向量回归的上边界模型构建[J]. 华南理工大学学报(自然科学版), 2024 , 52(12) : 139 -150 . DOI: 10.12141/j.issn.1000-565X.240283
At present, traditional data-driven nonlinear system modeling methods primarily focus on model fitting and application. In this context, this paper constructs an upper boundary model based on least squares support vector regression (LSSVR) for the maximum tolerable output of a critical parameter from the system, which is influenced by uncertainty. The study delves into the relationship between the balance of model accuracy and sparsity, and its effect on the model output. First, by utilizing the optimization problem of LSSVR, the original equality linear constraints are transformed into inequality constraints that satisfy the upper boundary model. Next, to improve the model’s accuracy, an inequality constraint based on the approximation error between the predicted output of the upper bound model and the actual output is introduced. Meanwhile, the LSSVR’s weight L2-norm is employed to control the complexity of the upper boundary model’s structure, thereby constructing a new objective function and establishing a new optimization problem that satisfies the inequality constraints of the upper bound model. Finally, the Lagrangian function is introduced into the optimization problem, and the Karush-Kuhn-Tucker conditions are used to derive the corresponding dual optimization problem, which is then converted into a standard quadratic programming problem to solve for the parameters of the upper bound model. Since the new optimization problem satisfies convexity, the solution for the model coefficients is globally optimal. The effectiveness and superiority of the proposed method are validated through experimental analysis, where the maximum approximation error, root mean square error, and sparsity-related metrics are used to reflect the model’s accuracy and sparsity characteristics.
| 1 | SHANG D Y, LI X P, YIN M,et al .Neural network model identification control of dual-inertia system with a flexible load considering payload mass variation and nonlinear deformation[J].Engineering Applications of Artificial Intelligence,2024,128:107503/1-19. |
| 2 | MENG X, ZHANG Y, QUAN L,et al .A self-organizing fuzzy neural network with hybrid learning algorithm for nonlinear system modeling[J].Information Sciences,2023,642:119145/1-20. |
| 3 | LI J, SONG G, YAN J,et al .Data-driven fault detection and classification for MTDC systems by integrating HCTSA and soft max regression[J].IEEE Transactions on Power Delivery,2022,37(2):893-904. |
| 4 | FANG X, QU J, LIU B .Overcoming limited fault data:intermittent fault detection in analog circuits via improved GAN[J].IEEE Transactions on Instrumentation and Measurement,2024,73:1-11. |
| 5 | 哀微,朱学峰 .基于单边SPSA的动态偏差数据驱动控制算法[J].华南理工大学学报(自然科学版),2012,40(9):81-86,92. |
| AI Wei, ZHU Xue-feng .One-sided SPSA-based data-driven control algorithm with dynamic deviations[J].Journal of South China University of Technology (Natural Science Edition),2012,40(9):81-86,92. | |
| 6 | 李中奇,周靓,杨辉 .高速动车组数据驱动无模型自适应积分滑模预测控制[J].自动化学报,2024,50(1):194-210. |
| LI Zhongqi, ZHOU Liang, YANG Hui .Data-driven model-free adaptive integral sliding mode predictive control for high-speed electric multiple unit[J].Acta Automatica Sinica,2024,50(1):194-210. | |
| 7 | BOUBOULIS P, THEODORIDIS S, MAVROFORAKIS C,et al .Complex support vector machines for regression and quaternary classification[J].IEEE Transactions on Neural Networks and Learning Systems,2015,26(6):1260-1274. |
| 8 | 黄伟,黄遥,牛继强,等 .基于多尺度LBP和复合核的高光谱图像分类方法[J].信阳师范学院学报(自然科学版),2021,34(2):300-307. |
| HUANG Wei, HUANG Yao, NIU Jiqiang,et al .Hyperspectral image classification based on multi-scale local binary pattern and composite kernel function[J].Journal of Xinyang Normal University(Natural Science Edition),2021,34(2):300-307. | |
| 9 | BARANYI P .How to vary the input pace of a T-S fuzzy model: a TP model transformation-based approach[J].IEEE Transactions on Fuzzy Systems,2022,30(2):345-356. |
| 10 | KALTWANG S, TODOROVIC S, PANTIC M .Doub-ly sparse relevance vector machine for continuous facial behavior estimation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(9):1748-1761. |
| 11 | WANG X, JIANG B, WU S .Multivariate relevance vector regression based degradation modeling and remaining useful life prediction[J].IEEE Transactions on Industrial Electronics,2021,69(9):9514-9523. |
| 12 | HAN H, LIN Z, QIAO J .Modeling of nonlinear systems using the self-organizing fuzzy neural network with adaptive gradient algorithm[J].Neurocomputing,2017,266(29):566-578. |
| 13 | VAPNIK V .An overview of statistical learning theory[J].IEEE Transactions on Neural Networks,1999,10(5):988-999. |
| 14 | BURGES C .A tutorial on support vector machines for pattern recognition[J].Data Mining and Knowledge Discovery,1998,2(2):121-167. |
| 15 | SUYKENS J, VAN GESTEL T, DE BRABANTER J .Least squares support vector machines[M].Singapore:World Scientific,2002. |
| 16 | SUYKENS J A K, VANDEWALLE J .Least squares support vector machine classifiers[J].Neural Processing Letter,1999,9(3):293-300. |
| 17 | GOETHALS I, PELCKMANS K, SUYKENS J .Subspace identification of Hammerstein systems using least squares support vector machines[J].IEEE Transactions on Automatic Control,2005,50(10):1509-1519. |
| 18 | BAKO L, MERCERE G, LECOEUCHE S .Recursive subspace identification of Hammerstein models based on least squares support vector machines[J].IET Control Theory & Applications,2009,3(9):1209-1216. |
| 19 | LIU X, FANG H, CHEN Z .A novel cost function based on decomposing least-square support vector machine for Takagi-Sugeno fuzzy system identification[J].IET Control Theory & Applications,2014,8(5):338-347. |
| 20 | ZHENG W, WANG C, LIU D .Combustion process modeling based on deep sparse least squares support vector regression[J].Engineering Applications of Artificial Intelligence,2024,132:107869/1-14. |
| 21 | ZHANG H, WU Q, LI F .Application of online multitask learning based on least squares support vector regression in the financial market[J].Applied Soft Computing,2022,121:108754/1-26. |
| 22 | KAMM S, SUSHMA S, TIMO M .A survey on machine learning based analysis of heterogeneous data in industrial automation[J].Computers in Industry,2023,149:103930/1-14. |
| 23 | LI H, ZHANG Z, LI T .A review on physics-informed data-driven remaining useful life prediction:challenges and opportunities[J].Mechanical Systems and Signal Processing,2024,209(1):111120/1-32. |
| 24 | 刘道华,冯宸,赵岩松,等 .多尺度特征融合网络的磁共振快速重建方法[J].信阳师范学院学报(自然科学版),2023,36(1):127-131. |
| LIU Daohua, FENG Chen, ZHAO Yansong,et al .A fast MRI reconstruction method based on multi-scale feature fusion network[J].Journal of Xinyang Normal University (Natural Science Edition),2023,36(1):127-131. | |
| 25 | LI S, FANG H, LIU X .Parameter optimization of support vector regression based on sine cosine algorithm[J].Expert Systems with Applications,2018,91:63-77. |
| 26 | TIPPING M .Sparse Bayesian learning and the re-levance vector machine[J].Journal of Machine Learning Research,2001,1(3):211-244. |
/
| 〈 |
|
〉 |