Soft-Sensor Modeling Method Based on Ensemble Kalman Filter-Elman Neural Network

doi:10.12141/j.issn.1000-565X.220625

Abstract

Abstract:

Wastewater treatment system is a dynamic system with complex nonlinearity and large time delay. Due to the complexity of the process, the incompleteness of the testing equipment and the constraint of economic cost, some important effluent indicators cannot be detected accurately. To solve this problem, this paper proposes a soft-sensor method based on an ensemble Kalman filter-Elman neural network. The traditional dynamic neural network has the dynamic memory ability to process time-delay data, so it can be used in data-driven soft sensing modeling. However, the conventional training method is easy to trap in a local minimum, resulting in poor prediction performance. This paper introduces the ensemble Kalman filter and the dual finite-size ensemble Kalman filter, and, together with the Elman neural network for gradient-free training, to construct two soft sensor models, which not only improve the prediction performance of Elman neural network but also provide a simple and gradient-free training method for neural network. The two models are then applied to a dataset of the University of California, Irvine (UCI data). The results show that the proposed method based on ensemble Kalman filter-Elman neural network possesses good prediction performance, and that the ensemble Kalman filter can be used as an alternative gradient-free method to train neural networks.

Key words: soft sensor, ensemble Kalman filter, Elman neural network, wastewater treatment

CLC Number:

TP 277

FANG Gang, YUAN Longhua, WANG Xiaoming, et al. Soft-Sensor Modeling Method Based on Ensemble Kalman Filter-Elman Neural Network[J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(8): 126-136.

Figures/Tables 6

Fig.1

Table 1

UCI auxiliary variables"

序号	变量符号	变量含义	单位
1	BOD_E	进水生物需氧量	mg/L
2	COD_E	进水化学需氧量	mg/L
3	BOD_P	预处理池生物需氧量	mg/L
4	pH_D	二沉池进水pH值
5	BOD_D	二沉池进水生物需氧量	mg/L
6	COD_D	二沉池进水化学需氧量	mg/L
7	$ρ S S, D$	二沉池进水固体悬浮物质量浓度	mg/L
8	$ρ S E D, D$	二沉池可降解固体质量浓度	mg/L
9	ρ_BOD，RD-P	初沉池生物需氧量的性能输入	mg/L
10	ρ_SS，RD-P	初沉池固体悬浮物的性能输入	mg/L
11	ρ_BOD，RD-S	性能输入的二沉池生物需氧量	mg/L
12	ρ_COD，RD-S	性能输入的二沉池化学需氧量	mg/L
13	ρ_BOD，RD-G	全局性能输入的生物需氧量	mg/L
14	ρ_COD，RD-G	全局性能输入的化学需氧量	mg/L
15	ρ_SS，RD-G	全局性能输入的固体悬浮物质量浓度	mg/L
16	ρ_SED，RD-G	全局性能输入的沉积物质量浓度	mg/L
17	pH_S	出水pH值
18	$ρ S E D, S$	出水可降解固体的质量浓度	mg/L

Table 1

Table 2

Fig.2

Fig.3

Fig.4

References 35

1	黄道平，刘乙奇，李艳．软测量在污水处理过程中的研究与应用［J］．化工学报，2011，62（1）：1-9．
	HUANG Daoping， LIU Yiqi， LI Yan ．Soft sensor research and its application in wastewater treatment［J］．CIESC Journal，2011，62（1）：1-9．
2	GENG Z， CHEN Z， MENG Q，et al ．Novel transformer based on gated convolutional neural network for dynamic soft sensor modeling of industrial processes［J］．IEEE Transactions on Industrial Informatics，2021，18（3）：1521-1529．
3	KADLEC P， GABRYS B， STRANDT S ．Data-driven soft sensors in the process industry［J］．Computers & Chemical Engineering，2009，33（4）：795-814．
4	KAY S，KAY H， MOWBRAY M，et al ．Integrating autoencoder and heteroscedastic noise neural networks for the batch process soft-sensor design［J］．Industrial & Engineering Chemistry Research，2022，61（36）：13559-13569．
5	李东，黄道平，许翀，等．基于协同训练的集成自适应GPR-RVM多输出模型研究［J］．华南理工大学学报（自然科学版），2021，49（6）：100-108．
	LI Dong， HUANG Daoping， XU Chong，et al ．On integrated adaptive GPR-RVM multi-output model based on co-training algorithm［J］．Journal of South China University of Technology （Natural Science Edition），2021，49（6）：100-108．
6	YUAN X， LI L， WANG Y，et al ．Deep learning for quality prediction of nonlinear dynamic processes with variable attention‐based long short‐term memory network［J］．The Canadian Journal of Chemical Engineering，2020，98（6）：1377-1389．
7	GENCAY R， LIU T ．Nonlinear modelling and prediction with feedforward and recurrent networks［J］．Physica D：Nonlinear Phenomena，1997，108（1/2）：119-134．
8	杨马英，周芳芹，李军．基于Elman神经网络的城市污水处理水质参数软测量［J］．东南大学学报（自然科学版），2006，36（S1）：119-123．
	YANG Maying， ZHOU Fangqin， LI Jun ．Soft sensing model for city wastewater treatment process parameter based on Elman neural network［J］．Journal of Southeast University （Natural Science Edition），2006，36（S1）：119-123．
9	关学忠，宋韬略，徐延海，等．污水处理中BP神经网络与Elman神经网络的预测比较［J］．自动化技术与应用，2014，33（10）：1-3，25．
	GUAN Xuezhong， SONG Taolue， XU Yanhai，et al ．Comparison of prediction on BP neural networks and Elman neural networks in wastewater treatment［J］．Techniques of Automation and Applications，2014，33（10）：1-3，25．
10	WERBOS P J ．Backpropagation through time：what it does and how to do it［J］．Proceedings of the IEEE，1990，78（10）：1550-1560．
11	HAGAN M T， MENHAJ M B ．Training feedforward networks with the Marquardt algorithm［J］．IEEE Transactions on Neural Networks，1994，5（6）：989-993．
12	HOWARD B D， MARK H B， ORLANDO De J，et al ．Neural network design［M］．Stillwater：Oklahoma State University，2014：335-468．
13	ZHANG F， REYNOLDS A C， OLIVER D S ．An initial guess for the Levenberg-Marquardt algorithm for conditioning a stochastic channel to pressure data［J］．Mathematical Geology，2003，35（1）：67-88．
14	ZHU J， JIANG Q， SHEN Y，et al ．Application of recurrent neural network to mechanical fault diagnosis：a review［J］．Journal of Mechanical Science and Technology，2022，36：527-542．
15	WELCH G， BISHOP G ．An introduction to the kalman filter［R］．Raleigh：University of North Carolina，2006．
16	NOVI T， CAPITANI R， ANNICCHIARICO C ．An integrated artificial neural network-unscented Kalman filter vehicle sideslip angle estimation based on inertial measurement unit measurements［J］．Proceedings of the Institution of Mechanical Engineers，Part D：Journal of Automobile Engineering，2019，233（7）：1864-1878．
17	HAO Y， QIU X ．Performance analysis of wireless location and velocity tracking of digital broadcast signals based on extended Kalman filter algorithm［J］．Complexity，2021（2021）：1-10．
18	李小兵，田玉松，邱天爽．基于Kalman滤波的基本Elman网络训练新算法［J］．大连理工大学学报，2009，49（2）：276-281．
	LI Xiao-bing， TIAN Yu-song， QIU Tian-shuang ．A new training algorithm based on Kalman filter of essential Elman network［J］．Journal of Dalian University of Technology，2009，49（2）：276-281．
19	WANG J， ZHANG J， WANG Y，et al ．Nonlinear identification of one-stage spur gearbox based on pseudo-linear neural network［J］．Neurocomputing，2018，308：75-86．
20	CHEN C， DOU Y， CHEN J，et al ．A novel neural network training framework with data assimilation［J］．Journal of Supercomputing，2022，78（17）：19020-19045．
21	ROTH M， HENDEBY G， FRITSCHE C，et al ．The ensemble Kalman filter：a signal processing perspective［J］．EURASIP Journal on Advances in Signal Proce-ssing，2017，2017（1）：1-16．
22	EVENSEN G ．Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics［J］．Journal of Geophysical Research，1994，99：10143-10162．
23	LEGLER S， JANJIĆ T ．Combining data assimilation and machine learning to estimate parameters of a convective-scale model［J］．Quarterly Journal of the Royal Meteorological Society，2022，148（743）：860-874．
24	MIRIKITANI D T， NIKOLAEV N ．Dynamic modeling with ensemble Kalman filter trained recurrent neural networks［C］∥Proceedings of 2008 Seventh International Conference on Machine Learning and Applications．San Diego：IEEE，2008：843-848．
25	CHEN C， LIN X， HUANG Y，et al ．Approximate bayesian neural network trained with ensemble Kalman filter［C］∥Proceedings of 2019 International Joint Conference on Neural Networks （IJCNN）．Budapest：IEEE，2019：1-8．
26	ELMAN J L ．Finding structure in time［J］．Cognitive Science，1990，14（2）：179-211．
27	EVENSEN G ．The ensemble Kalman filter：theoretical formulation and practical implementation［J］．Ocean Dynamics，2003，53（4）：343-367．
28	MANDEL J ．Efficient implementation of the ensemble Kalman filter［R］．Denver：University of Colorado at Denver，2006．
29	ZUPANSKI M ．Maximum likelihood ensemble filter：theoretical aspects［J］．Monthly Weather Review，2005，133（6）：1710-1726．
30	BOCQUET M ．Ensemble Kalman filtering without the intrinsic need for inflation［J］．Nonlinear Processes in Geophysics，2011，18（5）：735-750．
31	BOCQUET M， SAKOV P ．Combining inflation-free and iterative ensemble Kalman filters for strongly nonlinear systems［J］．Nonlinear Processes in Geophysics，2012，19（3）：383-399．
32	BOCQUET M， RAANES P N， HANNART A ．Expanding the validity of the ensemble Kalman filter without the intrinsic need for inflation［J］．Nonlinear Processes in Geophysics，2015，22（6）：645-662．
33	LIU Y， HUANG D， LI Y ．Development of interval soft sensors using enhanced just-in-time learning and inductive confidence predictor［J］．Industrial & Engineering Chemistry Research，2012，51（8）：3356-3367．
34	LIU Y， CHEN J， SUN Z，et al ．A probabilistic self-validating soft-sensor with application to wastewater treatment［J］．Computers & Chemical Engineering，2014，71：263-280．
35	LIU Y， YUAN L， LI D，et al ．Process monitoring of quality-related variables in wastewater treatment using Kalman-Elman neural network-based soft-sensor modeling［J］．Water，2021，13（24）：1-21．

模型	预测变量	E_RMSE	R	E_MAPE/%	E_RMSSD	R_MR
BPTT-Elman	ρ_SS，S	4.957	0.622	24.6	17.271	0.688
	BOD₅	3.591	0.709	17.2
	COD_S	16.125	0.732	19.0
GDM-Elman	ρ_SS，S	4.598	0.705	22.4	15.569	0.781
	BOD₅	3.308	0.799	15.4
	COD_S	14.479	0.834	16.6
LM-Elman	ρ_SS，S	25.961	0.471	127.3	31.356	0.765
	BOD₅	3.358	0.906	14.5
	COD_S	13.531	0.917	13.5
EnKF-Elman-500^2）	ρ_SS，S	3.379	0.848	14.43	9.499	0.911
	BOD₅	1.897	0.941	8.53
	COD_S	8.667	0.943	8.77
D-EnKF-N-Elman-500	ρ_SS，S	3.349	0.850	14.22	9.453	0.913
	BOD₅	1.858	0.943	8.51
	COD_S	8.638	0.944	8.69

[1]	LI Dong, HUANG Daoping, XU Chong, et al. On Integrated Adaptive GPR-RVM Multi-Output Model Based on Co-Training Algorithm [J]. Journal of South China University of Technology (Natural Science Edition), 2021, 49(6): 100-108.
[2]	CHENG Hongchao, WU Jing, LIU Yiqi, et al. Forecast Components-RVM Fault Detection Modeling for Wastewater Treatment [J]. Journal of South China University of Technology (Natural Science Edition), 2020, 48(3): 10-17.
[3]	XU Yuge LAI Chunling LUO Fei . Bagging Ensemble Fault Diagnosis Modeling with Imbalanced classification in Wastewater Treatment Plant [J]. Journal of South China University of Technology (Natural Science Edition), 2018, 46(8): 107-115.
[4]	XU Yu-ge LIU Li LUO Fei. A Self-Optimal Relevance Vector Machine with Multiple-Attribute Gaussian Kernel Functions [J]. Journal of South China University of Technology (Natural Science Edition), 2017, 45(1): 88-94.
[5]	PU Yue-wu XU Xiao-ma WANG Si-yao WU Jie-jie WEI Cheng. Investigation into Biofilm Colonization on a New-Type Biofilm Reactor for Domestic Wastewater Treatment in Traditional Villages [J]. Journal of South China University of Technology (Natural Science Edition), 2016, 44(3): 142-148.
[6]	XIAO Hong-jun LIU Yi-qi HUANG Dao-ping. Application of Gaussian Process Modeling Method in Industrial Processes [J]. Journal of South China University of Technology (Natural Science Edition), 2016, 44(12): 36-43,52.
[7]	Xu Yu- ge Cao Tao Luo Fei. Wastewater Effluent Quality Prediction Model Based on Relevance Vector Machine [J]. Journal of South China University of Technology (Natural Science Edition), 2014, 42(5): 103-108.
[8]	Lin Mei- jin Luo Fei Xu Yu- ge. Optimization Control of Biochemical Process of Wastewater Based on SFLA and PSO [J]. Journal of South China University of Technology (Natural Science Edition), 2013, 41(9): 51-57.
[9]	Xu Yu- ge Song Ya- ling Luo Fei Zhang Yong- tao Cao Tao. Energy- Saving Optimization of Wastewater Treatment System Based on Artificial Immune Algorithm [J]. Journal of South China University of Technology (Natural Science Edition), 2013, 41(8): 34-40.
[10]	Mo Li- huan Zhou Zhi- ming Wang Yu- feng Lu Li- cheng. Photocatalysis- Fenton Degradation of Papermaking- Reconstituted Tobacco Slice Wastewater Using Fe- Bent [J]. Journal of South China University of Technology (Natural Science Edition), 2013, 41(12): 43-48.
[11]	Shen Wen-hao Ning Li Chen Xiao-quan Li Cui-cui. Flocculation-Photocatalysis Treatment of Secondary-Fiber Papermaking Wastewater [J]. Journal of South China University of Technology (Natural Science Edition), 2011, 39(6): 90-94.
[12]	Liu Yi-qi Huang Dao-ping Li Yan. Enhanced JIT-Based Soft-Sensing Modeling and Its Application to Wastewater Treatment [J]. Journal of South China University of Technology (Natural Science Edition), 2011, 39(5): 55-60,67.
[13]	Xiao Kai-jun Wang Xin Yin Yu-rong . Degradation of Nitrobenzene Wastewater by Three-Dimension Electrode Method Coupling Electro-Fenton [J]. Journal of South China University of Technology (Natural Science Edition), 2010, 38(8): 131-136.
[14]	Ye Hong-tao Luo Fei Xu Yu-ge Yang Hong Liang Jin. Optimal Control of Wastewater Treatment Process Based on Novel Immune Algorithm [J]. Journal of South China University of Technology (Natural Science Edition), 2010, 38(2): 137-141.
[15]	Yu Wei Luo Fei Yang Hong Xu Yu-ge. Multiple Neural Network-Based Model to Predict Ammonia Nitrogen Content in Wastewater [J]. Journal of South China University of Technology (Natural Science Edition), 2010, 38(12): 79-83.