Design and Implementation of Hardware Structure for Online Learning of Spiking Neural Networks Based on FPGA Parallel Acceleration

doi:10.12141/j.issn.1000-565X.220623

Abstract

Abstract:

Currently, the hardware design of spiking neural networks based on digital circuits has a low synaptic parallel nature in terms of learning function, leading to a large overall hardware delay, which limits the speed of online learning of spiking neural network models to some extent. To address the above problems, this paper proposed an efficient spiking neural network online learning hardware architecture based on FPGA parallel acceleration, which accelerates the training and inference process of the model through the dual parallel design of neurons and synapses. Firstly, a synaptic structure with parallel spike delivery function and parallel spike time-dependent plasticity learning function was designed; then, the learning layers of input encoding layer and winner-take-all structure were built, and the implementation of lateral inhibition of the winner-take-all network was optimized, forming an impulsive neural network model with a scale of 784~400. The experiments show, the hardware has a training speed of 1.61 ms/image and an energy consumption of about 3.18 mJ/image for the SNN model and an inference speed of 1.19 ms/image and an energy consumption of about 2.37 mJ/image on the MNIST dataset, with an accuracy rate of 87.51%. Based on the hardware framework designed in this paper, the synaptic parallel structure can improve the training speed by more than 38%, and reduce the hardware energy consumption by about 24.1%, which can help to promote the development of edge intelligent computing devices and technologies.

Key words: neural network, learning algorithm, acceleration, parallel architecture

CLC Number:

TP389.1

LIU Yijun, CAO Yu, YE Wujian, et al. Design and Implementation of Hardware Structure for Online Learning of Spiking Neural Networks Based on FPGA Parallel Acceleration[J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(5): 104-113.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Table 1

Table 2

Table 3

References 16

1	STIMBERG M， BRETTE R， GOODMAN D F M ．Brian 2，an intuitive and efficient neural simulator［J］．Elife，2019，8：e47314/1-10.
2	CHOU T S， KASHYAP H J， XING J，et al ．CARLsim 4：an open source library for large scale，biologically detailed spiking neural network simulation using heterogeneous clusters［C］∥ Proceedings of 2018 International Joint Conference on Neural Networks．Rio de Janeiro：IEEE，2018：1158-1165.
3	LI S， ZHANG Z， MAO R，et al ．A fast and energy-efficient SNN processor with adaptive clock/event-driven computation scheme and online learning［J］．IEEE Transactions on Circuits and Systems I：Regular Papers，2021，68（4）：1543-1552.
4	SOLEIMANI H， AHMADI A， BAVANDPOUR M ．Biologically inspired spiking neurons：piecewise linear models and digital implementation［J］．IEEE Transactions on Circuits and Systems I：Regular Papers，2012，59（12）：2991-3004.
5	HEIDARPUR M， AHMADI A， AHMADI M，et al ．CORDIC-SNN：on-FPGA STDP learning with Izhikevich neurons［J］．IEEE Transactions on Circuits and Systems I：Regular Papers，2019，66（7）：2651-2661.
6	JOKAR E， SOLEIMANI H ．Digital multiplierless realization of a calcium-based plasticity model［J］．IEEE Transactions on Circuits and Systems Ⅱ：Express Briefs，2017，64（7）：832-836.
7	QUINTANA F M， PEREZ-PEÑA F， GALINDO P L ．Bio-plausible digital implementation of a reward modulated STDP synapse［J］．Neural Computing and Applications，2022，34：15649-15660.
8	WU J， ZHAN Y， PENG Z，et al ．Efficient design of spiking neural network with STDP learning based on fast CORDIC［J］．IEEE Transactions on Circuits and Systems I：Regular Papers，2021，68（6）：2522-2534.
9	WAN L， LUO Y， SONG S，et al ．Efficient neuron architecture for FPGA-based spiking neural networks［C］∥ Proceedings of 2016 the 27th Irish Signals and Systems Conference．Londonderry：IEEE，2016：1-6.
10	LIU Y， CHEN Y， YE W，et al ．FPGA-NHAP：a general FPGA-based neuromorphic hardware acceleration platform with high speed and low power［J］．IEEE Transactions on Circuits and Systems I：Regular Papers，2022，69（6）：2553-2566.
11	WANG Q， LI Y， SHAO B，et al ．Energy efficient parallel neuromorphic architectures with approximate arithmetic on FPGA ［J］．Neurocomputing，2017，221：146-158.
12	HE Z， SHI C， WANG T，et al ．A low-cost FPGA implementation of spiking extreme learning machine with on-chip reward-modulated STDP learning［J］．IEEE Transactions on Circuits and Systems Ⅱ：Express Briefs，2022，69（3）：1657-1661.
13	DIEHL P U， COOK M ．Unsupervised learning of digit recognition using spike-timing-dependent plasticity［J］．Frontiers in Computational Neuroscience，2015，9：99/1-9.
14	VIGNERON A， MARTINET J ．A critical survey of STDP in spiking neural networks for pattern recognition［C］∥ Proceedings of 2020 International Joint Conference on Neural Networks．Glasgow：IEEE，2020：1-9.
15	GUO W， YANTIR H E， FOUDA M E，et al ．Toward the optimal design and FPGA implementation of spiking neural networks［J］．IEEE Transactions on Neural Networks and Learning Systems，2021，33（8）：3988-4002.
16	MORRISON A， AERTSEN A， DIESMANN M ．Spike-timing-dependent plasticity in balanced random networks［J］．Neural Computation，2007，19（6）：1437-1467.

并行情况	使用数量				使用率/%				片上总功耗/W
并行情况	LUT	LUTRAM	FF	BRAM	LUT	LUTRAM	FF	BRAM	片上总功耗/W
1×8	21 335	915	26 323	266.0	10.47	1.43	6.46	59.78	1.322
2×8	26 436	946	31 685	282.0	12.97	1.48	7.77	63.37	1.636
4×8	36 536	1 018	40 690	314.0	17.93	1.59	9.98	70.56	1.986
8×8	56 842	1 279	62 547	379.5	27.89	2.00	15.35	85.28	2.906

任务	并行情况	样本数/10⁴	训练用时/s	输入脉冲数量	脉冲事件处理速度	单幅图像处理时间/ms	单幅图像处理能耗/mJ	准确率/%
在线学习	1×8	6	179.623	152 777 357	850 544.5	2.994	3.958	86.80
	2×8	6	110.189	150 372 047	1 364 673.8	1.836	3.004	87.41
	4×8	6	96.291	150 255 976	1 560 436.3	1.605	3.181	86.92
	8×8	6	90.802	146 492 996	1 613 323.4	1.513	4.398	87.51
推理	1×8	1	19.596	26 248 957	1 339 505.8	1.960	2.591	86.80
	2×8	1	12.914	20 794 115	1 610 199.3	1.291	2.113	87.41
	4×8	1	11.941	25 913 646	2 170 140.3	1.194	2.367	86.92
	8×8	1	11.714	25 262 790	2 156 632.2	1.171	3.404	87.51

硬件结构	系统时钟/MHz	数据格式	学习算法	FPGA 设备	神经元模型	神经元总数	突触数量	单幅图像处理时间/ms		单幅图像处理能耗/mJ		准确率/%
硬件结构	系统时钟/MHz	数据格式	学习算法	FPGA 设备	神经元模型	神经元总数	突触数量	在线学习	推理	在线学习	推理	准确率/%
文献［11］	120	8位固定	STDP	Virtex 6	LIF	1 591	638 208	16 800.00	8 400.00	1 330.00	1 120.00	89.10
文献［3］	100	16位浮点	STDP	Virtex 7	LIF	984	88 400	16.30	3.15	26.32	5.04	85.28
文献［15］	100	8位固定	STDP	Virtex 7	LIF	1 184	313 600	34.00	28.00	2.43	1.73	89.70
文中设计	200	16位固定	STDP	Kintex 7	LIF	1 184	313 600	1.61	1.19	3.18	2.37	87.51

[1]	. Dynamic Neighborhood Sampling for Social Recommendation [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(2): 32-41.
[2]	FANG Gang, YUAN Longhua, WANG Xiaoming, et al. Soft-Sensor Modeling Method Based on Ensemble Kalman Filter-Elman Neural Network [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(8): 126-136.
[3]	MA Xiaoliang, AN Lingling, DENG Congjian, et al. Translation Optimization Technology of Automatic Speech Recognition Based on Industry-Specific Vocabulary [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(8): 118-125.
[4]	ZHU Zhengyu, LUO Chao, HE Qianhua, et al. Multi-View Lip Motion and Voice Consistency Judgment Based on Lip Reconstruction and Three-Dimensional Coupled CNN [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(5): 70-77.
[5]	YE Feng, CHEN Biao, LAI Yizong. Contrastive Knowledge Distillation Method Based on Feature Space Embedding [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(5): 13-23.
[6]	LENG Sheng, FU Youwei, MA Wantai, et al. Study on the Morphology Control Technology of Spray Forming Ingot Billets Based on GA-BP Neural Network [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(2): 27-34.
[7]	ZHAI Jingmei, LU Dongwei. Research on Optimized Teaching Strategy and BPNN-DMPs Trajectory Learning Model of Massage Robot [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(12): 1-8.
[8]	ZHAO Rongchao, WU Baili, CHEN Zhuyun, WEN Kairu, ZHANG Shaohui, LI Weihua. Graph Neural Network for Fault Diagnosis with Multi-Scale Time-Spatial Information Fusion Mechanism [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(12): 42-52.
[9]	LUO Yutao, GAO Qiang. Traffic Sign Detection Based on Channel Attention and Feature Enhancement [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(12): 64-72.
[10]	YANG Xufeng, LIU Zeqing, ZHANG Yi. Estimation of P-S-N Curve of Metal Materials Based on Bayesian Neural Network [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(11): 82-92.
[11]	XU Lunhui, YU Jiaxin, PEI Mingyang, et al. Repositioning Strategy for Ride-Hailing Vehicles Based on Geometric Road Network Structure and Reinforcement Learning [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(10): 99-109.
[12]	LAN Fengchong, ZHANG Yue, CHEN Jiqing, et al. Correlation Analysis and Prediction of Pedestrian Casualty Risk in Car-Pedestrian Collision Accident [J]. Journal of South China University of Technology(Natural Science Edition), 2022, 50(5): 1-10.
[13]	YANG Chunling YANG Yajing. A Deep Neural Network Based on Layer-by-Layer Fusion of Multi-Scale Features for No-Reference Image Quality Assessment [J]. Journal of South China University of Technology(Natural Science Edition), 2022, 50(4): 81-89,141.
[14]	WANG Weifa, ZHANG Daming, DAI Yi, et al. Research on Survivability Technology for Software Defined Network Based on Q-Learning Algorithm [J]. Journal of South China University of Technology(Natural Science Edition), 2022, 50(4): 65-72.
[15]	FU Xinsha, ZENG Yanjie, MA Li, et al. Weather Recognition of Highway Surveillance Scenes Based on Light-Weight Deep Neural Network [J]. Journal of South China University of Technology(Natural Science Edition), 2022, 50(3): 1-8.