关键词: 深度学习, 灾难性遗忘, 增量学习, 神经元正则, 资源释放机制, 容量有限环境

Abstract: Aimimg at the catastrophic forgetting problem caused by the image classification of deep learning systems in an incremental scene. A incremental learning method which based on neuron regularization and resource releasing mechanism was proposed. This method is based on the framework of Bayesian neural network. Firstly, grouping the input weights by neurons and restrict the standard deviation of weights to the same value according to the groups. Then in the training process, the corresponding strength regularization was performed for the weights of each group according to the standard deviation after unification. Finally, in order to improve model's continuous learning ability, a resource releasing mechanism was proposed. This mechanism maintains model's learning ability by guiding the model to selectively dilute the regularization strengths of some weights. Experiments on several common datasets show that the proposed method can explore the continuous learning capability of the model more effectively, and a better model can be learned even in a fixed capacity environment.

Key words: compressed sensing, deep learning, multi-hypothesis prediction, adaptive hypothesis weight,
multi-frame reference reconstruction