Aimimg at the catastrophic forgetting problem caused by the image classification of deep learning systems in an incremental scene. A incremental learning method which based on neuron regularization and resource releasing mechanism was proposed. This method is based on the framework of Bayesian neural network. Firstly, grouping the input weights by neurons and restrict the standard deviation of weights to the same value according to the groups. Then in the training process, the corresponding strength regularization was performed for the weights of each group according to the standard deviation after unification. Finally, in order to improve model's continuous learning ability, a resource releasing mechanism was proposed. This mechanism maintains model's learning ability by guiding the model to selectively dilute the regularization strengths of some weights. Experiments on several common datasets show that the proposed method can explore the continuous learning capability of the model more effectively, and a better model can be learned even in a fixed capacity environment.