Emotion classification of images is a challenging task regarding the complexity of various images and the subjectivity of human’s emotion perception． Most existing deep learning methods didn’t consider image prior in- formation fully． A new multi-level deep convolutional neural network was proposed to predict the emotion based on the multi-level prior information from global and local view． Extensive experiments on both the large scale and small scale emotion datasets demonstrate the effectiveness of our method． The average classification accuracy of our meth- od is 2. 8% higher than the state-of-art method，especially 15% higher in the category“disgust”．