In order to effectively classify vehicles, a new classification method based on image analysis is proposed. In this method, the road background is set up according to video sequence images, and the moment invariant fea- tures of the vehicle region segmented via the background division are calculated. Then, in order to speed up the ex- traction, a Canny operator is used to detect the edge of the vehicle region and to extract the vehicle outline, and the moment invariant is directly obtained as the characteristic quantity of vehicle classification. Afterwards, a three-layer BP neural network, with the moment invariant as the input, is established, and the vehicle is then classified according to the network output. Some experiments are finally carried out to verify the effectiveness of the proposed method.