Abstract:

In this paper,by taking the visual-only lipreading system as the research objective,a method to extract the visual lip feature based on the lip gray energy image ( LGEI) is proposed. In this method,the image sequences of a word are projected to the 2D lip gray energy image to unify the dimension of input data and maintain most motion information of image sequences. In order to eliminate the dependence of the template matching method on the
template,the LGEI of the single-training sample is extended to the multi-training sample. Moreover,a lip location method based on the lip center is also proposed. Experimental results show that,as compared with the conventional methods that extract features for each image of the sequence,the proposed method greatly improves the recognition rate and significantly decreases the computation time in the same dimension of features for a single image,that the recognition rate of double-training samples averagely improves by 11.29%,as compared with that of single-training samples,and that,after an accurate lip location,the recognition rate improves by more than 2%,with its maximum being up to 90.63%.

Key words: lipreading, lip gray energy image, gait energy image, feature extraction