关键词: 音唇一致性判决方法, 韵母发音事件, 音唇相关度, 韵母分割

Abstract: The traditional lip motion and voice consistency recognition method is to analyze the whole sentence without filtering the content，which is complicate in computation and its results are vulnerable to weak related segments such as mute． The vowels which with significant lip shape changes were researched in depth． By analyzing the audio and lip motion correlation of each vowel category clustered by lip sequence features，a more representative specific phonological pronunciation unit was selected as the analysis object． Combined with audio-visual delay analysis，a consistent recognition method based on specific vowel pronunciation events analysis was proposed．Firstly，the selected unit was segmented and identified． Then the correlation degree of each specific vowel event was obtained，and the delay distribution of each specific vowel occurrence position was statistically scored． Finally，a consistency judgment was made by combining the vowel pronunciation event audio-visual correlation score with the position delay analysis score． Compared with other methods through experiments，results show that the proposed method is superior in recognition performance and reduces the amount of computation．

Key words: lip motion and voice consistency recognition method, vowel pronunciation events, correlation of lip motion and voice consistency, vowel segmentation