Electronics, Communication & Automation Technology

Lip Motion and Voice Consistency Recognition based on Specific Vowel Pronunciation Events Analysis

Expand
  • 1. School of Electronic and Information Engineering,South China University of Technology,Guangzhou 510640,Guangdong,China; 2. School of Electronics and Information,Guangdong Polytechnic Normal University,Guangzhou 510665,Guangdong,China
朱铮宇(1984-),男,博士后,讲师,主要从事音视频多模态信号处理研究。E-mail: zhuzhengyu0701@163. com

Received date: 2019-05-16

  Revised date: 2019-07-03

  Online published: 2019-12-01

Supported by

Supported by the National Natural Science Foundation of China (61672173)

Abstract

The traditional lip motion and voice consistency recognition method is to analyze the whole sentence without filtering the content,which is complicate in computation and its results are vulnerable to weak related segments such as mute. The vowels which with significant lip shape changes were researched in depth. By analyzing the audio and lip motion correlation of each vowel category clustered by lip sequence features,a more representative specific phonological pronunciation unit was selected as the analysis object. Combined with audio-visual delay analysis,a consistent recognition method based on specific vowel pronunciation events analysis was proposed.Firstly,the selected unit was segmented and identified. Then the correlation degree of each specific vowel event was obtained,and the delay distribution of each specific vowel occurrence position was statistically scored. Finally,a consistency judgment was made by combining the vowel pronunciation event audio-visual correlation score with the position delay analysis score. Compared with other methods through experiments,results show that the proposed method is superior in recognition performance and reduces the amount of computation.

Cite this article

ZHU Zhengyu, QIU Huayu, YANG Chunling, et al . Lip Motion and Voice Consistency Recognition based on Specific Vowel Pronunciation Events Analysis[J]. Journal of South China University of Technology(Natural Science), 2020 , 48(1) : 139 -146 . DOI: 10.12141/j.issn.1000-565X.190287

Outlines

/