A Study on the Communication System using 3D Lip Information

A Study on the Communication System using 3D Lip Information
lip reading; 3D reconstruction; communication system
Issue Date
The 7th Asian Pacific Conference on Biomechanics
Communication is an essential factor for human social interaction. However, severely disabled people have significant difficulty exchanging their thoughts, emotions or desires with others owing to congenital or acquired handicaps. To improve their quality of life, various studies on the Human Machine Interface (HMI), which can provide useful communication tools, have been actively conducted. Among the HMI, the Auditory Speech Recognition (ASR) system is widely accepted as the effective communication tool. However, ASR is vulnerable to noisy environments. To overcome this limitation, we focused on a communication approach that uses lip information (lip reading). In the lip reading, the lip feature extraction method is the most important step to get the lip movement information. Recent studies have reported successful results for extracting lip feature using image or model based methods. However, these techniques based on two-dimensional (2D) images are unable to extract lip movement features in a robust manner since they are affected strongly by environmental changes such as facial pose and illumination variations [1]. Therefore, this paper presents a communication system using three-dimensional (3D) lip information to improve the performance in natural environments. The proposed method comprises five steps. First, the lip region is distinguished from the facial image using color information. Second, four feature points on the outer lip contour are extracted using a curve fitting algorithm. Third, the feature points-pair extracted from the two stereo images are used to reconstruct a 3D lip shape. Fourth, lip movement features are extracted from the 3D lip shape. Finally, the extracted lip movement features are classified by a Support Vector Machine (SVM) classifier to obtain lip reading results. The proposed system is expected to become a useful tool for interpreting the user’s intended meaning witho
Appears in Collections:
KIST Publication > Conference Paper
Files in This Item:
There are no files associated with this item.
RIS (EndNote)
XLS (Excel)


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.