3D facial feature extraction and global motion recovery using multi-modal information

Abstract

Robust extraction of 3D facial features and global motion information from 2D image sequence for the MPEG-4 SNHC face model encoding is described. The facial regions are detected from image sequence using multi-modal fusion technique that combines range, color and motion information. 23 facial features among the MPEC-4 FDP(Face Definition Parameters) are extracted automatically inside the facial region using morphological processing. The extracted facial features are used to recover the 3D shape and global motion of the object using paraperspective factorization method. Stereo view and averaging technique are used to reduce the depth estimation error caused by the inherent paraperspective camera model. The recovered 3D motion information is transformed into global motion parameters of FAP(Face Animation Parameters) of the MPEG-4 to synchronize a generic face model with a real face.