Implementation of Real-Time 3D Video Avatar for Immersive Telecommunication

Abstract

Immersive telecommunication is a new challenging field that enables a user to share a virtual space with remote participants providing users with a sense of telepresence and interaction. This paper presents an implementation of realtime dynamic 3D avatar from multi-view cameras for the immersive telecommunication. The main objective is to offer rich communication modalities, as similar as those used in the face-to-face meetings like gestures, gaze awareness, realistic images, and correct sound direction. Active segmentation algorithm is suggested to generate realistic 3D video avatars in real-time and also to augment the avatars naturally into 3D collaborative virtual environment. The proposed algorithm has been integrated to the framework of distributed VR, called NAVER, and an example of application, Heritage Alive!, has been implemented to demonstrate the concept. As a result, the user can be immersed and has natural interaction with remote participant. This would overcome the limitations both of the conventional video-based telecommunication and also the VR-based collaborative virtual environment approaches.