Human-robot interaction in real environments by audio-visual integration

Abstract

In this paper, we developed not only a reliable sound localization system including a VAD (Voice Activity Detection) component using three microphones but also a face tracking system using a vision camera. Moreover, we proposed a way to integrate three systems in the human-robot interaction to compensate errors in the localization of a speaker and to reject unnecessary speech or noise signals entering from undesired directions effectively. For the put-pose of verifying Our system's performances, we installed the proposed audio-visual system in a prototype robot, called IROBAA (Intelligent ROBot for Active Audition), and demonstrated how to integrate the audio-visual system.