Particle filter algorithm for single speaker tracking with audio-video data fusion

Abstract

In this paper, a single speaker tracking mechanism with particle filter is presented for a reliable interaction with robot. To have a smooth tracking result, we use not only audio data but also video data. For sound observation, we use Time Delay Of Arrival (TDOA) between two microphones, and in order to expect high reliability in the likelihood of localization, we apply variable sensitivity values according to the angle of arrival for a triangle arrangement of microphones to calculate the likelihood for audio observation. For video observation, we use face detection routine of OpenCV which is widely available through the internet. Together with audio and video data, we found out that more reliable source tracking is possible if we apply data fusion mechanism at every possible environment on a robot platform than single information. ？2007 IEEE.