Skip to main content

Audiovisual Head Orientation Estimation with Particle Filtering in Multisensor Scenarios

Abstract

This article presents a multimodal approach to head pose estimation of individuals in environments equipped with multiple cameras and microphones, such as SmartRooms or automatic video conferencing. Determining the individuals head orientation is the basis for many forms of more sophisticated interactions between humans and technical devices and can also be used for automatic sensor selection (camera, microphone) in communications or video surveillance systems. The use of particle filters as a unified framework for the estimation of the head orientation for both monomodal and multimodal cases is proposed. In video, we estimate head orientation from color information by exploiting spatial redundancy among cameras. Audio information is processed to estimate the direction of the voice produced by a speaker making use of the directivity characteristics of the head radiation pattern. Furthermore, two different particle filter multimodal information fusion schemes for combining the audio and video streams are analyzed in terms of accuracy and robustness. In the first one, fusion is performed at a decision level by combining each monomodal head pose estimation, while the second one uses a joint estimation system combining information at data level. Experimental results conducted over the CLEAR 2006 evaluation database are reported and the comparison of the proposed multimodal head pose estimation algorithms with the reference monomodal approaches proves the effectiveness of the proposed approach.

Publisher note

To access the full article, please see PDF.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Cristian Canton-Ferrer.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Canton-Ferrer, C., Segura, C., Casas, J.R. et al. Audiovisual Head Orientation Estimation with Particle Filtering in Multisensor Scenarios. EURASIP J. Adv. Signal Process. 2008, 276846 (2007). https://doi.org/10.1155/2008/276846

Download citation

Keywords

  • Particle Filter
  • Video Surveillance
  • Video Conferencing
  • Head Orientation
  • Orientation Estimation