Open Access

Audiovisual Head Orientation Estimation with Particle Filtering in Multisensor Scenarios

  • Cristian Canton-Ferrer1Email author,
  • Carlos Segura2,
  • Josep R. Casas1,
  • Montse Pardàs1 and
  • Javier Hernando2
EURASIP Journal on Advances in Signal Processing20072008:276846

Received: 1 February 2007

Accepted: 7 June 2007

Published: 12 June 2007


This article presents a multimodal approach to head pose estimation of individuals in environments equipped with multiple cameras and microphones, such as SmartRooms or automatic video conferencing. Determining the individuals head orientation is the basis for many forms of more sophisticated interactions between humans and technical devices and can also be used for automatic sensor selection (camera, microphone) in communications or video surveillance systems. The use of particle filters as a unified framework for the estimation of the head orientation for both monomodal and multimodal cases is proposed. In video, we estimate head orientation from color information by exploiting spatial redundancy among cameras. Audio information is processed to estimate the direction of the voice produced by a speaker making use of the directivity characteristics of the head radiation pattern. Furthermore, two different particle filter multimodal information fusion schemes for combining the audio and video streams are analyzed in terms of accuracy and robustness. In the first one, fusion is performed at a decision level by combining each monomodal head pose estimation, while the second one uses a joint estimation system combining information at data level. Experimental results conducted over the CLEAR 2006 evaluation database are reported and the comparison of the proposed multimodal head pose estimation algorithms with the reference monomodal approaches proves the effectiveness of the proposed approach.

Publisher note

To access the full article, please see PDF.

Authors’ Affiliations

Image Processing Group, Universitat Politècnica de Catalunya
TALP Research center, Universitat Politècnica de Catalunya


© Cristian Canton-Ferrer et al. 2008

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.