Skip to content

Advertisement

Open Access

An Iterative Decoding Algorithm for Fusion of Multimodal Information

  • Shankar T. Shivappa1Email author,
  • Bhaskar D. Rao1 and
  • Mohan M. Trivedi1
EURASIP Journal on Advances in Signal Processing20072008:478396

https://doi.org/10.1155/2008/478396

Received: 16 February 2007

Accepted: 26 October 2007

Published: 13 November 2007

Abstract

Human activity analysis in an intelligent space is typically based on multimodal informational cues. Use of multiple modalities gives us a lot of advantages. But information fusion from different sources is a problem that has to be addressed. In this paper, we propose an iterative algorithm to fuse information from multimodal sources. We draw inspiration from the theory of turbo codes. We draw an analogy between the redundant parity bits of the constituent codes of a turbo code and the information from different sensors in a multimodal system. A hidden Markov model is used to model the sequence of observations of individual modalities. The decoded state likelihoods from one modality are used as additional information in decoding the states of the other modalities. This procedure is repeated until a certain convergence criterion is met. The resulting iterative algorithm is shown to have lower error rates than the individual models alone. The algorithm is then applied to a real-world problem of speech segmentation using audio and visual cues.

Keywords

Markov ModelHide Markov ModelIterative AlgorithmTurbo CodeFuse Information

Publisher note

To access the full article, please see PDF.

Authors’ Affiliations

(1)
Department of Electrical and Computer Engineering, University of California, La Jolla, USA

Copyright

© Shankar T. Shivappa et al. 2008

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement