Skip to main content

Robust Recognition and Segmentation of Human Actions Using HMMs with Missing Observations

Abstract

This paper describes the integration of missing observation data with hidden Markov models to create a framework that is able to segment and classify individual actions from a stream of human motion using an incomplete 3D human pose estimation. Based on this framework, a model is trained to automatically segment and classify an activity sequence into its constituent subactions during inferencing. This is achieved by introducing action labels into the observation vector and setting these labels as missing data during inferencing, thus forcing the system to infer the probability of each action label. Additionally, missing data provides recognition-level support for occlusions and imperfect silhouette segmentation, permitting the use of a fast (real-time) pose estimation that delegates the burden of handling undetected limbs onto the action recognition system. Findings show that the use of missing data to segment activities is an accurate and elegant approach. Furthermore, action recognition can be accurate even when almost half of the pose feature data is missing due to occlusions, since not all of the pose data is important all of the time.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Patrick Peursum.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Peursum, P., Bui, H.H., Venkatesh, S. et al. Robust Recognition and Segmentation of Human Actions Using HMMs with Missing Observations. EURASIP J. Adv. Signal Process. 2005, 870416 (2005). https://doi.org/10.1155/ASP.2005.2110

Download citation

  • Received:

  • Revised:

  • Published:

  • DOI: https://doi.org/10.1155/ASP.2005.2110

Keywords and phrases

  • human motion analysis
  • action segmentation
  • HMMs