Skip to main content

A Human Body Analysis System

Abstract

This paper describes a system for human body analysis (segmentation, tracking, face/hands localisation, posture recognition) from a single view that is fast and completely automatic. The system first extracts low-level data and uses part of the data for high-level interpretation. It can detect and track several persons even if they merge or are completely occluded by another person from the camera's point of view. For the high-level interpretation step, static posture recognition is performed using a belief theory-based classifier. The belief theory is considered here as a new approach for performing posture recognition and classification using imprecise and/or conflicting data. Four different static postures are considered: standing, sitting, squatting, and lying. The aim of this paper is to give a global view and an evaluation of the performances of the entire system and to describe in detail each of its processing steps, whereas our previous publications focused on a single part of the system. The efficiency and the limits of the system have been highlighted on a database of more than fifty video sequences where a dozen different individuals appear. This system allows real-time processing and aims at monitoring elderly people in video surveillance applications or at the mixing of real and virtual worlds in ambient intelligence systems.

References

  1. 1.

    Haritaoglu I, Harwood D, Davis L: W4: who? when? where? what? a real time system for detecting and tracking people. Proceedings of the 3rd International Conference on Conference on Automatic Face and Gesture Recognition (CAFGR '98), April 1998, Nara, Japan 222–227.

    Google Scholar 

  2. 2.

    Wren CR, Azarbayejani A, Darrell TJ, Pentland AP: Pfinder: real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997, 19(7):780–785. 10.1109/34.598236

    Article  Google Scholar 

  3. 3.

    Architecture and authoring tools prototype for living images and new video experiments website of the art.live project: IST Project 10942, 2002, https://doi.org/www.transfiction.net/artlive/

  4. 4.

    Website of SIMILAR Network of excellence: the European taskforce creating human-machine interfaces similar to human-human communication, 2003, https://doi.org/www.similar.cc/

  5. 5.

    Aizawa K, Huang TS: Model-based image coding: advanced video coding techniques for very low bit-rate applications. Proceedings of the IEEE 1995, 83(2):259–271. 10.1109/5.364463

    Article  Google Scholar 

  6. 6.

    Doulamis ND, Doulamis AD, Kollias SD: Efficient content-based retrieval of humans from video databases. Proceedings of the 2nd International Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-Time Systems (RATFG '99), September 1999, Corfu, Greece 89–95.

    Google Scholar 

  7. 7.

    Gehrig N, Lepetit V, Fua P: Golf club visual tracking for enhanced swing analysis. Proceedings of the British Machine Vision Conference (BMVC '03), September 2003, Norwich, UK

    Google Scholar 

  8. 8.

    Köhle M, Merkl D, Kastner J: Clinical gait analysis by neural networks: issues and experiences. Proceedings of the 10th IEEE Symposium on Computer-Based Medical Systems (CBMS '97), June 1997, Maribor, Slovenia 138–143.

    Google Scholar 

  9. 9.

    Maes P, Darrell TJ, Blumberg B, Pentland AP: The ALIVE system: wireless, full-body interaction with autonomous agents. ACM Multimedia Systems 1997, 5(2):105–112. 10.1007/s005300050046

    Article  Google Scholar 

  10. 10.

    Wren CR, Sparacino F, Azarbayejani AJ, et al.: Perceptive spaces for performance and entertainment: untethered interaction using computer vision and audition. Applied Artificial Intelligence 1997, 11(4):267–284. 10.1080/088395197118154

    Article  Google Scholar 

  11. 11.

    Gavrila DM: The visual analysis of human movement: a survey. Computer Vision and Image Understanding 1999, 73(1):82–98. 10.1006/cviu.1998.0716

    MATH  Article  Google Scholar 

  12. 12.

    Aggarwal JK, Cai Q: Human motion analysis: a review. Computer Vision and Image Understanding 1999, 73(3):428–440. 10.1006/cviu.1998.0744

    Article  Google Scholar 

  13. 13.

    Pentland A: Looking at people: sensing for ubiquitous and wearable computing. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000, 22(1):107–119. 10.1109/34.824823

    Article  Google Scholar 

  14. 14.

    Moeslund TB, Granum E: A survey of computer vision-based human motion capture. Computer Vision and Image Understanding 2001, 81(3):231–268. 10.1006/cviu.2000.0897

    MATH  Article  Google Scholar 

  15. 15.

    Wang L, Hu WM, Tan TN: Recent developments in human motion analysis. Pattern Recognition 2003, 36(3):585–601. 10.1016/S0031-3203(02)00100-0

    Article  Google Scholar 

  16. 16.

    Wang JJ, Singh S: Video analysis of human dynamics: a survey. Real-Time Imaging 2003, 9(5):321–346. 10.1016/j.rti.2003.08.001

    Article  Google Scholar 

  17. 17.

    Collins RT, Lipton AJ, Kanade T: A system for video surveillance and monitoring. In Tech. Rep. CMU-RI-TR-00-12. Carnegie Mellon University, Pittsburgh, Pa, USA; May 2000.

    Google Scholar 

  18. 18.

    Nair V, Clark JJ: Automated visual surveillance using hidden markov models. Proceedings of the The 15th International Conference on Vision Interface (VI '02), May 2002, Calgary, Canada 88–94.

    Google Scholar 

  19. 19.

    Mitiche A, Bouthemy P: Computation and analysis of image motion: a synopsis of current problems and methods. International Journal of Computer Vision 1996, 19(1):29–55. 10.1007/BF00131147

    Article  Google Scholar 

  20. 20.

    Nagel HH: Formation of an object concept by analysis of systematic time variations in the optically perceptible environment. Computer Graphics and Image Processing 1978, 7(2):149–194. 10.1016/0146-664X(78)90111-9

    Article  Google Scholar 

  21. 21.

    Sangi P, Heikkilä J, Silvén O: Motion analysis using frame differences with spatial gradient measures. Proceedings of the 17th International Conference on Pattern Recognition (ICPR '04), August 2004, Cambridge, UK 4: 733–736.

    Article  Google Scholar 

  22. 22.

    Aach T, Kaup A, Mester R: Statistical model-based detection in moving videos. Signal Processing 1993, 31(2):165–180. 10.1016/0165-1684(93)90063-G

    MATH  Article  Google Scholar 

  23. 23.

    Lee D-S: Effective Gaussian mixture learning for video background subtraction. IEEE Transactions on Pattern Analysis and Machine Intelligence 2005, 27(5):827–832.

    Article  Google Scholar 

  24. 24.

    Long W, Yang YH: Stationary background generation: an alternative to the difference of two images. Pattern Recognition 1990, 23(12):1351–1359. 10.1016/0031-3203(90)90081-U

    Article  Google Scholar 

  25. 25.

    Seki M, Wada T, Fujiwara H, Sumi K: Background subtraction based on cooccurrence of image variations. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '03), June 2003, Madison, Wis, USA 2: 65–72.

    Google Scholar 

  26. 26.

    Luthon F, Caplier A, Liévin M: Spatiotemporal MRF approach to video segmentation: application to motion detection and lip segmentation. Signal Processing 1999, 76(1):61–80. 10.1016/S0165-1684(98)00247-3

    MATH  Article  Google Scholar 

  27. 27.

    Caplier A, Bonnaud L, Chassery J-M: Robust fast extraction of video objects combining frame differences and adaptive reference image. Proceedings of International Conference on Image Processing (ICIP '01), October 2001, Thessaloniki, Greece 2(2):785–788.

    Google Scholar 

  28. 28.

    Geman S, Geman D: Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6(6):721–741.

    MATH  Article  Google Scholar 

  29. 29.

    Besag J: On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society 1986, B-48(3):259–302.

    MathSciNet  MATH  Google Scholar 

  30. 30.

    McKenna SJ, Jabri S, Duric Z, Rosenfeld A, Wechsler H: Tracking groups of people. Computer Vision and Image Understanding 2000, 80(1):42–56. 10.1006/cviu.2000.0870

    MATH  Article  Google Scholar 

  31. 31.

    Chellappa R, Wilson CL, Sirohey S: Human and machine recognition of faces: a survey. Proceedings of the IEEE 1995, 83(5):705–740. 10.1109/5.381842

    Article  Google Scholar 

  32. 32.

    Fromherz T, Stucki P, Bichsel M: A survey of face recognition. In Tech. Rep. 97.01. Department of Computer Science, University of Zurich, Zurich, Switzerland; 1997.

    Google Scholar 

  33. 33.

    Yang M-H, Kriegman DJ, Ahuja N: Detecting faces in images: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(1):34–58. 10.1109/34.982883

    Article  Google Scholar 

  34. 34.

    Hjelmås E, Low BK: Face detection: a survey. Computer Vision and Image Understanding 2001, 83(3):236–274. 10.1006/cviu.2001.0921

    MATH  Article  Google Scholar 

  35. 35.

    Zhao W, Chellappa R, Rosenfeld A, Phillips PJ: Face recognition: a literature survey. In Tech. Rep. TR4167R. UMD University of Maryland, College Park, Md, USA ; 2002.

    Google Scholar 

  36. 36.

    Yang J, Lu W, Waibel A: Skin-color modeling and adaptation. Proceedings of Asian Conference on Computer Vision (ACCV '98), January 1998, Hong Kong 2: 687–694.

    Google Scholar 

  37. 37.

    Terrillon J-C, Shirazi MN, Fukamachi H, Akamatsu S: Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images. Proceedingsof the 4th IEEE International Conference on Automatic Face and Gesture Recognition (AFGR '00), March 2000, Grenoble, France 54–61.

    Google Scholar 

  38. 38.

    Brunelli R, Poggio T: Face recognition: features versus templates. IEEE Transactions on Pattern Analysis and Machine Intelligence 1993, 15(10):1042–1052. 10.1109/34.254061

    Article  Google Scholar 

  39. 39.

    Pentland AP, Moghaddam B, Starner TE: View-based and modular eigenspace for face recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '94), June 1994, Washington, DC, USA 84–91.

    Google Scholar 

  40. 40.

    Girondel V, Bonnaud L, Caplier A: Hands detection and tracking for interactive multimedia applications. International Conference on Computer Vision and Graphics (ICCVG~'02), September 2002, Zakopane, Poland 1: 282–287.

    Google Scholar 

  41. 41.

    Girondel V: Détection de peau, suivi de tête et de mains pour des applications multimédia. In SIPT Master's Technical Report. Laboratoire des Images et des Signaux (LIS), Institut National Polytechnique, Grenoble, France; July 2002.

    Google Scholar 

  42. 42.

    Chai D, Ngan KN: Face segmentation using skin-color map in videophone applications. IEEE Transactions on Circuits and Systems for Video Technology 1999, 9(4):551–564. 10.1109/76.767122

    Article  Google Scholar 

  43. 43.

    Dockstader SL, Tekalp AM: On the tracking of articulated and occluded video object motion. Real-Time Imaging 2001, 7(5):415–432. 10.1006/rtim.2000.0210

    MATH  Article  Google Scholar 

  44. 44.

    Capellades MB, Doermann D, DeMenthon D, Chellappa R: An appearance based approach for human and object tracking. Proceedings of IEEE International Conference on Image Processing (ICIP '03), September 2003, Barcelona, Spain 2: 85–88.

    Google Scholar 

  45. 45.

    Girondel V, Caplier A, Bonnaud L: Real time tracking of multiple persons by kalman filtering and face pursuit for multimedia applications. Proceedings of the IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI '04), March 2004, Lake Tahoe, Nev, USA 6: 201–205.

    Google Scholar 

  46. 46.

    Kalman RE: A new approach to linear filtering and prediction problems. Transactions of the ASME - Journal of Basic Engineering 1960, 82: 35–45. 10.1115/1.3662552

    MathSciNet  Article  Google Scholar 

  47. 47.

    Bobick AF, Wilson AD: A state-based approach to the representation and recognition of gesture. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997, 19(12):1325–1337. 10.1109/34.643892

    Article  Google Scholar 

  48. 48.

    Yamato J, Ohya J, Ishii K: Recognizing human action in time-sequential images using hidden Markov model. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '92), June 1992, Champaign, Ill, USA 379–385.

    Google Scholar 

  49. 49.

    Guo Y, Xu G, Tsuji S: Understanding human motion patterns. Proceedings of the 17th International Conference on Pattern Recognition (ICPR '94), October 1994, Jerusalem, Israel B: 325–329.

    Article  Google Scholar 

  50. 50.

    Girondel V, Bonnaud L, Caplier A, Rombaut M: Static human body postures recognition in video sequences using the belief theory. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy 45–48.

    Google Scholar 

  51. 51.

    Girondel V, Bonnaud L, Caplier A: A belief theory-based static posture recognition system for real-time video surveillance applications. Proceedings of IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS '05), September 2005, Como, Italy 10–15.

    Google Scholar 

  52. 52.

    Hammal Z, Caplier A, Rombaut M: Classification d'expressions faciales par la théorie de l'évidence. Proceedings of the 12es rencontres francophones sur la Logique Floue et ses Applications (LFA '04), November 2004, Nantes, France 173–180.

    Google Scholar 

  53. 53.

    Hammal Z, Couvreur L, Caplier A, Rombaut M: Facial expression recognition based on the belief theory: comparison with different classifiers. Proceedings of the 13th International Conference on Image Analysis and Processing (ICIAP~'05), September 2005, Cagliari, Italy 743–752.

    Google Scholar 

  54. 54.

    Smets P, Kennes R: The transferable belief model. Artificial Intelligence 1994, 66(2):191–234. 10.1016/0004-3702(94)90026-4

    MathSciNet  MATH  Article  Google Scholar 

  55. 55.

    Smets P: The transferable belief model for quantified belief representation. In Handbook of Defeasible Reasoning and Uncertainty Management Systems. Volume 1. Edited by: Gabbay DM, Smets P. Kluwer Academic, Dordrecht, The Netherlands; 1998:267–301.

    Google Scholar 

  56. 56.

    Dempster A: A generalization of Bayesian inference. Journal of the Royal Statistical Society 1968, 30: 205–245.

    MathSciNet  MATH  Google Scholar 

  57. 57.

    Shafer G: A Mathematical Theory of Evidence. Princeton University Press, Princeton, NJ, USA; 1976.

    Google Scholar 

  58. 58.

    Salvador E, Cavalarro A, Ebrahimi T: Shadow identification and classification using invariant color models. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '01), May 2001, Salt Lake City, Utah, USA 3: 1545–1548.

    Google Scholar 

  59. 59.

    Hernandez PC, Czyz J, Umeda T, Marques F, Marichal X, Macq B: Silhouette based probabilistic 2d human motion estimation for real time applications. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy

    Google Scholar 

  60. 60.

    Hammal Z, Massot C, Bedoya G, Caplier A: Eyes segmentation applied to gaze direction and vigilance estimation. Proceedings of the 3rd International Conference on Advances in Pattern Recognition (ICAPR '05), August 2005, Bath, UK 236–246.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Vincent Girondel.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Girondel, V., Bonnaud, L. & Caplier, A. A Human Body Analysis System. EURASIP J. Adv. Signal Process. 2006, 061927 (2006). https://doi.org/10.1155/ASP/2006/61927

Download citation

Keywords

  • Video Sequence
  • Static Posture
  • Intelligence System
  • Virtual World
  • Video Surveillance