- Research Article
- Open Access
A Human Body Analysis System
EURASIP Journal on Advances in Signal Processing volume 2006, Article number: 061927 (2006)
This paper describes a system for human body analysis (segmentation, tracking, face/hands localisation, posture recognition) from a single view that is fast and completely automatic. The system first extracts low-level data and uses part of the data for high-level interpretation. It can detect and track several persons even if they merge or are completely occluded by another person from the camera's point of view. For the high-level interpretation step, static posture recognition is performed using a belief theory-based classifier. The belief theory is considered here as a new approach for performing posture recognition and classification using imprecise and/or conflicting data. Four different static postures are considered: standing, sitting, squatting, and lying. The aim of this paper is to give a global view and an evaluation of the performances of the entire system and to describe in detail each of its processing steps, whereas our previous publications focused on a single part of the system. The efficiency and the limits of the system have been highlighted on a database of more than fifty video sequences where a dozen different individuals appear. This system allows real-time processing and aims at monitoring elderly people in video surveillance applications or at the mixing of real and virtual worlds in ambient intelligence systems.
Haritaoglu I, Harwood D, Davis L: W4: who? when? where? what? a real time system for detecting and tracking people. Proceedings of the 3rd International Conference on Conference on Automatic Face and Gesture Recognition (CAFGR '98), April 1998, Nara, Japan 222–227.
Wren CR, Azarbayejani A, Darrell TJ, Pentland AP: Pfinder: real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997, 19(7):780–785. 10.1109/34.598236
Architecture and authoring tools prototype for living images and new video experiments website of the art.live project: IST Project 10942, 2002, https://doi.org/www.transfiction.net/artlive/
Website of SIMILAR Network of excellence: the European taskforce creating human-machine interfaces similar to human-human communication, 2003, https://doi.org/www.similar.cc/
Aizawa K, Huang TS: Model-based image coding: advanced video coding techniques for very low bit-rate applications. Proceedings of the IEEE 1995, 83(2):259–271. 10.1109/5.364463
Doulamis ND, Doulamis AD, Kollias SD: Efficient content-based retrieval of humans from video databases. Proceedings of the 2nd International Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-Time Systems (RATFG '99), September 1999, Corfu, Greece 89–95.
Gehrig N, Lepetit V, Fua P: Golf club visual tracking for enhanced swing analysis. Proceedings of the British Machine Vision Conference (BMVC '03), September 2003, Norwich, UK
Köhle M, Merkl D, Kastner J: Clinical gait analysis by neural networks: issues and experiences. Proceedings of the 10th IEEE Symposium on Computer-Based Medical Systems (CBMS '97), June 1997, Maribor, Slovenia 138–143.
Maes P, Darrell TJ, Blumberg B, Pentland AP: The ALIVE system: wireless, full-body interaction with autonomous agents. ACM Multimedia Systems 1997, 5(2):105–112. 10.1007/s005300050046
Wren CR, Sparacino F, Azarbayejani AJ, et al.: Perceptive spaces for performance and entertainment: untethered interaction using computer vision and audition. Applied Artificial Intelligence 1997, 11(4):267–284. 10.1080/088395197118154
Gavrila DM: The visual analysis of human movement: a survey. Computer Vision and Image Understanding 1999, 73(1):82–98. 10.1006/cviu.1998.0716
Aggarwal JK, Cai Q: Human motion analysis: a review. Computer Vision and Image Understanding 1999, 73(3):428–440. 10.1006/cviu.1998.0744
Pentland A: Looking at people: sensing for ubiquitous and wearable computing. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000, 22(1):107–119. 10.1109/34.824823
Moeslund TB, Granum E: A survey of computer vision-based human motion capture. Computer Vision and Image Understanding 2001, 81(3):231–268. 10.1006/cviu.2000.0897
Wang L, Hu WM, Tan TN: Recent developments in human motion analysis. Pattern Recognition 2003, 36(3):585–601. 10.1016/S0031-3203(02)00100-0
Wang JJ, Singh S: Video analysis of human dynamics: a survey. Real-Time Imaging 2003, 9(5):321–346. 10.1016/j.rti.2003.08.001
Collins RT, Lipton AJ, Kanade T: A system for video surveillance and monitoring. In Tech. Rep. CMU-RI-TR-00-12. Carnegie Mellon University, Pittsburgh, Pa, USA; May 2000.
Nair V, Clark JJ: Automated visual surveillance using hidden markov models. Proceedings of the The 15th International Conference on Vision Interface (VI '02), May 2002, Calgary, Canada 88–94.
Mitiche A, Bouthemy P: Computation and analysis of image motion: a synopsis of current problems and methods. International Journal of Computer Vision 1996, 19(1):29–55. 10.1007/BF00131147
Nagel HH: Formation of an object concept by analysis of systematic time variations in the optically perceptible environment. Computer Graphics and Image Processing 1978, 7(2):149–194. 10.1016/0146-664X(78)90111-9
Sangi P, Heikkilä J, Silvén O: Motion analysis using frame differences with spatial gradient measures. Proceedings of the 17th International Conference on Pattern Recognition (ICPR '04), August 2004, Cambridge, UK 4: 733–736.
Aach T, Kaup A, Mester R: Statistical model-based detection in moving videos. Signal Processing 1993, 31(2):165–180. 10.1016/0165-1684(93)90063-G
Lee D-S: Effective Gaussian mixture learning for video background subtraction. IEEE Transactions on Pattern Analysis and Machine Intelligence 2005, 27(5):827–832.
Long W, Yang YH: Stationary background generation: an alternative to the difference of two images. Pattern Recognition 1990, 23(12):1351–1359. 10.1016/0031-3203(90)90081-U
Seki M, Wada T, Fujiwara H, Sumi K: Background subtraction based on cooccurrence of image variations. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '03), June 2003, Madison, Wis, USA 2: 65–72.
Luthon F, Caplier A, Liévin M: Spatiotemporal MRF approach to video segmentation: application to motion detection and lip segmentation. Signal Processing 1999, 76(1):61–80. 10.1016/S0165-1684(98)00247-3
Caplier A, Bonnaud L, Chassery J-M: Robust fast extraction of video objects combining frame differences and adaptive reference image. Proceedings of International Conference on Image Processing (ICIP '01), October 2001, Thessaloniki, Greece 2(2):785–788.
Geman S, Geman D: Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6(6):721–741.
Besag J: On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society 1986, B-48(3):259–302.
McKenna SJ, Jabri S, Duric Z, Rosenfeld A, Wechsler H: Tracking groups of people. Computer Vision and Image Understanding 2000, 80(1):42–56. 10.1006/cviu.2000.0870
Chellappa R, Wilson CL, Sirohey S: Human and machine recognition of faces: a survey. Proceedings of the IEEE 1995, 83(5):705–740. 10.1109/5.381842
Fromherz T, Stucki P, Bichsel M: A survey of face recognition. In Tech. Rep. 97.01. Department of Computer Science, University of Zurich, Zurich, Switzerland; 1997.
Yang M-H, Kriegman DJ, Ahuja N: Detecting faces in images: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(1):34–58. 10.1109/34.982883
Hjelmås E, Low BK: Face detection: a survey. Computer Vision and Image Understanding 2001, 83(3):236–274. 10.1006/cviu.2001.0921
Zhao W, Chellappa R, Rosenfeld A, Phillips PJ: Face recognition: a literature survey. In Tech. Rep. TR4167R. UMD University of Maryland, College Park, Md, USA ; 2002.
Yang J, Lu W, Waibel A: Skin-color modeling and adaptation. Proceedings of Asian Conference on Computer Vision (ACCV '98), January 1998, Hong Kong 2: 687–694.
Terrillon J-C, Shirazi MN, Fukamachi H, Akamatsu S: Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images. Proceedingsof the 4th IEEE International Conference on Automatic Face and Gesture Recognition (AFGR '00), March 2000, Grenoble, France 54–61.
Brunelli R, Poggio T: Face recognition: features versus templates. IEEE Transactions on Pattern Analysis and Machine Intelligence 1993, 15(10):1042–1052. 10.1109/34.254061
Pentland AP, Moghaddam B, Starner TE: View-based and modular eigenspace for face recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '94), June 1994, Washington, DC, USA 84–91.
Girondel V, Bonnaud L, Caplier A: Hands detection and tracking for interactive multimedia applications. International Conference on Computer Vision and Graphics (ICCVG~'02), September 2002, Zakopane, Poland 1: 282–287.
Girondel V: Détection de peau, suivi de tête et de mains pour des applications multimédia. In SIPT Master's Technical Report. Laboratoire des Images et des Signaux (LIS), Institut National Polytechnique, Grenoble, France; July 2002.
Chai D, Ngan KN: Face segmentation using skin-color map in videophone applications. IEEE Transactions on Circuits and Systems for Video Technology 1999, 9(4):551–564. 10.1109/76.767122
Dockstader SL, Tekalp AM: On the tracking of articulated and occluded video object motion. Real-Time Imaging 2001, 7(5):415–432. 10.1006/rtim.2000.0210
Capellades MB, Doermann D, DeMenthon D, Chellappa R: An appearance based approach for human and object tracking. Proceedings of IEEE International Conference on Image Processing (ICIP '03), September 2003, Barcelona, Spain 2: 85–88.
Girondel V, Caplier A, Bonnaud L: Real time tracking of multiple persons by kalman filtering and face pursuit for multimedia applications. Proceedings of the IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI '04), March 2004, Lake Tahoe, Nev, USA 6: 201–205.
Kalman RE: A new approach to linear filtering and prediction problems. Transactions of the ASME - Journal of Basic Engineering 1960, 82: 35–45. 10.1115/1.3662552
Bobick AF, Wilson AD: A state-based approach to the representation and recognition of gesture. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997, 19(12):1325–1337. 10.1109/34.643892
Yamato J, Ohya J, Ishii K: Recognizing human action in time-sequential images using hidden Markov model. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '92), June 1992, Champaign, Ill, USA 379–385.
Guo Y, Xu G, Tsuji S: Understanding human motion patterns. Proceedings of the 17th International Conference on Pattern Recognition (ICPR '94), October 1994, Jerusalem, Israel B: 325–329.
Girondel V, Bonnaud L, Caplier A, Rombaut M: Static human body postures recognition in video sequences using the belief theory. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy 45–48.
Girondel V, Bonnaud L, Caplier A: A belief theory-based static posture recognition system for real-time video surveillance applications. Proceedings of IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS '05), September 2005, Como, Italy 10–15.
Hammal Z, Caplier A, Rombaut M: Classification d'expressions faciales par la théorie de l'évidence. Proceedings of the 12es rencontres francophones sur la Logique Floue et ses Applications (LFA '04), November 2004, Nantes, France 173–180.
Hammal Z, Couvreur L, Caplier A, Rombaut M: Facial expression recognition based on the belief theory: comparison with different classifiers. Proceedings of the 13th International Conference on Image Analysis and Processing (ICIAP~'05), September 2005, Cagliari, Italy 743–752.
Smets P, Kennes R: The transferable belief model. Artificial Intelligence 1994, 66(2):191–234. 10.1016/0004-3702(94)90026-4
Smets P: The transferable belief model for quantified belief representation. In Handbook of Defeasible Reasoning and Uncertainty Management Systems. Volume 1. Edited by: Gabbay DM, Smets P. Kluwer Academic, Dordrecht, The Netherlands; 1998:267–301.
Dempster A: A generalization of Bayesian inference. Journal of the Royal Statistical Society 1968, 30: 205–245.
Shafer G: A Mathematical Theory of Evidence. Princeton University Press, Princeton, NJ, USA; 1976.
Salvador E, Cavalarro A, Ebrahimi T: Shadow identification and classification using invariant color models. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '01), May 2001, Salt Lake City, Utah, USA 3: 1545–1548.
Hernandez PC, Czyz J, Umeda T, Marques F, Marichal X, Macq B: Silhouette based probabilistic 2d human motion estimation for real time applications. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy
Hammal Z, Massot C, Bedoya G, Caplier A: Eyes segmentation applied to gaze direction and vigilance estimation. Proceedings of the 3rd International Conference on Advances in Pattern Recognition (ICAPR '05), August 2005, Bath, UK 236–246.
About this article
Cite this article
Girondel, V., Bonnaud, L. & Caplier, A. A Human Body Analysis System. EURASIP J. Adv. Signal Process. 2006, 061927 (2006). https://doi.org/10.1155/ASP/2006/61927
- Video Sequence
- Static Posture
- Intelligence System
- Virtual World
- Video Surveillance