Open Access

Robust Background Subtraction with Shadow and Highlight Removal for Indoor Surveillance

EURASIP Journal on Advances in Signal Processing20072007:082931

Received: 1 March 2006

Accepted: 29 October 2006

Published: 4 January 2007


This work describes a robust background subtraction scheme involving shadow and highlight removal for indoor environmental surveillance. Foreground regions can be precisely extracted by the proposed scheme despite illumination variations and dynamic background. The Gaussian mixture model (GMM) is applied to construct a color-based probabilistic background model (CBM). Based on CBM, the short-term color-based background model (STCBM) and the long-term color-based background model (LTCBM) can be extracted and applied to build the gradient-based version of the probabilistic background model (GBM). Furthermore, a new dynamic cone-shape boundary in the RGB color space, called a cone-shape illumination model (CSIM), is proposed to distinguish pixels among shadow, highlight, and foreground. A novel scheme combining the CBM, GBM, and CSIM is proposed to determine the background which can be used to detect abnormal conditions. The effectiveness of the proposed method is demonstrated via experiments with several video clips collected in a complex indoor environment.


Authors’ Affiliations

Department of Electrical and Control Engineering, National Chiao-Tung University


  1. Friedman N, Russell S: Image segmentation in video sequences: a probabilistic approach. Proceedings of the 13th Conference Uncertainty in Artificial Intelligence (UAI '97), August 1997, Providence, RI, USA 175-181.Google Scholar
  2. Wren CR, Azarbayejani A, Darrell T, Pentland AP: Pfinder: real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997,19(7):780-785. 10.1109/34.598236View ArticleGoogle Scholar
  3. Kaew P, Trakul P, Bowden R: An improved adaptive background mixture model for real-time tracking with shadow detection. Proceedings of the 2nd European Workshop on Advanced Video-Based Surveillance Systems, September 2001, Kingston, UKGoogle Scholar
  4. Stauffer C, Grimson WEL: Adaptive background mixture models for real-time tracking. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 1999, Fort Collins, Colo, USA 2: 246-252.Google Scholar
  5. Koller D, Weber J, Huang T, et al.: Towards robust automatic traffic scene analysis in real-time. Proceedings of the 33rd IEEE Conference on Decision and Control, December 1994, Lake Buena Vista, Fla, USA 4: 3776-3781.Google Scholar
  6. Elgammal A, Duraiswami R, Harwood D, Davis LS: Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proceedings of the IEEE 2002,90(7):1151-1163. 10.1109/JPROC.2002.801448View ArticleGoogle Scholar
  7. Stockham TG Jr.: Image processing in the context of a visual model. Proceedings of the IEEE 1972,60(7):828-842.View ArticleGoogle Scholar
  8. Rosin PL, Ellis T: Image difference threshold strategies and shadow detection. Proceedings of the 6th British Machine Vision Conference, September 1995, Birmingham, Ala, USA 347-356.Google Scholar
  9. Elgammal A, Harwood D, Davis LS: Non-parametric Model for Background Subtraction. Proceedings of the 6th European Conference on Computer Vision, June 2000, Dublin, Ireland 751-767.Google Scholar
  10. Su T-M, Hu J-S: Background removal in vision servo system using Gaussian mixture model framework. Proceeding of IEEE International Conference on Networking, Sensing and Control, March 2004, Taipei, Taiwan 1: 70-75.Google Scholar
  11. Javed O, Shafique K, Shah M: A hierarchical approach to robust background subtraction using color and gradient information. Proceedings of IEEE Workshop on Motion and Video Computing (MOTION '02), December 2002, Orlando, Fla, USA 22-27.Google Scholar
  12. MacQueen JB: Some methods for classification and analysis of multivariate observations. Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, 1967, Berkeley, Calif, USA 1: 281-297.MathSciNetMATHGoogle Scholar
  13. Hoprasert T, Harwood D, Davis LS: A statistical approach for real-time robust background subtraction and shadow detection. Proceedings of the 7th IEEE International Conference on Computer Vision, Frame Rate Workshop (ICCV '99), September 1999, Kerkyra, Greece 1-19.Google Scholar


© J.-S. Hu and T.-M. Su. 2007

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.