Outdoor shadow detection by combining tricolor attenuation and intensity
© Tian et al; licensee Springer. 2012
Received: 17 January 2012
Accepted: 28 May 2012
Published: 28 May 2012
Shadow detection is of broad interest in computer vision. In this article, a new shadow detection method for single color images in outdoor scenes is proposed. Shadows attenuate pixel intensity, and the degrees of attenuation are different in the three RGB color channels. Previously, we proposed the Tricolor Attenuation Model (TAM) that describes the attenuation relationship between shadows and their non-shadow backgrounds in the three color channels. TAM can provide strong information on shadow detection; however, our previous study needs a rough segmentation as the pre-processing step and requires four thresholds. These shortcomings can be overcome by adding intensity information. This article addresses the problem of how to combine TAM and intensity and meanwhile to obtain a threshold for shadow segmentation. Simple and complicated shadow images are used to test the proposed method. The experimental results and comparisons validate its effectiveness.
Light sources and intensity of shadow and non-shadow region
Part sunlight + skylight
where m=1.31 and n=1.19.
Segmenting the original image and calculating TAM in each segmented sub-region.
Simply using the mean value over each sub-region to binarizate the TAM images and to obtain initial shadows.
Simply using the mean values in three color channels, in each sub-region, as the thresholds to verify and refine the initial shadows (to obtain detailed and more accurate results).
It needs segmentation. Although the method is not sensitive to little segmentation error, it is not an easy work to get a satisfying segmentation result (shadows and their non-shadow backgrounds are segmented into same regions). For some images, serious segmentation errors may lead to bad shadow detection results.
It uses four simple mean values as thresholds in the two key steps (steps 2 and 3). One threshold is used for initial shadow segmentation and three thresholds are used to obtain accuracy boundaries and details. The thresholds sometimes have noticeable influence on the final results, i.e., simple thresholds are insufficient for some images.
In this article, we try to solve the above-mentioned two problems; we combine TAM and intensity information to avoid the segmentation step and derive only one threshold to substitute previous four simple ones. The new proposed method in this article is simpler and meanwhile can achieve similar or better results.
2 Previous studies
Shadows, a common phenomenon in most outdoor scenes, take extensive effects in computer vision and pattern recognition. It brings many difficulties to computer vision applications such as segmentation, tracking, retrieval, recognition. On the other hand, shadows in an image also provide useful information about the scene: they provide cues about the location of the sun as well as the shape and the geometry of the occluder. Overall, dealing with shadows is an important and challenging task in computer vision and pattern recognition.
The most straightforward feature of shadow is that it darkens the surface it casts on, and this feature is adopted by some methods directly [2, 3] or indirectly [4, 5]. Many methods assume that shadow pixels mainly change luminance but less chrominance. For example, in , the authors assume hue and saturation components change within a certain limit in HSV space. In , multiple cues including color, luminance, and texture are applied to detect moving shadows. Another commonly used feature for shadow detection is intrinsic feature. Intrinsic features locate shadows by comparing the intrinsic image and the original one. Salvador et al.  employed c1c2c3 feature to derive intrinsic images. Finlayson et al.  developed a method to generate a 1D illumination invariant image by finding a special direction in a 2D chromaticity feature space. Tian and Tang  proposed a method to generate illumination invariant image by using the linearity between shadow and non-shadow paired regions. The intrinsic image is useful for shadow detection. However, it cannot totally eliminate the illumination effect and thus is often used in the simple scenes.
Most shadow detection methods focus on detecting moving shadows. Moving shadow detection methods can employ the frame difference technique to locate moving objects and their moving shadows. Then, the problem of shadow detection becomes differentiating the moving objects and the moving shadows. Prati et al.  provided a good review for shadow detection methods in video sequences. To adapt to background changes, learning approaches have proven useful. Huang and Chen  employed Gaussian mixture model to learn the color features and to model the background appearance variations under cast shadows. Brisson and Zaccarin  presented an unsupervised kernel-based approach to estimate the cast shadow direction. Siala et al.  described a moving shadow detection algorithm by training the manually segmented shadow regions. Joshi and Papanikolopoulos  used SVM and co-training technique to detect shadows. Compared with static shadow detection methods, moving shadow detection methods can employ the powerful background subtraction techniques. Therefore, the majority of moving shadow detection methods cannot be directly used to detect static shadows in single images.
As detecting moving shadows has made great progress, detecting it from a single image remains a difficult problem. Wu and Tang  used the Bayesian approach to extract shadows from a single image, but it requires user's intervention as the input. Panagopoulos et al.  used the Fisher distribution to model shadows, but this approach needs 3D geometry information. As a special application of shadow detection in single image, literatures [3, 18, 19] focus on detecting shadows in the remote sensing images. Lalonde et al.  proposed a learning approach to train a decision tree classifier on a set of shadow sensitive features to detect ground shadows in consumer-grade photographs. Guo et al.  proposed a learning-based shadow detection method by using paired regions (shadow and non-shadow) for a single image. Learning methods can achieve good performance if the parameters are trained well. However, they will fail when the test image is vastly different from the images in the training set . In the previous study , we proposed the TAM-based shadow detection algorithm. The algorithm is automatic and simple but it depends more or less upon priori segmentation and the four simply chosen thresholds. The improved algorithm described in Section 3 can address these two problems.
3 Method description
where denotes the k th pixel of image F in R channel, and M is the number of pixels.
As mentioned above, though TAM can provide information for shadow detection, it may suffer from false detection and details missing problems. These problems caused by luminance information are lost during the channel-subtraction procedure. Fortunately, the lost information in the TAM image can be compensated by intensity (grayscale) image. The problem then becomes how to combine intensity image with TAM image. In the following, we will give a method to address it meanwhile to derive a threshold for shadow segmentation.
Given T, S can be determined by using Equation (6).
α is initialized with . Repeating (4)-(9) to update T and α until .
4 Experimental results
In this article, we propose a shadow detection method based on combining TAM image and intensity image. In previous study , TAM information and intensity information are used separately. Shadow detection only relies on TAM information, and it needs a rough segmentation preprocessing step; intensity information is simply used to improve the boundary accuracy and details of the detected shadows. The effective combination of them in this article allows that the new method is free from segmentation. Furthermore, the new method only requires one threshold to detect shadows and handle the details simultaneously. These advantages make the proposed method easier to use and more robust in applications.
This study was supported by the National Natural Science Foundation of China (Grant No. 61102116).
- Tian J, Sun J, Tang Y: Tricolor attenuation model for shadow detection. IEEE Trans Image Process 2009, 18(10):2355-2363.MathSciNetView ArticleGoogle Scholar
- Barnard K, Finlayson G: Shadow identification using colour ratios. In Proceedings of the IS&T/SID Eighth Color Imaging Conference: Color Science, Systems and Applications. Volume 8. Scottsdale, Arizona, USA; 2000:97-101.Google Scholar
- Chung K, Lin Y, Huang Y: Efficient shadow detection of color aerial images based on successive thresholding scheme. IEEE Trans Geosci Remote Sensing 2009, 47(2):671-682.View ArticleGoogle Scholar
- Zhang W, Fang X, Yang X, Wu Q: Moving cast shadows detection using ratio edge. IEEE Trans Multim 2007, 9(6):1202-1214.View ArticleGoogle Scholar
- Leone A, Distante C: Shadow detection for moving objects based on texture analysis. Pattern Recognit 2007, 40(4):1222-1233. 10.1016/j.patcog.2006.09.017View ArticleGoogle Scholar
- Cucchiara R, Grana C, Piccardi M, Prati A: Detecting moving objects, ghosts, and shadows in video streams. IEEE Trans PAMI 2003, 25(10):1337-1342. 10.1109/TPAMI.2003.1233909View ArticleGoogle Scholar
- Yang M, Lo K, Chiang C, Tai W: Moving cast shadow detection by exploiting multiple cues. IET Image Process 2007, 2(2):95-104.View ArticleGoogle Scholar
- Salvador E, Cavallaro A, Ebrahimi T: Cast shadow segmentation using invariant color features. Comput Vis Image Understand 2004, 95(2):238-259. 10.1016/j.cviu.2004.03.008View ArticleGoogle Scholar
- Finlayson G, Drew MS, Lu C: Entropy minimization for shadow removal. Int J Comput Vis 2009, 85(1):35-57. 10.1007/s11263-009-0243-zView ArticleGoogle Scholar
- Tian J, Tang Y: Linearity of each channel pixel values from a surface in and out of shadows and its applications. In IEEE Conference on Computer Vision and Pattern Recognition. Springs, Colorado, USA; 2011:985-992.Google Scholar
- Prati A, Cucchiara R, Mikic I, Trivedi MM: Analysis and detection of shadows in video streams: a comparative evaluation. In IEEE Conference on Computer Vision and Pattern Recognition. Volume 2. Kauai, Hawaii, USA; 2001:571-576.Google Scholar
- Huang J, Chen C: Moving cast shadow detection using physics-based features. In IEEE Conference on Computer Vision and Pattern Recognition. Miami, Florida, USA; 2009:2310-2317.Google Scholar
- Brisson N, Zaccarin A: Learning and removing cast shadows through a multi-distribution approach. IEEE Trans PAMI 2007, 29(7):1133-1146.View ArticleGoogle Scholar
- Siala K, Chakchouk M, Besbes O, Chaieb F: Moving shadow detection with support vector domain description in the color ratios space. In International Conference on Pattern Recognition. Volume 4. Cambridge, UK; 2004:384-387.Google Scholar
- Joshi A, Papanikolopoulos N: Learning to detect moving shadows in dynamic environments. IEEE Trans PAMI 2008, 30(11):2055-2063.View ArticleGoogle Scholar
- Wu T, Tang C: A Bayesian approach for shadow extraction from a single image. IEEE International Conference on Computer Vision 2005, 1: Beijing, China 480-487.Google Scholar
- Panagopoulos A, Samaras D, Paragios N: Robust shadow and illumination estimation using a mixture model. In IEEE Conference on Computer Vision and Pattern Recognition. Miami, Florida, USA; 2009:651-658.Google Scholar
- Makarau A, Richter R, Müller R, Reinartz P: Adaptive shadow detection using a blackbody radiator model. IEEE Trans Geosci Remote Sensing 2011, 49(6):2049-2059.View ArticleGoogle Scholar
- Yao J, Zhang ZM: Hierarchical shadow detection for color aerial images. Comput Vis Image Understand 2006, 102(1):60-69. 10.1016/j.cviu.2005.09.003View ArticleGoogle Scholar
- Lalonde J, Efros A, Narasimhan S: Detecting ground shadows in outdoor consumer photographs. In European Conference on Computer Vision. Volume 2. Crete, Greece; 2010:322-335.Google Scholar
- Guo R, Dai Q, Hoiem D: Single-image shadow detection and removal using paired regions. In IEEE Conference on Computer Vision and Pattern Recognition. Springs, Colorado, USA; 2011:2033-2040.Google Scholar
- Littmann E, Ritter H: Adaptive color segmentation-a comparison of neural and statistical methods. IEEE Trans Neural Netw 1997, 8(1):175-185. 10.1109/72.554203View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.