- Research Article
- Open access
- Published:
Unsupervised Performance Evaluation of Image Segmentation
EURASIP Journal on Advances in Signal Processing volume 2006, Article number: 096306 (2006)
Abstract
We present in this paper a study of unsupervised evaluation criteria that enable the quantification of the quality of an image segmentation result. These evaluation criteria compute some statistics for each region or class in a segmentation result. Such an evaluation criterion can be useful for different applications: the comparison of segmentation results, the automatic choice of the best fitted parameters of a segmentation method for a given image, or the definition of new segmentation methods by optimization. We first present the state of art of unsupervised evaluation, and then, we compare six unsupervised evaluation criteria. For this comparative study, we use a database composed of 8400 synthetic gray-level images segmented in four different ways. Vinet's measure (correct classification rate) is used as an objective criterion to compare the behavior of the different criteria. Finally, we present the experimental results on the segmentation evaluation of a few gray-level natural images.
References
Freixenet J, Muñoz X, Raba D, Marti J, Cufi X: Yet another survey on image segmentation: region and boundary information integration. Proceedings of the European Conference on Computer Vision (ECCV '02), May 2002, Copenhagen, Denmark 408–422.
Haralick RM, Shapiro LG: Image segmentation techniques. Computer Vision, Graphics, & Image Processing 1985, 29(1):100–132. 10.1016/S0734-189X(85)90153-7
Zhang YJ: A survey on evaluation methods for image segmentation. Pattern Recognition 1996, 29(8):1335–1346. 10.1016/0031-3203(95)00169-7
Nasab NM, Analoui M, Delp EJ: Robust and efficient image segmentation approaches using Markov random field models. Journal of Electronic Imaging 2003, 12(1):50–58. 10.1117/1.1525280
Baddeley AJ: An error metric for binary images. In Robust Computer Vision. Wichmann, Karlsruhe, Germany; 1992:59–78.
Vinet L: Segmentation et mise en correspondance de régions de paires d'images stéréoscopiques, M.S. thesis. Université de Paris IX Dauphine, Paris, France; 1991.
Huttenlocher DP, Rucklidge WJ: Multi-resolution technique for comparing images using the Hausdorff distance. Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR '93), June 1993, New York, NY, USA 705–706.
Martin D, Fowlkes C, Tal D, Malik J: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of the IEEE International Conference on Computer Vision (ICCV '01), July 2001, Vancouver, BC, Canada 2: 416–423.
Weszka JS, Rosenfeld A: Threshold evaluation techniques. IEEE Transactions on Systems, Man and Cybernetics 1978, 8(8):622–629.
Levine MD, Nazif AM: Dynamic measurement of computer generated image segmentations. IEEE Transactions on Pattern Analysis and Machine Intelligence 1985, 7(2):155–164.
Sezgin M, Sankur B: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 2004, 13(1):146–168. 10.1117/1.1631315
Cochran WG:Some methods for strengthening the common tests. Biometrics 1954, 10: 417–451. 10.2307/3001616
Pal NR, Pal SK: Entropic thresholding . Signal Processing 1989, 16(2):97–108. 10.1016/0165-1684(89)90090-X
Rosenberger C: Mise en oeuvre d'un système adaptatif de segmentation d'images, M.S. thesis. Université de Rennes 1, Rennes, France; 1999.
Borsotti M, Campadelli P, Schettini R: Quantitative evaluation of color image segmentation results. Pattern Recognition Letters 1998, 19(8):741–747. 10.1016/S0167-8655(98)00052-X
Liu J, Yang Y-H: Multiresolution color image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 1994, 16(7):689–700. 10.1109/34.297949
Zeboudj R: Filtrage, seuillage automatique, contraste et contours: du pré-traitement à l'analyse d'image, M.S. thesis. Université de Saint Etienne, Saint Etienne, France; 1988.
Chabrier S, Rosenberger C, Laurent H, Emile B, Marché P: Evaluating the segmentation result of a gray-level image. Proceedings of 12th European Signal Processing Conference (EUSIPCO '04), September 2004, Vienna, Austria 953–956.
Chabrier S, Emile B, Laurent H, Rosenberger C, Marché P: Unsupervised evaluation of image segmentation application to multi-spectral images. Proceedings of International Conference on Pattern Recognition (ICPR '04), August 2004, Cambridge, UK 1: 576–579.
Krishnapuram R, Keller JM:Possibilistic-means algorithm: insights and recommendations. IEEE Transactions on Fuzzy Systems 1996, 4(3):385–393. 10.1109/91.531779
Comaniciu D, Meer P: Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(5):603–619. 10.1109/34.1000236
Monawer HA: Image vector quantization using a modified LBG algorithm with approximated centroids. Electronics Letters 1995, 31(3):174–175. 10.1049/el:19950100
Erdem ÇE, Sankur B, Tekalp AM: Performance measures for video object segmentation and tracking. IEEE Transactions on Image Processing 2004, 13(7):937–951. 10.1109/TIP.2004.828427
Nazif AM, Levine MD: Low level image segmentation: an expert system. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6(5):555–577.
Krishnapuram R, Keller JM: Possibilistic approach to clustering. IEEE Transactions on Fuzzy Systems 1993, 1(2):98–110. 10.1109/91.227387
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Chabrier, S., Emile, B., Rosenberger, C. et al. Unsupervised Performance Evaluation of Image Segmentation. EURASIP J. Adv. Signal Process. 2006, 096306 (2006). https://doi.org/10.1155/ASP/2006/96306
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1155/ASP/2006/96306