Skip to content


  • Research Article
  • Open Access

Unsupervised Performance Evaluation of Image Segmentation

  • 1Email author,
  • 1,
  • 1 and
  • 1
EURASIP Journal on Advances in Signal Processing20062006:096306

  • Received: 1 March 2005
  • Accepted: 21 January 2006
  • Published:


We present in this paper a study of unsupervised evaluation criteria that enable the quantification of the quality of an image segmentation result. These evaluation criteria compute some statistics for each region or class in a segmentation result. Such an evaluation criterion can be useful for different applications: the comparison of segmentation results, the automatic choice of the best fitted parameters of a segmentation method for a given image, or the definition of new segmentation methods by optimization. We first present the state of art of unsupervised evaluation, and then, we compare six unsupervised evaluation criteria. For this comparative study, we use a database composed of 8400 synthetic gray-level images segmented in four different ways. Vinet's measure (correct classification rate) is used as an objective criterion to compare the behavior of the different criteria. Finally, we present the experimental results on the segmentation evaluation of a few gray-level natural images.


  • Information Technology
  • Performance Evaluation
  • Evaluation Criterion
  • Quantum Information
  • Image Segmentation

Authors’ Affiliations

Laboratoire Vision et Robotique, UPRES EA 2078,ENSI de Bourges, Universit$#233; d' Orl$#233;ans, 10 boulevard Lahitolle, Bourges cedex, 18020, France


  1. Freixenet J, Muñoz X, Raba D, Marti J, Cufi X: Yet another survey on image segmentation: region and boundary information integration. Proceedings of the European Conference on Computer Vision (ECCV '02), May 2002, Copenhagen, Denmark 408–422.MATHGoogle Scholar
  2. Haralick RM, Shapiro LG: Image segmentation techniques. Computer Vision, Graphics, & Image Processing 1985, 29(1):100–132. 10.1016/S0734-189X(85)90153-7View ArticleGoogle Scholar
  3. Zhang YJ: A survey on evaluation methods for image segmentation. Pattern Recognition 1996, 29(8):1335–1346. 10.1016/0031-3203(95)00169-7View ArticleGoogle Scholar
  4. Nasab NM, Analoui M, Delp EJ: Robust and efficient image segmentation approaches using Markov random field models. Journal of Electronic Imaging 2003, 12(1):50–58. 10.1117/1.1525280View ArticleGoogle Scholar
  5. Baddeley AJ: An error metric for binary images. In Robust Computer Vision. Wichmann, Karlsruhe, Germany; 1992:59–78.Google Scholar
  6. Vinet L: Segmentation et mise en correspondance de régions de paires d'images stéréoscopiques, M.S. thesis. Université de Paris IX Dauphine, Paris, France; 1991.Google Scholar
  7. Huttenlocher DP, Rucklidge WJ: Multi-resolution technique for comparing images using the Hausdorff distance. Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR '93), June 1993, New York, NY, USA 705–706.View ArticleGoogle Scholar
  8. Martin D, Fowlkes C, Tal D, Malik J: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of the IEEE International Conference on Computer Vision (ICCV '01), July 2001, Vancouver, BC, Canada 2: 416–423.Google Scholar
  9. Weszka JS, Rosenfeld A: Threshold evaluation techniques. IEEE Transactions on Systems, Man and Cybernetics 1978, 8(8):622–629.View ArticleGoogle Scholar
  10. Levine MD, Nazif AM: Dynamic measurement of computer generated image segmentations. IEEE Transactions on Pattern Analysis and Machine Intelligence 1985, 7(2):155–164.View ArticleGoogle Scholar
  11. Sezgin M, Sankur B: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 2004, 13(1):146–168. 10.1117/1.1631315View ArticleGoogle Scholar
  12. Cochran WG:Some methods for strengthening the common tests. Biometrics 1954, 10: 417–451. 10.2307/3001616MathSciNetMATHGoogle Scholar
  13. Pal NR, Pal SK: Entropic thresholding . Signal Processing 1989, 16(2):97–108. 10.1016/0165-1684(89)90090-XMathSciNetView ArticleGoogle Scholar
  14. Rosenberger C: Mise en oeuvre d'un système adaptatif de segmentation d'images, M.S. thesis. Université de Rennes 1, Rennes, France; 1999.Google Scholar
  15. Borsotti M, Campadelli P, Schettini R: Quantitative evaluation of color image segmentation results. Pattern Recognition Letters 1998, 19(8):741–747. 10.1016/S0167-8655(98)00052-XMATHView ArticleGoogle Scholar
  16. Liu J, Yang Y-H: Multiresolution color image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 1994, 16(7):689–700. 10.1109/34.297949View ArticleGoogle Scholar
  17. Zeboudj R: Filtrage, seuillage automatique, contraste et contours: du pré-traitement à l'analyse d'image, M.S. thesis. Université de Saint Etienne, Saint Etienne, France; 1988.Google Scholar
  18. Chabrier S, Rosenberger C, Laurent H, Emile B, Marché P: Evaluating the segmentation result of a gray-level image. Proceedings of 12th European Signal Processing Conference (EUSIPCO '04), September 2004, Vienna, Austria 953–956.Google Scholar
  19. Chabrier S, Emile B, Laurent H, Rosenberger C, Marché P: Unsupervised evaluation of image segmentation application to multi-spectral images. Proceedings of International Conference on Pattern Recognition (ICPR '04), August 2004, Cambridge, UK 1: 576–579.View ArticleGoogle Scholar
  20. Krishnapuram R, Keller JM:Possibilistic -means algorithm: insights and recommendations. IEEE Transactions on Fuzzy Systems 1996, 4(3):385–393. 10.1109/91.531779Google Scholar
  21. Comaniciu D, Meer P: Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(5):603–619. 10.1109/34.1000236View ArticleGoogle Scholar
  22. Monawer HA: Image vector quantization using a modified LBG algorithm with approximated centroids. Electronics Letters 1995, 31(3):174–175. 10.1049/el:19950100View ArticleGoogle Scholar
  23. Erdem ÇE, Sankur B, Tekalp AM: Performance measures for video object segmentation and tracking. IEEE Transactions on Image Processing 2004, 13(7):937–951. 10.1109/TIP.2004.828427View ArticleGoogle Scholar
  24. Nazif AM, Levine MD: Low level image segmentation: an expert system. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6(5):555–577.View ArticleGoogle Scholar
  25. Krishnapuram R, Keller JM: Possibilistic approach to clustering. IEEE Transactions on Fuzzy Systems 1993, 1(2):98–110. 10.1109/91.227387View ArticleGoogle Scholar


© Chabrier et al. 2006