Skip to main content


Unsupervised Performance Evaluation of Image Segmentation


We present in this paper a study of unsupervised evaluation criteria that enable the quantification of the quality of an image segmentation result. These evaluation criteria compute some statistics for each region or class in a segmentation result. Such an evaluation criterion can be useful for different applications: the comparison of segmentation results, the automatic choice of the best fitted parameters of a segmentation method for a given image, or the definition of new segmentation methods by optimization. We first present the state of art of unsupervised evaluation, and then, we compare six unsupervised evaluation criteria. For this comparative study, we use a database composed of 8400 synthetic gray-level images segmented in four different ways. Vinet's measure (correct classification rate) is used as an objective criterion to compare the behavior of the different criteria. Finally, we present the experimental results on the segmentation evaluation of a few gray-level natural images.


  1. 1.

    Freixenet J, Muñoz X, Raba D, Marti J, Cufi X: Yet another survey on image segmentation: region and boundary information integration. Proceedings of the European Conference on Computer Vision (ECCV '02), May 2002, Copenhagen, Denmark 408–422.

  2. 2.

    Haralick RM, Shapiro LG: Image segmentation techniques. Computer Vision, Graphics, & Image Processing 1985, 29(1):100–132. 10.1016/S0734-189X(85)90153-7

  3. 3.

    Zhang YJ: A survey on evaluation methods for image segmentation. Pattern Recognition 1996, 29(8):1335–1346. 10.1016/0031-3203(95)00169-7

  4. 4.

    Nasab NM, Analoui M, Delp EJ: Robust and efficient image segmentation approaches using Markov random field models. Journal of Electronic Imaging 2003, 12(1):50–58. 10.1117/1.1525280

  5. 5.

    Baddeley AJ: An error metric for binary images. In Robust Computer Vision. Wichmann, Karlsruhe, Germany; 1992:59–78.

  6. 6.

    Vinet L: Segmentation et mise en correspondance de régions de paires d'images stéréoscopiques, M.S. thesis. Université de Paris IX Dauphine, Paris, France; 1991.

  7. 7.

    Huttenlocher DP, Rucklidge WJ: Multi-resolution technique for comparing images using the Hausdorff distance. Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR '93), June 1993, New York, NY, USA 705–706.

  8. 8.

    Martin D, Fowlkes C, Tal D, Malik J: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of the IEEE International Conference on Computer Vision (ICCV '01), July 2001, Vancouver, BC, Canada 2: 416–423.

  9. 9.

    Weszka JS, Rosenfeld A: Threshold evaluation techniques. IEEE Transactions on Systems, Man and Cybernetics 1978, 8(8):622–629.

  10. 10.

    Levine MD, Nazif AM: Dynamic measurement of computer generated image segmentations. IEEE Transactions on Pattern Analysis and Machine Intelligence 1985, 7(2):155–164.

  11. 11.

    Sezgin M, Sankur B: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 2004, 13(1):146–168. 10.1117/1.1631315

  12. 12.

    Cochran WG:Some methods for strengthening the common tests. Biometrics 1954, 10: 417–451. 10.2307/3001616

  13. 13.

    Pal NR, Pal SK: Entropic thresholding . Signal Processing 1989, 16(2):97–108. 10.1016/0165-1684(89)90090-X

  14. 14.

    Rosenberger C: Mise en oeuvre d'un système adaptatif de segmentation d'images, M.S. thesis. Université de Rennes 1, Rennes, France; 1999.

  15. 15.

    Borsotti M, Campadelli P, Schettini R: Quantitative evaluation of color image segmentation results. Pattern Recognition Letters 1998, 19(8):741–747. 10.1016/S0167-8655(98)00052-X

  16. 16.

    Liu J, Yang Y-H: Multiresolution color image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 1994, 16(7):689–700. 10.1109/34.297949

  17. 17.

    Zeboudj R: Filtrage, seuillage automatique, contraste et contours: du pré-traitement à l'analyse d'image, M.S. thesis. Université de Saint Etienne, Saint Etienne, France; 1988.

  18. 18.

    Chabrier S, Rosenberger C, Laurent H, Emile B, Marché P: Evaluating the segmentation result of a gray-level image. Proceedings of 12th European Signal Processing Conference (EUSIPCO '04), September 2004, Vienna, Austria 953–956.

  19. 19.

    Chabrier S, Emile B, Laurent H, Rosenberger C, Marché P: Unsupervised evaluation of image segmentation application to multi-spectral images. Proceedings of International Conference on Pattern Recognition (ICPR '04), August 2004, Cambridge, UK 1: 576–579.

  20. 20.

    Krishnapuram R, Keller JM:Possibilistic-means algorithm: insights and recommendations. IEEE Transactions on Fuzzy Systems 1996, 4(3):385–393. 10.1109/91.531779

  21. 21.

    Comaniciu D, Meer P: Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(5):603–619. 10.1109/34.1000236

  22. 22.

    Monawer HA: Image vector quantization using a modified LBG algorithm with approximated centroids. Electronics Letters 1995, 31(3):174–175. 10.1049/el:19950100

  23. 23.

    Erdem ÇE, Sankur B, Tekalp AM: Performance measures for video object segmentation and tracking. IEEE Transactions on Image Processing 2004, 13(7):937–951. 10.1109/TIP.2004.828427

  24. 24.

    Nazif AM, Levine MD: Low level image segmentation: an expert system. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6(5):555–577.

  25. 25.

    Krishnapuram R, Keller JM: Possibilistic approach to clustering. IEEE Transactions on Fuzzy Systems 1993, 1(2):98–110. 10.1109/91.227387

Download references

Author information

Correspondence to Sebastien Chabrier.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Chabrier, S., Emile, B., Rosenberger, C. et al. Unsupervised Performance Evaluation of Image Segmentation. EURASIP J. Adv. Signal Process. 2006, 096306 (2006).

Download citation


  • Information Technology
  • Performance Evaluation
  • Evaluation Criterion
  • Quantum Information
  • Image Segmentation