Skip to main content

Distance Measures for Image Segmentation Evaluation


The task considered in this paper is performance evaluation of region segmentation algorithms in the ground-truth-based paradigm. Given a machine segmentation and a ground-truth segmentation, performance measures are needed. We propose to consider the image segmentation problem as one of data clustering and, as a consequence, to use measures for comparing clusterings developed in statistics and machine learning. By doing so, we obtain a variety of performance measures which have not been used before in image processing. In particular, some of these measures have the highly desired property of being a metric. Experimental results are reported on both synthetic and real data to validate the measures and compare them with others.


  1. 1.

    Jiang X: Performance evaluation of image segmentation algorithms. In Handbook of Pattern Recognition and Computer Vision. 3rd edition. Edited by: Chen CH, Wang PSP. World Scientific, Singapore; 2005:525–542.

    Google Scholar 

  2. 2.

    Jiang X, Mojon D: Supervised evaluation methodology for curvilinear structure detection algorithms. Proceedings of 16th International Conference on Pattern Recognition (ICPR~'02), August 2002, Quebec, Canada 1: 103–106.

    Article  Google Scholar 

  3. 3.

    Prieto MS, Allen AR: A similarity metric for edge images. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003, 25(10):1265–1273. 10.1109/TPAMI.2003.1233900

    Article  Google Scholar 

  4. 4.

    Sezgin M, Sankur B: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 2004, 13(1):146–165. 10.1117/1.1631315

    Article  Google Scholar 

  5. 5.

    Jiang X, Marti C, Irniger C, Bunke H: Image segmentation evaluation by techniques of comparing clusterings. Proceedings of 13th International Conference on Image Analysis and Processing (ICIAP '05), September 2005, Cagliari, Italy

    Google Scholar 

  6. 6.

    Hoover A, Jean-Baptiste G, Jiang X, et al.: An experimental comparison of range image segmentation algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 1996, 18(7):673–689. 10.1109/34.506791

    Article  Google Scholar 

  7. 7.

    Chang KI, Bowyer KW, Sivagurunath M: Evaluation of texture segmentation algorithms. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '99), June 1999, Fort Collins, Colo, USA 1: 294–299.

    Article  Google Scholar 

  8. 8.

    Jiang X: An adaptive contour closure algorithm and its experimental evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000, 22(11):1252–1265. 10.1109/34.888710

    Article  Google Scholar 

  9. 9.

    Jiang X, Bowyer KW, Morioka Y, et al.: Some further results of experimental comparison of range image segmentation algorithms. Proceedings of 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 4: 877–881.

    Article  Google Scholar 

  10. 10.

    Min J, Powell MW, Bowyer KW: Automated performance evaluation of range image segmentation algorithms. IEEE Transactions on Systems, Man and Cybernetics—Part B: Cybernetics 2004, 34(1):263–271. 10.1109/TSMCB.2003.811118

    Article  Google Scholar 

  11. 11.

    Powell MW, Bowyer KW, Jiang X, Bunke H: Comparing curved-surface range image segmenters. Proceedings of 6th IEEE International Conference on Computer Vision (ICCV '98), January 1998, Bombay, India 286–291.

    Google Scholar 

  12. 12.

    Huang Q, Dom B: Quantitative methods of evaluating image segmentation. Proceedings of International Conference on Image Processing (ICIP '95), October 1995, Washington, DC, USA 3: 53–56.

    Article  Google Scholar 

  13. 13.

    Freixenet J, Muñoz X, Raba D, Martí J, Cufí X: Yet another survey on image segmentation: region and boundary information integration. Proceedings of 7th European Conference on Computer Vision-Part III (ECCV '02), May 2002, Copenhagen, Denmark 408–422.

    Google Scholar 

  14. 14.

    Martin D, Fowlkes C, Tal D, Malik J: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of 8th IEEE International Conference on Computer Vision (ICCV '01), July 2001, Vancouver, BC, Canada 2: 416–423.

    Google Scholar 

  15. 15.

    Jain AK, Murty MN, Flynn PJ: Data clustering: a review. ACM Computing Surveys (CSUR) 1999, 31(3):264–323. 10.1145/331499.331504

    Article  Google Scholar 

  16. 16.

    Rand WM: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 1971, 66(336):846–850. 10.2307/2284239

    Article  Google Scholar 

  17. 17.

    Fowlkes EB, Mallows CL: A Method for comparing two hierarchical clusterings. Journal of the American Statistical Association 1983, 78(383):553–569. 10.2307/2288117

    Article  Google Scholar 

  18. 18.

    Ben-Hur A, Elisseeff A, Guyon I: A stability based method for discovering structure in clustered data. Proceedings of 7th Pacific Symposium on Biocomputing (PSB '02), January 2002, Lihue, Hawaii, USA 7: 6–17.

    Google Scholar 

  19. 19.

    van Dongen S: Performance criteria for graph clustering and Markov cluster experiments. In Tech. Rep. INS-R0012. Centrum voor Wiskunde en Informatica (CWI), Amsterdam, The Netherlands; 2000.

    Google Scholar 

  20. 20.

    Khuller S, Raghavachari B: Advanced combinatorial algorithms. In Algorithms and Theory of Computation Handbook. Edited by: Atallah MJ. CRC Press, Boca Raton, Fla, USA; 1999:1–23. chapter 7

    Google Scholar 

  21. 21.

    Cover TM, Thomas JA: Elements of Information Theory. John Wiley & Sons, Chichester, UK; 1991.

    Google Scholar 

  22. 22.

    Strehl A, Ghosh J, Mooney R: Impact of similarity measures on web-page clustering. Proceedings of 17th National Conference on Artificial Intelligence: Workshop of Artificial Intelligence for Web Search (AAAI '00), July 2000, Austin, Tex, USA 58–64.

    Google Scholar 

  23. 23.

    Meila M: Comparing clusterings by the variation of information. Proceedings of 16th Annual Conference on Computational Learning Theory and 7th Workshop on Kernel Machines (COLT/Kernel '03), August 2003, Washington, DC, USA 173–187.

    Google Scholar 

  24. 24.

    Cinque L, Levialdi S, Pignalberi G, Cucchiara R, Martinz S: Optimal range segmentation parameters through genetic algorithms. Proceedings of 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 474–477.

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Xiaoyi Jiang.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Jiang, X., Marti, C., Irniger, C. et al. Distance Measures for Image Segmentation Evaluation. EURASIP J. Adv. Signal Process. 2006, 035909 (2006).

Download citation


  • Image Processing
  • Information Technology
  • Machine Learning
  • Performance Evaluation
  • Real Data