Skip to main content
EURASIP Journal on Advances in Signal Processing
Download PDF
  • Research Article
  • Open access
  • Published:

Distance Measures for Image Segmentation Evaluation

EURASIP Journal on Advances in Signal Processing volume 2006, Article number: 035909 (2006) Cite this article

Abstract

The task considered in this paper is performance evaluation of region segmentation algorithms in the ground-truth-based paradigm. Given a machine segmentation and a ground-truth segmentation, performance measures are needed. We propose to consider the image segmentation problem as one of data clustering and, as a consequence, to use measures for comparing clusterings developed in statistics and machine learning. By doing so, we obtain a variety of performance measures which have not been used before in image processing. In particular, some of these measures have the highly desired property of being a metric. Experimental results are reported on both synthetic and real data to validate the measures and compare them with others.

References

  1. Jiang X: Performance evaluation of image segmentation algorithms. In Handbook of Pattern Recognition and Computer Vision. 3rd edition. Edited by: Chen CH, Wang PSP. World Scientific, Singapore; 2005:525–542.

    Chapter  Google Scholar 

  2. Jiang X, Mojon D: Supervised evaluation methodology for curvilinear structure detection algorithms. Proceedings of 16th International Conference on Pattern Recognition (ICPR~'02), August 2002, Quebec, Canada 1: 103–106.

    Article  Google Scholar 

  3. Prieto MS, Allen AR: A similarity metric for edge images. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003, 25(10):1265–1273. 10.1109/TPAMI.2003.1233900

    Article  Google Scholar 

  4. Sezgin M, Sankur B: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 2004, 13(1):146–165. 10.1117/1.1631315

    Article  Google Scholar 

  5. Jiang X, Marti C, Irniger C, Bunke H: Image segmentation evaluation by techniques of comparing clusterings. Proceedings of 13th International Conference on Image Analysis and Processing (ICIAP '05), September 2005, Cagliari, Italy

    Google Scholar 

  6. Hoover A, Jean-Baptiste G, Jiang X, et al.: An experimental comparison of range image segmentation algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 1996, 18(7):673–689. 10.1109/34.506791

    Article  Google Scholar 

  7. Chang KI, Bowyer KW, Sivagurunath M: Evaluation of texture segmentation algorithms. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '99), June 1999, Fort Collins, Colo, USA 1: 294–299.

    Article  Google Scholar 

  8. Jiang X: An adaptive contour closure algorithm and its experimental evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000, 22(11):1252–1265. 10.1109/34.888710

    Article  Google Scholar 

  9. Jiang X, Bowyer KW, Morioka Y, et al.: Some further results of experimental comparison of range image segmentation algorithms. Proceedings of 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 4: 877–881.

    Article  Google Scholar 

  10. Min J, Powell MW, Bowyer KW: Automated performance evaluation of range image segmentation algorithms. IEEE Transactions on Systems, Man and Cybernetics—Part B: Cybernetics 2004, 34(1):263–271. 10.1109/TSMCB.2003.811118

    Article  Google Scholar 

  11. Powell MW, Bowyer KW, Jiang X, Bunke H: Comparing curved-surface range image segmenters. Proceedings of 6th IEEE International Conference on Computer Vision (ICCV '98), January 1998, Bombay, India 286–291.

    Google Scholar 

  12. Huang Q, Dom B: Quantitative methods of evaluating image segmentation. Proceedings of International Conference on Image Processing (ICIP '95), October 1995, Washington, DC, USA 3: 53–56.

    Article  Google Scholar 

  13. Freixenet J, Muñoz X, Raba D, Martí J, Cufí X: Yet another survey on image segmentation: region and boundary information integration. Proceedings of 7th European Conference on Computer Vision-Part III (ECCV '02), May 2002, Copenhagen, Denmark 408–422.

    MATH  Google Scholar 

  14. Martin D, Fowlkes C, Tal D, Malik J: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of 8th IEEE International Conference on Computer Vision (ICCV '01), July 2001, Vancouver, BC, Canada 2: 416–423.

    Google Scholar 

  15. Jain AK, Murty MN, Flynn PJ: Data clustering: a review. ACM Computing Surveys (CSUR) 1999, 31(3):264–323. 10.1145/331499.331504

    Article  Google Scholar 

  16. Rand WM: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 1971, 66(336):846–850. 10.2307/2284239

    Article  Google Scholar 

  17. Fowlkes EB, Mallows CL: A Method for comparing two hierarchical clusterings. Journal of the American Statistical Association 1983, 78(383):553–569. 10.2307/2288117

    Article  Google Scholar 

  18. Ben-Hur A, Elisseeff A, Guyon I: A stability based method for discovering structure in clustered data. Proceedings of 7th Pacific Symposium on Biocomputing (PSB '02), January 2002, Lihue, Hawaii, USA 7: 6–17.

    Google Scholar 

  19. van Dongen S: Performance criteria for graph clustering and Markov cluster experiments. In Tech. Rep. INS-R0012. Centrum voor Wiskunde en Informatica (CWI), Amsterdam, The Netherlands; 2000.

    Google Scholar 

  20. Khuller S, Raghavachari B: Advanced combinatorial algorithms. In Algorithms and Theory of Computation Handbook. Edited by: Atallah MJ. CRC Press, Boca Raton, Fla, USA; 1999:1–23. chapter 7

    MATH  Google Scholar 

  21. Cover TM, Thomas JA: Elements of Information Theory. John Wiley & Sons, Chichester, UK; 1991.

    Book  Google Scholar 

  22. Strehl A, Ghosh J, Mooney R: Impact of similarity measures on web-page clustering. Proceedings of 17th National Conference on Artificial Intelligence: Workshop of Artificial Intelligence for Web Search (AAAI '00), July 2000, Austin, Tex, USA 58–64.

    Google Scholar 

  23. Meila M: Comparing clusterings by the variation of information. Proceedings of 16th Annual Conference on Computational Learning Theory and 7th Workshop on Kernel Machines (COLT/Kernel '03), August 2003, Washington, DC, USA 173–187.

    Chapter  Google Scholar 

  24. Cinque L, Levialdi S, Pignalberi G, Cucchiara R, Martinz S: Optimal range segmentation parameters through genetic algorithms. Proceedings of 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 474–477.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Vision and Pattern Recognition Group, Department of Computer Science, University of Münster, Einsteinstrasse 62, Münster, D-48149, Germany

    Xiaoyi Jiang

  2. Institute of Computer Science and Applied Mathematics, University of Bern, Neubrückstrasse 10, Bern, CH-3012, Switzerland

    Cyril Marti, Christophe Irniger & Horst Bunke

Authors
  1. Xiaoyi Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cyril Marti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christophe Irniger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Horst Bunke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoyi Jiang.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Jiang, X., Marti, C., Irniger, C. et al. Distance Measures for Image Segmentation Evaluation. EURASIP J. Adv. Signal Process. 2006, 035909 (2006). https://doi.org/10.1155/ASP/2006/35909

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/ASP/2006/35909

Keywords