Open Access

Distance Measures for Image Segmentation Evaluation

  • Xiaoyi Jiang1Email author,
  • Cyril Marti2,
  • Christophe Irniger2 and
  • Horst Bunke2
EURASIP Journal on Advances in Signal Processing20062006:035909

https://doi.org/10.1155/ASP/2006/35909

Received: 17 March 2005

Accepted: 31 July 2005

Published: 18 March 2006

Abstract

The task considered in this paper is performance evaluation of region segmentation algorithms in the ground-truth-based paradigm. Given a machine segmentation and a ground-truth segmentation, performance measures are needed. We propose to consider the image segmentation problem as one of data clustering and, as a consequence, to use measures for comparing clusterings developed in statistics and machine learning. By doing so, we obtain a variety of performance measures which have not been used before in image processing. In particular, some of these measures have the highly desired property of being a metric. Experimental results are reported on both synthetic and real data to validate the measures and compare them with others.

[123456789101112131415161718192021222324]

Authors’ Affiliations

(1)
Computer Vision and Pattern Recognition Group, Department of Computer Science, University of Münster
(2)
Institute of Computer Science and Applied Mathematics, University of Bern

References

  1. Jiang X: Performance evaluation of image segmentation algorithms. In Handbook of Pattern Recognition and Computer Vision. 3rd edition. Edited by: Chen CH, Wang PSP. World Scientific, Singapore; 2005:525-542.View ArticleGoogle Scholar
  2. Jiang X, Mojon D: Supervised evaluation methodology for curvilinear structure detection algorithms. Proceedings of 16th International Conference on Pattern Recognition (ICPR~'02), August 2002, Quebec, Canada 1: 103-106.Google Scholar
  3. Prieto MS, Allen AR: A similarity metric for edge images. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003, 25(10):1265-1273. 10.1109/TPAMI.2003.1233900View ArticleGoogle Scholar
  4. Sezgin M, Sankur B: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 2004, 13(1):146-165. 10.1117/1.1631315View ArticleGoogle Scholar
  5. Jiang X, Marti C, Irniger C, Bunke H: Image segmentation evaluation by techniques of comparing clusterings. Proceedings of 13th International Conference on Image Analysis and Processing (ICIAP '05), September 2005, Cagliari, ItalyGoogle Scholar
  6. Hoover A, Jean-Baptiste G, Jiang X, et al.: An experimental comparison of range image segmentation algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 1996, 18(7):673-689. 10.1109/34.506791View ArticleGoogle Scholar
  7. Chang KI, Bowyer KW, Sivagurunath M: Evaluation of texture segmentation algorithms. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '99), June 1999, Fort Collins, Colo, USA 1: 294-299.View ArticleGoogle Scholar
  8. Jiang X: An adaptive contour closure algorithm and its experimental evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000, 22(11):1252-1265. 10.1109/34.888710View ArticleGoogle Scholar
  9. Jiang X, Bowyer KW, Morioka Y, et al.: Some further results of experimental comparison of range image segmentation algorithms. Proceedings of 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 4: 877-881.View ArticleGoogle Scholar
  10. Min J, Powell MW, Bowyer KW: Automated performance evaluation of range image segmentation algorithms. IEEE Transactions on Systems, Man and Cybernetics—Part B: Cybernetics 2004, 34(1):263-271. 10.1109/TSMCB.2003.811118View ArticleGoogle Scholar
  11. Powell MW, Bowyer KW, Jiang X, Bunke H: Comparing curved-surface range image segmenters. Proceedings of 6th IEEE International Conference on Computer Vision (ICCV '98), January 1998, Bombay, India 286-291.Google Scholar
  12. Huang Q, Dom B: Quantitative methods of evaluating image segmentation. Proceedings of International Conference on Image Processing (ICIP '95), October 1995, Washington, DC, USA 3: 53-56.View ArticleGoogle Scholar
  13. Freixenet J, Muñoz X, Raba D, Martí J, Cufí X: Yet another survey on image segmentation: region and boundary information integration. Proceedings of 7th European Conference on Computer Vision-Part III (ECCV '02), May 2002, Copenhagen, Denmark 408-422.Google Scholar
  14. Martin D, Fowlkes C, Tal D, Malik J: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of 8th IEEE International Conference on Computer Vision (ICCV '01), July 2001, Vancouver, BC, Canada 2: 416-423.Google Scholar
  15. Jain AK, Murty MN, Flynn PJ: Data clustering: a review. ACM Computing Surveys (CSUR) 1999, 31(3):264-323. 10.1145/331499.331504View ArticleGoogle Scholar
  16. Rand WM: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 1971, 66(336):846-850. 10.2307/2284239View ArticleGoogle Scholar
  17. Fowlkes EB, Mallows CL: A Method for comparing two hierarchical clusterings. Journal of the American Statistical Association 1983, 78(383):553-569. 10.2307/2288117View ArticleMATHGoogle Scholar
  18. Ben-Hur A, Elisseeff A, Guyon I: A stability based method for discovering structure in clustered data. Proceedings of 7th Pacific Symposium on Biocomputing (PSB '02), January 2002, Lihue, Hawaii, USA 7: 6-17.Google Scholar
  19. van Dongen S: Performance criteria for graph clustering and Markov cluster experiments. In Tech. Rep. INS-R0012. Centrum voor Wiskunde en Informatica (CWI), Amsterdam, The Netherlands; 2000.Google Scholar
  20. Khuller S, Raghavachari B: Advanced combinatorial algorithms. In Algorithms and Theory of Computation Handbook. Edited by: Atallah MJ. CRC Press, Boca Raton, Fla, USA; 1999:1-23. chapter 7Google Scholar
  21. Cover TM, Thomas JA: Elements of Information Theory. John Wiley & Sons, Chichester, UK; 1991.View ArticleMATHGoogle Scholar
  22. Strehl A, Ghosh J, Mooney R: Impact of similarity measures on web-page clustering. Proceedings of 17th National Conference on Artificial Intelligence: Workshop of Artificial Intelligence for Web Search (AAAI '00), July 2000, Austin, Tex, USA 58-64.Google Scholar
  23. Meila M: Comparing clusterings by the variation of information. Proceedings of 16th Annual Conference on Computational Learning Theory and 7th Workshop on Kernel Machines (COLT/Kernel '03), August 2003, Washington, DC, USA 173-187.View ArticleGoogle Scholar
  24. Cinque L, Levialdi S, Pignalberi G, Cucchiara R, Martinz S: Optimal range segmentation parameters through genetic algorithms. Proceedings of 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 474-477.View ArticleGoogle Scholar

Copyright

© Jiang et al. 2006