Skip to main content

Advertisement

Distance Measures for Image Segmentation Evaluation

Abstract

The task considered in this paper is performance evaluation of region segmentation algorithms in the ground-truth-based paradigm. Given a machine segmentation and a ground-truth segmentation, performance measures are needed. We propose to consider the image segmentation problem as one of data clustering and, as a consequence, to use measures for comparing clusterings developed in statistics and machine learning. By doing so, we obtain a variety of performance measures which have not been used before in image processing. In particular, some of these measures have the highly desired property of being a metric. Experimental results are reported on both synthetic and real data to validate the measures and compare them with others.

References

  1. 1.

    Jiang X: Performance evaluation of image segmentation algorithms. In Handbook of Pattern Recognition and Computer Vision. 3rd edition. Edited by: Chen CH, Wang PSP. World Scientific, Singapore; 2005:525–542.

  2. 2.

    Jiang X, Mojon D: Supervised evaluation methodology for curvilinear structure detection algorithms. Proceedings of 16th International Conference on Pattern Recognition (ICPR~'02), August 2002, Quebec, Canada 1: 103–106.

  3. 3.

    Prieto MS, Allen AR: A similarity metric for edge images. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003, 25(10):1265–1273. 10.1109/TPAMI.2003.1233900

  4. 4.

    Sezgin M, Sankur B: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 2004, 13(1):146–165. 10.1117/1.1631315

  5. 5.

    Jiang X, Marti C, Irniger C, Bunke H: Image segmentation evaluation by techniques of comparing clusterings. Proceedings of 13th International Conference on Image Analysis and Processing (ICIAP '05), September 2005, Cagliari, Italy

  6. 6.

    Hoover A, Jean-Baptiste G, Jiang X, et al.: An experimental comparison of range image segmentation algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 1996, 18(7):673–689. 10.1109/34.506791

  7. 7.

    Chang KI, Bowyer KW, Sivagurunath M: Evaluation of texture segmentation algorithms. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '99), June 1999, Fort Collins, Colo, USA 1: 294–299.

  8. 8.

    Jiang X: An adaptive contour closure algorithm and its experimental evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000, 22(11):1252–1265. 10.1109/34.888710

  9. 9.

    Jiang X, Bowyer KW, Morioka Y, et al.: Some further results of experimental comparison of range image segmentation algorithms. Proceedings of 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 4: 877–881.

  10. 10.

    Min J, Powell MW, Bowyer KW: Automated performance evaluation of range image segmentation algorithms. IEEE Transactions on Systems, Man and Cybernetics—Part B: Cybernetics 2004, 34(1):263–271. 10.1109/TSMCB.2003.811118

  11. 11.

    Powell MW, Bowyer KW, Jiang X, Bunke H: Comparing curved-surface range image segmenters. Proceedings of 6th IEEE International Conference on Computer Vision (ICCV '98), January 1998, Bombay, India 286–291.

  12. 12.

    Huang Q, Dom B: Quantitative methods of evaluating image segmentation. Proceedings of International Conference on Image Processing (ICIP '95), October 1995, Washington, DC, USA 3: 53–56.

  13. 13.

    Freixenet J, Muñoz X, Raba D, Martí J, Cufí X: Yet another survey on image segmentation: region and boundary information integration. Proceedings of 7th European Conference on Computer Vision-Part III (ECCV '02), May 2002, Copenhagen, Denmark 408–422.

  14. 14.

    Martin D, Fowlkes C, Tal D, Malik J: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of 8th IEEE International Conference on Computer Vision (ICCV '01), July 2001, Vancouver, BC, Canada 2: 416–423.

  15. 15.

    Jain AK, Murty MN, Flynn PJ: Data clustering: a review. ACM Computing Surveys (CSUR) 1999, 31(3):264–323. 10.1145/331499.331504

  16. 16.

    Rand WM: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 1971, 66(336):846–850. 10.2307/2284239

  17. 17.

    Fowlkes EB, Mallows CL: A Method for comparing two hierarchical clusterings. Journal of the American Statistical Association 1983, 78(383):553–569. 10.2307/2288117

  18. 18.

    Ben-Hur A, Elisseeff A, Guyon I: A stability based method for discovering structure in clustered data. Proceedings of 7th Pacific Symposium on Biocomputing (PSB '02), January 2002, Lihue, Hawaii, USA 7: 6–17.

  19. 19.

    van Dongen S: Performance criteria for graph clustering and Markov cluster experiments. In Tech. Rep. INS-R0012. Centrum voor Wiskunde en Informatica (CWI), Amsterdam, The Netherlands; 2000.

  20. 20.

    Khuller S, Raghavachari B: Advanced combinatorial algorithms. In Algorithms and Theory of Computation Handbook. Edited by: Atallah MJ. CRC Press, Boca Raton, Fla, USA; 1999:1–23. chapter 7

  21. 21.

    Cover TM, Thomas JA: Elements of Information Theory. John Wiley & Sons, Chichester, UK; 1991.

  22. 22.

    Strehl A, Ghosh J, Mooney R: Impact of similarity measures on web-page clustering. Proceedings of 17th National Conference on Artificial Intelligence: Workshop of Artificial Intelligence for Web Search (AAAI '00), July 2000, Austin, Tex, USA 58–64.

  23. 23.

    Meila M: Comparing clusterings by the variation of information. Proceedings of 16th Annual Conference on Computational Learning Theory and 7th Workshop on Kernel Machines (COLT/Kernel '03), August 2003, Washington, DC, USA 173–187.

  24. 24.

    Cinque L, Levialdi S, Pignalberi G, Cucchiara R, Martinz S: Optimal range segmentation parameters through genetic algorithms. Proceedings of 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 474–477.

Download references

Author information

Correspondence to Xiaoyi Jiang.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Jiang, X., Marti, C., Irniger, C. et al. Distance Measures for Image Segmentation Evaluation. EURASIP J. Adv. Signal Process. 2006, 035909 (2006). https://doi.org/10.1155/ASP/2006/35909

Download citation

Keywords

  • Image Processing
  • Information Technology
  • Machine Learning
  • Performance Evaluation
  • Real Data