Skip to main content

Advertisement

Combining Global and Local Information for Knowledge-Assisted Image Analysis and Classification

Article metrics

  • 911 Accesses

  • 15 Citations

Abstract

A learning approach to knowledge-assisted image analysis and classification is proposed that combines global and local information with explicitly defined knowledge in the form of an ontology. The ontology specifies the domain of interest, its subdomains, the concepts related to each subdomain as well as contextual information. Support vector machines (SVMs) are employed in order to provide image classification to the ontology subdomains based on global image descriptions. In parallel, a segmentation algorithm is applied to segment the image into regions and SVMs are again employed, this time for performing an initial mapping between region low-level visual features and the concepts in the ontology. Then, a decision function, that receives as input the computed region-concept associations together with contextual information in the form of concept frequency of appearance, realizes image classification based on local information. A fusion mechanism subsequently combines the intermediate classification results, provided by the local- and global-level information processing, to decide on the final image classification. Once the image subdomain is selected, final region-concept association is performed using again SVMs and a genetic algorithm (GA) for optimizing the mapping between the image regions and the selected subdomain concepts taking into account contextual information in the form of spatial relations. Application of the proposed approach to images of the selected domain results in their classification (i.e., their assignment to one of the defined subdomains) and the generation of a fine granularity semantic representation of them (i.e., a segmentation map with semantic concepts attached to each segment). Experiments with images from the personal collection domain, as well as comparative evaluation with other approaches of the literature, demonstrate the performance of the proposed approach.

References

  1. 1.

    Smeulders AWM, Worring M, Santini S, Gupta A, Jain R: Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000,22(12):1349-1380. 10.1109/34.895972

  2. 2.

    Bloehdorn S, Petridis K, Saathoff C, et al.: Semantic annotation of images and videos for multimedia analysis. Proceedings of the 2nd European Semantic Web Conference (ESWC '05), May-June 2005, Herakleion, Greece 592–607.

  3. 3.

    Barla A, Odone F, Verri A: Old fashioned state-of-the-art image classification. Proceedings of the 12th International Conference on Image Analysis and Processing (ICIAP '03), September 2003, Mantova, Italy 566–571.

  4. 4.

    Tang J, Zhang C-Y, Luo B: A graph and PNN-based approach to image classification. Proceedings of International Conference on Machine Learning and Cybernetics (ICMLC '05), August 2005, Guangzhou, China 8: 5122–5126.

  5. 5.

    Tanaka H, Sakano H, Ohtsuka S: Retrieval method for multi-category images. Proceedings of the 17th International Conference on Pattern Recognition (ICPR '04), August 2004, Cambridge, UK 2: 965–968.

  6. 6.

    Ahmad I, Ibrahim MT: Image classification and retrieval using correlation. Proccedings of the 3rd Canadian Conference on Computer and Robot Vision (CRV '06), June 2006, Quebec City, Canada 60.

  7. 7.

    Papadopoulos S, Mezaris V, Kompatsiaris I, Strintzis MG: A region-based approach to conceptual image classification. Proceedings of IEE International Conference on Visual Information Engineering (VIE '05), April 2005, Glasgow, UK 141–147.

  8. 8.

    Tseng VS, Wang M-H, Su J-H: A new method for image classification by using multilevel association rules. Proceedings of the 21st International Conference on Data Engineering Workshops (ICDE '05), April 2005, Tokyo, Japan 1180.

  9. 9.

    Blaschke T: Object-based contextual image classification built on image segmentation. Proceedings of IEEE Workshop on Advances in Techniques for Analysis of Remotely Sensed Data (WARSD '03), October 2003, Washington, DC, USA 113–119.

  10. 10.

    Wang Z, Feng D, Chi Z: Region-based binary tree representation for image classification. Proceedings of International Conference on Neural Networks and Signal Processing (ICNNSP '03), December 2003, Nanjing, China 1: 232–235.

  11. 11.

    Dasiopoulou S, Mezaris V, Kompatsiaris I, Papastathis V-K, Strintzis MG: Knowledge-assisted semantic video object detection. IEEE Transactions on Circuits and Systems for Video Technology 2005,15(10):1210-1224.

  12. 12.

    Hollink L, Little S, Hunter J: Evaluating the application of semantic inferencing rules to image annotation. Proceedings of the 3rd International Conference on Knowledge Capture (K-CAP '05), October 2005, Banff, Canada 91–98.

  13. 13.

    Maillot N, Thonnat M, Hudelot C: Ontology based object learning and recognition : application to image retrieval. Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI '04), November 2004, Boca Raton, Fla, USA 620–625.

  14. 14.

    Kompatsiaris I, Mezaris V, Strintzis MG: Multimedia content indexing and retrieval using an object ontology. In Multimedia Content and Semantic Web-Methods, Standards and Tools. John Wiley & Sons, New York, NY, USA; 2004:339-371.

  15. 15.

    Tansley R, Bird C, Hall W, Lewis P, Weal M: Automating the linking of content and concept. Proceedings of the 8th ACM International Multimedia Conference and Exhibition (MULTIMEDIA '00), October-November 2000, Los Angeles, Calif, USA 445–447.

  16. 16.

    Kim KI, Jung K, Park SH, Kim HJ: Support vector machines for texture classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002,24(11):1542-1550. 10.1109/TPAMI.2002.1046177

  17. 17.

    Chapelle O, Haffner P, Vapnik VN: Support vector machines for histogram-based image classification. IEEE Transactions on Neural Networks 1999,10(5):1055-1064. 10.1109/72.788646

  18. 18.

    Mitchell M: An Introduction to Genetic Algorithms. MIT Press, Cambridge, Mass, USA; 1995.

  19. 19.

    Voisine N, Dasiopoulou S, Precioso F, Mezaris V, Kompatsiaris I, Strintzis MG: A genetic algorithm-based approach to knowledge-assisted video analysis. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy 3: 441–444.

  20. 20.

    Papadopoulos GTh, Panagi P, Dasiopoulou S, Mezaris V, Kompatsiaris I: A learning approach to semantic image analysis. Proceedings of the 2nd International Mobile Multimedia Communications Conference (MobiMedia '06), September 2006, Alghero, Italy

  21. 21.

    MPEG-7 Visual Experimentation Model (XM) Version 10.0, ISO/IEC/JTC1/SC29/WG11, Doc. N4062, March 2001

  22. 22.

    Adamek T, O'Connor N, Murphy N: Region-based segmentation of images using syntactic visual features. Proceedings of the 6th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS '05), April 2005, Montreux, Switzerland

  23. 23.

    Skiadopoulos S, Giannoukos C, Sarkas N, Vassiliadis P, Sellis T, Koubarakis M: Computing and managing cardinal direction relations. IEEE Transactions on Knowledge and Data Engineering 2005,17(12):1610-1623.

  24. 24.

    Wang Y, Makedon F, Ford J, Shen L, Goldin D: Generating fuzzy semantic metadata describing spatial relations from images using the R-Histogram. Proceedings of the 4th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '04), June 2004, Tucson, Ariz, USA 202–211.

  25. 25.

    Staab S, Studer R (Eds): Handbook on Ontologies, International Handbooks on Information Systems. Springer, Berlin, Germany; 2004.

  26. 26.

    Tax DMJ, Duin RPW: Using two-class classifiers for multiclass classification. Proceedings of the 16th International Conference on Pattern Recognition (ICPR '02), August 2002, Quebec City, Canada 2: 124–127.

  27. 27.

    Chang C-C, Lin C-J: LIBSVM: a library for support vector machines. 2001.https://doi.org/www.csie.ntu.edu.tw/~cjlin/libsvm/

  28. 28.

    Goldberg DE, Deb K: A comparative analysis of selection schemes used in genetic algorithms. In Foundations of Genetic Algorithms. Edited by: Rawlins G. Morgan Kaufmann Publishers, San Mateo, Calif, USA; 1991:69-93.

  29. 29.

    Wall M: GAlib: A C++ Genetic Algorithm Library. MIT, 2005, https://doi.org/lancet.mit.edu/ga/

  30. 30.

    Flickr https://doi.org/www.flickr.com/

  31. 31.

    Ren J, Shen Y, Ma S, Guo L: Applying multi-class SVMs into scene image classification. Proceedings of the 17th International Conference on Innovations in Applied Artificial Intelligence, May 2004, Ottawa, Canada 924–934.

  32. 32.

    Spyrou E, Le Borgne H, Mailis T, Cooke E, Avrithis Y, O'Connor N: Fusing MPEG-7 visual descriptors for image classification. Proceedings of International Conference on Artificial Neural Networks (ICANN '05), September 2005, Warsaw, Poland 847–852.

Download references

Author information

Correspondence to G. Th. Papadopoulos.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://doi.org/creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Papadopoulos, G.T., Mezaris, V., Kompatsiaris, I. et al. Combining Global and Local Information for Knowledge-Assisted Image Analysis and Classification. EURASIP J. Adv. Signal Process. 2007, 045842 (2007) doi:10.1155/2007/45842

Download citation

Keywords

  • Genetic Algorithm
  • Support Vector Machine
  • Local Information
  • Contextual Information
  • Image Classification