Open Access

Combining Global and Local Information for Knowledge-Assisted Image Analysis and Classification

  • G. Th. Papadopoulos1, 2Email author,
  • V. Mezaris2,
  • I. Kompatsiaris2 and
  • M. G. Strintzis1, 2
EURASIP Journal on Advances in Signal Processing20072007:045842

https://doi.org/10.1155/2007/45842

Received: 8 September 2006

Accepted: 2 April 2007

Published: 10 July 2007

Abstract

A learning approach to knowledge-assisted image analysis and classification is proposed that combines global and local information with explicitly defined knowledge in the form of an ontology. The ontology specifies the domain of interest, its subdomains, the concepts related to each subdomain as well as contextual information. Support vector machines (SVMs) are employed in order to provide image classification to the ontology subdomains based on global image descriptions. In parallel, a segmentation algorithm is applied to segment the image into regions and SVMs are again employed, this time for performing an initial mapping between region low-level visual features and the concepts in the ontology. Then, a decision function, that receives as input the computed region-concept associations together with contextual information in the form of concept frequency of appearance, realizes image classification based on local information. A fusion mechanism subsequently combines the intermediate classification results, provided by the local- and global-level information processing, to decide on the final image classification. Once the image subdomain is selected, final region-concept association is performed using again SVMs and a genetic algorithm (GA) for optimizing the mapping between the image regions and the selected subdomain concepts taking into account contextual information in the form of spatial relations. Application of the proposed approach to images of the selected domain results in their classification (i.e., their assignment to one of the defined subdomains) and the generation of a fine granularity semantic representation of them (i.e., a segmentation map with semantic concepts attached to each segment). Experiments with images from the personal collection domain, as well as comparative evaluation with other approaches of the literature, demonstrate the performance of the proposed approach.

[1234567891011121314151617181920212223242526272829303132]

Authors’ Affiliations

(1)
Department of Electrical and Computer Engineering, Aristotle University of Thessaloniki
(2)
Centre for Research and Technology Hellas (CERTH), Informatics and Telematics Institute

References

  1. Smeulders AWM, Worring M, Santini S, Gupta A, Jain R: Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000,22(12):1349-1380. 10.1109/34.895972View ArticleGoogle Scholar
  2. Bloehdorn S, Petridis K, Saathoff C, et al.: Semantic annotation of images and videos for multimedia analysis. Proceedings of the 2nd European Semantic Web Conference (ESWC '05), May-June 2005, Herakleion, Greece 592-607.Google Scholar
  3. Barla A, Odone F, Verri A: Old fashioned state-of-the-art image classification. Proceedings of the 12th International Conference on Image Analysis and Processing (ICIAP '03), September 2003, Mantova, Italy 566-571.Google Scholar
  4. Tang J, Zhang C-Y, Luo B: A graph and PNN-based approach to image classification. Proceedings of International Conference on Machine Learning and Cybernetics (ICMLC '05), August 2005, Guangzhou, China 8: 5122-5126.Google Scholar
  5. Tanaka H, Sakano H, Ohtsuka S: Retrieval method for multi-category images. Proceedings of the 17th International Conference on Pattern Recognition (ICPR '04), August 2004, Cambridge, UK 2: 965-968.View ArticleGoogle Scholar
  6. Ahmad I, Ibrahim MT: Image classification and retrieval using correlation. Proccedings of the 3rd Canadian Conference on Computer and Robot Vision (CRV '06), June 2006, Quebec City, Canada 60.View ArticleGoogle Scholar
  7. Papadopoulos S, Mezaris V, Kompatsiaris I, Strintzis MG: A region-based approach to conceptual image classification. Proceedings of IEE International Conference on Visual Information Engineering (VIE '05), April 2005, Glasgow, UK 141-147.View ArticleGoogle Scholar
  8. Tseng VS, Wang M-H, Su J-H: A new method for image classification by using multilevel association rules. Proceedings of the 21st International Conference on Data Engineering Workshops (ICDE '05), April 2005, Tokyo, Japan 1180.View ArticleGoogle Scholar
  9. Blaschke T: Object-based contextual image classification built on image segmentation. Proceedings of IEEE Workshop on Advances in Techniques for Analysis of Remotely Sensed Data (WARSD '03), October 2003, Washington, DC, USA 113-119.View ArticleGoogle Scholar
  10. Wang Z, Feng D, Chi Z: Region-based binary tree representation for image classification. Proceedings of International Conference on Neural Networks and Signal Processing (ICNNSP '03), December 2003, Nanjing, China 1: 232-235.Google Scholar
  11. Dasiopoulou S, Mezaris V, Kompatsiaris I, Papastathis V-K, Strintzis MG: Knowledge-assisted semantic video object detection. IEEE Transactions on Circuits and Systems for Video Technology 2005,15(10):1210-1224.View ArticleGoogle Scholar
  12. Hollink L, Little S, Hunter J: Evaluating the application of semantic inferencing rules to image annotation. Proceedings of the 3rd International Conference on Knowledge Capture (K-CAP '05), October 2005, Banff, Canada 91-98.View ArticleGoogle Scholar
  13. Maillot N, Thonnat M, Hudelot C: Ontology based object learning and recognition : application to image retrieval. Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI '04), November 2004, Boca Raton, Fla, USA 620-625.View ArticleGoogle Scholar
  14. Kompatsiaris I, Mezaris V, Strintzis MG: Multimedia content indexing and retrieval using an object ontology. In Multimedia Content and Semantic Web-Methods, Standards and Tools. John Wiley & Sons, New York, NY, USA; 2004:339-371.Google Scholar
  15. Tansley R, Bird C, Hall W, Lewis P, Weal M: Automating the linking of content and concept. Proceedings of the 8th ACM International Multimedia Conference and Exhibition (MULTIMEDIA '00), October-November 2000, Los Angeles, Calif, USA 445-447.View ArticleGoogle Scholar
  16. Kim KI, Jung K, Park SH, Kim HJ: Support vector machines for texture classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002,24(11):1542-1550. 10.1109/TPAMI.2002.1046177View ArticleGoogle Scholar
  17. Chapelle O, Haffner P, Vapnik VN: Support vector machines for histogram-based image classification. IEEE Transactions on Neural Networks 1999,10(5):1055-1064. 10.1109/72.788646View ArticleGoogle Scholar
  18. Mitchell M: An Introduction to Genetic Algorithms. MIT Press, Cambridge, Mass, USA; 1995.MATHGoogle Scholar
  19. Voisine N, Dasiopoulou S, Precioso F, Mezaris V, Kompatsiaris I, Strintzis MG: A genetic algorithm-based approach to knowledge-assisted video analysis. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy 3: 441-444.Google Scholar
  20. Papadopoulos GTh, Panagi P, Dasiopoulou S, Mezaris V, Kompatsiaris I: A learning approach to semantic image analysis. Proceedings of the 2nd International Mobile Multimedia Communications Conference (MobiMedia '06), September 2006, Alghero, ItalyGoogle Scholar
  21. MPEG-7 Visual Experimentation Model (XM) Version 10.0, ISO/IEC/JTC1/SC29/WG11, Doc. N4062, March 2001Google Scholar
  22. Adamek T, O'Connor N, Murphy N: Region-based segmentation of images using syntactic visual features. Proceedings of the 6th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS '05), April 2005, Montreux, SwitzerlandGoogle Scholar
  23. Skiadopoulos S, Giannoukos C, Sarkas N, Vassiliadis P, Sellis T, Koubarakis M: Computing and managing cardinal direction relations. IEEE Transactions on Knowledge and Data Engineering 2005,17(12):1610-1623.View ArticleGoogle Scholar
  24. Wang Y, Makedon F, Ford J, Shen L, Goldin D: Generating fuzzy semantic metadata describing spatial relations from images using the R-Histogram. Proceedings of the 4th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '04), June 2004, Tucson, Ariz, USA 202-211.Google Scholar
  25. Staab S, Studer R (Eds): Handbook on Ontologies, International Handbooks on Information Systems. Springer, Berlin, Germany; 2004.Google Scholar
  26. Tax DMJ, Duin RPW: Using two-class classifiers for multiclass classification. Proceedings of the 16th International Conference on Pattern Recognition (ICPR '02), August 2002, Quebec City, Canada 2: 124-127.Google Scholar
  27. Chang C-C, Lin C-J: LIBSVM: a library for support vector machines. 2001.http://www.csie.ntu.edu.tw/~cjlin/libsvm/Google Scholar
  28. Goldberg DE, Deb K: A comparative analysis of selection schemes used in genetic algorithms. In Foundations of Genetic Algorithms. Edited by: Rawlins G. Morgan Kaufmann Publishers, San Mateo, Calif, USA; 1991:69-93.Google Scholar
  29. Wall M: GAlib: A C++ Genetic Algorithm Library. MIT, 2005, http://lancet.mit.edu/ga/
  30. Flickr http://www.flickr.com/
  31. Ren J, Shen Y, Ma S, Guo L: Applying multi-class SVMs into scene image classification. Proceedings of the 17th International Conference on Innovations in Applied Artificial Intelligence, May 2004, Ottawa, Canada 924-934.View ArticleGoogle Scholar
  32. Spyrou E, Le Borgne H, Mailis T, Cooke E, Avrithis Y, O'Connor N: Fusing MPEG-7 visual descriptors for image classification. Proceedings of International Conference on Artificial Neural Networks (ICANN '05), September 2005, Warsaw, Poland 847-852.Google Scholar

Copyright

© G. Th. Papadopoulos et al. 2007

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.