Skip to content


  • Research Article
  • Open Access

Video Object Relevance Metrics for Overall Segmentation Quality Evaluation

EURASIP Journal on Advances in Signal Processing20062006:082195

  • Received: 28 February 2005
  • Accepted: 31 July 2005
  • Published:


Video object segmentation is a task that humans perform efficiently and effectively, but which is difficult for a computer to perform. Since video segmentation plays an important role for many emerging applications, as those enabled by the MPEG-4 and MPEG-7 standards, the ability to assess the segmentation quality in view of the application targets is a relevant task for which a standard, or even a consensual, solution is not available. This paper considers the evaluation of overall segmentation partitions quality, highlighting one of its major components: the contextual relevance of the segmented objects. Video object relevance metrics are presented taking into account the behaviour of the human visual system and the visual attention mechanisms. In particular, contextual relevance evaluation takes into account the context where an object is found, exploiting, for instance, the contrast to neighbours or the position in the image. Most of the relevance metrics proposed in this paper can also be used in contexts other than segmentation quality evaluation, such as object-based rate control algorithms, description creation, or image and video quality evaluation.


  • Video Quality
  • Human Visual System
  • Contextual Relevance
  • Object Segmentation
  • Relevant Task

Authors’ Affiliations

Instituto Superior Técnico – Instituto de Telecomunicações, Av. Rovisco Pais, Lisboa, 1049-001, Portugal


  1. ISO/IEC 14496 : Information technology—Coding of Audio-Visual Objects. 1999.Google Scholar
  2. ISO/IEC 15938 : Multimedia Content Description Interface. 2001.Google Scholar
  3. Rui Y, Huang TS, Mehrotra S: Relevance feedback techniques in interactive content-based image retrieval. Proceedings of IS&T SPIE Storage and Retrieval for Image and Video Databases VI, January 1998, San Jose, Calif, USA, Proceedings of SPIE 3312: 25–36.View ArticleGoogle Scholar
  4. ITU-R : Methodology for the Subjective Assessment of the Quality of Television Pictures. Recommendation BT.500-7, 1995Google Scholar
  5. ITU-T : Subjective Video Quality Assessment Methods for Multimedia Applications. Recommendation P.910, August 1996Google Scholar
  6. Correia PL, Pereira F: Objective evaluation of video segmentation quality. IEEE Transactions on Image Processing 2003, 12(2):186–200. 10.1109/TIP.2002.807355View ArticleGoogle Scholar
  7. Erdem CE, Tekalp AM, Sankur B: Metrics for performance evaluation of video object segmentation and tracking without ground-truth. Proceedings of IEEE International Conference on Image Processing (ICIP '01), October 2001, Thessaloniki, Greece 2: 69–72.Google Scholar
  8. Villegas P, Marichal X: Perceptually-weighted evaluation criteria for segmentation masks in video sequences. IEEE Transactions on Image Processing 2004, 13(8):1092–1103. 10.1109/TIP.2004.828433View ArticleGoogle Scholar
  9. COST211quat European Project : Call for AM Comparisons. available at:
  10. Osberger W, Bergmann N, Maeder A: A technique for image quality assessment based on a human visual system model. Proceedings of 9th European Signal Processing Conference (EUSIPCO '98), September 1998, Rhodes, Greece 1049–1052.Google Scholar
  11. Correia PL, Pereira F: Estimation of video object's relevance. Proceedings of 10th European Signal Processing Conference (EUSIPCO '00), September 2000, Tampere, Finland 925–928.Google Scholar
  12. Hamada T, Miyaji S, Matsumoto S: Picture quality assessment system by three-layered bottom-up noise weighting considering human visual perception. Proceedings of 139th SMPTE Technical Conference, November 1997, New York, NY, USA 179–192.Google Scholar
  13. Itti L, Koch C, Niebur E: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998, 20(11):1254–1259. 10.1109/34.730558View ArticleGoogle Scholar
  14. Marichal X, Delmot T, Vleeschouwer C, Warscotte V, Macq B: Automatic detection of interest areas of an image or of a sequence of images. Proceedings of IEEE International Conference on Image Processing (ICIP '96), September 1996, Lausanne, Switzerland 3: 371–374.View ArticleGoogle Scholar
  15. Stentiford FWM: An estimator for visual attention through competitive novelty with application to image compression. Proceedings of Picture Coding Symposium, April 2001, Seoul, Korea 101–104.Google Scholar
  16. Zhao J, Shimazu Y, Ohta K, Hayasaka R, Matsushita Y: A JPEG codec adaptive to region importance. Proceedings of 4th ACM International Conference on Multimedia (ACM Multimedia '96), November 1996, Boston, Mass, USA 209–218.View ArticleGoogle Scholar
  17. Sonka M, Hlavac V, Boyle R: Image Processing, Analysis and Machine Vision. Chapman & Hall, London, UK; 1993.View ArticleGoogle Scholar
  18. Serra J: Image Analysis and Mathematical Morphology. Volume 1. Academic Press, London, UK; 1982.MATHGoogle Scholar
  19. Wolf S, Webster A: Subjective and objective measures of scene criticality. Proceedings of ITU Meeting on Subjective and Objective Audiovisual Quality Assessment Methods, October 1997, Turin, ItalyGoogle Scholar


© Correia and Pereira 2006