Skip to main content

Video Object Relevance Metrics for Overall Segmentation Quality Evaluation


Video object segmentation is a task that humans perform efficiently and effectively, but which is difficult for a computer to perform. Since video segmentation plays an important role for many emerging applications, as those enabled by the MPEG-4 and MPEG-7 standards, the ability to assess the segmentation quality in view of the application targets is a relevant task for which a standard, or even a consensual, solution is not available. This paper considers the evaluation of overall segmentation partitions quality, highlighting one of its major components: the contextual relevance of the segmented objects. Video object relevance metrics are presented taking into account the behaviour of the human visual system and the visual attention mechanisms. In particular, contextual relevance evaluation takes into account the context where an object is found, exploiting, for instance, the contrast to neighbours or the position in the image. Most of the relevance metrics proposed in this paper can also be used in contexts other than segmentation quality evaluation, such as object-based rate control algorithms, description creation, or image and video quality evaluation.


  1. 1.

    ISO/IEC 14496 : Information technology—Coding of Audio-Visual Objects. 1999.

  2. 2.

    ISO/IEC 15938 : Multimedia Content Description Interface. 2001.

  3. 3.

    Rui Y, Huang TS, Mehrotra S: Relevance feedback techniques in interactive content-based image retrieval. Proceedings of IS&T SPIE Storage and Retrieval for Image and Video Databases VI, January 1998, San Jose, Calif, USA, Proceedings of SPIE 3312: 25–36.

    Article  Google Scholar 

  4. 4.

    ITU-R : Methodology for the Subjective Assessment of the Quality of Television Pictures. Recommendation BT.500-7, 1995

  5. 5.

    ITU-T : Subjective Video Quality Assessment Methods for Multimedia Applications. Recommendation P.910, August 1996

  6. 6.

    Correia PL, Pereira F: Objective evaluation of video segmentation quality. IEEE Transactions on Image Processing 2003, 12(2):186–200. 10.1109/TIP.2002.807355

    Article  Google Scholar 

  7. 7.

    Erdem CE, Tekalp AM, Sankur B: Metrics for performance evaluation of video object segmentation and tracking without ground-truth. Proceedings of IEEE International Conference on Image Processing (ICIP '01), October 2001, Thessaloniki, Greece 2: 69–72.

    Google Scholar 

  8. 8.

    Villegas P, Marichal X: Perceptually-weighted evaluation criteria for segmentation masks in video sequences. IEEE Transactions on Image Processing 2004, 13(8):1092–1103. 10.1109/TIP.2004.828433

    Article  Google Scholar 

  9. 9.

    COST211quat European Project : Call for AM Comparisons. available at:

  10. 10.

    Osberger W, Bergmann N, Maeder A: A technique for image quality assessment based on a human visual system model. Proceedings of 9th European Signal Processing Conference (EUSIPCO '98), September 1998, Rhodes, Greece 1049–1052.

    Google Scholar 

  11. 11.

    Correia PL, Pereira F: Estimation of video object's relevance. Proceedings of 10th European Signal Processing Conference (EUSIPCO '00), September 2000, Tampere, Finland 925–928.

    Google Scholar 

  12. 12.

    Hamada T, Miyaji S, Matsumoto S: Picture quality assessment system by three-layered bottom-up noise weighting considering human visual perception. Proceedings of 139th SMPTE Technical Conference, November 1997, New York, NY, USA 179–192.

    Google Scholar 

  13. 13.

    Itti L, Koch C, Niebur E: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998, 20(11):1254–1259. 10.1109/34.730558

    Article  Google Scholar 

  14. 14.

    Marichal X, Delmot T, Vleeschouwer C, Warscotte V, Macq B: Automatic detection of interest areas of an image or of a sequence of images. Proceedings of IEEE International Conference on Image Processing (ICIP '96), September 1996, Lausanne, Switzerland 3: 371–374.

    Article  Google Scholar 

  15. 15.

    Stentiford FWM: An estimator for visual attention through competitive novelty with application to image compression. Proceedings of Picture Coding Symposium, April 2001, Seoul, Korea 101–104.

    Google Scholar 

  16. 16.

    Zhao J, Shimazu Y, Ohta K, Hayasaka R, Matsushita Y: A JPEG codec adaptive to region importance. Proceedings of 4th ACM International Conference on Multimedia (ACM Multimedia '96), November 1996, Boston, Mass, USA 209–218.

    Google Scholar 

  17. 17.

    Sonka M, Hlavac V, Boyle R: Image Processing, Analysis and Machine Vision. Chapman & Hall, London, UK; 1993.

    Google Scholar 

  18. 18.

    Serra J: Image Analysis and Mathematical Morphology. Volume 1. Academic Press, London, UK; 1982.

    Google Scholar 

  19. 19.

    Wolf S, Webster A: Subjective and objective measures of scene criticality. Proceedings of ITU Meeting on Subjective and Objective Audiovisual Quality Assessment Methods, October 1997, Turin, Italy

    Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Paulo Correia.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Correia, P., Pereira, F. Video Object Relevance Metrics for Overall Segmentation Quality Evaluation. EURASIP J. Adv. Signal Process. 2006, 082195 (2006).

Download citation


  • Video Quality
  • Human Visual System
  • Contextual Relevance
  • Object Segmentation
  • Relevant Task