Skip to main content

Motion Estimation and Signaling Techniques for 2D+t Scalable Video Coding

Abstract

We describe a fully scalable wavelet-based 2D+t (in-band) video coding architecture. We propose new coding tools specifically designed for this framework aimed at two goals: reduce the computational complexity at the encoder without sacrificing compression; improve the coding efficiency, especially at low bitrates. To this end, we focus our attention on motion estimation and motion vector encoding. We propose a fast motion estimation algorithm that works in the wavelet domain and exploits the geometrical properties of the wavelet subbands. We show that the computational complexity grows linearly with the size of the search window, yet approaching the performance of a full search strategy. We extend the proposed motion estimation algorithm to work with blocks of variable sizes, in order to better capture local motion characteristics, thus improving in terms of rate-distortion behavior. Given this motion field representation, we propose a motion vector coding algorithm that allows to adaptively scale the motion bit budget according to the target bitrate, improving the coding efficiency at low bitrates. Finally, we show how to optimally scale the motion field when the sequence is decoded at reduced spatial resolution. Experimental results illustrate the advantages of each individual coding tool presented in this paper. Based on these simulations, we define the best configuration of coding parameters and we compare the proposed codec with MC-EZBC, a widely used reference codec implementing the t+2D framework.

References

  1. 1.

    Taubam D, Marcellin MW: JPEG2000: Image Compression Fundamentals, Standards and Practice. Kluwer Academic, Boston, Mass, USA; 2002.

    Google Scholar 

  2. 2.

    Said A, Pearlman WA: A new, fast, and efficient image codec based on set partitioning in hierarchical trees. IEEE Transactions on Circuits and Systems for Video Technology 1996, 6(3):243–250. 10.1109/76.499834

    Article  Google Scholar 

  3. 3.

    Hsiang S-T, Woods JW: Embedded image coding using zeroblocks of subband/wavelet coefficients and context modeling. Proceedings of IEEE International Symposium on Circuits and Systems (ISCAS '00), May 2000, Geneva, Switzerland 3: 662–665.

    Google Scholar 

  4. 4.

    Xu J, Xiong Z, Li S, Zhang Y-Q: Three-dimensional embedded subband coding with optimized truncation (3-D ESCOT). Applied and Computational Harmonic Analysis 2001, 10(3):290–315. 10.1006/acha.2000.0345

    MathSciNet  Article  Google Scholar 

  5. 5.

    Chen P, Woods JW: Bidirectional MC-EZBC with lifting implementation. IEEE Transactions on Circuits and Systems for Video Technology 2004, 14(10):1183–1194. 10.1109/TCSVT.2004.833165

    Article  Google Scholar 

  6. 6.

    Secker A, Taubman D: Lifting-based invertible motion adaptive transform (LIMAT) framework for highly scalable video compression. IEEE Transactions on Image Processing 2003, 12(12):1530–1542. 10.1109/TIP.2003.819433

    Article  Google Scholar 

  7. 7.

    Pau G, Tillier C, Pesquet-Popescu B, Heijmans H: Motion compensation and scalability in lifting-based video coding. Signal Processing: Image Communication 2004, 19(7):577–600. 10.1016/j.image.2004.05.003

    Google Scholar 

  8. 8.

    Ohm J-R: Three-dimensional subband coding with motion compensation. IEEE Transactions on Image Processing 1994, 3(5):559–571. 10.1109/83.334985

    Article  Google Scholar 

  9. 9.

    Andreopoulos Y, Munteanu A, Barbarien J, van der Schaar M, Cornelis J, Schelkens P: In-band motion compensated temporal filtering. Signal Processing: Image Communication 2004, 19(7):653–673. 10.1016/j.image.2004.05.007

    Google Scholar 

  10. 10.

    Wang Y, Cui S, Fowler JE: 3D video coding using redundant-wavelet multihypothesis and motion-compensated temporal filtering. Proceedings of IEEE International Conference on Image Processing (ICIP '03), September 2003, Barcelona, Spain 2: 755–758.

    Google Scholar 

  11. 11.

    Park H-W, Kim H-S: Motion estimation using low-band-shift method for wavelet-based moving-picture coding. IEEE Transactions on Image Processing 2000, 9(4):577–587. 10.1109/83.841935

    Article  Google Scholar 

  12. 12.

    Ye JC, van der Schaar M: Fully scalable 3D overcomplete wavelet video coding using adaptive motion compensated temporal filtering. In Visual Communications and Image Processing (VCIP '03), July 2003, Lugano, Switzerland, Proceedings of SPIE Edited by: Ebrahimi T, Sikora T. 5150: 1169–1180.

    Google Scholar 

  13. 13.

    Mallat S: A Wavelet Tour of Signal Processing. Academic Press, San Diego, Calif, USA; 1998.

    Google Scholar 

  14. 14.

    Mehrseresht N, Taubman D: A flexible structure for fully scalable motion compensated 3D-DWT with emphasis on the impact of spatial scalability. submitted to IEEE Transactions on Image Processing

  15. 15.

    Mehrseresht N, Taubman D: An efficient content-adaptive motion compensated 3D-DWT with enhanced spatial and temporal scalability. submitted to IEEE Transactions on Image Processing

  16. 16.

    ITU-T : Information technology—Coding of audio-visual objects—Part 10: Advanced video coding. May 2003, ISO/IEC International Standard 14496-10:2003

  17. 17.

    Maestroni D, Sarti A, Tagliasacchi M, Tubaro S: Fast in-band motion estimation with variable size block matching. Proceedings of IEEE International Conference on Image Processing (ICIP '04), October 2004, Singapore 4: 2287–2290.

    Google Scholar 

  18. 18.

    Maestroni D, Sarti A, Tagliasacchi M, Tubaro S: Scalable coding of variable size blocks motion vectors. Proceedings of IEEE International Conference on Image Processing (ICIP~'04), October 2004, Singapore 2: 1333–1336.

    Google Scholar 

  19. 19.

    Maestroni D, Sarti A, Tagliasacchi M, Tubaro S: Wavelet-based video coding: optimal use of motion information for the decoding of spatially scaled video sequences. Proceedings of the 12th European Signal Processing Conference (EUSIPCO '04), September 2004, Vienna, Austria

    Google Scholar 

  20. 20.

    Andreopoulos Y, Munteanu A, Van der Auwera G, Schelkens P, Cornelis J: A new method for complete-to-overcomplete discrete wavelet transforms. Proceedings of the 14th International Conference on Digital Signal Processing (DSP '02), July 2002, Santorini, Greece 2: 501–504.

    Article  Google Scholar 

  21. 21.

    Andreopoulos Y, van der Schaar M, Munteanu A, Barbarien J, Schelkens P, Cornelis J: Fully-scalable wavelet video coding using in-band motion compensated temporal filtering. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '03), April 2003, Hong Kong, China 3: 417–420.

    Google Scholar 

  22. 22.

    Cai W, Adjouadi M: An efficient approach of fast motion estimation and compensation in wavelet domain video compression. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '04), May 2004, Montreal, Quebec, Canada 2: 977–980.

    Google Scholar 

  23. 23.

    Choi S-J, Woods JW: Motion-compensated 3-D subband coding of video. IEEE Transactions on Image Processing 1999, 8(2):155–167. 10.1109/83.743851

    Article  Google Scholar 

  24. 24.

    Xiong R, Wu F, Li S, Xiong Z, Zhang Y-Q: Exploiting temporal correlation with adaptive block-size motion alignment for 3D wavelet coding. Visual Communications and Image Processing 2004, January 2004, San Jose, Calif, USA, Proceedings of SPIE 5308: 144–155.

    Google Scholar 

  25. 25.

    Taubman D, Secker A: Highly scalable video compression with scalable motion coding. Proceedings of IEEE International Conference on Image Processing (ICIP '03), September 2003, Barcelona, Spain 3: 273–276.

    Google Scholar 

  26. 26.

    Secker A, Taubman D: Highly scalable video compression with scalable motion coding. IEEE Transactions on Image Processing 2004, 13(8):1029–1041. 10.1109/TIP.2004.826089

    Article  Google Scholar 

  27. 27.

    Barbarien J, Munteanu A, Verdicchio F, Andreopoulos Y, Cornelis J, Schelkens P: Scalable motion vector coding. Electronics Letters 2004, 40(15):932–934. 10.1049/el:20040490

    Article  Google Scholar 

  28. 28.

    Boisson G, Francois E, Guillemot C: Accuracy-scalable motion coding for efficient scalable video compression. Proceedings of IEEE International Conference on Image Processing (ICIP '04), October 2004, Singapore 2: 1309–1312.

    Google Scholar 

  29. 29.

    Xiong R, Xu J, Wu F, Li S, Zhang Y-Q: Layered motion estimation and coding for fully scalable 3D wavelet video coding. Proceeding of IEEE International Conference on Image Processing (ICIP '04), October 2004, Singapore 4: 2271–2274.

    Google Scholar 

  30. 30.

    Lucas BD, Kanade T: An iterative image registration technique with an application to stereo vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI '81), August 1981, Vancouver, BC, Canada 674–679.

    Google Scholar 

  31. 31.

    Horn BKP, Schunck BG: Determining optical flow. Artificial Intelligence 1981, 17(1–3):185–203.

    Article  Google Scholar 

  32. 32.

    Tekalp MA: Digital Video Processing, Prentice-Hall Signal Processing Series. Prentice-Hall, Upper Saddle River, NJ, USA; 1995.

    Google Scholar 

  33. 33.

    Benzler U, Wien M: Results of SVC CE3 (Quality Evaluation). ISO/IEC JTC1/SC29/WG11 MPEG Document M10931, July 2004

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to M. Tagliasacchi.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Tagliasacchi, M., Maestroni, D., Tubaro, S. et al. Motion Estimation and Signaling Techniques for 2D+t Scalable Video Coding. EURASIP J. Adv. Signal Process. 2006, 057308 (2006). https://doi.org/10.1155/ASP/2006/57308

Download citation

Keywords

  • Motion Vector
  • Motion Estimation
  • Video Code
  • Scalable Video Code
  • Motion Field