Skip to main content

Advertisement

Ordinal Regression Based Subpixel Shift Estimation for Video Super-Resolution

Article metrics

  • 1309 Accesses

  • 4 Citations

Abstract

We present a supervised learning-based approach for subpixel motion estimation which is then used to perform video super-resolution. The novelty of this work is the formulation of the problem of subpixel motion estimation in a ranking framework. The ranking formulation is a variant of classification and regression formulation, in which the ordering present in class labels namely, the shift between patches is explicitly taken into account. Finally, we demonstrate the applicability of our approach on superresolving synthetically generated images with global subpixel shifts and enhancing real video frames by accounting for both local integer and subpixel shifts.

References

  1. 1.

    Lucas B, Kanade T: An iterative image registration technique with an application to stereo vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI '81), August 1981, Vancouver, BC, Canada 674–679.

  2. 2.

    Aggarwal JK, Nandhakumar N: On the computation of motion from sequences of images—a review. Proceedings of the IEEE 1988,76(8):917-935. 10.1109/5.5965

  3. 3.

    Mitiche A, Bouthemy P: Computation and analysis of image motion: a synopsis of current problems and methods. International Journal of Computer Vision 1996,19(1):29-55. 10.1007/BF00131147

  4. 4.

    Nagel H-H: Image sequence evaluation: 30 years and still going strong. Proceedings of the 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 149–158.

  5. 5.

    Rajaram S, Garg A, Zhou XS, Huang TS: Classification approach towards ranking and sorting problems. Proceedings of the 14th European Conference on Machine Learning (ECML '03), September 2003, Cavtat-Dubrovnik, Croatia 301–312.

  6. 6.

    Chiang M-C, Boult T: Efficient image warping and super-resolution. Proceedings of the 3rd Workshop on Applications of Computer Vision (WACV '96), December 1996, Sarasota, Fla, USA 56–61.

  7. 7.

    Dellaert F, Thrun S, Thorpe C: Jacobian images of super-resolved texture maps for model-based motion estimation and tracking. Proceedings of the 4th IEEE Workshop on Applications of Computer Vision (WACV '98), October 1998, Princeton, NJ, USA 2–7.

  8. 8.

    Elad M, Feuer A: Super-resolution restoration of an image sequence: adaptive filtering approach. IEEE Transactions on Image Processing 1999,8(3):387-395. 10.1109/83.748893

  9. 9.

    Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116

  10. 10.

    Patti AJ, Sezan MI, Tekalp AM: Super-resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Transactions on Image Processing 1997,6(8):1064-1076. 10.1109/83.605404

  11. 11.

    Huang TS, Tsai R: Multi-frame image restoration and registration. In Advances in Computer Vision and Image Processing. Volume 1. Edited by: Huang TS. JAI Press, Greenwich, Conn, USA; 1984:317-339.

  12. 12.

    Kim SP, Bose NK, Valenzuela HM: Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Transactions on Acoustics, Speech, and Signal Processing 1990,38(6):1013-1027. 10.1109/29.56062

  13. 13.

    Cohen WW, Schapire RE, Singer Y: Learning to order things. Journal of Artificial Intelligence Research 1999, 10: 243–270.

  14. 14.

    Herbrich R, Graepel T, Obermayer K: Large margin rank boundaries for ordinal regression. In Advances in Large Margin Classifiers. MIT Press, Cambridge, Mass, USA; 2000:115-132.

  15. 15.

    Roth D, Har-Paled S, Zimak D: Constraint classification: a new approach to multiclass classification. Proceedings of the 13th Interntional Conference on Algorithmic Learning Theory (ALT '02), November 2002, Lübeck, Germany 365–379.

  16. 16.

    Smola A, Schlkopf B: A tutorial on support vector regression. In Tech. Rep. NC2-TR-1998-030. Neural and Computational Learning 2 (NeuroCOLT2), London, UK; 1998.

  17. 17.

    Baker S, Kanade T: Super-resolution optical flow. In Tech. Rep. CMU-RI-TR-99-36. Carnegie Mellon University, Pittsburgh, Pa, USA; 1999.

  18. 18.

    Baker S, Matthews I: Lucas-kanade 20 years on: a unifying framework. International Journal of Computer Vision 2004,56(3):221-255.

  19. 19.

    Wolberg G: Digital Image Warping. IEEE Computer Society Press, Los Alamitos, Calif, USA; 1992.

  20. 20.

    Li X, Zhu Y, Han C: Unified optimal linear estimation fusion—I: unified models and fusion rules. Proceedings of the 3rd International Conference on Information Fusion (FUSION '00), July 2000, Paris, France 1: 10–17.

  21. 21.

    Julier S, Uhlmann J: A non-divergent estimation algorithm in the presence of unknown correlations. Proceedings of the IEEE American Control Conference (ACC '97), June 1997, Alberquerque, NM, USA 4: 2369–2373 .

  22. 22.

    Comaniciu D: Nonparametric information fusion for motion estimation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '03), June 2003, Madison, Wis, USA 1: 59–66.

  23. 23.

    Aysal TC, Barner KE: Quadratic weighted median filters for edge enhancement of noisy images. IEEE Transactions on Image Processing 2006,15(11):3294-3310.

  24. 24.

    Aysal TC, Barner KE: Hybrid polynomial filters for Gaussian and non-Gaussian noise environments. IEEE Transactions on Signal Processing 2006,54(12):4644-4661.

  25. 25.

    Petrovic N, Jojic N, Huang TS: Hierarchical video clustering. Proceedings of the 6th IEEE Workshop on Multimedia Signal Processing (MMSP '04), September 2004, Siena, Italy 423–426.

  26. 26.

    Jojic N, Petrovic N, Frey BJ, Huang TS: Transformed hidden Markov models: estimating mixture models of images and inferring spatial transformations in video sequences. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '00), June 2000, Hilton Head, SC, USA 2: 26–33.

Download references

Author information

Correspondence to Mithun Das Gupta.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://doi.org/creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Das Gupta, M., Rajaram, S., Huang, T.S. et al. Ordinal Regression Based Subpixel Shift Estimation for Video Super-Resolution. EURASIP J. Adv. Signal Process. 2007, 085963 (2007) doi:10.1155/2007/85963

Download citation

Keywords

  • Information Technology
  • Quantum Information
  • Class Label
  • Motion Estimation
  • Video Frame