Skip to main content
  • Research Article
  • Open access
  • Published:

Ordinal Regression Based Subpixel Shift Estimation for Video Super-Resolution

Abstract

We present a supervised learning-based approach for subpixel motion estimation which is then used to perform video super-resolution. The novelty of this work is the formulation of the problem of subpixel motion estimation in a ranking framework. The ranking formulation is a variant of classification and regression formulation, in which the ordering present in class labels namely, the shift between patches is explicitly taken into account. Finally, we demonstrate the applicability of our approach on superresolving synthetically generated images with global subpixel shifts and enhancing real video frames by accounting for both local integer and subpixel shifts.

References

  1. Lucas B, Kanade T: An iterative image registration technique with an application to stereo vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI '81), August 1981, Vancouver, BC, Canada 674–679.

    Google Scholar 

  2. Aggarwal JK, Nandhakumar N: On the computation of motion from sequences of images—a review. Proceedings of the IEEE 1988,76(8):917-935. 10.1109/5.5965

    Article  Google Scholar 

  3. Mitiche A, Bouthemy P: Computation and analysis of image motion: a synopsis of current problems and methods. International Journal of Computer Vision 1996,19(1):29-55. 10.1007/BF00131147

    Article  Google Scholar 

  4. Nagel H-H: Image sequence evaluation: 30 years and still going strong. Proceedings of the 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 149–158.

    Article  Google Scholar 

  5. Rajaram S, Garg A, Zhou XS, Huang TS: Classification approach towards ranking and sorting problems. Proceedings of the 14th European Conference on Machine Learning (ECML '03), September 2003, Cavtat-Dubrovnik, Croatia 301–312.

    Google Scholar 

  6. Chiang M-C, Boult T: Efficient image warping and super-resolution. Proceedings of the 3rd Workshop on Applications of Computer Vision (WACV '96), December 1996, Sarasota, Fla, USA 56–61.

    Chapter  Google Scholar 

  7. Dellaert F, Thrun S, Thorpe C: Jacobian images of super-resolved texture maps for model-based motion estimation and tracking. Proceedings of the 4th IEEE Workshop on Applications of Computer Vision (WACV '98), October 1998, Princeton, NJ, USA 2–7.

    Google Scholar 

  8. Elad M, Feuer A: Super-resolution restoration of an image sequence: adaptive filtering approach. IEEE Transactions on Image Processing 1999,8(3):387-395. 10.1109/83.748893

    Article  Google Scholar 

  9. Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116

    Article  Google Scholar 

  10. Patti AJ, Sezan MI, Tekalp AM: Super-resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Transactions on Image Processing 1997,6(8):1064-1076. 10.1109/83.605404

    Article  Google Scholar 

  11. Huang TS, Tsai R: Multi-frame image restoration and registration. In Advances in Computer Vision and Image Processing. Volume 1. Edited by: Huang TS. JAI Press, Greenwich, Conn, USA; 1984:317-339.

    Google Scholar 

  12. Kim SP, Bose NK, Valenzuela HM: Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Transactions on Acoustics, Speech, and Signal Processing 1990,38(6):1013-1027. 10.1109/29.56062

    Article  Google Scholar 

  13. Cohen WW, Schapire RE, Singer Y: Learning to order things. Journal of Artificial Intelligence Research 1999, 10: 243–270.

    Article  MathSciNet  Google Scholar 

  14. Herbrich R, Graepel T, Obermayer K: Large margin rank boundaries for ordinal regression. In Advances in Large Margin Classifiers. MIT Press, Cambridge, Mass, USA; 2000:115-132.

    Google Scholar 

  15. Roth D, Har-Paled S, Zimak D: Constraint classification: a new approach to multiclass classification. Proceedings of the 13th Interntional Conference on Algorithmic Learning Theory (ALT '02), November 2002, Lübeck, Germany 365–379.

    Google Scholar 

  16. Smola A, Schlkopf B: A tutorial on support vector regression. In Tech. Rep. NC2-TR-1998-030. Neural and Computational Learning 2 (NeuroCOLT2), London, UK; 1998.

    Google Scholar 

  17. Baker S, Kanade T: Super-resolution optical flow. In Tech. Rep. CMU-RI-TR-99-36. Carnegie Mellon University, Pittsburgh, Pa, USA; 1999.

    Google Scholar 

  18. Baker S, Matthews I: Lucas-kanade 20 years on: a unifying framework. International Journal of Computer Vision 2004,56(3):221-255.

    Article  Google Scholar 

  19. Wolberg G: Digital Image Warping. IEEE Computer Society Press, Los Alamitos, Calif, USA; 1992.

    Google Scholar 

  20. Li X, Zhu Y, Han C: Unified optimal linear estimation fusion—I: unified models and fusion rules. Proceedings of the 3rd International Conference on Information Fusion (FUSION '00), July 2000, Paris, France 1: 10–17.

    Google Scholar 

  21. Julier S, Uhlmann J: A non-divergent estimation algorithm in the presence of unknown correlations. Proceedings of the IEEE American Control Conference (ACC '97), June 1997, Alberquerque, NM, USA 4: 2369–2373 .

    Google Scholar 

  22. Comaniciu D: Nonparametric information fusion for motion estimation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '03), June 2003, Madison, Wis, USA 1: 59–66.

    Google Scholar 

  23. Aysal TC, Barner KE: Quadratic weighted median filters for edge enhancement of noisy images. IEEE Transactions on Image Processing 2006,15(11):3294-3310.

    Article  Google Scholar 

  24. Aysal TC, Barner KE: Hybrid polynomial filters for Gaussian and non-Gaussian noise environments. IEEE Transactions on Signal Processing 2006,54(12):4644-4661.

    Article  Google Scholar 

  25. Petrovic N, Jojic N, Huang TS: Hierarchical video clustering. Proceedings of the 6th IEEE Workshop on Multimedia Signal Processing (MMSP '04), September 2004, Siena, Italy 423–426.

    Google Scholar 

  26. Jojic N, Petrovic N, Frey BJ, Huang TS: Transformed hidden Markov models: estimating mixture models of images and inferring spatial transformations in video sequences. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '00), June 2000, Hilton Head, SC, USA 2: 26–33.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mithun Das Gupta.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://doi.org/creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Das Gupta, M., Rajaram, S., Huang, T.S. et al. Ordinal Regression Based Subpixel Shift Estimation for Video Super-Resolution. EURASIP J. Adv. Signal Process. 2007, 085963 (2007). https://doi.org/10.1155/2007/85963

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/2007/85963

Keywords