Open Access

Ordinal Regression Based Subpixel Shift Estimation for Video Super-Resolution

  • Mithun Das Gupta1Email author,
  • Shyamsundar Rajaram1,
  • Thomas S. Huang1 and
  • Nemanja Petrovic2
EURASIP Journal on Advances in Signal Processing20072007:085963

https://doi.org/10.1155/2007/85963

Received: 2 October 2006

Accepted: 3 May 2007

Published: 5 July 2007

Abstract

We present a supervised learning-based approach for subpixel motion estimation which is then used to perform video super-resolution. The novelty of this work is the formulation of the problem of subpixel motion estimation in a ranking framework. The ranking formulation is a variant of classification and regression formulation, in which the ordering present in class labels namely, the shift between patches is explicitly taken into account. Finally, we demonstrate the applicability of our approach on superresolving synthetically generated images with global subpixel shifts and enhancing real video frames by accounting for both local integer and subpixel shifts.

[1234567891011121314151617181920212223242526]

Authors’ Affiliations

(1)
Department of Electrical and Computer Engineering, University of Illinois, Urbana Champaign
(2)
Google Inc

References

  1. Lucas B, Kanade T: An iterative image registration technique with an application to stereo vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI '81), August 1981, Vancouver, BC, Canada 674-679.Google Scholar
  2. Aggarwal JK, Nandhakumar N: On the computation of motion from sequences of images—a review. Proceedings of the IEEE 1988,76(8):917-935. 10.1109/5.5965View ArticleGoogle Scholar
  3. Mitiche A, Bouthemy P: Computation and analysis of image motion: a synopsis of current problems and methods. International Journal of Computer Vision 1996,19(1):29-55. 10.1007/BF00131147View ArticleGoogle Scholar
  4. Nagel H-H: Image sequence evaluation: 30 years and still going strong. Proceedings of the 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 149-158.View ArticleGoogle Scholar
  5. Rajaram S, Garg A, Zhou XS, Huang TS: Classification approach towards ranking and sorting problems. Proceedings of the 14th European Conference on Machine Learning (ECML '03), September 2003, Cavtat-Dubrovnik, Croatia 301-312.Google Scholar
  6. Chiang M-C, Boult T: Efficient image warping and super-resolution. Proceedings of the 3rd Workshop on Applications of Computer Vision (WACV '96), December 1996, Sarasota, Fla, USA 56-61.View ArticleGoogle Scholar
  7. Dellaert F, Thrun S, Thorpe C: Jacobian images of super-resolved texture maps for model-based motion estimation and tracking. Proceedings of the 4th IEEE Workshop on Applications of Computer Vision (WACV '98), October 1998, Princeton, NJ, USA 2-7.Google Scholar
  8. Elad M, Feuer A: Super-resolution restoration of an image sequence: adaptive filtering approach. IEEE Transactions on Image Processing 1999,8(3):387-395. 10.1109/83.748893View ArticleGoogle Scholar
  9. Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116View ArticleGoogle Scholar
  10. Patti AJ, Sezan MI, Tekalp AM: Super-resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Transactions on Image Processing 1997,6(8):1064-1076. 10.1109/83.605404View ArticleGoogle Scholar
  11. Huang TS, Tsai R: Multi-frame image restoration and registration. In Advances in Computer Vision and Image Processing. Volume 1. Edited by: Huang TS. JAI Press, Greenwich, Conn, USA; 1984:317-339.Google Scholar
  12. Kim SP, Bose NK, Valenzuela HM: Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Transactions on Acoustics, Speech, and Signal Processing 1990,38(6):1013-1027. 10.1109/29.56062View ArticleGoogle Scholar
  13. Cohen WW, Schapire RE, Singer Y: Learning to order things. Journal of Artificial Intelligence Research 1999, 10: 243-270.MathSciNetMATHGoogle Scholar
  14. Herbrich R, Graepel T, Obermayer K: Large margin rank boundaries for ordinal regression. In Advances in Large Margin Classifiers. MIT Press, Cambridge, Mass, USA; 2000:115-132.Google Scholar
  15. Roth D, Har-Paled S, Zimak D: Constraint classification: a new approach to multiclass classification. Proceedings of the 13th Interntional Conference on Algorithmic Learning Theory (ALT '02), November 2002, Lübeck, Germany 365-379.Google Scholar
  16. Smola A, Schlkopf B: A tutorial on support vector regression. In Tech. Rep. NC2-TR-1998-030. Neural and Computational Learning 2 (NeuroCOLT2), London, UK; 1998.Google Scholar
  17. Baker S, Kanade T: Super-resolution optical flow. In Tech. Rep. CMU-RI-TR-99-36. Carnegie Mellon University, Pittsburgh, Pa, USA; 1999.Google Scholar
  18. Baker S, Matthews I: Lucas-kanade 20 years on: a unifying framework. International Journal of Computer Vision 2004,56(3):221-255.View ArticleGoogle Scholar
  19. Wolberg G: Digital Image Warping. IEEE Computer Society Press, Los Alamitos, Calif, USA; 1992.Google Scholar
  20. Li X, Zhu Y, Han C: Unified optimal linear estimation fusion—I: unified models and fusion rules. Proceedings of the 3rd International Conference on Information Fusion (FUSION '00), July 2000, Paris, France 1: 10-17.Google Scholar
  21. Julier S, Uhlmann J: A non-divergent estimation algorithm in the presence of unknown correlations. Proceedings of the IEEE American Control Conference (ACC '97), June 1997, Alberquerque, NM, USA 4: 2369-2373 .View ArticleGoogle Scholar
  22. Comaniciu D: Nonparametric information fusion for motion estimation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '03), June 2003, Madison, Wis, USA 1: 59-66.Google Scholar
  23. Aysal TC, Barner KE: Quadratic weighted median filters for edge enhancement of noisy images. IEEE Transactions on Image Processing 2006,15(11):3294-3310.View ArticleGoogle Scholar
  24. Aysal TC, Barner KE: Hybrid polynomial filters for Gaussian and non-Gaussian noise environments. IEEE Transactions on Signal Processing 2006,54(12):4644-4661.View ArticleGoogle Scholar
  25. Petrovic N, Jojic N, Huang TS: Hierarchical video clustering. Proceedings of the 6th IEEE Workshop on Multimedia Signal Processing (MMSP '04), September 2004, Siena, Italy 423-426.Google Scholar
  26. Jojic N, Petrovic N, Frey BJ, Huang TS: Transformed hidden Markov models: estimating mixture models of images and inferring spatial transformations in video sequences. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '00), June 2000, Hilton Head, SC, USA 2: 26-33.Google Scholar

Copyright

© Mithun Das Gupta et al. 2007

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.