Skip to content

Advertisement

Open Access

Learning-Based Nonparametric Image Super-Resolution

  • Shyamsundar Rajaram1Email author,
  • Mithun Das Gupta2,
  • Nemanja Petrovic3 and
  • Thomas S. Huang2
EURASIP Journal on Advances in Signal Processing20062006:051306

https://doi.org/10.1155/ASP/2006/51306

Received: 1 December 2004

Accepted: 25 April 2005

Published: 18 January 2006

Abstract

We present a novel learning-based framework for zooming and recognizing images of digits obtained from vehicle registration plates, which have been blurred using an unknown kernel. We model the image as an undirected graphical model over image patches in which the compatibility functions are represented as nonparametric kernel densities. The crucial feature of this work is an iterative loop that alternates between super-resolution and restoration stages. A machine-learning-based framework has been used for restoration which also models spatial zooming. Image segmentation is done by a column-variance estimation-based "dissection" algorithm. Initially, the compatibility functions are learned by nonparametric kernel density estimation, using random samples from the training data. Next, we solve the inference problem by using an extended version of the nonparametric belief propagation algorithm, in which we introduce the notion of partial messages. Finally, we recognize the super-resolved and restored images. The resulting confidence scores are used to sample from the training set to better learn the compatibility functions.

Keywords

Image SegmentationKernel Density EstimationConfidence ScoreImage PatchInference Problem

[1234567891011121314151617181920212223]

Authors’ Affiliations

(1)
Beckman Institute, University of Illinois at Urbana-Champaign, USA
(2)
Beckman Institute, at Urbana-Champaign, University of Illinois Urbana, USA
(3)
Siemens Corporate Research, Princeton, USA

References

  1. Bascle B, Blake A, Zisserman A: Motion deblurring and super-resolution from an image sequence. Proceedings of 4th European Conference on Computer Vision (ECCV '96), April 1996, Cambridge, UK 2: 573-582.Google Scholar
  2. Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997, 6(12):1621-1633. 10.1109/83.650116View ArticleGoogle Scholar
  3. Irani M, Peleg S: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991, 53(3):231-239. 10.1016/1049-9652(91)90045-LGoogle Scholar
  4. Kim SP, Bose NK, Valenzuela HM: Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Transactions on Acoustics, Speech and Signal Processing 1990, 38(6):1013-1027. 10.1109/29.56062View ArticleGoogle Scholar
  5. Schultz RR, Stevenson RL: Extraction of high-resolution frames from video sequences. IEEE Transactions on Image Processing 1996, 5(6):996-1011. 10.1109/83.503915View ArticleGoogle Scholar
  6. Stark H, Oskoui P: High-resolution image recovery from image-plane arrays, using convex projections. Journal of the Optical Society of America. A, Optics and image science. 1989, 6(11):1715-1726. 10.1364/JOSAA.6.001715View ArticleGoogle Scholar
  7. Chiang M-C, Boult TE: Local blur estimation and super-resolution. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '97), June 1997, San Juan, Puerto Rico, USA 821-826.View ArticleGoogle Scholar
  8. Baker S, Kanade T: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(9):1167-1183. 10.1109/TPAMI.2002.1033210View ArticleGoogle Scholar
  9. Bishop CM, Blake A, Marthi B: Super-resolution enhancement of video. Proceedings of 9th International Workshop on Artificial Intelligence and Statistics (AISTATS '03), January 2003, Key West, Fla, USAGoogle Scholar
  10. Capel D, Zisserman A: Super-resolution from multiple views using learnt image models. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 2: II-627-II-634.Google Scholar
  11. Freeman WT, Pasztor EC, Carmichael OT: Learning low-level vision. International Journal of Computer Vision 2000, 40(1):25-47. 10.1023/A:1026501619075MATHView ArticleGoogle Scholar
  12. Liu C, Shum H-Y, Zhang C-S: A two-step approach to hallucinating faces: global parametric model and local nonparametric model. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 1: I-192-I-198.Google Scholar
  13. Dedeoglu G, Kanade T, August J: High-zoom video hallucination by exploiting spatio-temporal regularities. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '04), June—July 2004, Washington, DC, USA 2: II-151-II-158.Google Scholar
  14. Gupta MD, Rajaram S, Petrovic N, Huang TS: Restoration and recognition in a loop. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '05), June 2005, San Diego, Calif, USA 1: 638-644.Google Scholar
  15. Pearl J: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco, Calif, USA; 1988.MATHGoogle Scholar
  16. Yedidia JS, Freeman WT, Weiss Y: Understanding belief propagation and its generalizations. In Exploring Artificial Intelligence in the New Millennium. Morgan Kaufmann, San Francisco, Calif, USA; 2003:239-269.Google Scholar
  17. Sudderth EB, Ihler AT, Freeman WT, Willsky AS: Nonparametric belief propagation. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '03), June 2003, Madison, Wis, USA 1: I-605-I-612.Google Scholar
  18. Isard M: PAMPAS: real-valued graphical models for computer vision. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '03), June 2003, Madison, Wis, USA 1: I-613-I-620.Google Scholar
  19. Geman S, Geman D: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6(6):721-741.MATHView ArticleGoogle Scholar
  20. Hinton GE: Training products of experts by minimizing contrastive divergence. Neural Computation 2002, 14(8):1771-1800. 10.1162/089976602760128018MATHMathSciNetView ArticleGoogle Scholar
  21. Silverman BW: Density Estimation for Statistics and Data Analysis. Chapman & Hall/CRC, Boca Raton, Fla, USA; 1986.MATHView ArticleGoogle Scholar
  22. Hoffman RL, McCullough JW: Segmentation methods for recognition of machine-printed characters. IBM Journal of Research and Development 1971, 15(2):153-165.View ArticleGoogle Scholar
  23. Gupta MD, Rajaram S, Petrovic N, Huang TS: Non-parametric image super-resolution using multiple images. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, ItalyGoogle Scholar

Copyright

© Rajaram et al. 2006

Advertisement