Skip to main content

Learning-Based Nonparametric Image Super-Resolution

Abstract

We present a novel learning-based framework for zooming and recognizing images of digits obtained from vehicle registration plates, which have been blurred using an unknown kernel. We model the image as an undirected graphical model over image patches in which the compatibility functions are represented as nonparametric kernel densities. The crucial feature of this work is an iterative loop that alternates between super-resolution and restoration stages. A machine-learning-based framework has been used for restoration which also models spatial zooming. Image segmentation is done by a column-variance estimation-based "dissection" algorithm. Initially, the compatibility functions are learned by nonparametric kernel density estimation, using random samples from the training data. Next, we solve the inference problem by using an extended version of the nonparametric belief propagation algorithm, in which we introduce the notion of partial messages. Finally, we recognize the super-resolved and restored images. The resulting confidence scores are used to sample from the training set to better learn the compatibility functions.

References

  1. 1.

    Bascle B, Blake A, Zisserman A: Motion deblurring and super-resolution from an image sequence. Proceedings of 4th European Conference on Computer Vision (ECCV '96), April 1996, Cambridge, UK 2: 573–582.

    Google Scholar 

  2. 2.

    Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997, 6(12):1621–1633. 10.1109/83.650116

    Article  Google Scholar 

  3. 3.

    Irani M, Peleg S: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991, 53(3):231–239. 10.1016/1049-9652(91)90045-L

    Google Scholar 

  4. 4.

    Kim SP, Bose NK, Valenzuela HM: Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Transactions on Acoustics, Speech and Signal Processing 1990, 38(6):1013–1027. 10.1109/29.56062

    Article  Google Scholar 

  5. 5.

    Schultz RR, Stevenson RL: Extraction of high-resolution frames from video sequences. IEEE Transactions on Image Processing 1996, 5(6):996–1011. 10.1109/83.503915

    Article  Google Scholar 

  6. 6.

    Stark H, Oskoui P: High-resolution image recovery from image-plane arrays, using convex projections. Journal of the Optical Society of America. A, Optics and image science. 1989, 6(11):1715–1726. 10.1364/JOSAA.6.001715

    Article  Google Scholar 

  7. 7.

    Chiang M-C, Boult TE: Local blur estimation and super-resolution. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '97), June 1997, San Juan, Puerto Rico, USA 821–826.

    Google Scholar 

  8. 8.

    Baker S, Kanade T: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(9):1167–1183. 10.1109/TPAMI.2002.1033210

    Article  Google Scholar 

  9. 9.

    Bishop CM, Blake A, Marthi B: Super-resolution enhancement of video. Proceedings of 9th International Workshop on Artificial Intelligence and Statistics (AISTATS '03), January 2003, Key West, Fla, USA

    Google Scholar 

  10. 10.

    Capel D, Zisserman A: Super-resolution from multiple views using learnt image models. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 2: II-627–II-634.

    Google Scholar 

  11. 11.

    Freeman WT, Pasztor EC, Carmichael OT: Learning low-level vision. International Journal of Computer Vision 2000, 40(1):25–47. 10.1023/A:1026501619075

    Article  Google Scholar 

  12. 12.

    Liu C, Shum H-Y, Zhang C-S: A two-step approach to hallucinating faces: global parametric model and local nonparametric model. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 1: I-192–I-198.

    Google Scholar 

  13. 13.

    Dedeoglu G, Kanade T, August J: High-zoom video hallucination by exploiting spatio-temporal regularities. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '04), June—July 2004, Washington, DC, USA 2: II-151–II-158.

    Google Scholar 

  14. 14.

    Gupta MD, Rajaram S, Petrovic N, Huang TS: Restoration and recognition in a loop. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '05), June 2005, San Diego, Calif, USA 1: 638–644.

    Google Scholar 

  15. 15.

    Pearl J: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco, Calif, USA; 1988.

    Google Scholar 

  16. 16.

    Yedidia JS, Freeman WT, Weiss Y: Understanding belief propagation and its generalizations. In Exploring Artificial Intelligence in the New Millennium. Morgan Kaufmann, San Francisco, Calif, USA; 2003:239–269.

    Google Scholar 

  17. 17.

    Sudderth EB, Ihler AT, Freeman WT, Willsky AS: Nonparametric belief propagation. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '03), June 2003, Madison, Wis, USA 1: I-605–I-612.

    Google Scholar 

  18. 18.

    Isard M: PAMPAS: real-valued graphical models for computer vision. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '03), June 2003, Madison, Wis, USA 1: I-613–I-620.

    Google Scholar 

  19. 19.

    Geman S, Geman D: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6(6):721–741.

    Article  Google Scholar 

  20. 20.

    Hinton GE: Training products of experts by minimizing contrastive divergence. Neural Computation 2002, 14(8):1771–1800. 10.1162/089976602760128018

    Article  Google Scholar 

  21. 21.

    Silverman BW: Density Estimation for Statistics and Data Analysis. Chapman & Hall/CRC, Boca Raton, Fla, USA; 1986.

    Google Scholar 

  22. 22.

    Hoffman RL, McCullough JW: Segmentation methods for recognition of machine-printed characters. IBM Journal of Research and Development 1971, 15(2):153–165.

    Article  Google Scholar 

  23. 23.

    Gupta MD, Rajaram S, Petrovic N, Huang TS: Non-parametric image super-resolution using multiple images. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Shyamsundar Rajaram.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Rajaram, S., Gupta, M.D., Petrovic, N. et al. Learning-Based Nonparametric Image Super-Resolution. EURASIP J. Adv. Signal Process. 2006, 051306 (2006). https://doi.org/10.1155/ASP/2006/51306

Download citation

Keywords

  • Image Segmentation
  • Kernel Density Estimation
  • Confidence Score
  • Image Patch
  • Inference Problem