- Research Article
- Open Access
Learning-Based Nonparametric Image Super-Resolution
EURASIP Journal on Advances in Signal Processing volume 2006, Article number: 051306 (2006)
We present a novel learning-based framework for zooming and recognizing images of digits obtained from vehicle registration plates, which have been blurred using an unknown kernel. We model the image as an undirected graphical model over image patches in which the compatibility functions are represented as nonparametric kernel densities. The crucial feature of this work is an iterative loop that alternates between super-resolution and restoration stages. A machine-learning-based framework has been used for restoration which also models spatial zooming. Image segmentation is done by a column-variance estimation-based "dissection" algorithm. Initially, the compatibility functions are learned by nonparametric kernel density estimation, using random samples from the training data. Next, we solve the inference problem by using an extended version of the nonparametric belief propagation algorithm, in which we introduce the notion of partial messages. Finally, we recognize the super-resolved and restored images. The resulting confidence scores are used to sample from the training set to better learn the compatibility functions.
Bascle B, Blake A, Zisserman A: Motion deblurring and super-resolution from an image sequence. Proceedings of 4th European Conference on Computer Vision (ECCV '96), April 1996, Cambridge, UK 2: 573–582.
Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997, 6(12):1621–1633. 10.1109/83.650116
Irani M, Peleg S: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991, 53(3):231–239. 10.1016/1049-9652(91)90045-L
Kim SP, Bose NK, Valenzuela HM: Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Transactions on Acoustics, Speech and Signal Processing 1990, 38(6):1013–1027. 10.1109/29.56062
Schultz RR, Stevenson RL: Extraction of high-resolution frames from video sequences. IEEE Transactions on Image Processing 1996, 5(6):996–1011. 10.1109/83.503915
Stark H, Oskoui P: High-resolution image recovery from image-plane arrays, using convex projections. Journal of the Optical Society of America. A, Optics and image science. 1989, 6(11):1715–1726. 10.1364/JOSAA.6.001715
Chiang M-C, Boult TE: Local blur estimation and super-resolution. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '97), June 1997, San Juan, Puerto Rico, USA 821–826.
Baker S, Kanade T: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(9):1167–1183. 10.1109/TPAMI.2002.1033210
Bishop CM, Blake A, Marthi B: Super-resolution enhancement of video. Proceedings of 9th International Workshop on Artificial Intelligence and Statistics (AISTATS '03), January 2003, Key West, Fla, USA
Capel D, Zisserman A: Super-resolution from multiple views using learnt image models. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 2: II-627–II-634.
Freeman WT, Pasztor EC, Carmichael OT: Learning low-level vision. International Journal of Computer Vision 2000, 40(1):25–47. 10.1023/A:1026501619075
Liu C, Shum H-Y, Zhang C-S: A two-step approach to hallucinating faces: global parametric model and local nonparametric model. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 1: I-192–I-198.
Dedeoglu G, Kanade T, August J: High-zoom video hallucination by exploiting spatio-temporal regularities. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '04), June—July 2004, Washington, DC, USA 2: II-151–II-158.
Gupta MD, Rajaram S, Petrovic N, Huang TS: Restoration and recognition in a loop. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '05), June 2005, San Diego, Calif, USA 1: 638–644.
Pearl J: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco, Calif, USA; 1988.
Yedidia JS, Freeman WT, Weiss Y: Understanding belief propagation and its generalizations. In Exploring Artificial Intelligence in the New Millennium. Morgan Kaufmann, San Francisco, Calif, USA; 2003:239–269.
Sudderth EB, Ihler AT, Freeman WT, Willsky AS: Nonparametric belief propagation. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '03), June 2003, Madison, Wis, USA 1: I-605–I-612.
Isard M: PAMPAS: real-valued graphical models for computer vision. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '03), June 2003, Madison, Wis, USA 1: I-613–I-620.
Geman S, Geman D: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6(6):721–741.
Hinton GE: Training products of experts by minimizing contrastive divergence. Neural Computation 2002, 14(8):1771–1800. 10.1162/089976602760128018
Silverman BW: Density Estimation for Statistics and Data Analysis. Chapman & Hall/CRC, Boca Raton, Fla, USA; 1986.
Hoffman RL, McCullough JW: Segmentation methods for recognition of machine-printed characters. IBM Journal of Research and Development 1971, 15(2):153–165.
Gupta MD, Rajaram S, Petrovic N, Huang TS: Non-parametric image super-resolution using multiple images. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy
About this article
Cite this article
Rajaram, S., Gupta, M.D., Petrovic, N. et al. Learning-Based Nonparametric Image Super-Resolution. EURASIP J. Adv. Signal Process. 2006, 051306 (2006) doi:10.1155/ASP/2006/51306
- Image Segmentation
- Kernel Density Estimation
- Confidence Score
- Image Patch
- Inference Problem