- Research Article
- Open Access
Overcoming Registration Uncertainty in Image Super-Resolution: Maximize or Marginalize?
EURASIP Journal on Advances in Signal Processing volume 2007, Article number: 023565 (2007)
In multiple-image super-resolution, a high-resolution image is estimated from a number of lower-resolution images. This usually involves computing the parameters of a generative imaging model (such as geometric and photometric registration, and blur) and obtaining a MAP estimate by minimizing a cost function including an appropriate prior. Two alternative approaches are examined. First, both registrations and the super-resolution image are found simultaneously using a joint MAP optimization. Second, we perform Bayesian integration over the unknown image registration parameters, deriving a cost function whose only variables of interest are the pixel values of the super-resolution image. We also introduce a scheme to learn the parameters of the image prior as part of the super-resolution algorithm. We show examples on a number of real sequences including multiple stills, digital video, and DVDs of movies.
Irani M, Peleg S: Super resolution from image sequences. Proceedings of the 10th International Conference on Pattern Recognition (ICPR '90), June 1990, Atlantic City, NJ, USA 2: 115–120.
Patti AJ, Sezan MI, Tekalp AM: Robust methods for high-quality stills from interlaced video in the presence of dominant motion. IEEE Transactions on Circuits and Systems for Video Technology 1997,7(2):328-342. 10.1109/76.564111
Salient Stills https://doi.org/www.salientstills.com/
Cheeseman P, Kanefsky B, Kraft R, Stutz J, Hanson R: Super-resolved surface reconstruction from multiple images. In Maximum Entropy and Bayesian Methods. Edited by: Heidbreder GR. Kluwer Academic Publishers, Dordrecht, The Netherlands; 1996:293-308.
Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116
Joshi MV, Chaudhuri S, Panuganti R: A learning-based method for image super-resolution from zoomed observations. IEEE Transactions on Systems, Man, and Cybernetics, Part B 2005,35(3):527-537. 10.1109/TSMCB.2005.846647
Capel DP, Zisserman A: Automated mosaicing with super-resolution zoom. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '98), June 1998, Santa Barbara, Calif, USA 885–891.
Altunbasak Y, Patti AJ, Mersereau RM: Super-resolution still and video reconstruction from MPEG-coded video. IEEE Transactions on Circuits and Systems for Video Technology 2002,12(4):217-226. 10.1109/76.999200
Baker S, Kanade T: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002,24(9):1167-1183. 10.1109/TPAMI.2002.1033210
Bascle B, Blake A, Zisserman A: Motion deblurring and super-resolution from an image sequence. In Proceedings of the 4th European Conference on Computer Vision (ECCV '96), April 1996, Cambridge, UK. Volume 2. Springer; 573–582.
Elad M, Feuer A: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Transactions on Image Processing 1997,6(12):1646-1658. 10.1109/83.650118
Nguyen N, Milanfar P, Golub G: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Transactions on Image Processing 2001,10(9):1299-1308. 10.1109/83.941854
Schultz RR, Stevenson RL: A Bayesian approach to image expansion for improved definition. IEEE Transactions on Image Processing 1994,3(3):233-242. 10.1109/83.287017
Robinson D, Milanfar P: Fundamental performance limits in image registration. IEEE Transactions on Image Processing 2004,13(9):1185-1199. 10.1109/TIP.2004.832923
Tipping ME, Bishop CM: Bayesian image super-resolution. Proceedings of Advances in Neural Information Processing Systems 15 (NIPS '02), December 2002, Vancouver, British Columbia, Canada 1279–1286.
Borman S: Topics in multiframe superresolution restoration, Ph.D. thesis. University of Notre Dame, Notre Dame, Ind, USA; 2004.
Borman S, Stevenson RL: Simultaneous multi-frame MAP super-resolution video enhancement using spatio-temporal priors. Proceedings of International Conference on Image Processing (ICIP '99), October 1999, Kobe, Japan 3: 469–473.
Capel DP: Image Mosaicing and Super-Resolution, Distinguished Dissertations. Springer, New York, NY, USA; 2004.
Farsiu S, Elad M, Milanfar P: A practical approach to super-resolution. Visual Communications and Image Processing, January 2006, San Jose, Calif, USA, Proceedings of SPIE 6077:
Pickup LC, Roberts SJ, Zisserman A: A sampled texture prior for image super-resolution. Proceedings of Advances in Neural Information Processing Systems 16 (NIPS '03), December 2004, Vancouver, British Columbia, Canada 1587–1594.
Pickup LC, Capel DP, Roberts SJ, Zisserman A: Bayesian image super-resolution, continued. Advances in Neural Information Processing Systems 19, December 2006, Cambridge, Mass, USA 1089–1096.
Pickup LC, Roberts SJ, Zisserman A: Optimizing and learning for super-resolution. Proceedings of the 17th British Machine Vision Conference (BMVC '06), September 2006, Edinburgh, UK
Hartley RI, Zisserman A: Multiple View Geometry in Computer Vision. 2nd edition. Cambridge University Press, Cambridge, UK; 2004.
Nabney I: NETLAB: Algorithms for Pattern Recognition. Springer, New York, NY, USA; 2002.
Triggs B, McLauchlan PF, Hartley RI, Fitzgibbon AW: Bundle adjustment—a modern synthesis. In Proceedings of International Workshop on Vision Algorithms on Vision Algorithms: Theory and Practice, September 1999, Corfu, Greece, Lecture Notes in Computer Science. Volume 1883. Edited by: Triggs B, Zisserman A, Szeliski R. Springer; 298–372.
Hardie RC, Barnard KJ, Bognar JG, Armstrong EE, Watson EA: High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Optical Engineering 1998,37(1):247-260. 10.1117/1.601623
Reeves SJ, Mersereau RM: Blur identification by the method of generalized cross-validation. IEEE Transactions on Image Processing 1992,1(3):301-311. 10.1109/83.148604
Lagendijk RL, Tekalp AM, Biemond J: Maximum likelihood image and blur identification: a unifying approach. Optical Engineering 1990,29(5):422-435. 10.1117/12.55611
Kundur D, Hatzinakos D: Blind image deconvolution. IEEE Signal Processing Magazine 1996,13(3):43-64. 10.1109/79.489268
Freeman WT, Jones TR, Pasztor EC: Example-based super-resolution. IEEE Computer Graphics and Applications 2002,22(2):56-65. 10.1109/38.988747