Skip to content

Advertisement

  • Research Article
  • Open Access

Overcoming Registration Uncertainty in Image Super-Resolution: Maximize or Marginalize?

  • 1Email author,
  • 1,
  • 1 and
  • 1
EURASIP Journal on Advances in Signal Processing20072007:023565

https://doi.org/10.1155/2007/23565

  • Received: 15 September 2006
  • Accepted: 4 May 2007
  • Published:

Abstract

In multiple-image super-resolution, a high-resolution image is estimated from a number of lower-resolution images. This usually involves computing the parameters of a generative imaging model (such as geometric and photometric registration, and blur) and obtaining a MAP estimate by minimizing a cost function including an appropriate prior. Two alternative approaches are examined. First, both registrations and the super-resolution image are found simultaneously using a joint MAP optimization. Second, we perform Bayesian integration over the unknown image registration parameters, deriving a cost function whose only variables of interest are the pixel values of the super-resolution image. We also introduce a scheme to learn the parameters of the image prior as part of the super-resolution algorithm. We show examples on a number of real sequences including multiple stills, digital video, and DVDs of movies.

Keywords

  • Cost Function
  • Image Registration
  • Digital Video
  • Imaging Model
  • Generative Imaging

[123456789101112131415161718192021222324252627282930]

Authors’ Affiliations

(1)
Information Engineering Building, Department of Engineering Science, Parks Road, Oxford, OX1 3PJ, UK

References

  1. Irani M, Peleg S: Super resolution from image sequences. Proceedings of the 10th International Conference on Pattern Recognition (ICPR '90), June 1990, Atlantic City, NJ, USA 2: 115-120.View ArticleGoogle Scholar
  2. Patti AJ, Sezan MI, Tekalp AM: Robust methods for high-quality stills from interlaced video in the presence of dominant motion. IEEE Transactions on Circuits and Systems for Video Technology 1997,7(2):328-342. 10.1109/76.564111View ArticleGoogle Scholar
  3. Salient Stills http://www.salientstills.com/
  4. Cheeseman P, Kanefsky B, Kraft R, Stutz J, Hanson R: Super-resolved surface reconstruction from multiple images. In Maximum Entropy and Bayesian Methods. Edited by: Heidbreder GR. Kluwer Academic Publishers, Dordrecht, The Netherlands; 1996:293-308.View ArticleGoogle Scholar
  5. Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116View ArticleGoogle Scholar
  6. Joshi MV, Chaudhuri S, Panuganti R: A learning-based method for image super-resolution from zoomed observations. IEEE Transactions on Systems, Man, and Cybernetics, Part B 2005,35(3):527-537. 10.1109/TSMCB.2005.846647View ArticleGoogle Scholar
  7. Capel DP, Zisserman A: Automated mosaicing with super-resolution zoom. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '98), June 1998, Santa Barbara, Calif, USA 885-891.Google Scholar
  8. Altunbasak Y, Patti AJ, Mersereau RM: Super-resolution still and video reconstruction from MPEG-coded video. IEEE Transactions on Circuits and Systems for Video Technology 2002,12(4):217-226. 10.1109/76.999200View ArticleGoogle Scholar
  9. Baker S, Kanade T: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002,24(9):1167-1183. 10.1109/TPAMI.2002.1033210View ArticleGoogle Scholar
  10. Bascle B, Blake A, Zisserman A: Motion deblurring and super-resolution from an image sequence. In Proceedings of the 4th European Conference on Computer Vision (ECCV '96), April 1996, Cambridge, UK. Volume 2. Springer; 573-582.Google Scholar
  11. Elad M, Feuer A: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Transactions on Image Processing 1997,6(12):1646-1658. 10.1109/83.650118View ArticleGoogle Scholar
  12. Nguyen N, Milanfar P, Golub G: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Transactions on Image Processing 2001,10(9):1299-1308. 10.1109/83.941854MathSciNetView ArticleMATHGoogle Scholar
  13. Schultz RR, Stevenson RL: A Bayesian approach to image expansion for improved definition. IEEE Transactions on Image Processing 1994,3(3):233-242. 10.1109/83.287017View ArticleGoogle Scholar
  14. Robinson D, Milanfar P: Fundamental performance limits in image registration. IEEE Transactions on Image Processing 2004,13(9):1185-1199. 10.1109/TIP.2004.832923View ArticleGoogle Scholar
  15. Tipping ME, Bishop CM: Bayesian image super-resolution. Proceedings of Advances in Neural Information Processing Systems 15 (NIPS '02), December 2002, Vancouver, British Columbia, Canada 1279-1286.Google Scholar
  16. Borman S: Topics in multiframe superresolution restoration, Ph.D. thesis. University of Notre Dame, Notre Dame, Ind, USA; 2004.Google Scholar
  17. Borman S, Stevenson RL: Simultaneous multi-frame MAP super-resolution video enhancement using spatio-temporal priors. Proceedings of International Conference on Image Processing (ICIP '99), October 1999, Kobe, Japan 3: 469-473.View ArticleGoogle Scholar
  18. Capel DP: Image Mosaicing and Super-Resolution, Distinguished Dissertations. Springer, New York, NY, USA; 2004.View ArticleMATHGoogle Scholar
  19. Farsiu S, Elad M, Milanfar P: A practical approach to super-resolution. Visual Communications and Image Processing, January 2006, San Jose, Calif, USA, Proceedings of SPIE 6077:Google Scholar
  20. Pickup LC, Roberts SJ, Zisserman A: A sampled texture prior for image super-resolution. Proceedings of Advances in Neural Information Processing Systems 16 (NIPS '03), December 2004, Vancouver, British Columbia, Canada 1587-1594.Google Scholar
  21. Pickup LC, Capel DP, Roberts SJ, Zisserman A: Bayesian image super-resolution, continued. Advances in Neural Information Processing Systems 19, December 2006, Cambridge, Mass, USA 1089-1096.Google Scholar
  22. Pickup LC, Roberts SJ, Zisserman A: Optimizing and learning for super-resolution. Proceedings of the 17th British Machine Vision Conference (BMVC '06), September 2006, Edinburgh, UKGoogle Scholar
  23. Hartley RI, Zisserman A: Multiple View Geometry in Computer Vision. 2nd edition. Cambridge University Press, Cambridge, UK; 2004.View ArticleMATHGoogle Scholar
  24. Nabney I: NETLAB: Algorithms for Pattern Recognition. Springer, New York, NY, USA; 2002.MATHGoogle Scholar
  25. Triggs B, McLauchlan PF, Hartley RI, Fitzgibbon AW: Bundle adjustment—a modern synthesis. In Proceedings of International Workshop on Vision Algorithms on Vision Algorithms: Theory and Practice, September 1999, Corfu, Greece, Lecture Notes in Computer Science. Volume 1883. Edited by: Triggs B, Zisserman A, Szeliski R. Springer; 298-372.View ArticleGoogle Scholar
  26. Hardie RC, Barnard KJ, Bognar JG, Armstrong EE, Watson EA: High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Optical Engineering 1998,37(1):247-260. 10.1117/1.601623View ArticleGoogle Scholar
  27. Reeves SJ, Mersereau RM: Blur identification by the method of generalized cross-validation. IEEE Transactions on Image Processing 1992,1(3):301-311. 10.1109/83.148604View ArticleGoogle Scholar
  28. Lagendijk RL, Tekalp AM, Biemond J: Maximum likelihood image and blur identification: a unifying approach. Optical Engineering 1990,29(5):422-435. 10.1117/12.55611View ArticleGoogle Scholar
  29. Kundur D, Hatzinakos D: Blind image deconvolution. IEEE Signal Processing Magazine 1996,13(3):43-64. 10.1109/79.489268View ArticleGoogle Scholar
  30. Freeman WT, Jones TR, Pasztor EC: Example-based super-resolution. IEEE Computer Graphics and Applications 2002,22(2):56-65. 10.1109/38.988747View ArticleGoogle Scholar

Copyright

© Lyndsey C. Pickup et al.s 2007

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement