Skip to main content
EURASIP Journal on Advances in Signal Processing
Download PDF

Overcoming Registration Uncertainty in Image Super-Resolution: Maximize or Marginalize?

EURASIP Journal on Advances in Signal Processing volume 2007, Article number: 023565 (2007) Cite this article

Abstract

In multiple-image super-resolution, a high-resolution image is estimated from a number of lower-resolution images. This usually involves computing the parameters of a generative imaging model (such as geometric and photometric registration, and blur) and obtaining a MAP estimate by minimizing a cost function including an appropriate prior. Two alternative approaches are examined. First, both registrations and the super-resolution image are found simultaneously using a joint MAP optimization. Second, we perform Bayesian integration over the unknown image registration parameters, deriving a cost function whose only variables of interest are the pixel values of the super-resolution image. We also introduce a scheme to learn the parameters of the image prior as part of the super-resolution algorithm. We show examples on a number of real sequences including multiple stills, digital video, and DVDs of movies.

References

  1. Irani M, Peleg S: Super resolution from image sequences. Proceedings of the 10th International Conference on Pattern Recognition (ICPR '90), June 1990, Atlantic City, NJ, USA 2: 115–120.

    Article  Google Scholar 

  2. Patti AJ, Sezan MI, Tekalp AM: Robust methods for high-quality stills from interlaced video in the presence of dominant motion. IEEE Transactions on Circuits and Systems for Video Technology 1997,7(2):328-342. 10.1109/76.564111

    Article  Google Scholar 

  3. Salient Stills https://doi.org/www.salientstills.com/

  4. Cheeseman P, Kanefsky B, Kraft R, Stutz J, Hanson R: Super-resolved surface reconstruction from multiple images. In Maximum Entropy and Bayesian Methods. Edited by: Heidbreder GR. Kluwer Academic Publishers, Dordrecht, The Netherlands; 1996:293-308.

    Chapter  Google Scholar 

  5. Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116

    Article  Google Scholar 

  6. Joshi MV, Chaudhuri S, Panuganti R: A learning-based method for image super-resolution from zoomed observations. IEEE Transactions on Systems, Man, and Cybernetics, Part B 2005,35(3):527-537. 10.1109/TSMCB.2005.846647

    Article  Google Scholar 

  7. Capel DP, Zisserman A: Automated mosaicing with super-resolution zoom. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '98), June 1998, Santa Barbara, Calif, USA 885–891.

    Google Scholar 

  8. Altunbasak Y, Patti AJ, Mersereau RM: Super-resolution still and video reconstruction from MPEG-coded video. IEEE Transactions on Circuits and Systems for Video Technology 2002,12(4):217-226. 10.1109/76.999200

    Article  Google Scholar 

  9. Baker S, Kanade T: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002,24(9):1167-1183. 10.1109/TPAMI.2002.1033210

    Article  Google Scholar 

  10. Bascle B, Blake A, Zisserman A: Motion deblurring and super-resolution from an image sequence. In Proceedings of the 4th European Conference on Computer Vision (ECCV '96), April 1996, Cambridge, UK. Volume 2. Springer; 573–582.

    Google Scholar 

  11. Elad M, Feuer A: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Transactions on Image Processing 1997,6(12):1646-1658. 10.1109/83.650118

    Article  Google Scholar 

  12. Nguyen N, Milanfar P, Golub G: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Transactions on Image Processing 2001,10(9):1299-1308. 10.1109/83.941854

    Article  MathSciNet  Google Scholar 

  13. Schultz RR, Stevenson RL: A Bayesian approach to image expansion for improved definition. IEEE Transactions on Image Processing 1994,3(3):233-242. 10.1109/83.287017

    Article  Google Scholar 

  14. Robinson D, Milanfar P: Fundamental performance limits in image registration. IEEE Transactions on Image Processing 2004,13(9):1185-1199. 10.1109/TIP.2004.832923

    Article  Google Scholar 

  15. Tipping ME, Bishop CM: Bayesian image super-resolution. Proceedings of Advances in Neural Information Processing Systems 15 (NIPS '02), December 2002, Vancouver, British Columbia, Canada 1279–1286.

    Google Scholar 

  16. Borman S: Topics in multiframe superresolution restoration, Ph.D. thesis. University of Notre Dame, Notre Dame, Ind, USA; 2004.

    Google Scholar 

  17. Borman S, Stevenson RL: Simultaneous multi-frame MAP super-resolution video enhancement using spatio-temporal priors. Proceedings of International Conference on Image Processing (ICIP '99), October 1999, Kobe, Japan 3: 469–473.

    Article  Google Scholar 

  18. Capel DP: Image Mosaicing and Super-Resolution, Distinguished Dissertations. Springer, New York, NY, USA; 2004.

    Book  Google Scholar 

  19. Farsiu S, Elad M, Milanfar P: A practical approach to super-resolution. Visual Communications and Image Processing, January 2006, San Jose, Calif, USA, Proceedings of SPIE 6077:

    Google Scholar 

  20. Pickup LC, Roberts SJ, Zisserman A: A sampled texture prior for image super-resolution. Proceedings of Advances in Neural Information Processing Systems 16 (NIPS '03), December 2004, Vancouver, British Columbia, Canada 1587–1594.

    Google Scholar 

  21. Pickup LC, Capel DP, Roberts SJ, Zisserman A: Bayesian image super-resolution, continued. Advances in Neural Information Processing Systems 19, December 2006, Cambridge, Mass, USA 1089–1096.

    Google Scholar 

  22. Pickup LC, Roberts SJ, Zisserman A: Optimizing and learning for super-resolution. Proceedings of the 17th British Machine Vision Conference (BMVC '06), September 2006, Edinburgh, UK

    Google Scholar 

  23. Hartley RI, Zisserman A: Multiple View Geometry in Computer Vision. 2nd edition. Cambridge University Press, Cambridge, UK; 2004.

    Book  Google Scholar 

  24. Nabney I: NETLAB: Algorithms for Pattern Recognition. Springer, New York, NY, USA; 2002.

    MATH  Google Scholar 

  25. Triggs B, McLauchlan PF, Hartley RI, Fitzgibbon AW: Bundle adjustment—a modern synthesis. In Proceedings of International Workshop on Vision Algorithms on Vision Algorithms: Theory and Practice, September 1999, Corfu, Greece, Lecture Notes in Computer Science. Volume 1883. Edited by: Triggs B, Zisserman A, Szeliski R. Springer; 298–372.

    Chapter  Google Scholar 

  26. Hardie RC, Barnard KJ, Bognar JG, Armstrong EE, Watson EA: High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Optical Engineering 1998,37(1):247-260. 10.1117/1.601623

    Article  Google Scholar 

  27. Reeves SJ, Mersereau RM: Blur identification by the method of generalized cross-validation. IEEE Transactions on Image Processing 1992,1(3):301-311. 10.1109/83.148604

    Article  Google Scholar 

  28. Lagendijk RL, Tekalp AM, Biemond J: Maximum likelihood image and blur identification: a unifying approach. Optical Engineering 1990,29(5):422-435. 10.1117/12.55611

    Article  Google Scholar 

  29. Kundur D, Hatzinakos D: Blind image deconvolution. IEEE Signal Processing Magazine 1996,13(3):43-64. 10.1109/79.489268

    Article  Google Scholar 

  30. Freeman WT, Jones TR, Pasztor EC: Example-based super-resolution. IEEE Computer Graphics and Applications 2002,22(2):56-65. 10.1109/38.988747

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Information Engineering Building, Department of Engineering Science, Parks Road, Oxford, OX1 3PJ, UK

    Lyndsey C Pickup, David P Capel, Stephen J Roberts & Andrew Zisserman

Authors
  1. Lyndsey C Pickup
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David P Capel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stephen J Roberts
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrew Zisserman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lyndsey C Pickup.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://doi.org/creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Pickup, L.C., Capel, D.P., Roberts, S.J. et al. Overcoming Registration Uncertainty in Image Super-Resolution: Maximize or Marginalize?. EURASIP J. Adv. Signal Process. 2007, 023565 (2007). https://doi.org/10.1155/2007/23565

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/2007/23565

Keywords

  • Cost Function
  • Image Registration
  • Digital Video
  • Imaging Model
  • Generative Imaging