Skip to content


  • Research Article
  • Open Access

Overcoming Registration Uncertainty in Image Super-Resolution: Maximize or Marginalize?

  • 1Email author,
  • 1,
  • 1 and
  • 1
EURASIP Journal on Advances in Signal Processing20072007:023565

  • Received: 15 September 2006
  • Accepted: 4 May 2007
  • Published:


In multiple-image super-resolution, a high-resolution image is estimated from a number of lower-resolution images. This usually involves computing the parameters of a generative imaging model (such as geometric and photometric registration, and blur) and obtaining a MAP estimate by minimizing a cost function including an appropriate prior. Two alternative approaches are examined. First, both registrations and the super-resolution image are found simultaneously using a joint MAP optimization. Second, we perform Bayesian integration over the unknown image registration parameters, deriving a cost function whose only variables of interest are the pixel values of the super-resolution image. We also introduce a scheme to learn the parameters of the image prior as part of the super-resolution algorithm. We show examples on a number of real sequences including multiple stills, digital video, and DVDs of movies.


  • Cost Function
  • Image Registration
  • Digital Video
  • Imaging Model
  • Generative Imaging


Authors’ Affiliations

Information Engineering Building, Department of Engineering Science, Parks Road, Oxford, OX1 3PJ, UK


  1. Irani M, Peleg S: Super resolution from image sequences. Proceedings of the 10th International Conference on Pattern Recognition (ICPR '90), June 1990, Atlantic City, NJ, USA 2: 115-120.View ArticleGoogle Scholar
  2. Patti AJ, Sezan MI, Tekalp AM: Robust methods for high-quality stills from interlaced video in the presence of dominant motion. IEEE Transactions on Circuits and Systems for Video Technology 1997,7(2):328-342. 10.1109/76.564111View ArticleGoogle Scholar
  3. Salient Stills
  4. Cheeseman P, Kanefsky B, Kraft R, Stutz J, Hanson R: Super-resolved surface reconstruction from multiple images. In Maximum Entropy and Bayesian Methods. Edited by: Heidbreder GR. Kluwer Academic Publishers, Dordrecht, The Netherlands; 1996:293-308.View ArticleGoogle Scholar
  5. Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116View ArticleGoogle Scholar
  6. Joshi MV, Chaudhuri S, Panuganti R: A learning-based method for image super-resolution from zoomed observations. IEEE Transactions on Systems, Man, and Cybernetics, Part B 2005,35(3):527-537. 10.1109/TSMCB.2005.846647View ArticleGoogle Scholar
  7. Capel DP, Zisserman A: Automated mosaicing with super-resolution zoom. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '98), June 1998, Santa Barbara, Calif, USA 885-891.Google Scholar
  8. Altunbasak Y, Patti AJ, Mersereau RM: Super-resolution still and video reconstruction from MPEG-coded video. IEEE Transactions on Circuits and Systems for Video Technology 2002,12(4):217-226. 10.1109/76.999200View ArticleGoogle Scholar
  9. Baker S, Kanade T: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002,24(9):1167-1183. 10.1109/TPAMI.2002.1033210View ArticleGoogle Scholar
  10. Bascle B, Blake A, Zisserman A: Motion deblurring and super-resolution from an image sequence. In Proceedings of the 4th European Conference on Computer Vision (ECCV '96), April 1996, Cambridge, UK. Volume 2. Springer; 573-582.Google Scholar
  11. Elad M, Feuer A: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Transactions on Image Processing 1997,6(12):1646-1658. 10.1109/83.650118View ArticleGoogle Scholar
  12. Nguyen N, Milanfar P, Golub G: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Transactions on Image Processing 2001,10(9):1299-1308. 10.1109/83.941854MathSciNetView ArticleMATHGoogle Scholar
  13. Schultz RR, Stevenson RL: A Bayesian approach to image expansion for improved definition. IEEE Transactions on Image Processing 1994,3(3):233-242. 10.1109/83.287017View ArticleGoogle Scholar
  14. Robinson D, Milanfar P: Fundamental performance limits in image registration. IEEE Transactions on Image Processing 2004,13(9):1185-1199. 10.1109/TIP.2004.832923View ArticleGoogle Scholar
  15. Tipping ME, Bishop CM: Bayesian image super-resolution. Proceedings of Advances in Neural Information Processing Systems 15 (NIPS '02), December 2002, Vancouver, British Columbia, Canada 1279-1286.Google Scholar
  16. Borman S: Topics in multiframe superresolution restoration, Ph.D. thesis. University of Notre Dame, Notre Dame, Ind, USA; 2004.Google Scholar
  17. Borman S, Stevenson RL: Simultaneous multi-frame MAP super-resolution video enhancement using spatio-temporal priors. Proceedings of International Conference on Image Processing (ICIP '99), October 1999, Kobe, Japan 3: 469-473.View ArticleGoogle Scholar
  18. Capel DP: Image Mosaicing and Super-Resolution, Distinguished Dissertations. Springer, New York, NY, USA; 2004.View ArticleMATHGoogle Scholar
  19. Farsiu S, Elad M, Milanfar P: A practical approach to super-resolution. Visual Communications and Image Processing, January 2006, San Jose, Calif, USA, Proceedings of SPIE 6077:Google Scholar
  20. Pickup LC, Roberts SJ, Zisserman A: A sampled texture prior for image super-resolution. Proceedings of Advances in Neural Information Processing Systems 16 (NIPS '03), December 2004, Vancouver, British Columbia, Canada 1587-1594.Google Scholar
  21. Pickup LC, Capel DP, Roberts SJ, Zisserman A: Bayesian image super-resolution, continued. Advances in Neural Information Processing Systems 19, December 2006, Cambridge, Mass, USA 1089-1096.Google Scholar
  22. Pickup LC, Roberts SJ, Zisserman A: Optimizing and learning for super-resolution. Proceedings of the 17th British Machine Vision Conference (BMVC '06), September 2006, Edinburgh, UKGoogle Scholar
  23. Hartley RI, Zisserman A: Multiple View Geometry in Computer Vision. 2nd edition. Cambridge University Press, Cambridge, UK; 2004.View ArticleMATHGoogle Scholar
  24. Nabney I: NETLAB: Algorithms for Pattern Recognition. Springer, New York, NY, USA; 2002.MATHGoogle Scholar
  25. Triggs B, McLauchlan PF, Hartley RI, Fitzgibbon AW: Bundle adjustment—a modern synthesis. In Proceedings of International Workshop on Vision Algorithms on Vision Algorithms: Theory and Practice, September 1999, Corfu, Greece, Lecture Notes in Computer Science. Volume 1883. Edited by: Triggs B, Zisserman A, Szeliski R. Springer; 298-372.View ArticleGoogle Scholar
  26. Hardie RC, Barnard KJ, Bognar JG, Armstrong EE, Watson EA: High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Optical Engineering 1998,37(1):247-260. 10.1117/1.601623View ArticleGoogle Scholar
  27. Reeves SJ, Mersereau RM: Blur identification by the method of generalized cross-validation. IEEE Transactions on Image Processing 1992,1(3):301-311. 10.1109/83.148604View ArticleGoogle Scholar
  28. Lagendijk RL, Tekalp AM, Biemond J: Maximum likelihood image and blur identification: a unifying approach. Optical Engineering 1990,29(5):422-435. 10.1117/12.55611View ArticleGoogle Scholar
  29. Kundur D, Hatzinakos D: Blind image deconvolution. IEEE Signal Processing Magazine 1996,13(3):43-64. 10.1109/79.489268View ArticleGoogle Scholar
  30. Freeman WT, Jones TR, Pasztor EC: Example-based super-resolution. IEEE Computer Graphics and Applications 2002,22(2):56-65. 10.1109/38.988747View ArticleGoogle Scholar