Skip to main content


We're creating a new version of this page. See preview

  • Research Article
  • Open Access

A Total Variation Regularization Based Super-Resolution Reconstruction Algorithm for Digital Video

  • 1Email author,
  • 1, 2,
  • 3 and
  • 2
EURASIP Journal on Advances in Signal Processing20072007:074585

  • Received: 13 September 2006
  • Accepted: 21 April 2007
  • Published:


Super-resolution (SR) reconstruction technique is capable of producing a high-resolution image from a sequence of low-resolution images. In this paper, we study an efficient SR algorithm for digital video. To effectively deal with the intractable problems in SR video reconstruction, such as inevitable motion estimation errors, noise, blurring, missing regions, and compression artifacts, the total variation (TV) regularization is employed in the reconstruction model. We use the fixed-point iteration method and preconditioning techniques to efficiently solve the associated nonlinear Euler-Lagrange equations of the corresponding variational problem in SR. The proposed algorithm has been tested in several cases of motion and degradation. It is also compared with the Laplacian regularization-based SR algorithm and other TV-based SR algorithms. Experimental results are presented to illustrate the effectiveness of the proposed algorithm.


  • Reconstruction Algorithm
  • Motion Estimation
  • Reconstruction Technique
  • Digital Video
  • Reconstruction Model

Authors’ Affiliations

Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong
The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, Hubei, China
Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong


  1. Tsai RY, Huang TS: Multi-frame image restoration and registration. Advances in Computer Vision and Image Processing 1984,1(2):317-339.Google Scholar
  2. Kim SP, Bose NK, Valenzuela HM: Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Transactions on Acoustics, Speech, and Signal Processing 1990,38(6):1013-1027. 10.1109/29.56062Google Scholar
  3. Kim SP, Su W-Y: Recursive high-resolution reconstruction of blurred multiframe images. IEEE Transactions on Image Processing 1993,2(4):534-539. 10.1109/83.242363Google Scholar
  4. Rhee S, Kang MG: Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Optical Engineering 1999,38(8):1348-1356. 10.1117/1.602177Google Scholar
  5. Chan RH, Chan TF, Shen L, Shen Z: Wavelet algorithms for high-resolution image reconstruction. SIAM Journal of Scientific Computing 2003,24(4):1408-1432. 10.1137/S1064827500383123MathSciNetMATHGoogle Scholar
  6. Ng MK, Sze CK, Yung SP: Wavelet algorithms for deblurring models. International Journal of Imaging Systems and Technology 2004,14(3):113-121. 10.1002/ima.20014Google Scholar
  7. Nguyen N, Milanfar P: A wavelet-based interpolation-restoration method for superresolution (wavelet superresolution). Circuits, Systems, and Signal Processing 2000,19(4):321-338. 10.1007/BF01200891MATHGoogle Scholar
  8. Ur H, Gross D: Improved resolution from subpixel shifted pictures. CVGIP: Graphical Models and Image Processing 1992,54(2):181-186. 10.1016/1049-9652(92)90065-6Google Scholar
  9. Irani M, Peleg S: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991,53(3):231-239. 10.1016/1049-9652(91)90045-LGoogle Scholar
  10. Stark H, Oskoui P: High-resolution image recovery from image-plane arrays, using convex projections. Journal of the Optical Society of America A: Optics and Image Science, and Vision 1989,6(11):1715-1726. 10.1364/JOSAA.6.001715Google Scholar
  11. Tekalp AM, Ozkan MK, Sezan MI: High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '92), March 1992, San Francisco, Calif, USA 3: 169–172.Google Scholar
  12. Patti AJ, Sezan MI, Tekalp AM: High-resolution image reconstruction from a low-resolution image sequence in the presence of time-varying motion blur. Proceedings of IEEE International Conference Image Processing (ICIP '94), November 1994, Austin, Tex, USA 1: 343–347.Google Scholar
  13. Patti AJ, Sezan MI, Tekalp AM: Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Transactions on Image Processing 1997,6(8):1064-1076. 10.1109/83.605404Google Scholar
  14. Tom BC, Katsaggelos AK: Reconstruction of a high-resolution image from multiple-degraded misregistered low-resolution images. Visual Communications and Image Processing, September 1994, Chicago, Ill, USA, Proceedings of SPIE 2308: 971–981.Google Scholar
  15. Schultz RR, Stevenson RL: Extraction of high-resolution frames from video sequences. IEEE Transactions on Image Processing 1996,5(6):996-1011. 10.1109/83.503915Google Scholar
  16. Hardie RC, Tuinstra TR, Bognar J, Barnard KJ, Armstrong EE: High resolution image reconstruction from digital video with global and non-global scene motion. Proceedings of IEEE International Conference on Image Processing (ICIP '97), October 1997, Santa Barbara, Calif, USA 1: 153–156.Google Scholar
  17. Elad M, Feuer A: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Transactions on Image Processing 1997,6(12):1646-1658. 10.1109/83.650118Google Scholar
  18. Elad M, Feuer A: Superresolution restoration of an image sequence: adaptive filtering approach. IEEE Transactions on Image Processing 1999,8(3):387-395. 10.1109/83.748893Google Scholar
  19. Chung J, Haber E, Nagy J: Numerical methods for coupled super-resolution. Inverse Problems 2006,22(4):1261-1272. 10.1088/0266-5611/22/4/009MathSciNetMATHGoogle Scholar
  20. Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116Google Scholar
  21. Woods NA, Galatsanos NP, Katsaggelos AK: Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images. IEEE Transactions on Image Processing 2006,15(1):201-213.MathSciNetGoogle Scholar
  22. Shen H, Zhang L, Huang B, Li P: A MAP approach for joint motion estimation, segmentation, and super resolution. IEEE Transactions on Image Processing 2007,16(2):479-490.MathSciNetGoogle Scholar
  23. Sasahara R, Hasegawa H, Yamada I, Sakaniwa K: A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy 1: 857–860.Google Scholar
  24. Farsiu S, Elad M, Milanfar P: Multiframe demosaicing and super-resolution of color images. IEEE Transactions on Image Processing 2006,15(1):141-159.Google Scholar
  25. Akgun T, Altunbasak Y, Mersereau RM: Super-resolution reconstruction of hyperspectral images. IEEE Transactions on Image Processing 2005,14(11):1860-1875.Google Scholar
  26. Segall CA, Katsaggelos AK, Molina R, Mateos J: Bayesian resolution enhancement of compressed video. IEEE Transactions on Image Processing 2004,13(7):898-910. 10.1109/TIP.2004.827230Google Scholar
  27. Segall CA, Molina R, Katsaggelos AK: High-resolution images from low-resolution compressed video. IEEE Signal Processing Magazine 2003,20(3):37-48. 10.1109/MSP.2003.1203208Google Scholar
  28. Capel D, Zisserman A: Super-resolution enhancement of text image sequences. Proceedings of the 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 600–605.Google Scholar
  29. Han YB, Wu LN: Super resolution reconstruction of video sequence based on total variation. Proceedings of International Symposium on Intelligent Multimedia, Video and Speech Processing (ISIMP '04), October 2004, Hong Kong 575–578.Google Scholar
  30. Vazquez C, Aly H, Dubois E, Mitiche A: Motion compensated super-resolution of video by level sets evolution. Proceedings of IEEE International Conference on Image Processing (ICIP '04), October 2004, Singapore 3: 1767–1770.Google Scholar
  31. Farsiu S, Robinson MD, Elad M, Milanfar P: Fast and robust multiframe super resolution. IEEE Transactions on Image Processing 2004,13(10):1327-1344. 10.1109/TIP.2004.834669Google Scholar
  32. Chan TF, Shen J: Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics 2002,62(3):1019-1043. 10.1137/S0036139900368844MathSciNetMATHGoogle Scholar
  33. Borman S, Stevenson RL: Spatial resolution enhancement of low-resolution image sequences: a comprehensive review with directions for future research. Laboratory for Image and Signal Analysis (LISA), University of Notre Dame, Notre Dame, Ind, USA; 1998.Google Scholar
  34. Park SC, Park MK, Kang MG: Super-resolution image reconstruction: a technical overview. IEEE Signal Processing Magazine 2003,20(3):21-36. 10.1109/MSP.2003.1203207Google Scholar
  35. Nguyen N, Milanfar P, Golub G: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Transactions on Image Processing 2001,10(9):1299-1308. 10.1109/83.941854MathSciNetMATHGoogle Scholar
  36. Capel D, Zisserman A: Computer vision applied to super resolution. IEEE Signal Processing Magazine 2003,20(3):75-86. 10.1109/MSP.2003.1203211Google Scholar
  37. Schultz RR, Meng L, Stevenson RL: Subpixel motion estimation for super-resolution image sequence enhancement. Journal of Visual Communication and Image Representation 1998,9(1):38-50. 10.1006/jvci.1997.0370Google Scholar
  38. Tekalp AM: Digital Video Processing. Prentice-Hall, Englewood Clliffs, NJ, USA; 1995.Google Scholar
  39. Farsiu S, Robinson D, Elad M, Milanfar P: Advances and challenges in super-resolution. International Journal of Imaging Systems and Technology 2004,14(2):47-57. 10.1002/ima.20007Google Scholar
  40. Rudin L, Osher S, Fatemi E: Nonlinear total variation based noise removal algorithms. Physica D 1992,60(1–4):259-268.MathSciNetMATHGoogle Scholar
  41. Vogel CR, Oman ME: Fast, robust total variation-based reconstruction of noisy, blurred images. IEEE Transactions on Image Processing 1998,7(6):813-824. 10.1109/83.679423MathSciNetMATHGoogle Scholar
  42. Chambolle A: An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision 2004,20(1-2):89-97.MathSciNetMATHGoogle Scholar
  43. Li Y, Santosa F: A computational algorithm for minimizing total variation in image restoration. IEEE Transactions on Image Processing 1996,5(6):987-995. 10.1109/83.503914Google Scholar
  44. Bioucas-Dias JM, Figueiredo MAT, Oliveira JP: Total variation-based image deconvolution: a majorization-minimization approach. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '06), May 2006, Toulouse, France 2: 861–864.Google Scholar
  45. Vogel CR, Oman ME: Iterative methods for total variation denoising. SIAM Journal of Scientific Computing 1996,17(1):227-238. 10.1137/0917016MathSciNetMATHGoogle Scholar
  46. Vogel CR: Computational Methods for Inverse Problems, Frontiers in Applied Mathematics. SIAM, Philadelphia, Pa, USA; 2002.Google Scholar
  47. Lin F-R, Ng MK, Ching W-K: Factorized banded inverse preconditioners for matrices with Toeplitz structure. SIAM Journal of Scientific Computing 2005,26(6):1852-1870. 10.1137/030601272MathSciNetMATHGoogle Scholar
  48. Chan RH, Chan TF, Wong C-K: Cosine transform based preconditioners for total variation deblurring. IEEE Transactions on Image Processing 1999,8(10):1472-1478. 10.1109/83.791976Google Scholar
  49. Ng MK, Chan RH, Chan TF, Yip AM: Cosine transform preconditioners for high resolution image reconstruction. Linear Algebra and Its Applications 2000,316(1–3):89-104.MathSciNetMATHGoogle Scholar
  50. Kolotilina LY, Yeremin AY: Factorized sparse approximate inverse preconditionings I: theory. SIAM Journal on Matrix Analysis and Applications 1993,14(1):45-58. 10.1137/0614004MathSciNetMATHGoogle Scholar
  51. Nguyen N, Milanfar P, Golub G: A computationally efficient superresolution image reconstruction algorithm. IEEE Transactions on Image Processing 2001,10(4):573-583. 10.1109/83.913592MATHGoogle Scholar


© Michael K. Ng et al. 2007

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.