- Research Article
- Open Access
Video-to-Video Dynamic Super-Resolution for Grayscale and Color Sequences
EURASIP Journal on Advances in Signal Processing volume 2006, Article number: 061859 (2006)
We address the dynamic super-resolution (SR) problem of reconstructing a high-quality set of monochromatic or color super-resolved images from low-quality monochromatic, color, or mosaiced frames. Our approach includes a joint method for simultaneous SR, deblurring, and demosaicing, this way taking into account practical color measurements encountered in video sequences. For the case of translational motion and common space-invariant blur, the proposed method is based on a very fast and memory efficient approximation of the Kalman filter (KF). Experimental results on both simulated and real data are supplied, demonstrating the presented algorithms, and their strength.
Huang TS, Tsai RY: Multi-frame image restoration and registration. In Advances in Computer Vision and Image Processing. Volume 1. JAI Press, Greenwich, Conn, USA; 1984:317–339. chapter 7
Nguyen N, Milanfar P, Golub GH: A computationally efficient super-resolution image reconstruction algorithm. Transactions on Image Processing 2001, 10(4):573–583. 10.1109/83.913592
Irani M, Peleg S: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991, 53(3):231–239. 10.1016/1049-9652(91)90045-L
Elad M, Feuer A: Restoration of a single super-resolution image from several blurred, noisy, and undersampled measured images. Transactions on Image Processing 1997, 6(12):1646–1658. 10.1109/83.650118
Zomet A, Peleg S: Efficient super-resolution and applications to mosaics. Proceedings of IEEE 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 579–583.
Ng MK, Bose NK: Mathematical analysis of super-resolution methodology. IEEE Signal Processing Magazine 2003, 20(3):62–74. 10.1109/MSP.2003.1203210
Altunbasak Y, Patti AJ, Mersereau RM: Super-resolution still and video reconstruction from MPEG-coded video. IEEE Transactions on Circuits and Systems for Video Technology 2002, 12(4):217–226. 10.1109/76.999200
Farsiu S, Robinson MD, Elad M, Milanfar P: Fast and robust multi-frame super-resolution. IEEE Transactions on Image Processing 2004, 13(10):1327–1344. 10.1109/TIP.2004.834669
Segall CA, Katsaggelos AK, Molina R, Mateos J: Bayesian resolution enhancement of compressed video. IEEE Transactions on Image Processing 2004, 13(7):898–911. 10.1109/TIP.2004.827230
Borman S, Stevenson RL: Super-resolution from image sequences—a review. Proceedings of Midwest Symposium on Circuits and Systems (MWSCAS '98), August 1998, Notre Dame, Ind, USA 374–378.
Park SC, Park MK, Kang MG: Super-resolution image reconstruction: a technical overview. IEEE Signal Processing Magazine 2003, 20(3):21–36. 10.1109/MSP.2003.1203207
Farsiu S, Robinson MD, Elad M, Milanfar P: Advances and challenges in super-resolution. International Journal of Imaging Systems and Technology 2004, 14(2):47–57. 10.1002/ima.20007
Elad M, Feuer A: Super-resolution restoration of an image sequence: adaptive filtering approach. IEEE Transactions on Image Processing 1999, 8(3):387–395. 10.1109/83.748893
Elad M, Feuer A: Super-resolution reconstruction of image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 1999, 21(9):817–834. 10.1109/34.790425
Elad M, Hel-Or Y: A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur. IEEE Transactions on Image Processing 2001, 10(8):1187–1193. 10.1109/83.935034
Farsiu S, Elad M, Milanfar P: Multiframe demosaicing and super-resolution from undersampled color images. Computational Imaging II, January 2004, San Jose, Calif, USA, Proceedings of SPIE 5299: 222–233.
Farsiu S, Elad M, Milanfar P: Multiframe demosaicing and super-resolution of color images. IEEE Transactions on Image Processing 2006, 15(1):141–159.
Kay SM: Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory. Prentice-Hall, Englewood Cliffs, NJ, USA; 1993.
Jazwinski AH: Stochastic Processes and Filtering Theory. Academic Press, New York, NY, USA; 1970.
Elad M: Super-resolution reconstruction of continuous image sequence, Ph.D. dissertation. Technion-Israel Institute of Technology, Haifa, Israel; 1997.
Rauch HE, Striebel CT, Tung F: Maximum likelihood estimates of dynamic linear systems. American Institute of Aeronautics and Astronautics 1965, 3(8):1445–1450.
Harvey AC: Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge University Press, Cambridge, UK; 1990.
Bovik A: Handbook of Image and Video Processing. Academic Press, New York, NY, USA; 2000.
Rudin LI, Osher S, Fatemi E: Nonlinear total variation based noise removal algorithms. Physica D 1992, 60(1–4):259–268.
Li Y, Santosa F: A computational algorithm for minimizing total variation in image restoration. IEEE Transactions on Image Processing 1996, 5(6):987–995. 10.1109/83.503914
Chan TF, Osher S, Shen J: The digital TV filter and nonlinear denoising. IEEE Transactions on Image Processing 2001, 10(2):231–241. 10.1109/83.902288
Tomasi C, Manduchi R: Bilateral filtering for gray and color images. Proceedings of IEEE 6th International Conference on Computer Vision (ICCV '98), January 1998, Bombay, India 839–846.
Elad M: On the origin of the bilateral filter and ways to improve it. IEEE Transactions on Image Processing 2002, 11(10):1141–1151. 10.1109/TIP.2002.801126
Laroche C, Prescott M: Apparatus and method for adaptive for adaptively interpolating a full color image utilizing chrominance gradients. United States Patent 5,373,322, 1994
Kimmel R: Demosaicing: image reconstruction from color CCD samples. IEEE Transactions on Image Processing 1999, 8(9):1221–1228. 10.1109/83.784434
Keren D, Osadchy M: Restoring subsampled color images. Machine Vision and Applications 1999, 11(4):197–202. 10.1007/s001380050102
Hel-Or Y, Keren D: Demosaicing of color images using steerable wavelets. In Tech. Rep. HPL-2002-206R1 20020830. HP Laboratories Israel, Haifa, Israel; 2002. Online, available: https://doi.org/citeseer.nj.nec.com/
Muresan DD, Parks TW: Optimal recovery demosaicing. Proceedings of IASTED International Conference on Signal and Image Processing (SIP '02), August 2002, Kauai, Hawaii, USA
Gunturk BK, Altunbasak Y, Mersereau RM: Color plane interpolation using alternating projections. IEEE Transactions on Image Processing 2002, 11(9):997–1013. 10.1109/TIP.2002.801121
Alleysson D, Süsstrunk S, Hérault J: Color demosaicing by estimating luminance and opponent chromatic signals in the Fourier domain. Proceedings of IS&T/SID 10th Color Imaging Conference, November 2002, Scottsdale, Ariz, USA 331–336.
Ramanath R, Snyder WE, Bilbro GL, Sander WA: Demosaicking methods for the Bayer color arrays. Journal of Electronic Imaging 2002, 11(3):306–315. 10.1117/1.1484495
Pei S-C, Tam I-K: Effective color interpolation in CCD color filter arrays using signal correlation. IEEE Transactions on Circuits and Systems for Video Technology 2003, 13(6):503–513. 10.1109/TCSVT.2003.813422
Zomet A, Peleg S: Multi-sensor super-resolution. Proceedings of IEEE 6th Workshop on Applications of Computer Vision (WACV '02), December 2002, Orlando, Fla, USA 27–31.
Gotoh T, Okutomi M: Direct super-resolution and registration using raw CFA images. Proceedings of IEEE Computer Society Conference on Computer Vision and Patern Recognition (CVPR '04), June–July 2004, Washington, DC, USA 2: 600–607.
Shah NR, Zakhor A: Resolution enhancement of color video sequences. IEEE Transactions on Image Processing 1999, 8(6):879–885. 10.1109/83.766865
Tom BC, Katsaggelos AK: Resolution enhancement of monochrome and color video using motion compensation. IEEE Transactions on Image Processing 2001, 10(2):278–287. 10.1109/83.902292
Pratt WK: Digital Image Processing. 3rd edition. John Wiley & Sons, New York, NY, USA; 2001.
Golland P, Bruckstein AM: Motion from color. Computer Vision and Image Understanding 1997, 68(3):346–362. 10.1006/cviu.1997.0553
Bergen JR, Anandan P, Hanna KJ, Hingorani R: Hierarchical model-based motion estimation. Proceedings of European Conference on Computer Vision (ECCV '92), May 1992, Santa Margherita Ligure, Italy 237–252.
About this article
Cite this article
Farsiu, S., Elad, M. & Milanfar, P. Video-to-Video Dynamic Super-Resolution for Grayscale and Color Sequences. EURASIP J. Adv. Signal Process. 2006, 061859 (2006). https://doi.org/10.1155/ASP/2006/61859
- Information Technology
- Real Data
- Quantum Information