Open Access

Robust Fusion of Irregularly Sampled Data Using Adaptive Normalized Convolution

  • Tuan Q. Pham1,
  • Lucas J. van Vliet1 and
  • Klamer Schutte2
EURASIP Journal on Advances in Signal Processing20062006:083268

https://doi.org/10.1155/ASP/2006/83268

Received: 1 December 2004

Accepted: 27 May 2005

Published: 20 February 2006

Abstract

We present a novel algorithm for image fusion from irregularly sampled data. The method is based on the framework of normalized convolution (NC), in which the local signal is approximated through a projection onto a subspace. The use of polynomial basis functions in this paper makes NC equivalent to a local Taylor series expansion. Unlike the traditional framework, however, the window function of adaptive NC is adapted to local linear structures. This leads to more samples of the same modality being gathered for the analysis, which in turn improves signal-to-noise ratio and reduces diffusion across discontinuities. A robust signal certainty is also adapted to the sample intensities to minimize the influence of outliers. Excellent fusion capability of adaptive NC is demonstrated through an application of super-resolution image reconstruction.

[12345678910111213141516171819202122232425262728293031]

Authors’ Affiliations

(1)
Quantitative Imaging Group, Department of Imaging Science and Technology, Faculty of Applied Sciences, Delft University of Technology
(2)
Electro Optics Group, TNO Defence, Security, and Safety

References

  1. Hardie RC, Barnard KJ, Bognar JG, Armstrong EE, Watson EA: High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Optical Engineering 1998, 37(1):247-260. 10.1117/1.601623View ArticleGoogle Scholar
  2. Irani M, Peleg S: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991, 53(3):231-239. 10.1016/1049-9652(91)90045-LGoogle Scholar
  3. Zomet A, Rav-Acha A, Peleg S: Robust super-resolution. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 1: 645-650.Google Scholar
  4. Lertrattanapanich S, Bose NK: High resolution image formation from low resolution frames using Delaunay triangulation. IEEE Transactions on Image Processing 2002, 11(12):1427-1441. 10.1109/TIP.2002.806234MathSciNetView ArticleGoogle Scholar
  5. Amidror I: Scattered data interpolation methods for electronic imaging systems: a survey. Journal of Electronic Imaging 2002, 11(2):157-176. 10.1117/1.1455013View ArticleGoogle Scholar
  6. Haralick RM, Watson L: A facet model for image data. Computer Graphics and Image Processing 1981, 15(2):113-129. 10.1016/0146-664X(81)90073-3View ArticleGoogle Scholar
  7. Farnebäck G: Polynomial expansion for orientation and motion estimation, M.S. thesis. Linköping University, Linköping, Sweden; 2002.Google Scholar
  8. van den Boomgaard R, van de Weijer J: Linear and robust estimation of local image structure. In Proceedings of 4th International Conference on Scale-Space Theories in Computer Vision (Scale-Space '03), Lecture Notes in Computer Science. Volume 2695. , Isle of Skye, Scotland, UK; 2003:237-254.Google Scholar
  9. Knutsson H, Westin C-F: Normalized and differential convolution. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '93), June 1993, New York, NY, USA 515-523.Google Scholar
  10. Young RA: The Gaussian derivative model for spatial vision: I. Retinal mechanisms. Spatial Vision 1987, 2(4):273-293. 10.1163/156856887X00222View ArticleGoogle Scholar
  11. Young IT, van Vliet LJ, van Ginkel M: Recursive Gabor filtering. IEEE Transactions on Signal Processing 2002, 50(11):2798-2805. 10.1109/TSP.2002.804095View ArticleGoogle Scholar
  12. Tomasi C, Manduchi R: Bilateral filtering for gray and color images. Proceedings of 6th International Conference on Computer Vision (ICCV '98), January 1998, Bombay, India 839-846.Google Scholar
  13. Farsiu S, Robinson MD, Elad M, Milanfar P: Fast and robust multiframe super resolution. IEEE Transactions on Image ProcessinG 2004, 13(10):1327-1344. 10.1109/TIP.2004.834669View ArticleGoogle Scholar
  14. Elad M: On the origin of the bilateral filter and ways to improve it. IEEE Transactions on Image Processing 2002, 11(10):1141-1151. 10.1109/TIP.2002.801126MathSciNetView ArticleGoogle Scholar
  15. Brownrigg DRK: The weighted median filter. Communications of the ACM 1984, 27(8):807-818. 10.1145/358198.358222View ArticleGoogle Scholar
  16. van de Weijer J, van den Boomgaard R: Local mode filtering. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 2: 428-433.Google Scholar
  17. Nitzberg M, Shiota T: Nonlinear image filtering with edge and corner enhancement. IEEE Transactions on Pattern Analysis and Machine Intelligence 1992, 14(8):826-833. 10.1109/34.149593View ArticleGoogle Scholar
  18. van de Weijer J, van Vliet LJ, Verbeek PW, van Ginkel M: Curvature estimation in oriented patterns using curvilinear models applied to gradient vector fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 2001, 23(9):1035-1042. 10.1109/34.955116View ArticleGoogle Scholar
  19. Borman S, Stevenson RL: Super-resolution from image sequences: a review. Proceedings of Midwest Symposium on Circuits and Systems (MWSCAS '98), August 1998, Notre Dame, Ind, USA 374-378.Google Scholar
  20. Capel D: Image Mosaicing and Super-Resolution. Springer, Berlin, Germany; 2004.MATHView ArticleGoogle Scholar
  21. Elad M, Hel-Or Y: A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur. IEEE Transactions on Image Processing 2001, 10(8):1187-1193. 10.1109/83.935034MATHView ArticleGoogle Scholar
  22. Schutte K, de Lange D-JJ, van den Broek SP: Signal conditioning algorithms for enhanced tactical sensor imagery. Infrared Imaging Systems: Design, Analysis, Modeling, and Testing XIV, April 2003, Orlando, Fla, USA, Proceedings of SPIE 5076: 92-100.Google Scholar
  23. Pham TQ, Bezuijen M, van Vliet LJ, Schutte K, Luengo Hendriks CL: Performance of optimal registration estimators. SPIE Defense and Security Symposium, Visual Information Processing XIV, March–April 2005, Orlando, Fla, USA, Proceedings of SPIE 5817: 133-144.View ArticleGoogle Scholar
  24. Pham TQ, van Vliet LJ, Schutte K: Influence of signal-to-noise ratio and point spread function on limits of superresolution. IS&T/SPIE's 17th Annual Symposium Electronic Imaging Science and Technology, Image Processing: Algorithms and Systems IV, January 2005, San Jose, Calif, USA, Proceedings of SPIE 5672: 169-180.Google Scholar
  25. Zuiderveld K: Contrast limited adaptive histogram equalization. In Graphics Gems IV. Edited by: Heckbert PS. Academic Press, Boston, Mass, USA; 1994:474-485.View ArticleGoogle Scholar
  26. Farsiu S, Robinson D, Elad M, Milanfar P: Robust shift and add approach to superresolution. Applications of Digital Image Processing XXVI, August 2003, San Diego, Calif, USA, Proceedings of SPIE 5203: 121-130.View ArticleGoogle Scholar
  27. Luxen M, Förstner W: Characterizing image quality: blind estimation of the point spread function from a single image. Proceedings of Photogrammetric Computer Vision (PCV~'02), September 2002, Graz, Austria 205-210.Google Scholar
  28. Rieger B, Timmermans FJ, van Vliet LJ, Verbeek PW: On curvature estimation of ISO surfaces in 3D gray-value images and the computation of shape descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence 2004, 26(8):1088-1094. 10.1109/TPAMI.2004.50View ArticleGoogle Scholar
  29. Rieger B, van Vliet LJ: Curvature of n-dimensional space curves in grey-value images. IEEE Transactions on Image Processing 2002, 11(7):738-745. 10.1109/TIP.2002.800885MathSciNetView ArticleGoogle Scholar
  30. Pham TQ, van Vliet LJ: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data. In Proceedings of 13th Scandinavian Conference on Image Analysis (SCIA '03), Lecture Notes in Computer Science. Volume 2749. , Göteborg, Sweden; 2003:485-492.Google Scholar
  31. Franke R: Smooth interpolation of scattered data by local thin plate splines. Computers & Mathematics with Applications 1982, 8(4):273-281. 10.1016/0898-1221(82)90009-8MATHMathSciNetView ArticleGoogle Scholar

Copyright

© Pham et al. 2006