Skip to main content

Advertisement

Robust Fusion of Irregularly Sampled Data Using Adaptive Normalized Convolution

Article metrics

Abstract

We present a novel algorithm for image fusion from irregularly sampled data. The method is based on the framework of normalized convolution (NC), in which the local signal is approximated through a projection onto a subspace. The use of polynomial basis functions in this paper makes NC equivalent to a local Taylor series expansion. Unlike the traditional framework, however, the window function of adaptive NC is adapted to local linear structures. This leads to more samples of the same modality being gathered for the analysis, which in turn improves signal-to-noise ratio and reduces diffusion across discontinuities. A robust signal certainty is also adapted to the sample intensities to minimize the influence of outliers. Excellent fusion capability of adaptive NC is demonstrated through an application of super-resolution image reconstruction.

References

  1. 1.

    Hardie RC, Barnard KJ, Bognar JG, Armstrong EE, Watson EA: High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Optical Engineering 1998, 37(1):247–260. 10.1117/1.601623

  2. 2.

    Irani M, Peleg S: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991, 53(3):231–239. 10.1016/1049-9652(91)90045-L

  3. 3.

    Zomet A, Rav-Acha A, Peleg S: Robust super-resolution. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 1: 645–650.

  4. 4.

    Lertrattanapanich S, Bose NK: High resolution image formation from low resolution frames using Delaunay triangulation. IEEE Transactions on Image Processing 2002, 11(12):1427–1441. 10.1109/TIP.2002.806234

  5. 5.

    Amidror I: Scattered data interpolation methods for electronic imaging systems: a survey. Journal of Electronic Imaging 2002, 11(2):157–176. 10.1117/1.1455013

  6. 6.

    Haralick RM, Watson L: A facet model for image data. Computer Graphics and Image Processing 1981, 15(2):113–129. 10.1016/0146-664X(81)90073-3

  7. 7.

    Farnebäck G: Polynomial expansion for orientation and motion estimation, M.S. thesis. Linköping University, Linköping, Sweden; 2002.

  8. 8.

    van den Boomgaard R, van de Weijer J: Linear and robust estimation of local image structure. In Proceedings of 4th International Conference on Scale-Space Theories in Computer Vision (Scale-Space '03), Lecture Notes in Computer Science. Volume 2695. , Isle of Skye, Scotland, UK; 2003:237–254.

  9. 9.

    Knutsson H, Westin C-F: Normalized and differential convolution. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '93), June 1993, New York, NY, USA 515–523.

  10. 10.

    Young RA: The Gaussian derivative model for spatial vision: I. Retinal mechanisms. Spatial Vision 1987, 2(4):273–293. 10.1163/156856887X00222

  11. 11.

    Young IT, van Vliet LJ, van Ginkel M: Recursive Gabor filtering. IEEE Transactions on Signal Processing 2002, 50(11):2798–2805. 10.1109/TSP.2002.804095

  12. 12.

    Tomasi C, Manduchi R: Bilateral filtering for gray and color images. Proceedings of 6th International Conference on Computer Vision (ICCV '98), January 1998, Bombay, India 839–846.

  13. 13.

    Farsiu S, Robinson MD, Elad M, Milanfar P: Fast and robust multiframe super resolution. IEEE Transactions on Image ProcessinG 2004, 13(10):1327–1344. 10.1109/TIP.2004.834669

  14. 14.

    Elad M: On the origin of the bilateral filter and ways to improve it. IEEE Transactions on Image Processing 2002, 11(10):1141–1151. 10.1109/TIP.2002.801126

  15. 15.

    Brownrigg DRK: The weighted median filter. Communications of the ACM 1984, 27(8):807–818. 10.1145/358198.358222

  16. 16.

    van de Weijer J, van den Boomgaard R: Local mode filtering. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 2: 428–433.

  17. 17.

    Nitzberg M, Shiota T: Nonlinear image filtering with edge and corner enhancement. IEEE Transactions on Pattern Analysis and Machine Intelligence 1992, 14(8):826–833. 10.1109/34.149593

  18. 18.

    van de Weijer J, van Vliet LJ, Verbeek PW, van Ginkel M: Curvature estimation in oriented patterns using curvilinear models applied to gradient vector fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 2001, 23(9):1035–1042. 10.1109/34.955116

  19. 19.

    Borman S, Stevenson RL: Super-resolution from image sequences: a review. Proceedings of Midwest Symposium on Circuits and Systems (MWSCAS '98), August 1998, Notre Dame, Ind, USA 374–378.

  20. 20.

    Capel D: Image Mosaicing and Super-Resolution. Springer, Berlin, Germany; 2004.

  21. 21.

    Elad M, Hel-Or Y: A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur. IEEE Transactions on Image Processing 2001, 10(8):1187–1193. 10.1109/83.935034

  22. 22.

    Schutte K, de Lange D-JJ, van den Broek SP: Signal conditioning algorithms for enhanced tactical sensor imagery. Infrared Imaging Systems: Design, Analysis, Modeling, and Testing XIV, April 2003, Orlando, Fla, USA, Proceedings of SPIE 5076: 92–100.

  23. 23.

    Pham TQ, Bezuijen M, van Vliet LJ, Schutte K, Luengo Hendriks CL: Performance of optimal registration estimators. SPIE Defense and Security Symposium, Visual Information Processing XIV, March–April 2005, Orlando, Fla, USA, Proceedings of SPIE 5817: 133–144.

  24. 24.

    Pham TQ, van Vliet LJ, Schutte K: Influence of signal-to-noise ratio and point spread function on limits of superresolution. IS&T/SPIE's 17th Annual Symposium Electronic Imaging Science and Technology, Image Processing: Algorithms and Systems IV, January 2005, San Jose, Calif, USA, Proceedings of SPIE 5672: 169–180.

  25. 25.

    Zuiderveld K: Contrast limited adaptive histogram equalization. In Graphics Gems IV. Edited by: Heckbert PS. Academic Press, Boston, Mass, USA; 1994:474–485.

  26. 26.

    Farsiu S, Robinson D, Elad M, Milanfar P: Robust shift and add approach to superresolution. Applications of Digital Image Processing XXVI, August 2003, San Diego, Calif, USA, Proceedings of SPIE 5203: 121–130.

  27. 27.

    Luxen M, Förstner W: Characterizing image quality: blind estimation of the point spread function from a single image. Proceedings of Photogrammetric Computer Vision (PCV~'02), September 2002, Graz, Austria 205–210.

  28. 28.

    Rieger B, Timmermans FJ, van Vliet LJ, Verbeek PW: On curvature estimation of ISO surfaces in 3D gray-value images and the computation of shape descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence 2004, 26(8):1088–1094. 10.1109/TPAMI.2004.50

  29. 29.

    Rieger B, van Vliet LJ: Curvature of n-dimensional space curves in grey-value images. IEEE Transactions on Image Processing 2002, 11(7):738–745. 10.1109/TIP.2002.800885

  30. 30.

    Pham TQ, van Vliet LJ: Normalized averaging using adaptive applicability functions with applications in image reconstruction from sparsely and randomly sampled data. In Proceedings of 13th Scandinavian Conference on Image Analysis (SCIA '03), Lecture Notes in Computer Science. Volume 2749. , Göteborg, Sweden; 2003:485–492.

  31. 31.

    Franke R: Smooth interpolation of scattered data by local thin plate splines. Computers & Mathematics with Applications 1982, 8(4):273–281. 10.1016/0898-1221(82)90009-8

Download references

Author information

Correspondence to Tuan Q. Pham.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Pham, T.Q., van Vliet, L.J. & Schutte, K. Robust Fusion of Irregularly Sampled Data Using Adaptive Normalized Convolution. EURASIP J. Adv. Signal Process. 2006, 083268 (2006) doi:10.1155/ASP/2006/83268

Download citation

Keywords

  • Image Reconstruction
  • Image Fusion
  • Linear Structure
  • Taylor Series Expansion
  • Window Function