Open Access

A MAP Estimator for Simultaneous Superresolution and Detector Nonunifomity Correction

EURASIP Journal on Advances in Signal Processing20072007:089354

https://doi.org/10.1155/2007/89354

Received: 31 August 2006

Accepted: 9 April 2007

Published: 3 May 2007

Abstract

During digital video acquisition, imagery may be degraded by a number of phenomena including undersampling, blur, and noise. Many systems, particularly those containing infrared focal plane array (FPA) sensors, are also subject to detector nonuniformity. Nonuniformity, or fixed pattern noise, results from nonuniform responsivity of the photodetectors that make up the FPA. Here we propose a maximum a posteriori (MAP) estimation framework for simultaneously addressing undersampling, linear blur, additive noise, and bias nonuniformity. In particular, we jointly estimate a superresolution (SR) image and detector bias nonuniformity parameters from a sequence of observed frames. This algorithm can be applied to video in a variety of ways including using a moving temporal window of frames to process successive groups of frames. By combining SR and nonuniformity correction (NUC) in this fashion, we demonstrate that superior results are possible compared with the more conventional approach of performing scene-based NUC followed by independent SR. The proposed MAP algorithm can be applied with or without SR, depending on the application and computational resources available. Even without SR, we believe that the proposed algorithm represents a novel and promising scene-based NUC technique. We present a number of experimental results to demonstrate the efficacy of the proposed algorithm. These include simulated imagery for quantitative analysis and real infrared video for qualitative analysis.

[123456789101112131415161718192021222324252627282930313233343536373839]

Authors’ Affiliations

(1)
Department of Electrical and Computer Engineering, University of Dayton
(2)
L-3 Communications Cincinnati Electronics

References

  1. Milton AF, Barone FR, Kruer MR: Influence of nonuniformity on infrared focal plane array performance. Optical Engineering 1985,24(5):855-862.View ArticleGoogle Scholar
  2. Gross W, Hierl T, Schultz M: Correctability and long-term stability of infrared focal plane arrays. Optical Engineering 1999,38(5):862-869. 10.1117/1.602055View ArticleGoogle Scholar
  3. Perry DL, Dereniak EL: Linear theory of nouniformity correction in infrared staring sensors. Optical Engineering 1993,32(8):1854-1859. 10.1117/12.145601View ArticleGoogle Scholar
  4. Nelson MD, Johnson JF, Lomheim TS: General noise processes in hybrid infrared focal plane arrays. Optical Engineering 1991,30(11):1682-1700. 10.1117/12.55996View ArticleGoogle Scholar
  5. El Gamal A, Eltoukhy H: CMOS image sensors. IEEE Circuits and Devices Magazine 2005,21(3):6-20. 10.1109/MCD.2005.1438751View ArticleGoogle Scholar
  6. Narendra PM, Foss NA: Shutterless fixed pattern noise correction for infrared imaging arrays. Technical Issues in Focal Plane Development, April 1981, Washington, DC, USA, Proceedings of SPIE 282: 44-51.View ArticleGoogle Scholar
  7. Harris JG: Continuous-time calibration of VLSI sensors for gain and offset variations. In Smart Focal Plane Arrays and Focal Plane Array Testing, April 1995, Orlando, Fla, USA, Proceedings of SPIE Edited by: Wigdor M, Massie MA. 2474: 23-33.View ArticleGoogle Scholar
  8. Harris JG, Chiang Y-M: Nonuniformity correction using the constant-statistics constraint: analog and digital implementations. In Infrared Technology and Applications XXIII, April 1997, Orlando, Fla, USA, Proceedings of SPIE Edited by: Andresen BF, Strojnik M. 3061: 895-905.Google Scholar
  9. Chiang Y-M, Harris JG: An analog integrated circuit for continuous-time gain and offset calibration of sensor arrays. Analog Integrated Circuits and Signal Processing 1997,12(3):231-238. 10.1023/A:1008297408871View ArticleGoogle Scholar
  10. Scribner DA, Sarkady KA, Caulfield JT, et al.: Nonuniformity correction for staring IR focal plane arrays using scene-based techniques. In Infrared Detectors and Focal Plane Arrays, April 1990, Orlando, Fla, USA, Proceedings of SPIE Edited by: Dereniak EL, Sampson RE. 1308: 224-233.Google Scholar
  11. Scribner DA, Sarkady KA, Kruer MR, Caulfield JT, Hunt JD, Herman C: Adaptive nonuniformity correction for IR focal-plane arrays using neural networks. In Infrared Sensors: Detectors, Electronics, and Signal Processing, July 1991, San Diego, Calif, USA, Proceedings of SPIE Edited by: Jayadev TS. 1541: 100-109.View ArticleGoogle Scholar
  12. Scribner DA, Sarkady KA, Kruer MR, et al.: Adaptive retina-like preprocessing for imaging detector arrays. Proceedings of IEEE International Conference on Neural Networks, March-April 1993, San Francisco, Calif, USA 3: 1955-1960.View ArticleGoogle Scholar
  13. Narayanan B, Hardie RC, Muse RA: Scene-based nonuniformity correction technique that exploits knowledge of the focal-plane array readout architecture. Applied Optics 2005,44(17):3482-3491. 10.1364/AO.44.003482View ArticleGoogle Scholar
  14. Hayat MM, Torres SN, Armstrong EE, Cain SC, Yasuda B: Statistical algorithm for nonuniformity correction in focal-plane arrays. Applied Optics 1999,38(5):772-780. 10.1364/AO.38.000772View ArticleGoogle Scholar
  15. Torres SN, Hayat MM: Kalman filtering for adaptive nonuniformity correction in infrared focal-plane arrays. Journal of the Optical Society of America A 2003,20(3):470-480. 10.1364/JOSAA.20.000470View ArticleGoogle Scholar
  16. Hardie RC, Hayat MM: A nonlinear-filter based approach to detector nonuniformity correction. Proceedings of IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing, June 2001, Baltimore, Md, USA 66-85.Google Scholar
  17. O'Neil WF: Dithered scan detector compensation. Proceedings of the Infrared Information Symposium (IRIS) Specialty Group on Passive Sensors, 1993, Ann Arbor, Mich, USAGoogle Scholar
  18. O'Neil WF: Experimental verification of dither scan non-uniformity correction. Proceedings of the Infrared Information Symposium (IRIS) Specialty Group on Passive Sensors, 1997, Monterey, Calif, USA 1: 329-339.Google Scholar
  19. Hardie RC, Hayat MM, Armstrong EE, Yasuda B: Scene-based nonuniformity correction with video sequences and registration. Applied Optics 2000,39(8):1241-1250. 10.1364/AO.39.001241View ArticleGoogle Scholar
  20. Ratliff BM, Hayat MM, Hardie RC: An algebraic algorithm for nonuniformity correction in focal-plane arrays. Journal of the Optical Society of America A 2002,19(9):1737-1747. 10.1364/JOSAA.19.001737View ArticleGoogle Scholar
  21. Ratliff BM, Hayat MM, Tyo JS: Radiometrically accurate scene-based nonuniformity correction for array sensors. Journal of the Optical Society of America A 2003,20(10):1890-1899. 10.1364/JOSAA.20.001890View ArticleGoogle Scholar
  22. Ratliff BM, Hayat MM, Tyo JS: Generalized algebraic scene-based nonuniformity correction algorithm. Journal of the Optical Society of America A 2005,22(2):239-249. 10.1364/JOSAA.22.000239View ArticleGoogle Scholar
  23. Sakoglu U, Hardie RC, Hayat MM, Ratliff BM, Tyo JS: An algebraic restoration method for estimating fixed-pattern noise in infrared imagery from a video sequence. Applications of Digital Image Processing XXVII, August 2004, Denver, Colo, USA, Proceedings of SPIE 5558: 69-79.View ArticleGoogle Scholar
  24. Park SC, Park MK, Kang MG: Super-resolution image reconstruction: a technical overview. IEEE Signal Processing Magazine 2003,20(3):21-36. 10.1109/MSP.2003.1203207View ArticleGoogle Scholar
  25. Borman S: Topics in multiframe superresolution restoration, Ph.D. dissertation.Google Scholar
  26. Schultz RR, Stevenson RL: A Bayesian approach to image expansion for improved definition. IEEE Transactions on Image Processing 1994,3(3):233-242. 10.1109/83.287017View ArticleGoogle Scholar
  27. Cheeseman P, Kanefsky B, Kraft R, Stutz J, Hanson R: Super-resolved surface reconstruction from multiple images. In Tech. Rep. FIA-94-12. NASA, Moffett Field, Calif, USA; 1994.Google Scholar
  28. Cain SC, Hardie RC, Armstrong EE: Restoration of aliased video sequences via a maximum-likelihood approach. Proceedings of National Infrared Information Symposium (IRIS) on Passive Sensors, March 1996, Monterey, Calif, USA 230-251.Google Scholar
  29. Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116View ArticleGoogle Scholar
  30. Segall CA, Katsaggelos AK, Molina R, Mateos J: Bayesian resolution enhancement of compressed video. IEEE Transactions on Image Processing 2004,13(7):898-910. 10.1109/TIP.2004.827230View ArticleGoogle Scholar
  31. Armstrong EE, Hayat MM, Hardie RC, Torres SN, Yasuda BJ: Nonuniformity correction for improved registration and high-resolution image reconstruction in IR imagery. In Applications of Digital Image Processing XXII, July 1999, Denver, Colo, USA, Proceedings of SPIE Edited by: Tescher AG. 3808: 150-161.View ArticleGoogle Scholar
  32. Armstrong EE, Hayat MM, Hardie RC, Torres SN, Yasuda B: The advantage of non-uniformity correction pre-processing on infrared image registration. Application of Digital Image Processing XXII, July 1999, Denver, Colo, USA, Proceedings of SPIE 3808:Google Scholar
  33. Cain S, Armstrong EE, Yasuda B: Joint estimation of image, shifts, and nonuniformities from IR images. In Infrared Information Symposium (IRIS) on Passive Sensors, 1997, Ann Arbor, Mich, USA. Infrared Information Analysis Center, ERIM International;Google Scholar
  34. Geman S, Geman D: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984,6(6):721-741.View ArticleMATHGoogle Scholar
  35. Besag J: Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society B 1974,36(2):192-236.MathSciNetMATHGoogle Scholar
  36. Derin H, Elliott E: Modeling and segmentation of noisy and textured images using Gibbs random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 1987,9(1):39-55.View ArticleGoogle Scholar
  37. Cain SC, Hayat MM, Armstrong EE: Projection-based image registration in the presence of fixed-pattern noise. IEEE Transactions on Image Processing 2001,10(12):1860-1872. 10.1109/83.974571View ArticleMATHGoogle Scholar
  38. Hardie RC, Barnard KJ, Bognar JG, Armstrong EE, Watson EA: High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Optical Engineering 1998,37(1):247-260. 10.1117/1.601623View ArticleGoogle Scholar
  39. Irani M, Peleg S: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991,53(3):231-239. 10.1016/1049-9652(91)90045-LGoogle Scholar

Copyright

© R. C. Hardie and D. R. Droege. 2007

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.