Skip to main content

Advertisement

A MAP Estimator for Simultaneous Superresolution and Detector Nonunifomity Correction

Article metrics

  • 2010 Accesses

  • 7 Citations

Abstract

During digital video acquisition, imagery may be degraded by a number of phenomena including undersampling, blur, and noise. Many systems, particularly those containing infrared focal plane array (FPA) sensors, are also subject to detector nonuniformity. Nonuniformity, or fixed pattern noise, results from nonuniform responsivity of the photodetectors that make up the FPA. Here we propose a maximum a posteriori (MAP) estimation framework for simultaneously addressing undersampling, linear blur, additive noise, and bias nonuniformity. In particular, we jointly estimate a superresolution (SR) image and detector bias nonuniformity parameters from a sequence of observed frames. This algorithm can be applied to video in a variety of ways including using a moving temporal window of frames to process successive groups of frames. By combining SR and nonuniformity correction (NUC) in this fashion, we demonstrate that superior results are possible compared with the more conventional approach of performing scene-based NUC followed by independent SR. The proposed MAP algorithm can be applied with or without SR, depending on the application and computational resources available. Even without SR, we believe that the proposed algorithm represents a novel and promising scene-based NUC technique. We present a number of experimental results to demonstrate the efficacy of the proposed algorithm. These include simulated imagery for quantitative analysis and real infrared video for qualitative analysis.

References

  1. 1.

    Milton AF, Barone FR, Kruer MR: Influence of nonuniformity on infrared focal plane array performance. Optical Engineering 1985,24(5):855-862.

  2. 2.

    Gross W, Hierl T, Schultz M: Correctability and long-term stability of infrared focal plane arrays. Optical Engineering 1999,38(5):862-869. 10.1117/1.602055

  3. 3.

    Perry DL, Dereniak EL: Linear theory of nouniformity correction in infrared staring sensors. Optical Engineering 1993,32(8):1854-1859. 10.1117/12.145601

  4. 4.

    Nelson MD, Johnson JF, Lomheim TS: General noise processes in hybrid infrared focal plane arrays. Optical Engineering 1991,30(11):1682-1700. 10.1117/12.55996

  5. 5.

    El Gamal A, Eltoukhy H: CMOS image sensors. IEEE Circuits and Devices Magazine 2005,21(3):6-20. 10.1109/MCD.2005.1438751

  6. 6.

    Narendra PM, Foss NA: Shutterless fixed pattern noise correction for infrared imaging arrays. Technical Issues in Focal Plane Development, April 1981, Washington, DC, USA, Proceedings of SPIE 282: 44–51.

  7. 7.

    Harris JG: Continuous-time calibration of VLSI sensors for gain and offset variations. In Smart Focal Plane Arrays and Focal Plane Array Testing, April 1995, Orlando, Fla, USA, Proceedings of SPIE Edited by: Wigdor M, Massie MA. 2474: 23–33.

  8. 8.

    Harris JG, Chiang Y-M: Nonuniformity correction using the constant-statistics constraint: analog and digital implementations. In Infrared Technology and Applications XXIII, April 1997, Orlando, Fla, USA, Proceedings of SPIE Edited by: Andresen BF, Strojnik M. 3061: 895–905.

  9. 9.

    Chiang Y-M, Harris JG: An analog integrated circuit for continuous-time gain and offset calibration of sensor arrays. Analog Integrated Circuits and Signal Processing 1997,12(3):231-238. 10.1023/A:1008297408871

  10. 10.

    Scribner DA, Sarkady KA, Caulfield JT, et al.: Nonuniformity correction for staring IR focal plane arrays using scene-based techniques. In Infrared Detectors and Focal Plane Arrays, April 1990, Orlando, Fla, USA, Proceedings of SPIE Edited by: Dereniak EL, Sampson RE. 1308: 224–233.

  11. 11.

    Scribner DA, Sarkady KA, Kruer MR, Caulfield JT, Hunt JD, Herman C: Adaptive nonuniformity correction for IR focal-plane arrays using neural networks. In Infrared Sensors: Detectors, Electronics, and Signal Processing, July 1991, San Diego, Calif, USA, Proceedings of SPIE Edited by: Jayadev TS. 1541: 100–109.

  12. 12.

    Scribner DA, Sarkady KA, Kruer MR, et al.: Adaptive retina-like preprocessing for imaging detector arrays. Proceedings of IEEE International Conference on Neural Networks, March-April 1993, San Francisco, Calif, USA 3: 1955–1960.

  13. 13.

    Narayanan B, Hardie RC, Muse RA: Scene-based nonuniformity correction technique that exploits knowledge of the focal-plane array readout architecture. Applied Optics 2005,44(17):3482-3491. 10.1364/AO.44.003482

  14. 14.

    Hayat MM, Torres SN, Armstrong EE, Cain SC, Yasuda B: Statistical algorithm for nonuniformity correction in focal-plane arrays. Applied Optics 1999,38(5):772-780. 10.1364/AO.38.000772

  15. 15.

    Torres SN, Hayat MM: Kalman filtering for adaptive nonuniformity correction in infrared focal-plane arrays. Journal of the Optical Society of America A 2003,20(3):470-480. 10.1364/JOSAA.20.000470

  16. 16.

    Hardie RC, Hayat MM: A nonlinear-filter based approach to detector nonuniformity correction. Proceedings of IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing, June 2001, Baltimore, Md, USA 66–85.

  17. 17.

    O'Neil WF: Dithered scan detector compensation. Proceedings of the Infrared Information Symposium (IRIS) Specialty Group on Passive Sensors, 1993, Ann Arbor, Mich, USA

  18. 18.

    O'Neil WF: Experimental verification of dither scan non-uniformity correction. Proceedings of the Infrared Information Symposium (IRIS) Specialty Group on Passive Sensors, 1997, Monterey, Calif, USA 1: 329–339.

  19. 19.

    Hardie RC, Hayat MM, Armstrong EE, Yasuda B: Scene-based nonuniformity correction with video sequences and registration. Applied Optics 2000,39(8):1241-1250. 10.1364/AO.39.001241

  20. 20.

    Ratliff BM, Hayat MM, Hardie RC: An algebraic algorithm for nonuniformity correction in focal-plane arrays. Journal of the Optical Society of America A 2002,19(9):1737-1747. 10.1364/JOSAA.19.001737

  21. 21.

    Ratliff BM, Hayat MM, Tyo JS: Radiometrically accurate scene-based nonuniformity correction for array sensors. Journal of the Optical Society of America A 2003,20(10):1890-1899. 10.1364/JOSAA.20.001890

  22. 22.

    Ratliff BM, Hayat MM, Tyo JS: Generalized algebraic scene-based nonuniformity correction algorithm. Journal of the Optical Society of America A 2005,22(2):239-249. 10.1364/JOSAA.22.000239

  23. 23.

    Sakoglu U, Hardie RC, Hayat MM, Ratliff BM, Tyo JS: An algebraic restoration method for estimating fixed-pattern noise in infrared imagery from a video sequence. Applications of Digital Image Processing XXVII, August 2004, Denver, Colo, USA, Proceedings of SPIE 5558: 69–79.

  24. 24.

    Park SC, Park MK, Kang MG: Super-resolution image reconstruction: a technical overview. IEEE Signal Processing Magazine 2003,20(3):21-36. 10.1109/MSP.2003.1203207

  25. 25.

    Borman S: Topics in multiframe superresolution restoration, Ph.D. dissertation.

  26. 26.

    Schultz RR, Stevenson RL: A Bayesian approach to image expansion for improved definition. IEEE Transactions on Image Processing 1994,3(3):233-242. 10.1109/83.287017

  27. 27.

    Cheeseman P, Kanefsky B, Kraft R, Stutz J, Hanson R: Super-resolved surface reconstruction from multiple images. In Tech. Rep. FIA-94-12. NASA, Moffett Field, Calif, USA; 1994.

  28. 28.

    Cain SC, Hardie RC, Armstrong EE: Restoration of aliased video sequences via a maximum-likelihood approach. Proceedings of National Infrared Information Symposium (IRIS) on Passive Sensors, March 1996, Monterey, Calif, USA 230–251.

  29. 29.

    Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116

  30. 30.

    Segall CA, Katsaggelos AK, Molina R, Mateos J: Bayesian resolution enhancement of compressed video. IEEE Transactions on Image Processing 2004,13(7):898-910. 10.1109/TIP.2004.827230

  31. 31.

    Armstrong EE, Hayat MM, Hardie RC, Torres SN, Yasuda BJ: Nonuniformity correction for improved registration and high-resolution image reconstruction in IR imagery. In Applications of Digital Image Processing XXII, July 1999, Denver, Colo, USA, Proceedings of SPIE Edited by: Tescher AG. 3808: 150–161.

  32. 32.

    Armstrong EE, Hayat MM, Hardie RC, Torres SN, Yasuda B: The advantage of non-uniformity correction pre-processing on infrared image registration. Application of Digital Image Processing XXII, July 1999, Denver, Colo, USA, Proceedings of SPIE 3808:

  33. 33.

    Cain S, Armstrong EE, Yasuda B: Joint estimation of image, shifts, and nonuniformities from IR images. In Infrared Information Symposium (IRIS) on Passive Sensors, 1997, Ann Arbor, Mich, USA. Infrared Information Analysis Center, ERIM International;

  34. 34.

    Geman S, Geman D: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984,6(6):721-741.

  35. 35.

    Besag J: Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society B 1974,36(2):192-236.

  36. 36.

    Derin H, Elliott E: Modeling and segmentation of noisy and textured images using Gibbs random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 1987,9(1):39-55.

  37. 37.

    Cain SC, Hayat MM, Armstrong EE: Projection-based image registration in the presence of fixed-pattern noise. IEEE Transactions on Image Processing 2001,10(12):1860-1872. 10.1109/83.974571

  38. 38.

    Hardie RC, Barnard KJ, Bognar JG, Armstrong EE, Watson EA: High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Optical Engineering 1998,37(1):247-260. 10.1117/1.601623

  39. 39.

    Irani M, Peleg S: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991,53(3):231-239. 10.1016/1049-9652(91)90045-L

Download references

Author information

Correspondence to Russell C. Hardie.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://doi.org/creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Hardie, R.C., Droege, D.R. A MAP Estimator for Simultaneous Superresolution and Detector Nonunifomity Correction. EURASIP J. Adv. Signal Process. 2007, 089354 (2007) doi:10.1155/2007/89354

Download citation

Keywords

  • Undersampling
  • Focal Plane Array
  • Fixed Pattern
  • Pattern Noise
  • Estimation Framework