- Research
- Open Access

# A parameter-adaptive iterative regularization model for image denoising

- Wenshu Li
^{1}Email author, - Chao Zhao
^{1}, - Qiegen Liu
^{2}, - Qingjiang Shi
^{1}and - Shen Xu
^{1}

**2012**:222

https://doi.org/10.1186/1687-6180-2012-222

© Li et al.; licensee Springer. 2012

**Received:**23 April 2012**Accepted:**11 September 2012**Published:**16 October 2012

## Abstract

In this article, an iterative regularization model (IRM) with adaptive parameter is addressed. IRM has gained a lot of attentions. But constant scale parameter becomes very sensitive for the fast convergence. It becomes very important to optimize the scale parameter adaptively. Therefore, we introduce a novel IRM with varying scale parameter because of the fact that when the scale parameter is smaller, the number of the iteration will enhance by IRM. A method to estimate a scale parameter is proposed according to the trend of the scale parameter. And the theoretical justification for this approach can be inferred. Numerical experiments show that the proposed methods with varying scale parameter can efficiently remove noise, reduce the number of iteration, and well preserve the details of images.

## Keywords

- Iterative regularization
- Total variation
- Variational methods
- Image denoising

## Introduction

During the last decade, in spite of the sophistication of the recently proposed methods, some algorithms have not yet attained a desirable level of applicability for image denoising, which is still a challenge at the crossing of functional analysis and statistics. The relations between variational regularization method and wavelet shrinkage have become one of the most active areas of research [1–5].

*f*(

*x*,

*y*): Ω→, where Ω is a bounded open subset of σ

^{2}, we want to obtain a decomposition equation:

where *g*(*x*,*y*) is the true image and *n*(*x*,*y*) is the noise with (*x*, *y*) ∈ Ω and *n* (*x*, *y*) (0, σ^{2})

*λ*> 0, where BV(Ω) denotes the space of functions with bounded variation on Ω, ·

_{2}is

*L*

_{2}norm.

*J*(

*u*) is the regularization item and ∥

*f*-

*u*∥

_{2}

^{2}is the fitting item.

*λ*is chosen to balance inconsistency (first term) and the deviation (second term) from the noise image

*f*(

*x*

*y*) and depends on the noise norm

*σ*. Therefore, a mass of researchers are concentrated on the regularization item

*J*(

*u*). The total variation model of Rudin–Osher–Fatemi (ROF) for image denoising is considered to be the better denoising model. But, there were two serious issues about the ROF model [6–11]. First, it was very complicated to compute the solutions of the optimization problems induced by the variational method. Second, it was difficult to extract textures from images by using the ROF model. For the first issue, Goldstein and Osher recently introduced the split Bregman method for

*L*1 regularized problems. The Bregman method gave rise to very efficient algorithms for solutions of the ROF model. Meyer [12] did some very interesting analysis by characterizing textures which he defines as “highly oscillatory patterns in image processing” as elements of the dual space of BV(Ω). An iterative regularization model (IRM) [13], which replaces the regularization term by a generalized Bregman distance [14, 15], was proposed. This model is formulated as

Large *λ* corresponds to very little noise removal, and hence *u*(*x* *y*) is quickly close to *f*(*x* *y*) and the quality of image denoising is not effective. Small *λ* yields an over-smoothed *u*(*x* *y*) and the iterated times will be enhanced. In spite of the sophistication of the recently proposed methods, most algorithms have not yet attained a desirable level of applicability [15–18].

In this article, we proposed a new denoising method with varying scale parameter where the regularization item is $J\left(u\right)=\underset{\mathrm{\Omega}}{{\displaystyle \iint}}\left|\nabla u\right|dxdy$. We deduce a method to gain the scale parameter from the iterative regularization. Finally, some numerical examples are presented and show that our method improves the quality of the image denoising and reduces the optimal number of iterations.

The remainder of this article is organized as follows. In “IRM” section, we mainly review IRM and its some attributes. The proposed method is introduced in “IRM with varying scale parameter” section; the experimental results of our method are given in “Result and discussion” section. This article is summarized in “Conclusion” section.

### IRM

*p*∈ ∂

*J*(

*v*), we define the non-negative quantity

where *u*_{0} = 0 and ${D}_{J}^{{p}_{k}}\left(u,{u}_{k}\right)$ are the Bregman distance between *u* and *u*_{
k
}. As the optimal number of iteration *k* increases, *u* is close to the noisy image *f*. The scale parameter *λ* tunes the weight between the regularization and fidelity terms. The iterated refinement method yields a well-defined sequence of minimizers {*u*_{
k
}} which satisfies ∥*u*_{
k
} − *f*∥_{2}^{2} ≤ *u*_{k − 1} − *f*_{2}^{2} and if *f* ∈ BV (Ω), then ${u}_{k}-{f}_{2}^{2}\le \frac{J\left(f\right)}{k}$, i.e., *u*_{
k
} converges monotonically to *f* in *L*^{2}(Ω) with a rate of $\frac{1}{\sqrt{k}}$. For *g* ∈ BV (Ω) and *γ* > 1, we have *D*(*g*, *u*_{
k
}) ≤ *D*(*g*, *u*_{k−1}) subject to *u*_{
k
} − *f*_{2} ≥ *γ∥g* − *f*_{2.}

Thus, the distance between a restored image *u*_{
k
} and a possible exact image *g* is decreasing until the *L*^{2}– distance of *f* and *u*_{
k
} is larger than the *L*^{2}– distance of *f* and *g*. This result can be used to construct a stopping rule for our iterative procedure [13].

It should be stressed that the Bregman-based methodology, in the last few years, has made rapid development due to the tireless efforts of Osher and collaborators [18, 21–23]. A key breakthrough among is that, with adequate initializations, the Bregman method equals to the augmented Lagrangian algorithm [7, 22]. Furthermore, many efficient algorithms are proposed to enable fast implementation [21, 24, 25].

### IRM with varying scale parameter

We know that for IRM the bigger the scale parameter *λ* is, the smaller the number of iteration is to the stop criterion, but *u* is quickly close to the noise image *f*, the quality of the image denoising is not ideal. When the scale parameter *λ* is smaller, the number of the iteration will enhance. Therefore, it is important to choose an optimal value *λ*.

### Varying scale parameter

*u*for Equation (3a) we have

*x*and

*y*, we get

*λ*

_{k+1}denotes

*λ*in Equation (8). Applying the proposed scale parameter to IRM with initial values

*u*

^{0}= 0,

*v*

^{0}= 0, we obtain different scale parameters

*λ*

_{k+1}for different iterations. Equation (3) should be written as

This gives us an adaptive value *λ*_{k+1}, which appears to converge as *k* → ∞. The theoretical justification for this approach comes from Appendices 1 and 2.

### Initial scale parameter

By the numerical experiment, we discover that the quality of image denoising is not ideal when initial values *u*_{0} = 0, *v*_{0} = 0. For example, if the initial condition holds, there is a question that Equation (9a) will be divided by zero.

*λ*

_{0}, we calculate

*λ*

_{ k }by

*λ*

_{ k }} and find that

*λ*

_{ k }has some properties as follows:

- (a)
the sequence vector {

*λ*_{ k }} is monotonically decreasing as the number of iteration*k*increases (see Figure 1c); - (b)
as the number of iteration

*k*increases, the sequence vector {*λ*_{ k }} will at first decrease, and then increase closely to*λ*_{0}(see Figure 1b); - (c)
the sequence vector {

*λ*_{ k }} is monotonically increasing as the number of iteration*k*increases (see Figure 1a).

*λ*

_{ k }} as follows:

- (1)
If the sequence vector {

*λ*_{ k }} is monotonically decreasing at first as the number of iteration*k*increases, we consider that the random selected*λ*_{0}is contented with the property of (a) or (b). Then, the initial scale parameter*λ*_{1}of our proposed method is equal to $\stackrel{-}{{\text{\lambda}}_{\text{k}}}$. Usually,*k*is equal to 3. - (2)
If the sequence vector {

*λ*_{ k }} is monotonically increasing as the number of iteration*k*increases, the random selected*λ*_{0}is contented with the property of (c). Then, the initial scale parameter of our proposed method ${\text{\lambda}}_{1}=\stackrel{-}{{\text{\lambda}}_{1}}$ or λ_{1}= λ^{0}/*p*with the constant*p*> 1. Usually,*p*= 2.

In Figure 1, as the example of ‘Barbara’ image, the trends of the sequence vector {*λ*_{
k
}} are gained when the scale parameter *λ*_{0} is 8.33, 4.34, and 0.013, respectively.

### IRM framework with varying scale parameter

*λ*

_{0}. Let

*u*

_{0}= 0,

*v*

_{0}= 0 and

*j*= 0, 1, 2…

- (1)
According to Equations (11) and (10), we calculate u

_{j+1}, v_{j+1}, and*λ*_{ j }by the number of iteration*j*. Generally*j*= 2. - (2)
We observe the trend of the sequence vector {

*λ*_{ k }}. According to the properties of “Initial scale parameter” section, we get the initial value*λ*_{1}of our proposed method.

*u*

_{0}= 0,

*v*

_{0}= 0 and

*k*= 1, 2…

- (1)
According to Equation (9) and the initial value

*λ*_{1}, we calculate u_{k+1}, v_{k+1}, and*λ*_{k+1}. - (2)
We get image

*u*_{ k }and stop the iteration when ∥*f*-*u*∥_{ k }≤ σ (as the stopping criterion).

## Result and discussion

*u*| ≈ 0. We fix this, as is usual, by perturbing $J\left(u\right)=\underset{\mathrm{\Omega}}{{\displaystyle \iint}}\left|\nabla u\right|dxdy$ to ${J}_{\epsilon}\left(u\right)=\underset{\mathrm{\Omega}}{{\displaystyle \iint}}\sqrt{{\left|\nabla u\right|}^{2}+{\epsilon}^{2}}dxdy$, where

*ε*is a small positive number. To be extent, the ‘stair-casing’ effect of this method can be decreased. In our calculations, we too

*k*= 10

^{−12}; the step of iteration unit

*τ*for Chambolle Algorithm is 0.2. Without loss of generality, the performance of the denoising algorithms is measured in terms of peak signal-to-noise-ratio (PSNR) [29], which can be defined as follows

where *f* is the original image and *u* is the denoising image.

### Convergence analysis

*σ*= 20 when

*λ*is smaller and bigger, respectively. In Figure 2, the first row results show that more iteration steps are required to stop criterion with smaller scale parameter

*λ*

_{0}= 0.67; the second row results show that our proposed methods require less iterations to get the optimal denoising results. At first, we used constant scale parameter

*λ*

_{0}= 0.67 to iterate three times and got a sequence vector {

*λ*

_{ k }} decreased in the first image of the third row. According to Equation (11), we got the initial value

*λ*

_{1}=

*λ*

_{2}= 5.74. The last two plots (

*i*) and (

*j*) show that ∥

*f*−

*u*∥

_{k 2}decreases monotonically with the iteration, first dropping below

*σ*at the optimal iterate

*k*= 12 and 2, respectively. It shows that our proposed method converge faster than IRM with the constant scale parameter. In Figure 3, as can be seen, with large scale parameter

*λ*

_{0}= 10 the original IRM convergences to the noisy image

*f*quickly, and only one iterative needed to reach the stop criterion. Obviously, the denoising result is not satisfied. However, promising result is obtained by our varying scale parameter strategy where the initial value λ

_{1}= λ

_{2}= 5.74 according to Equation (10).

### Preserved textures analysis

*λ*for IRM is 1. Compared with the constant scale parameter for IRM in Figure 4c–f, our proposed method can preserved more textures in Figure 4g–j. The last two plots (

*k*) and (

*l*) show that ∥

*f*−

*u*∥

_{k 2}decreases monotonically with the iterations, first dropping below

*σ*at the optimal iterate

*k*= 12 and 2, respectively. It shows that our proposed method converges faster than IRM with the constant scale parameter.

### Denoising analysis for MRI coronal brain

*λ*for IRM is 1. Compared with the constant scale parameter for IRM in Figure 5c–f, our proposed method can preserve more textures in Figure 5g–j. The last two plots (

*k*) and (

*l*) show that ∥

*f*−

*u*∥

_{k 2}decreases monotonically with the iterations, first dropping below

*σ*at the optimal iterate

*k*= 13 and 3, respectively. It shows that our proposed method converges faster than IRM with the constant scale parameter and has more texture details in the denoised image.

### Computational cost analysis

**Computer time of formula (9)**

Computing (9a) | Computing the sub-problem (9b) | |
---|---|---|

Computational time | 0.026 s | 1.281 s |

**Computer time of our method and the conventional IRM**

Our method | The conventional IRM | |
---|---|---|

Computational time | 3.136 s | 13.708 s |

**A variety of image denoising algorithms are compared for Lena image**

Gauss white noise σ | 10 | 15 | 20 | 25 | 30 |
---|---|---|---|---|---|

Wavelet + wiener | 30.65 | 28.06 | 26.94 | 25.48 | 24.39 |

Hard threshold of curvelet | 31.67 | 29.58 | 28.47 | 26.43 | 24.97 |

Soft threshold of curvelet | 31.99 | 30.76 | 29.78 | 28.79 | 26.34 |

Block threshold of curvelet | 32.76 | 32.33 | 31.09 | 29.57 | 28.08 |

Our method | 33.97 | 33.68 | 32.59 | 30.14 | 29.64 |

## Conclusion

A novel IRM with adaptive scale parameter is proposed in order to decrease the sensitivity of constant scale parameter, optimize the scale parameter adaptively in the IRM, and attain a desirable level of applicability for image denoising. We replace the classic regularization item and deduce the equation of the adaptive scale parameter, because we know that the scale parameter is smaller, the number of the iteration will enhance by IRM. Then, the rule of varying scale parameter by the trend of the sequence vector is attained. A new iterative scale parameter *λ* is obtained according to the trend of the sequence vector. In general, we can get the initial scale parameter *λ* just using three steps of iteration. We have seen with practical examples that our proposed method can reduce the number of iterations. Thus, a fast and robust method is got.

## Appendix 1

*N*

^{ T }

*X*−

*b*= 0. The basic assumption is that

*X*lies in the subspace tangent to the active constraints, i.e.,

*X*

_{i+1}=

*X*

_{ i }+

*αS*, where

*S*is the direction with the most negative directional derivative and

*α*is the iterative step length, both

*X*

_{ i }and

*X*

_{i+1}satisfy the constraint equations. Therefore, we obtain

*L*with respect to

*S*is

*N*

^{ T }

*S*= 0 in Equation (14) and multiplying Equation (17) by

*N*

^{ T }, we get

So, the proposition holds.

## Appendix 2

So, the proposition holds.

## Declarations

### Acknowledgment

This study was supported by the National Natural Science Foundation of China under the Grant nos. 60702069, 30300443 and 61105035; the Research Project of Department of Education of Zhejiang Province, China under the Grant no. 20060601; The Science Foundation of Zhejiang Sci-Tech University of China under the Grant no. 0604039-Y; the Natural Science Foundation of Zhejiang Province of China under the Grant no. Y1080851 and Y12H290045; the Research Project of 2011 overseas students of Zhejiang Province of China under the Grant no. 1104707-M; Qianjiang talents project of Science and Technology Department of Zhejiang province of China under the grant no. 2012R10054.

## Authors’ Affiliations

## References

- Rudin L, Osher S, Fatemi E: Nonlinear total variation based noise removal algorithms.
*Physica D*1992, 60: 259-268. 10.1016/0167-2789(92)90242-FView ArticleMathSciNetMATHGoogle Scholar - Devore R, Jawerth B, Lucier B: Image compression through wavelet transform coding.
*IEEE Trans. Inf. Theory*1992, 38: 719-746. 10.1109/18.119733MathSciNetView ArticleMATHGoogle Scholar - Donoho DL: De-noising by soft-threshold.
*IEEE Trans. Inf. Theory*1995, 41(3):613-627. 10.1109/18.382009MathSciNetView ArticleMATHGoogle Scholar - Dabov K, Foi A, Katkovnik V, Egiazarian K: Image denoising by sparse 3-D transform-domain collaborative filtering.
*IEEE Trans. Image Process.*2007, 16: 2080-2095.MathSciNetView ArticleGoogle Scholar - Yan R, Shao L, Cvetkovic SD, Klijn J: Improved nonlocal means based on pre-classification and invariant block matching.
*IEEE/OSA J. Disp. Technol.*2012, 8(4):212-218.View ArticleGoogle Scholar - Osher S, Solè A, Vese L: Image decomposition and restoration using total variation minimization and the H-1 norm.
*SIAM J. Multiscale Model. Simulat.*2003, 1(3):349-370. 10.1137/S1540345902416247View ArticleMATHGoogle Scholar - Burger M, Gilboa G, Osher S, Xu JJ: Nonlinear inverse scale space methods.
*Commun. Math. Sci.*2006, 4(1):175-208.MathSciNetGoogle Scholar - Chambolle A, DeVore R, Lee NY, Lucier B: Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage.
*IEEE Trans. Image Process.*1998, 7(3):319-335.MathSciNetView ArticleMATHGoogle Scholar - Chambolle A, Lucier BJ: Interpreting translation-invariant wavelet shrinkage as a new image smoothing scale space.
*IEEE Trans. Image Process.*2001, 10: 993-1000. 10.1109/83.931093MathSciNetView ArticleMATHGoogle Scholar - Steidl G, Weickert J, Brox T, MraZek P, Welk M: On the equivalence of soft wavelet shrinkage, total variation diffusion, total variation regularization, and SIDEs.
*SIAM J. Numer. Anal.*2004, 42(2):686-713. 10.1137/S0036142903422429MathSciNetView ArticleMATHGoogle Scholar - Xu J, Osher S: Iterative regularization and nonlinear inverse scale space applied to wavelet-based denoising.
*IEEE Trans. Image Process.*2007, 16(2):534-544.MathSciNetView ArticleGoogle Scholar - Meyer Y:
*Oscillating Patterns in Image Processing and Nonlinear Evolution Equations*. American Mathematical Society, Boston; 2002.Google Scholar - Osher S, Burger M, Goldfarb D, Xu J, Yin W: An iterative regularization method for total variation based image restoration.
*Multiscale Model. Simulat.*2005, 4: 460-489. 10.1137/040605412MathSciNetView ArticleMATHGoogle Scholar - Bregman L: The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming.
*U.S.S.R. Comput. Math. Math. Phys.*1967, 7: 200-217.View ArticleGoogle Scholar - Hao BB, Li M, Feng XC: Wavelet iterative regularization for image restoration with varying scale parameter.
*Signal Process. Image Commun.*2008, 23(6):433-441. 10.1016/j.image.2008.04.006View ArticleGoogle Scholar - Liu B, King K, Steckner M, Xie J, Sheng J, Ying L: Regularized sensitivity encoding (SENSE) reconstruction using Bregman iterations.
*Magn. Reson. Med.*2009, 61(1):145-152. 10.1002/mrm.21799View ArticleGoogle Scholar - Chambolle A: An algorithm for total variation minimization and applications.
*J. Math. Imag. Vis.*2004, 20: 89-97.MathSciNetView ArticleGoogle Scholar - Vese L, Osher S: Modeling textures with total variation minimization and oscillating patterns in image processing.
*J. Sci. Comput.*2003, 19: 553-572. 10.1023/A:1025384832106MathSciNetView ArticleMATHGoogle Scholar - Weissman T, Ordentlich E, Seroussi G, Verdu S, Weinberger M: Universal discrete denoising: known channel.
*IEEE Trans. Inf. Theory*2005, 51: 5-28.MathSciNetView ArticleMATHGoogle Scholar - Perona P, Malik J: Scale space and edge detection using anisotropic diffusion.
*IEEE Trans. Pattern Anal. Mach. Intell.*1990, 12: 629-639. 10.1109/34.56205View ArticleGoogle Scholar - Goldstein T, Osher S: The split Bregman method for L1-regularized problems.
*SIAM J. Imag. Sci.*2009, 2(2):323-343. 10.1137/080725891MathSciNetView ArticleMATHGoogle Scholar - Yin W, Osher S, Goldfarb D, Darbon J: Bregman iterative algorithms for l1-minimization with applications to compressed sensing.
*SIAM J. Imag. Sci.*2009, 1: 142-168.MathSciNetGoogle Scholar - He L, Chang TC, Osher S, Fang T, Speier P:
*MR image reconstruction by using the iterative refinement method and nonlinear inverse scale space methods*. 2006. Technical Report CAM Report 06–35, UCLAGoogle Scholar - Esser E:
*Applications of Lagrangian-based alternating direction methods and connections to split Bregman*. 2009. CAM Report 09–31, UCLAGoogle Scholar - Wang Y, Yang J, Yin W, Zhang Y: A new alternating minimization algorithm for total variation image reconstruction.
*SIAM J. Imag. Sci.*2008, 1(3):248-272. 10.1137/080724265MathSciNetView ArticleMATHGoogle Scholar - Ma J: Compressed sensing by inverse scale space and curvelet thresholding.
*Appl. Math. Comput.*2008, 206(2):980-988. 10.1016/j.amc.2008.10.011MathSciNetView ArticleMATHGoogle Scholar - Ma J, Le DF: Deblurring from highly incomplete measurements for remote sensing.
*IEEE Trans. Geosci. Remote Sensing (GeoRS)*2009, 3(47):792-802.Google Scholar - Gilboa G, Sochen N, Zeevi YY: Texture preserving variational denoising using an adaptive fidelity term.
*Proc VLSM Nice, France 1*2003, 137-144.Google Scholar - Shao L, Zhang H, de Haan G: An overview and performance evaluation of classification-based least squares trained filters.
*IEEE Trans. Image Process.*2008, 17: 1772-178.MathSciNetView ArticleGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.