Edge and contrast preserving in total variation image denoising

Abstract

Total variation (TV) regularization can very well remove noise and simultaneously preserve the sharp edges. But it has the drawback of the contrast loss in the restoration. In this paper, we first theoretically analyze the loss of contrast in the original TV regularization model, and then propose a forward-backward diffusion model in the framework of total variation, which can effectively preserve the edges and contrast in TV image denoising. A backward diffusion term based on a nonconvex and monotony decrease potential function is introduced in the TV energy, resulting in a forward-backward diffusion. In order to finely control the strength of the forward and backward diffusion, and separately design the efficient algorithm to numerically implement the forward and backward diffusion, we propose a two-step splitting method to iteratively solve the proposed model. We adopt the efficient projection algorithm in the dual framework to solve the forward diffusion in the first step, and then use the simple finite differences scheme to solve the backward diffusion to compensate the loss of contrast occurred in the previous step. At last, we test the models on both synthetic and real images. Compared with the classical TV, forward and backward diffusion (FBD), two-step methods (TSM), and TV-FF models, our model has the better performance in terms of peak signal-to-noise ratio (PSNR) and mean structural similarity (MSSIM) indexes.

1 Introduction

Image denoising plays an important role in various applied areas, such as pattern recognition, medical imaging, remote sensing, video processing, and so on. So far, image denoising has seen many experienced vigorous developments, and emerged in a number of important theories and research results [1], e.g., spatial filtering method [2–4]; transform domain filtering method [5–7]; and PDE-based method [8, 9].

In this paper, we focus on variational method that achieves image denoising by functional regularization and minimization [10, 11]. Since the total variation (TV) minimization model was proposed by Rudin, Osher, and Fatemi [12] in 1992, variational method has attracted more and more research attention due to its sound theoretical basis and good experimental results, where the TV of u TV (u) is defined as

Definition 1 Let Ω ∈ R² be an open subset with Lipschitz boundary, and u(x) : Ω → R. Then TV(u) is defined as

$$ \mathrm{T}\mathrm{V}(u)={\displaystyle {\int}_{\varOmega}\left|\nabla u\right|}d\boldsymbol{x}= \sup \left\{{\displaystyle {\int}_{\varOmega }u\mathrm{d}\mathrm{i}\mathrm{v}\left(\boldsymbol{\varphi} \right)d\boldsymbol{x},\boldsymbol{\varphi} \in {C}_C^1\left(\varOmega, {\mathrm{R}}^2\right),{\left\Vert \boldsymbol{\varphi} \right\Vert}_{\infty}\le 1}\right\} $$

where div represents divergence operator. Further, $\mathrm{T}\mathrm{V}(u)+{\left\Vert u\right\Vert}_{L^1\left(\varOmega \right)}$ is known as the bounded variation (BV) of u, and TV(u) is so called BV-seminorm.

Using TV(u) to measure the restoration, Rudin et al. [12] proposed the following constrained minimization problem called TV model,

$$ \underset{u}{ \min }{{\displaystyle {\int}_{\varOmega}\left|\nabla u\right|d\boldsymbol{x},\kern1em \mathrm{subject}\kern0.5em \mathrm{t}\mathrm{o}\kern0.5em \left\Vert u-{u}_0\right\Vert}}_{L^2\left(\varOmega \right)}^2={\sigma}^2 $$

(1)

where σ is the standard deviation of noise which is assumed to be known. In order to numerically solve the constrained minimization problem (1), Rudin et al. [12] transformed it to an unconstrained problem:

$$ \underset{u}{ \min}\left\{\alpha {\displaystyle {\int}_{\varOmega}\left|\nabla u\right|d\boldsymbol{x}+{\left\Vert u-{u}_0\right\Vert}_{L^2\left(\varOmega \right)}^2}\right\} $$

(2)

and solved it by a simple finite difference scheme. Chambolle and Lions [13] showed that problem (1) is equivalent to problem (2) when α = 1/λ, where λ is the Lagrange multiplier found in solving (1). TV model (2) is convex and easy to solve in practice. In addition, the solution u allows for discontinuities along curves; therefore, edges and contours can be preserved in the restored image.

Later on, many scholars researched into the TV model (2), and proposed lots of improved TV-based models (e.g., [14–17]). We note that many of the current TV-based models have good performance in noise removing and edge preserving. However, these models often cause “loss of contrast” effects in the restoration u [18, 19]. Two examples of loss of contrast in restorations from TV model (2) can be observed in Fig. 1 for the 1-d and 2-d cases.

Fig. 1

The two examples of “loss of contrast” in restorations from TV model

There are some improved methods to preserve the contrast in TV regularization, which can be categorized into three major classes: two-step methods, partial differential equation (PDE)-based methods and variation methods.

Two-step methods (TSM) remove the noise and enhance the contrast on two separate steps [20]. Some two-step methods firstly remove the noise by TV regularization, and then enhance the contrast in the restoration obtained from the previous regularization step by some classical enhancement methods, such as histogram equalization or gray-scale transformation algorithms. These methods have the drawbacks of losing weak edges. While, there are also some methods that implement the restoration and enhancement in the reverse order, i.e., first enhancement and then regularization. These methods have the drawbacks of poor denoising ability.

PDE-based methods enhance the contrast by evolving a PDE or PDEs. Here, we would like to list some classical PDE-based models in contrast enhancement which are the motivation of our present works. Osher and Rudin [21] proposed a shock filter to enhance the contrast.

$$ \left\{\begin{array}{l}{u}_t=-\mathrm{sign}\left({u}_{\eta \eta}\right)\left|\nabla u\right|\hfill \\ {}u\left(x,t=0\right)={u}_0\hfill \\ {}{u}_{\eta \eta }=\frac{u_{xx}{u}_x^2+2{u}_x{u}_y{u}_{xy}+{u}_{yy}{u}_y^2}{{\left|\nabla u\right|}^2}\hfill \end{array}\right\} $$

Alvarez and Mazorra [22] proposed a regularization shock filter to enhance the contrast for noisy image.

$$ \left\{\begin{array}{l}{u}_t=-\mathrm{sign}\left({G}_{\sigma}\ast {u}_{\eta \eta}\right)\left|\nabla u\right|\hfill \\ {}u\left(x,t=0\right)={u}_0\hfill \\ {}{u}_{\eta \eta }=\frac{u_{xx}{u}_x^2+2{u}_x{u}_y{u}_{xy}+{u}_{yy}{u}_y^2}{{\left|\nabla u\right|}^2}\hfill \end{array}\right\} $$

Weickert [9] proposed a tensor diffusion (i.e., anisotropic diffusion) model,

$$ \left\{\begin{array}{c}\hfill {u}_t=\mathrm{d}\mathrm{i}\mathrm{v}\left(\boldsymbol{D}\nabla u\right)\hfill \\ {}\hfill u\left(x,t=0\right)={u}_0\hfill \end{array}\right. $$

where D is diffusion tensor. Guy Gilboa et al. [23] proposed forward and backward diffusion (FBD) to simultaneously remove the noise and enhance the contrast.

$$ \left\{\begin{array}{l}{u}_t=\mathrm{d}\mathrm{i}\mathrm{v}\left(-g\left(\left|\nabla u\right|\right)\nabla u\right)\hfill \\ {}u\left(x,t=0\right)={u}_0\hfill \end{array}\right. $$

where $\mathit{\mathsf{g}}\left(\left|\nabla u\right|\right)$ is a decreasing function of the gradient. Actually, the above PDE-based methods all adapt the backward diffusion to enhance the contrast. i.e., the diffusion coefficients in these diffusion equations are negative. In addition, there also have some methods that implement the classical image enhancement methods by PDE. For example, in [24], the authors proposed a PDE-based approach to perform global histogram equalization. In [25], the authors implemented the scale transformation by PDE.

So far, variation methods achieve noise removing and contrast preserving mainly by adapting the fidelity term in TV regularization. For example, in [26], a stretching function f is employed in fidelity term in TV regulation to enhance the contrast (TV-FF), where the new fidelity term is defined as ${\left\Vert \mathit{\mathsf{g}}(u)-{u}_0\right\Vert}_{L^2\left(\varOmega \right)}^2$ with $\mathit{\mathsf{g}}={f}^{-1}$. In [27], the authors proposed a fidelity term based on image gradient, which is defined as ${\left\Vert \left|\nabla u\right|-k\left|\nabla {u}_0\right|\right\Vert}_{L^2\left(\varOmega \right)}^2$ where k is a constant and satisfies k ≥1. In [28], the authors proposed a common formulation using the image gradient, ${\left\Vert \left|\nabla u\right|-k{T}_{\varepsilon}\left(\left|\nabla {G}_{\sigma}\ast {u}_0\right|\right)\right\Vert}_{L^2\left(\varOmega \right)}^2$ where T _ε is a piecewise stretching function and G _σ is a Gaussian kernel with the standard deviation σ.

In this paper, based on variation method and backward diffusion, we propose a forward-backward diffusion model in the framework of TV (called TV-FBD). Compared to the tradition forward-backward diffusion models, our model has the following characteristics and advantages.

• (1) Tradition forward-backward diffusion models are PDE based, while our diffusion model is based on variation. So, our model has the better extensibility than the tradition PDE based models. Some image information (such as gradient, direction of edges, textures, and so on) can easily be incorporated into the energy to improve the restoration quality, while it is hard to do that in PDE based models.

• (2) Tradition forward-backward diffusion models are often implemented by finite difference scheme or multi-grid technique whose computing efficiency is always low in practice. While our variation model can be solved by some faster modern convex optimization algorithms that have higher efficiency.

The rest of this paper is organized as follows. In Section 2, we theoretically analyze the loss of contrast in TV regularization for piecewise constant functions. In Section 3, we present our forward-backward diffusion model in the framework of TV (i.e., TV-FBD model). In Section 4, we introduce a two-step splitting method for the proposed model. The numerical results showing the performance of the proposed model are given in Section 5. This paper is summarized in Section 6.

2 The loss of contrast in TV regularization

In this section, we analyze the loss of contrast in TV regularization for radially symmetric piecewise constant functions. We do this because image features are often partially or entirely composed by piecewise constant, or the ‘limit’ of piecewise constant functions. Another important reason is that we can find exact results for radially symmetric piecewise constant case, which are impossible to derive in the general case.

Firstly, with radially symmetry, we rewrite the TV regularization problem as a general mode,

$$ \underset{u}{ \min }{\displaystyle \int \frac{1}{2}{\left(u(r)-{u}_0(r)\right)}^2+\alpha \left|{u}_r(r)\right|d\varOmega (r)} $$

(3)

where u _r (r) is the directional derivative of u(r) with respect to r; and dΩ(r) is the infinitesimal element which satisfies dΩ(r) = dr in R¹, dΩ(r) = 2πrdr in R² and dΩ(r) = 4πr ² dr in R³, respectively. We note that the true image and its corresponding noisy version u ₀(r) are radially symmetric. So in this case, the noise present in the image is also supposed as radially symmetry. In general, noise is not radially symmetric. The reason why we make this assumption is to make the mathematical analysis and results possible.

Any piecewise constant functions are comprised of two types of features: “steps” and “extrema” regions, as illustrated in Fig. 2. So in what follows, we only study the effects of TV regularization when u ₀(r) is a monotonic steps function and a unimodal function, separately.

Fig. 2

The denoised results of TV regularization for noisy piecewise constant functions. a Monotonic steps function. b Unimodal function

Proposition 1 (Monotonic steps function) Let u ₀(r) be defined on [r ₀, r ₃]. In addition, suppose that:

• 1. u ₀(r) is a monotonic steps function, i.e., U _i  ≥ U _i + 1 for any 1 ≤ i ≤3, where

  $$ {U}_i=\frac{{\displaystyle {\int}_{r_{i-1}}^{r_i}{u}_0(r)d\varOmega (r)}}{{\displaystyle {\int}_{r_{i-1}}^{r_i}d\varOmega (r)}} $$

  that is actually the mean of u ₀(r) in the region of ${\varOmega}_{r_{i-1},{r}_i}={\varOmega}_{r_i}-{\varOmega}_{r_{i-1}}.$

• 2. U _i  + δ _i  ≥ U _i + 1 + δ _i + 1 for any 0 ≤ i ≤3.

• 3. max{|u ₀(r) − U|} ≥ |δ _i | ≥ max_{0 ≤ i ≤ 3}{|u ₀(r) − U _i |} for any 1 ≤ i ≤ 3, where

  $$ U=\frac{{\displaystyle {\int}_{r_0}^{r_3}{u}_0(r)d\varOmega (r)}}{{\displaystyle {\int}_{r_0}^{r_3}d\varOmega (r)}} $$

  that is actually the mean of u ₀(r) in the entire region Ω.

  Then the solution of (3) can be written as

  $$ u(r)={U}_i+{\delta}_i\kern0.5em \mathrm{f}\mathrm{o}\mathrm{r}\kern0.5em r\in \left[{r}_{i-1},{r}_i\right]\kern0.5em \mathrm{with}\kern0.5em 1\le i\le 3. $$

  where

  $$ {\delta}_i=\left\{\begin{array}{l}\frac{\alpha \left(\left|\partial {\varOmega}_{r_{i-1}}\right|-\left|\partial {\varOmega}_{r_i}\right|\right)}{\left|{\varOmega}_{r_{i-1},{r}_i}\right|},\ \mathrm{if}\frac{\left|\alpha \left(\left|\partial {\varOmega}_{r_{i-1}}\right|-\left|\partial {\varOmega}_{r_i}\right|\right)\right|}{\left|{\varOmega}_{r_{i-1},{r}_i}\right|}\le \max \left\{\left|{u}_0(r)-U\right|\right\}\hfill \\ {}U-{U}_i,\kern5.00em \mathrm{if}\frac{\left|\alpha \left(\left|\partial {\varOmega}_{r_{i-1}}\right|-\left|\partial {\varOmega}_{r_i}\right|\right)\right|}{\left|{\varOmega}_{r_{i-1},{r}_i}\right|}\ge \max \left\{\left|{u}_0(r)-U\right|\right\}\hfill \end{array}\right\} $$

  (4)

  We note that Ω is the symmetry interval in R¹, the circle in R², or the sphere in R³; |∂Ω| is the boundary of Ω; |Ω| is the length of the symmetry interval in R¹, the area of the circle in R², or the volume of the sphere in R³.

Remark 1 For Proposition 1, we make a few notes. (1) We refer the reader to Theorem 1 in [18] for a similar proof of this proposition. (2) If the mean of noise added to each region is zero, then the most ideal restoration (i.e., true image) is a monotonic steps function whose discontinuities are at {r _i }, and the value in region ${\varOmega}_{r_{i-1},{r}_i}$ is U _i . (3) Proposition 1 only shows the results for monotonically decreasing step function. Actually, the results for monotonically increasing step functions are analogous. The only difference between them is that the changes in intensity over each region are of opposite sign. (4) For α sufficiently large (but finite), from the Eq. (4), we note that the regularized image u(r) is constant and is simply the mean of the observed image u ₀(r) over the entire domain Ω. In this case, the contrast of the regularized image u(r) is zero.

Example 1 In the simplest case of R¹ function, we assume that (1) r ₀ = 0; (2) $\left|\partial {\Omega}_{r_0}\right|=0$ and $\left|\partial {\Omega}_{r_3}\right|=0$ (Neumann boundary conditions); and (3) |r ₀ r ₁| = |r ₁ r ₂| = |r ₂ r ₃| = r (equidistant interval). In this case, the change in function intensity is given by δ ₁ = − α/r, δ ₂ = 0 and δ ₃ = α/r; and the regularized image is represented as

$$ u(r)={U}_i+{\delta}_i=\left\{\begin{array}{l}{U}_1-\frac{\alpha }{r},i=1\hfill \\ {}{U}_{2,\kern1.8em }i=2\hfill \\ {}{U}_3+\frac{\alpha }{r},i=3\hfill \end{array}\right. $$

It is obvious that the contrasts (i.e., the scale of the discontinuities at {r _i }) are less than they were in the true image, as illustrated in Fig. 2a. And the total loss of contrasts is 2α/r.

Example 2 We extend our results in Example 1 to the R² functions. In this case, ${\varOmega}_{r_i}\mathrm{s}$ are the concentric circles centered at r = 0 and with radius of r _i . Under the same conditions as Example 1, by Proposition 1, we have δ ₁ = − 2α/r, δ ₂ = − 2α/3r, and δ ₃ = 4α/5r. Then the regularized image is represented as

$$ u(r)={U}_i+{\delta}_i=\left\{\begin{array}{c}\hfill {U}_1-\frac{2\alpha }{r},\ i=1\hfill \\ {}\hfill {U}_2-\frac{2\alpha }{3r},\ i=2\hfill \\ {}\hfill {U}_3+\frac{4\alpha }{5r},\ i=3\hfill \end{array}\right. $$

From the last equation, we obtain that the total loss of contrasts for concentric circles in R² is 14α/5r. Similarly, we deduce that the total loss of contrasts for concentric spheres in R³ is 69α/19r.

Proposition 2 (Unimodal function) Let u ₀(r) be defined in [r ₀, r ₃]. In addition, suppose that:

• 1) u ₀(r) is a step function with a single extremum, i.e., U ₁ ≤ U ₂ ≥ U ₃.

• 2) U ₂ + δ ₂ ≥ U _i  + δ _i for i = 1,3.

• 3)
  $$ {U}_2+{\delta}_2\le { \min}_{r\in \left[{r}_1,{r}_2\ \right]}{u}_0(r) $$

• 4) ${U}_i+{\delta}_i\ge { \max}_{r\in \left[{r}_{i-1},{r}_i\ \right]}{u}_0(r)$ for i = 1,3.

• 5) max{|u ₀(r) − U|} ≥ |δ _i | for any 1 ≤ i ≤ 3.

Then the solution of (3) can be written as

$$ u(r)={U}_i+{\delta}_i\kern0.5em \mathrm{f}\mathrm{o}\mathrm{r}\kern0.5em r\in \left[{r}_{i-1},{r}_i\right]\kern0.5em \mathrm{with}\kern0.5em 1\le i\le 3. $$

where

$$ {\delta}_i=\left\{\begin{array}{l}{\left(-1\right)}^{i-1}\frac{\alpha \left(\left|\partial {\Omega}_{r_{i-1}}\right|+\left|\partial {\Omega}_{r_i}\right|\right)}{\left|{\Omega}_{r_{i-1},{r}_i}\right|},\ \mathrm{if}\ \frac{\left|\alpha \left(\left|\partial {\Omega}_{r_{i-1}}\right|+\left|\partial {\Omega}_{r_i}\right|\right)\right|}{\left|{\Omega}_{r_{i-1},{r}_i}\right|}\le \max \left\{\left|{u}_0(r)-U\right|\right\}\hfill \\ {}U-{U}_i,\kern8.25em \mathrm{if}\frac{\left|\alpha \left(\left|\partial {\Omega}_{r_{i-1}}\right|+\left|\partial {\Omega}_{r_i}\right|\right)\right|}{\left|{\Omega}_{r_{i-1},{r}_i}\right|}\ge \max \left\{\left|{u}_0(r)-U\right|\right\}\hfill \end{array}\right. $$

(5)

This proposition can be proved by dividing the unimodal function u ₀(r) into two types of component ${u}_0^1(r)$ and ${u}_0^2(r)$ with the splitpoint $\frac{r_1+{r}_2}{2}$, where ${u}_0^1(r)$ and ${u}_0^2(r)$ are monotonically increasing and decreasing step functions, respectively, and then using the conclusion in Proposition 1.

Example 3. For the simplest case of R¹ unimodal function, we still assume that (1) r ₀ = 0; (2) $\left|\partial {\Omega}_{r_0}\right|=0$ and $\left|\partial {\Omega}_{r_3}\right|=0$; and (3) |r ₀ r ₁| = |r ₁ r ₂| = |r ₂ r ₃| = r. In this case, the change in function intensity is given by δ ₁ = α/r, δ ₂ = − 2α/r, and δ ₃ = α/r; and the regularized image is represented as

$$ u(r)={U}_i+{\delta}_i=\left\{\begin{array}{c}\hfill {U}_1+\frac{\alpha }{r},\kern0.5em i=1\hfill \\ {}\hfill {U}_2-\frac{2\alpha }{r},\kern0.5em i=2\hfill \\ {}\hfill {U}_3+\frac{\alpha }{r},\kern0.5em i=3\hfill \end{array}\right. $$

We can clearly see that the contrasts in the restoration are less than they were in the true image, as illustrated in Fig. 2b. And the total loss of contrasts is 3α/r.

Example 4. We extend our results to the unimodal functions in R² and R³. Under the same conditions as Example 3, by Proposition 2, we obtain that the total loss of contrasts is 4α/r in R² and 36α/7r in R³.

Examples 1–4 indicate that in TV regularization, the loss of contrast for piecewise constant functions is exactly inversely proportional to scale of local feature measured by r (which explains why TV regularization can remove smaller scaled noise, while preserving larger scaled features essentially intact), is independent of original intensity, and is directly proportional to the regularization parameter α.

To further show the loss of contrast in TV regularization, we use TV regulation model (3) to two synthetic images, one is a noisy monotonic step image, and another is a noisy step image with some extremums. The noisy images are obtained by adding the Gaussian noise with zero mean and 10 standard deviation to the corresponding clean versions. The results are show in the Figs. 3 and 4, respectively. From the results, it is obvious that the restorations (Figs. 3c and 4c) have a loss in contrast. The plots of the cross-section slice further illustrate this point. Overall, the agreement between the theory and the experiment results is compatible.

Fig. 3

The denoised results of TV regularization for noisy monotonic step image. a Noise-free image. b Noisy image. c Denoised image. e The plots of the cross-section slice. f The plots of the cross-section slice

Fig. 4

The denoised results of TV regularization for noisy step image with few extremums. a Noise-free image. b Noisy image. c Denoised image. e The plots of the cross-section slice. f The plots of the cross-section slice

3 Forward-backward diffusion in the framework of TV (TV-FBD)

3.1 Backward diffusion model

Let f ∈ L ²(Ω), we construct an energy with respect to u.

$$ {\displaystyle {\int}_{\varOmega}\varphi \left(\left|\nabla u\right|\right)dx+\frac{1}{2}{\left\Vert f-u\right\Vert}_2^2} $$

(6)

where ${\left\Vert \cdot \right\Vert}_2^2$ denotes the square of the L ²-norm; ∇ represents gradient operator; and potential function φ is defined in [0, + ∞) and satisfies that:

• (C.1) φ(s) ≥ 0 for any s ∈ [0, + ∞);

• (C.2) φ(s) is a monotony decreasing function in [0, + ∞);

• (C.3) φ(0) = 1 and lim _{s → + ∞}φ(s) = 0.

The corresponding gradient descent flow of energy (6) is

$$ \frac{\partial u}{\partial t}=\mathrm{d}\mathrm{i}\mathrm{v}\left(\frac{\varphi^{\hbox{'}}\left(\left|\nabla u\right|\right)}{\left|\nabla u\right|}\nabla u\right)+\left(u-f\right) $$

(7)

In the Eq. (7), the term div(φ ^'(|∇u|)/|∇u| ⋅ ∇u) is the diffusion term with the diffusion velocity φ^'(|∇u|)/|∇u|. Because φ(s) is a monotonically decreasing function in [0, + ∞), we have φ ' (s) < 0 in [0, + ∞). With the nonnegativity of |∇u|, it is obvious that φ ' (|∇u|)/|∇u| < 0. In this case, the gradient descent flow

$$ \frac{\partial u}{\partial t}=\mathrm{d}\mathrm{i}\mathrm{v}\left(\frac{\varphi^{\hbox{'}}\left(\left|\nabla u\right|\right)}{\left|\nabla u\right|}\nabla u\right) $$

(8)

is actually a backward diffusion equation with a negative diffusion velocity. Next, we study the diffusion behavior in Eq.(8) in theory. Eq.(8) can be rewritten as

$$ \frac{\partial u}{\partial t}=\frac{\varphi \hbox{'}\left(\left|\nabla u\right|\right)}{\left|\nabla u\right|}\varDelta u+\nabla \left(\frac{\varphi \hbox{'}\left(\left|\nabla u\right|\right)}{\left|\nabla u\right|}\right)\cdot \nabla u $$

(9)

More precisely, Eq.(9) is a reaction diffusion equation, in which, the term φ^'(|∇u|)/|∇u| ⋅ Δu is the diffusion term, and ∇(φ′(|∇u|)/|∇u|) ⋅ ∇u is the reaction term. Then, studying the diffusion behavior of Eq.(8) is equivalent to study the diffusion term φ′(|∇u|)/|∇u| ⋅ ∇u in Eq.(9), where ∆ is Laplacian operator. To simplify the notation, we denote φ′(|∇u|)/|∇u| = ψ(u). With the conclusion stated in above, we have

$$ \varPsi (u)<0 $$

We next study the diffusion behavior of ∂u/∂t = φ^'(|∇u|)/|∇u| ⋅ Δu = Ψ(u)Δu in Eq.(9) in the framework of difference. Using forward difference to approximate ∂u/∂t, and successively using backward difference and forward difference once to approximate Laplacian, we obtain

$$ {u}_s^{t+1}={u}_s^t+\frac{\varPsi \left({u}_s^t\right)}{\left|{N}_s\right|}{\displaystyle {\sum}_{p\in {N}_s}\nabla {u}_{s.p}^t} $$

(10)

where ${u}_s^t$ represents value of function u at the “s” point after t times iterations; N _s represents the neighborhood of s point; |N _s | is the cardinality of the neighborhood; and

$$ \nabla {u}_{s.p}^t={u}_p-{u}_s^t,\ p\in {N}_s $$

(11)

In combining Eq.(10) with Eq.(11), we obtain

$$ {u}_s^{t+1}={u}_s^t+\frac{\varPsi \left({u}_s^t\right)}{\left|{N}_s\right|}{\displaystyle {\sum}_{p\in {N}_s}\nabla {u}_{s.p}^t={u}_s^t+\frac{\varPsi \left({u}_s^t\right)}{\left|{N}_s\right|}}{\displaystyle {\sum}_{p\in {N}_s}\left({u}_p-{u}_s^t\right)} $$

From the last equation, we deduce that

$$ {u}_s^{t+1}-{u}_s^t=\left|\varPsi \left({u}_s^t\right)\right|\left({u}_s^t-\frac{1}{\left|{N}_s\right|}{\displaystyle {\sum}_{p\in {N}_s}{u}_p}\right) $$

(12)

where $\frac{1}{\left|{N}_s\right|}{\displaystyle {\sum}_{p\in {N}_s}{u}_p}$ is the mean of u the neighborhood N _s .

We suppose that ${u}_s^0>\frac{1}{\left|{N}_s\right|}{\displaystyle {\sum}_{p\in {N}_s}{u}_p}$, where ${u}_s^0$ is the initial value of u before iteration. By Eq.(12), we have

$$ {u}_s^1>{u}_s^0; $$

$$ {u}_s^2-{u}_s^1=\left|\varPsi \left({u}_s^1\right)\right|\left({u}_s^1-\frac{1}{\left|{N}_s\right|}{\displaystyle {\sum}_{p\in {N}_s}{u}_p}\right)>\left|\varPsi \left({u}_s^1\right)\right|\left({u}_s^0-\frac{1}{\left|{N}_s\right|}{\displaystyle {\sum}_{p\in {N}_s}{u}_p}\right)>0 $$

In successively using the last inequality, we deduce that

$$ {u}_s^{t+1}>{u}_s^t $$

Similarly, if ${u}_s^0<\frac{1}{\left|{N}_s\right|}{\displaystyle {\sum}_{p\in {N}_s}{u}_p}$, we have

$$ {u}_s^{t+1}<{u}_s^t $$

From the above analysis, we can conclude that if the function value of a point is larger (smaller) than the mean of its neighborhood, its function value is getting larger (smaller) and larger (smaller) during the iteration, which may further increase the difference between itself and its neighbors. So, backward diffusion model can enhance the contrast and sharpen edges in image processing. Two examples of contrast enhancement by backward diffusion model can be observed in Fig. 5 for the “monotonic step” and “extrema” cases.

Fig. 5

The contrast enhancement by backward diffusion model

3.2 The selection of φ

With the conditions (C.1)–(C.3), there are several different choices for potential function φ. We here only consider two typical real-valued functions: one is an exponential function, and the other is a rational function, which are defined as following,

$$ {\varphi}_1(s)={e}^{-s};\kern0.5em {\varphi}_2(s)=\frac{1}{1+s} $$

Figure 6a shows the plots of these two functions. It is obvious that both the functions meet the conditions (C.1)–(C.3), and also have the same variation trend in [0, + ∞). A little difference between them is that φ₁(s) has a faster descent velocity than φ₂(s). In the backward diffusion equation, the diffusion velocities with potential function φ₁ and φ₂ are

Fig. 6

The plots of the potential function φ and the corresponding backward diffusion velocity Ψ. a The plot of the potential function. b The corresponding backward diffusion velocity Ψ

$$ {\varPsi}_1(s)=\frac{\varphi_1\hbox{'}(s)}{s}=\frac{-{e}^{-s}}{s} $$

$$ {\varPsi}_2(s)=\frac{\varphi_2\hbox{'}(s)}{s}=\frac{-1}{s{\left(1+s\right)}^2} $$

, respectively. Figure 6b shows the plots of these two diffusion velocity functions Ψ ₁ and Ψ ₂. We have the following observations:

• (1) Both plots are below the line of s = 0, which implies that Ψ ₁(s) < 0 and Ψ ₂(s) < 0 for any s ∈ [0, + ∞). This further demonstrates that we can achieve the backward diffusion by using the above potential functions φ₁ and φ₂.

• (2) The plots show that the diffusion velocity decreases with the increase of the value of s, and the smaller the value of s is, the faster the diffusion viscosity is. That is to say, in image diffusion, the area with small variation in intensity has a large backward diffusion velocity. Conversely, the area with large intensity variation has a small backward diffusion velocity. In this case, we can enhance the contrast in the regions with small intensity variation, and preserve the sharp intensity variation in the other regions.

• (3) Ψ ₁ and Ψ ₂ have the same variation trend on [0, + ∞), and providing nearly the same function values at the same point of s. So, in image diffusion, using these two potential function φ₁ and φ₂, we can obtain nearly the same backward diffusion velocity. In what follows, we set potential function as φ(s) = e ^− s.

3.3 The TV-FBD model

In combining backward diffusion energy with TV energy, we have

$$ \alpha {\displaystyle {\int}_{\varOmega}\left|\nabla u\right|dx+\beta }{\displaystyle {\int}_{\varOmega}\varphi \left(\left|\nabla u\right|\right)dx+\frac{1}{2}{\left\Vert f-u\right\Vert}_2^2} $$

(13)

where α and β are two nonnegative tuning parameters which balance the strength of backward diffusion and forward diffusion. The corresponding gradient descent flow of energy (13) is

$$ \frac{\partial u}{\partial t}=\alpha \mathrm{d}\mathrm{i}\mathrm{v}\left(\frac{\nabla u}{\left|\nabla u\right|}\right)+\beta \mathrm{d}\mathrm{i}\mathrm{v}\left(\frac{\varphi \hbox{'}\left(\left|\nabla u\right|\right)}{\left|\nabla u\right|}\nabla u\right)+\left(u-f\right) $$

(14)

Because the div is a linear operator, the Eq.(14) can be rewritten as

$$ \frac{\partial u}{\partial t}=\mathrm{d}\mathrm{i}\mathrm{v}\left(\frac{\alpha +\beta \varphi \hbox{'}\left(\left|\nabla u\right|\right)}{\left|\nabla u\right|}\nabla u\right)+\left(u-f\right) $$

(15)

Equation (15) is actually a reaction diffusion equation, in which (α + βφ ' (|∇u|))/|∇u| is diffusion velocity. In fact, the term α/|∇u| > 0 is forward diffusion velocity, which measures the ability of denoising for diffusion equation (15); and βφ ' (|∇u|)/|∇u| < 0 is backward diffusion velocity, which controls the ability of contrast enhancing for diffusion equation (15). Here, the parameters α and β play the key role in our forward-backward diffusion to determinate magnitudes and directions of diffusion velocity. We next give a few observations.

First, we notice that if we take α = β, then α/s > β|φ ' (s)|/s, i.e., the forward diffusion velocity is larger than the backward diffusion velocity. Figure 7a shows the plots of α/s and β|φ ' (s)|/s with α = β = 1, which further demonstrates this point. So in this case, the forward-backward diffusion velocity satisfies that (α + βφ ' (|∇u|))/|∇u| > 0, which implies that the forward diffusion dominates the diffusion directions, and the forward-backward diffusion only turns into a pure forward diffusion (see Fig. 7b).

Fig. 7

The plots of the diffusion velocity when α = β = 1 a Forward and backward diffusion velocity. b Forward-backward diffusion velocity

Second, we notice that if we take α < β, then the forward-backward diffusion velocity (α + βφ ' (|∇u|))/|∇u| is not always greater than zero, or always less than zero. Let s ₀ be a zero point of (α + βφ ' (s))/s, i.e., (α + βφ ' (s ₀))/s ₀ = 0. It is obvious that

• (1) when |∇u| < s ₀, we have (α + βφ ' (|∇u|))/|∇u| < 0, Eq.(15) is a backward diffusion equation which can enhance the contrast in the regions where u satisfies |∇u| < s ₀;

• (2) when |∇u| > s ₀, we have (α + βφ ' (|∇u|))/|∇u| > 0, Eq.(15) is a forward diffusion equation which can remove the noise from the regions where u satisfies |∇u| > s ₀.

Figure 8 further demonstrates the above conclusions. In addition, from this figure, we deduce that the larger the difference between α and β is, the larger the value of s ₀ is.

Fig. 8

Plots of the forward-backward diffusion velocity when α < β

4 Numerical implementation

In this paper, we employ the two-step splitting (TSS) method to implement the proposed model (13), which allows for further fine tuning of strength of backward diffusion and forward diffusion. In addition, the TSS method splits the mixed diffusion model (13) into a backward diffusion and a forward diffusion, which enables us to seek the suitable fast algorithm for them, separately.

Firstly, we split the mixed diffusion model (13) into two sub-problems:

$$ \underset{u}{ \min}\left\{\alpha {\displaystyle {\int}_{\varOmega}\left|\nabla u\right|dx+\frac{1}{2}{\left\Vert f-u\right\Vert}_2^2}\right\} $$

(16)

and

$$ \underset{u}{ \min}\left\{\beta {\displaystyle {\int}_{\varOmega}\varphi \left(\left|\nabla u\right|\right)dx}\right\} $$

(17)

Then, the TSS method is stated as follows:

• Step 1: Solve the forward diffusion term u with initial condition u(x, t = 0) = u ⁿ in problem (16) till some time T _f to obtain the intermediate solution, denoted by u ^n + 1/2 = u(x, T _f );

• Step 2: Solve the backward diffusion term u with initial condition u(x, t = 0) = u ^n + 1/2 in problem (17) till some time T _b to obtain the final solution, denoted by u ⁿ = u(x, T _b ).

The first step is to remove the noise from the observation, while leading to a reduction of contrast in intensity. And the second step is a correction for the first step, which can make up the losses in contrast. Because the backward diffusion is ill-posed, the second step may have the risk of moving the diffusion term u far away from observation f. By choosing a small enough T _b compared to the spatial resolution (i.e., the number of grid points), and iteratively using the first step to smooth the backward diffusion term u during the iteration process, the deviation will fall within acceptable limits. So, by using forward diffusion (step 1) and backward diffusion (step 2) alternatively, we can remove the noise, simultaneously preserve the contrast.

We here adopt the projection algorithm in the dual framework proposed by Chambolle [29] to solve minimization problem (16). The solution can be represented as:

$$ u=f-{\mathrm{Proj}}_{G_{\alpha }}(f) $$

where ${\mathrm{Proj}}_{G_{\alpha }}(f)$ is the orthogonal projection of f on the closed convex set G _α  = {v : ||v|| _G  ≤ α}. In the discrete case, setting ${\mathrm{Proj}}_{G_{\alpha }}(f)=\mathrm{d}\mathrm{i}\mathrm{v}\left(\boldsymbol{\mathsf{g}}\right)$, the computation of this nonlinear projection amounts to solve the following constrained minimization problem with inequality constraints:

$$ \underset{\boldsymbol{g}\in {C}_c^1\left(\varOmega,\ {R}^2\right)}{ \min}\left\{{\left\Vert \alpha \mathrm{d}\mathrm{i}\mathrm{v}\left(\boldsymbol{\mathsf{g}}\right)-f\right\Vert}_2^2,\kern0.75em \left|{\boldsymbol{\mathsf{g}}}_{i,j}\right|\le 1,\kern0.75em i=1,2,\cdots, M;\ j=1,2,\cdots, N\right\} $$

where M × N is the image size; $\left|{\boldsymbol{\mathsf{g}}}_{i,j}\right|=\sqrt{{\mathit{\mathsf{g}}}_1^2+{\mathit{\mathsf{g}}}_2^2}$ with $\boldsymbol{\mathsf{g}}=\left({\mathit{\mathsf{g}}}_{1,\ }\ {\mathit{\mathsf{g}}}_2\right)$. The necessary condition (Euler-Lagrange equation) of the minimization problem (27) getting an extremum is:

$$ -\nabla {\left(\alpha \mathrm{d}\mathrm{i}\mathrm{v}\left(\boldsymbol{\mathsf{g}}\right)-f\right)}_{i,j}+{\lambda}_{i,j}{\boldsymbol{\mathsf{g}}}_{i,j}=0 $$

(18)

where λ _i,j is Lagrange multiplier. By the complementary slackness condition, we have

$$ {\lambda}_{i,j}=\left|\nabla {\left(\alpha \mathrm{d}\mathrm{i}\mathrm{v}\left(\boldsymbol{\mathsf{g}}\right)-f\right)}_{i,j}\right| $$

Then, using semi-implicit fixed point iteration strategy to solve Eq.(18) with respect to g, we obtain the following iteration scheme:

$$ {\boldsymbol{\mathsf{g}}}_{i,j}^0=0;\ {\boldsymbol{\mathsf{g}}}_{i,j}^{n+1/2}=\frac{{\boldsymbol{\mathsf{g}}}_{i,j}^n+\Delta {t}_1\left(\nabla {\left(\mathrm{d}\mathrm{i}\mathrm{v}\left({\boldsymbol{\mathsf{g}}}^n\right)-\frac{f}{\alpha}\right)}_{i,j}\right)}{1+\Delta {t}_1\left|\left(\nabla {\left(\mathrm{d}\mathrm{i}\mathrm{v}\left({\boldsymbol{\mathsf{g}}}^n\right)-\frac{f}{\alpha}\right)}_{i,j}\right)\right|} $$

(19)

The forward diffusion term u is represented as

$$ {u}^{n+1/2}=\alpha \mathrm{d}\mathrm{i}\mathrm{v}\left({\boldsymbol{\mathsf{g}}}_{i,j}^{n+1/2}\right) $$

(20)

By variation theory and gradient descent scheme, the minimum of (17) is the steady-state solution of the following PDE.

$$ \left\{\begin{array}{c}\hfill \frac{\partial u}{\partial t}=\beta \mathrm{d}\mathrm{i}\mathrm{v}\left(\frac{\varphi^{\hbox{'}}\left(\left|\nabla u\right|\right)}{\left|\nabla u\right|}\nabla u\right)\hfill \\ {}\hfill u\left(\boldsymbol{x},t=0\right)={u}_0\left(\boldsymbol{x}\right)\kern3em \hfill \end{array}\right. $$

(21)

In evolution Eq.(21), |∇u| is in the denominator. In order to avoid the singularity, it is common to use a slightly perturbed norm ${\left|\nabla u\right|}_{\varepsilon }=\sqrt{{\left|\nabla u\right|}^2+\varepsilon }$, where ε is a small positive constant, to replace |∇u|. This is equivalent to minimize the functional

$$ \beta {\displaystyle {\int}_{\varOmega}\varphi \left({\left|\nabla u\right|}_{\varepsilon}\right)dx} $$

(22)

In [30], it is shown that the solutions of the perturbed problems (22) converge to the solution of (17) when ε → 0. In our experiment, we set ε = 10^− 5. In this case, by variation theory and gradient descent scheme, the minimum of (17) is the steady-state solution of the following PDE.

$$ \left\{\begin{array}{c}\hfill \frac{\partial u}{\partial t}=\beta \mathrm{d}\mathrm{i}\mathrm{v}\left(\frac{\varphi^{\hbox{'}}\left({\left|\nabla u\right|}_{\varepsilon}\right)}{{\left|\nabla u\right|}_{\varepsilon }}\nabla u\right)\hfill \\ {}\hfill u\left(\boldsymbol{x},t=0\right)={u}_0\left(\boldsymbol{x}\right)\kern3.25em \hfill \end{array}\right. $$

(23)

Using finite difference method to solve Eq.(23) numerically, we obtain the following iteration scheme:

$$ {u}_{i,j}^{n+1}={u}_{i,j}^{n+1/2}+\Delta {t}_2{\left(\beta \mathrm{d}\mathrm{i}\mathrm{v}\left(\frac{\varphi^{\hbox{'}}\left({\left|\nabla {u}^{n+1/2}\right|}_{\varepsilon}\right)}{{\left|\nabla {u}^{n+1/2}\right|}_{\varepsilon }}\nabla {u}^{n+1/2}\right)\right)}_{i,j} $$

(24)

where the discrete version of the gradient predator (∇u) _i,j  = ((∂ _x u) _i,j , (∂ _y u) _i,j ) is computed by:

$$ {\left({\partial}_xu\right)}_{i,j}=\left\{\begin{array}{l}{u}_{i,j+1}-{u}_{i,j},\kern0.75em j<N\hfill \\ {}0,\kern4em j=N\hfill \end{array}\right.\kern0.5em \mathrm{and}\kern0.5em {\left({\partial}_yu\right)}_{i,j}=\left\{\begin{array}{l}{u}_{i+1,j}-{u}_{i,j},\kern0.75em i<M\hfill \\ {}0,\kern4.25em i=M\hfill \end{array}\right. $$

And the discrete version of the divergence predator div(ξ ¹, ξ ²) _i,j is computed by:

$$ \mathrm{d}\mathrm{i}\mathrm{v}{\left({\xi}^1,{\xi}^2\right)}_{i,j}=\left\{\begin{array}{c}\hfill {\xi}_{i,j}^1-{\xi}_{i,j-1}^1,\kern0.75em 1<j<N\hfill \\ {}\hfill {\xi}_{i,j}^1,\kern5.25em j=1\kern1.25em \hfill \\ {}\hfill -{\xi}_{i,j-1}^1,\kern3.5em j=N\kern1em \hfill \end{array}\right. + \left\{\begin{array}{c}\hfill {\xi}_{i,j}^2-{\xi}_{i-1,j}^2,\kern0.75em 1<i<M\hfill \\ {}\hfill {\xi}_{i,j}^2,\kern5.25em i=1\kern1.25em \hfill \\ {}\hfill -{\xi}_{i-1,j}^2,\kern3.5em i=M\kern1em \hfill \end{array}\right. $$

In the first step, we smooth the observation by forward diffusion Eqs. (19)–(20). While in the second step, we enhance the contrast in the smoothed version obtained from the previous step by backward diffusion Eq. (24). Then, the enhanced version obtained from the second step is used as the next input of the forward diffusion step Eqs. (19)–(20). We proceed with successive application of the above alternate steps, and obtain the alternate iteration algorithm for our TV-FBD model (see Algorithm 1).

5 Experimental results

In this section, we show the experimental results of image denoising on several synthetic and real images. The comparisons with TV model [12], TSM model [20], FBD model [23], and TV-FF model [26] are also performed in a forthcoming paper to show the superiority of the proposed model. The reason why we choose these models to compare in that the TV model [12] is the most original variational model in image denoising, which is the source of our study in this paper, and the other three models are representations of the three major classes of contrast preserving in TV regularization, respectively.

The algorithms are implemented by MATLAB software on a PC with an Intel Core (I5), CPU (2.50 GHz) and RAM (2.00 GB). We give also the peak signal-to-noise ratio (PSNR) and mean structural similarity (MSSIM) index [31] for quantitative analysis, which is defined as

$$ \mathrm{PSNR}\left(u,{u}_0\right)=10\cdot { \log}_{10}\left(\frac{255^2}{\mathrm{MSE}}\right)\kern0.5em \mathrm{with}\ \mathrm{M}\mathrm{S}\mathrm{E}=\frac{{\displaystyle {\sum}_{i=1}^M}{\displaystyle {\sum}_{j=1}^N}{\left(u-{u}_0\right)}^2}{M\times N} $$

and

$$ \mathrm{MSSIM}\left(u,{u}_0\right)=\frac{1}{\mathrm{M}}{\displaystyle {\sum}_{i=1}^M\mathrm{SSIM}\left({u}^i,\ {u}_0^i\right)} $$

with

$$ \mathrm{SSIM}\left({u}^i,\ {u}_0^i\right)=\frac{\left(2{\mu}_{u^i}{\mu}_{u_0^i}+{C}_1\right)\left(2{\sigma}_{u^i\ {u}_0^i}+{C}_2\right)}{\left({\mu}_{u^i}^2+{\mu}_{u_0^i}^2+{C}_1\right)\left({\sigma}_{u^i}^2+{\sigma}_{u_0^i}^2+{C}_2\right)}, $$

respectively. Here, u is the restored image from the observation u ₀, ${\mu}_{u^i}$ and ${\sigma}_{u^i}$ are the mean and the standard deviation of u at the i-th local image window, respectively; ${\sigma}_{u^i\ {u}_0^i}$ is the covariance between u ⁱ and ${u}_0^i$; M is the number of local windows in the image; C ₁ and C ₂ are two constants. In all experiments, we set C ₁ = (0.01 * 255)² and C ₂ = (0.03 * 255)².

5.1 The choice of parameters

The parameter α is to adjust the degree of smoothing. If it is too small, the model cannot effectively remove the noises. Conversely, if α is too large, a large amount of image details will be erased due to the oversmoothing of the image. How to choose an optimal smoothing parameter α in TV regularization is still an “open question”. In this paper, we use the “trial and error” technique to determine the value of smoothing parameter. The parameter β controls the velocity of backward diffusion, as discussed in Section 3.3, which may be larger than α to achieve forward and backward diffusion. In all experiments, we set β = 5α.

In [29], the authors introduced a sufficient condition to ensure the convergence of the iterative formula (13): if Δt ₁ ≤ 1/8, then $\alpha \mathrm{d}\mathrm{i}\mathrm{v}\left({\boldsymbol{\mathsf{g}}}^n\right)\to {\mathrm{Proj}}_{G_{\alpha }}(f)$ as n → + ∞. With consideration of the numerical stability, convergence, and diffusion efficiency, we set Δt ₁ = 0.12 in all experiments. Actually, the time step Δt ₂ also controls the backward diffusion in Step 2 as the weighting parameter β, and a large Δt ₂ has the risk of increasing the range of image intensity excessively, and resulting in intensity distortion. Therefore, we should use a small enough Δt ₂ compared to the time step Δt ₁ of the forward diffusion so that the range of image intensity is within acceptable limits. In our experiments, we set Δt ₂ = 0.01.

5.2 Test of synthetic images

In our first experiment, we demonstrate contrast preserving in denoising application for different synthetic images. The test noisy images are obtained by adding the Gaussian noise with standard deviation σ = 10 into the clean versions.

Figure 9 shows the denoised results of our model for two noisy step images shown in Figs. 3 and 4. The first row shows the denoised results for the monotonic steps function shown in Fig. 3, the corresponding plots of the cross-section slice are shown in Fig. 9d. The second row shows the denoised results for the extremal steps function shown in Fig. 4, and the corresponding plots of the cross-section slice are shown in Fig. 9e. Here, compared with the noise-free images, we observe that there is no significant loss of contrast in the denoised images. We can see this much more clearly by the plots of the cross-section slice.

Fig. 9

The denoised results of our model for noisy step images shown in Figs. 3 and 4. a Noise-free image. b Noisy image. c Denoised image. d The plots of the cross-section slice. e The plots of the cross-section slice

Figure 10 shows the denoised results of our model and TV regularization for a mixed step image comprised of monotonic steps and extremal steps. And Fig. 11 shows the denoised results of our model and TV regularization for a piecewise smooth image that can be seen as a “limit” of the piecewise constant. We can clearly see that in the both case, our model and TV regularization can successfully remove the noise and prevent the edges. But TV regularization obviously reduces the contrast in the denoised images, mainly in extremum regions, and starting and ending of the monotonic steps. Due to the case that the backward diffusion term is incorporated into the energy, our model can compensate the loss of contrast caused by TV regularization.

Fig. 10

The denoised results for noisy mixed step image. a Noise-free image. b Noisy image. c Denoised image (our). d Denoised image (TV). e The plots of the cross-section slice (our). f The plots of the cross-section slice (TV)

Fig. 11

The denoised results for noisy piecewise smooth image. a Noise-free image. b Noisy image. c Denoised image (our). d Denoised image (TV). e The plot of the cross-section slice (our). f The plot of the cross-section slice (TV)

5.3 Test of real images

In our second experiment, we evaluate the performance of the proposed model using the Barbara image contaminated by Gaussian noise with different standard deviations. In addition, we compare our model to TV, FBD, TSM, and TV-FF. We note that the parameters of each model are optimized to achieve the best restoration with respect to the PSNR.

Figure 12 shows the denoised results for noisy Barbara obtained by adding Gaussian noise with standard deviation σ = 5 to the clean one. Figure 12a shows the noise-free Barbara image; Fig. 12b shows the corresponding noisy version. The black solid line in the noisy Barbara represents the cross-section slice that will be plotted in the following experiments to show how noise is removed and how contrast changes. Figure 12c–g shows the denoised results using the five models, and Fig. 12h–l shows the corresponding local zoom-in of the plots of the cross-section slice (range is of horizontal axis ∈ [140, 160]; intensity ∈ [100, 200]). Figure 13 shows denoised results for noisy Barbara contaminated by Gaussian noise with standard deviation σ = 10.

Fig. 12

The denoised results of noisy Barbara (σ = 5). a Noise-free image. b Noisy image. c TV. d FBD. e TSM. f TV-FF. g TV-FBD. h TV. i FBD. j TSM. k TV-FF. l TV-FBD

Fig. 13

The denoised results of noisy Barbara (σ = 10). a Noisy image. b TV. c FBD. d TSM. e TV-FF. f TV-FBD. g TV. h FBD. i TSM. j TV-FF. k TV-FBD

From the results shown in Figs. 12 and 13, we see that the five models can be ranked according to the restoration quality: TSM < TV < TV-FF < FBD < TV-FBD. The TSM model performs worst since it applies histogram equalization to enhance the restoration obtained by TV regularization in the previous step. Histogram equalization can perform well in contrast enhancement and improve the visual quality. But it has the drawback of increasing the range of intensity, and resulting in visual distortion. So, the restoration obtained by TSM may be worse than that obtained by TV. The TV-FF model performs better but also has some loss of contrast near steps. The FBD model performs significantly better since it adopts a linear backward diffusion in the diffusion PDE, which can compensate the loss of contrast caused by forward diffusion. The proposed TV-FBD model performs best, which leads to a best approximation of the original data with appropriate contrast near steps. The reason why the TV-FBD model performs better than FBD model is that the TV-FBD model adopts a nonlinear backward diffusion, which allows for further fine tuning of velocity in backward diffusion (see Fig. 6). And the FBD model adopts linear backward diffusion which has a constant velocity at any position. The quantitative evaluation of the restoration emphasizes the visual impression of the results. See Table 1 for the exact PSNR and MSSIM values.

Table 1 PSNR and MSSIM for different models

Next, we test the denoising capabilities of the five models in case of severe noise. Figure 14 shows the denoised results for noisy Barbara containing Gaussian noise of standard deviation σ = 15; and Fig. 15 shows results for noisy Barbara containing Gaussian noise of standard deviation σ = 20. It is obvious that these five models can remove noise very well and preserve the sharp edges in the restorations. However, TV regularization leads to a darker result due to the loss of contrast. Due to the use of the contrast enhancement scheme, the other four models lead to significantly better results. Qualitatively, these models perform equally well, but the quantitative evaluation shows that the proposed TV-FBD model has higher PSNR and MSSIM values (see Table 1).

Fig. 14

The denoised results of noisy Barbara (σ = 15). a Noisy image. b TV. c FBD. d TSM. e TV-FF. f TV-FBD

Fig. 15

The denoised results of noisy Barbara (σ = 20).a Noisy image. b TV. c FBD. d TSM. e TV-FF. f TV-FBD

Finally, to further show the effectiveness and adaptability of the proposed model, we apply it to denoise four images of size 128 × 128 contaminated by Gaussian noise with different standard deviations (σ = 5,10,15, and 20). The first two test images are relatively simple. One is a panda image that contains large white areas and black areas (see Fig. 16a); the second is a cameraman in front of a blurry background, which contains large black areas and gray areas (see Fig. 17a). The other two are butterfly and Lenna images, respectively (see Figs. 18a and 19a, respectively). Compared to the first two images, these two are more complex, which contain a large amount of details, textures, and features of low contrast. Again, we compare our model to TV, FBD, TSM, and TV-FF.

Fig. 16

The denoised results of noisy panda (σ = 10). a Noise-free image. b Noisy image. c TV. d FBD. e TSM. f TV-FF. g TV-FBD

Fig. 17

The denoised results of noisy cameraman (σ = 10). a Noise-free image. b Noisy image. c TV. d FBD. e TSM. f TV-FF. g TV-FBD

Fig. 18

The denoised results of noisy butterfly (σ = 10). a Noise-free image. b Noisy image. c TV. d FBD. e TSM. f TV-FF. g TV-FBD

Fig. 19

The denoised results of noisy Lenna (σ = 10). a Noise-free image. b Noisy image. c TV. d FBD. e TSM. f TV-FF. g TV-FBD

We here only show the restoration results of the images containing Gaussian noise with standard deviations σ = 10 (see Figs. 16, 17, 18, 19). For the other cases, we only give the PSNR and MSSIM values in Table 2. From these figures, we can clearly see that all models can successfully remove noise and simultaneously preserve edges. But the quantitative evaluation shows that the proposed TV-FBD model has the highest PSNR and MSSIM values (see Table 2), which further demonstrates that our model has the best performance in these five models.

Table 2 PSNR and MSSIM for different models

From the above denoised results, we note that forward-backward diffusion is a good tool for noise removing and contrast preserving. Table 2 shows that the traditional FBD model and TV-FBD have the best performance in terms of PSNR and MSSIM indexes among the five models. We here compare the efficiency of the proposed TV- FBD to the traditional FBD. The CPU time for FBD and TV- FBD models are listed in Table 3, where the CPU time is the cost time that the restoration takes from the initiation to first achieve the local maximum of PSNR. One can clearly see that our model performs faster than traditional FBD since it adopt the more efficient dual projection algorithm rather than the finite differences scheme.

Table 3 CPU time for FBD and TV-FBD models (second)

6 Conclusions

In this paper, we proposed a forward-backward diffusion model in the framework of total variation (i.e., TV-FBD), which can effectively solve the problem of the contrast loss in TV regularization. New model was obtained by introducing a nonconvex and monotony decrease function with respect to total variation into the TV energy. A two-step splitting method was then proposed to effectively solve the TV-FBD model. We adopted the efficient projection algorithm in the dual framework to solve the forward diffusion in the first step, and then employed the simple finite differences scheme to solve the backward diffusion to compensate the loss of contrast occurred in the previous step. The experiments in both synthetic and real images demonstrated the promising performance of the proposed model. Compared with the classical TV, FBD, TSM, and TV-FF, our TV-FBD model has the highest PSNR and MSSIM values.

It should point out that our model cannot very well recover the texture in the restoration due to the use of TV regularization. And the TV minimization favors the solutions that are piecewise constant, which implies that the oscillatory components such as textures are removed. Actually, we have proposed a multi-scale variational image decomposition model to extract texture in our previous work [32]. This implies that if the extracted texture can be incorporated into the restoration, the quality of the restoration can be improved. Our successive research will focus on how to incorporate texture representation into our forward-backward diffusion function based on total variation.