Complementary tensor-driven image coherence diffusion for oriented structure enhancement

Oriented structure enhancement plays important role in computer vision tasks, where the diffusion is encouraged along the preferred direction instead of perpendicular to it. By analyzing the differential geometric property of the oriented structure, a complementary tensor is proposed by combining the first and the second-order structure tensors as complementary descriptors, which can precisely analyze not only the step edges, but also the weak edges such as narrow peak or ridge-like structures. Complementary diffusion tensor is constructed from the new structure tensor, which steers coherence diffusion for oriented structure enhancement. Furthermore, fast algorithm based on additive operator splitting scheme is used for numerical solution, which is much faster than usual approach. The experimental results on several images are provided. Experiments show that the image diffusion process steered by the new complementary tensor can strengthen the oriented structures and also close the interrupted lines as well. Both strong and weak edges are enhanced while noise is removed. Our approach is very promising and could be applied to many other images.


Introduction
Image enhancement is an important preprocessing step that removes noise while preserving semantically important structures such as edges and oriented structures.This may give great help for simplifying subsequent image analysis like segmentation and understanding.In recent years, nonlinear PDE-based diffusion for image enhancement has attracted much attention for its adaptive behavior in a purely data-driven way that is flexible enough to cope with the rich image structures [1].
Image diffusion by nonlinear partial differential equation (PDE) approach was first introduced by Perona and Malik [2], followed by their study, numerous nonlinear diffusion filters have been proposed [3][4][5][6][7][8].However, most of them use a scalar diffusivity thus the diffusion flux is along gradient direction, which may blur edges.Besides that, such isotropic diffusion cannot preserve the oriented structures precisely [1,7].
With the increasing appearance of oriented structures in many computer vision and image processing problems, for instance pulmonary vessels, trabecular bone, fingerprint, paintings, wood texture, etc.Many attempts on oriented patterns enhancement have been made [9][10][11][12].PDE-based coherence-enhancing diffusion has proved to be an attractive tool, which is an anisotropic diffusion steered by the diffusion tensor that allows a more flexible orientation-dependent filtering [10,11].
There are two important steps for coherence-enhancing diffusion: first analyze coherent structures and second enhance it.Structure tensor is an important descriptor for coherence analyzing, whose eigenvectors and eigenvalues can provide coherent direction and measurement, respectively.From the structure tensor, the diffusion tensor is constructed such that its eigenvalues encourage the diffusion along the coherent direction than across to it, which steers the coherent diffusion.
Precise coherence analysis is crucial to the diffusion behavior.The original structure tensor was proposed by Cottete and Germain [13], i.e., the tensor product of the gradient: J 0 = ∇u σ ⊗ ∇u T σ , where u s = G s *u is the slightly smooth version image by convolving u with Gauss kernel G s .The eigenvalues of J 0 provide coherent measurement and one of the eigenvectors provides the coherent direction.However, the simple structure tensor fails in analyzing corners or parallel structures.To solve this problem, Weickert [10,11] proposed the structure tensor: The eigenvalues of J ρ measure the variation of the gray values within a window size of order r.This structure tensor is useful to analyze strong edges, corners, and T-junctions.Brox and Weickert [14] proposed a nonlinear structure tensor by diffusing J 0 under the image gradient field ∇u, which may bring much computational cost.However, the above structure tensors are essentially linear smoothing of J 0 , and use only the local average of the first-order differential information.Many image features such as narrow peaks, ridge-like edges could not accurately be described by J r for the gradient is close to zero on these structures.
By analyzing the first-and second-order directional derivatives of the different edges, we propose a new structure tensor which is a desirable descriptor for analyzing not only strong edges, but also weak edges such as narrow peaks or ridge-like structures.The proposed structure tensor combines the first-and the second-order structure tensors as complementary descriptors that play important roles on detecting different kind of edges.The complementary diffusion tensor is constructed from the new structure tensors, which can steer the coherence diffusion controlled by a switch parameter.The diffusion can preserve not only strong edges, but also weak edges precisely while removing the noise.Furthermore, numerical implementation is solved by the additive operator splitting (AOS) scheme, resulting in fast convergence rate.
The remainder of the article is organized as follows.In Section 2, mathematical background of the coherence diffusion is outlined.In Section 3, edge structure tensor is proposed by combing the first-and the second-order structure tensors as complementary pairs.In Section 4, a complementary diffusion tensor is constructed from the new structure tensor.In Section 5, efficient numerical implementation of the diffusion PDE by fast AOS scheme is provided, and in Section 6, experimental results are provided and compared with that of the other methods.Finally, in Section 7, conclusions are reported.

Coherence diffusion
In image processing, the anisotropic diffusion process is equivalent to minimizing the following energy variational: where f denotes the original image defined on the image domain Ω and u the diffusion image.b is a small positive.ψ(s 2 ) denotes a penalizing function that is differential in its argument and convex in s.
From variational method, we can derive the following Euler Lagrange equations of anisotropic diffusion [15]: The detailed derivative can be found in [16].
It ψ (∇u • ∇u T ) = D(∇u • ∇u T ) , then Equation 2 is the steady-state solution of the following PDE with the reflecting boundary condition and the original f as the initial condition: where D(•) is the diffusion tensor and is a positive definite symmetric matrix.In the coherence diffusion case, it is desirable to rotate the diffusion tensor such that the diffusion is encouraged along the orientation of interesting features instead of perpendicular to it.The desired oriented direction is called coherent direction along which the diffusion performs more sufficient.
Designing diffusion tensor D is crucial to the diffusion process.To steer the coherence diffusion, the differential geometry property of oriented structure must be analyzed so as to detect the oriented structure and the coherent direction precisely.

Edge structure and its differential geometric property
The edge in image is defined as the position where the gray value changed suddenly.This kind of edge always refers to either step edge or the strong edge.The strong edge can easily be detected from the extreme of its first derivative.However, there are many oriented structures such as narrow peaks, ridge-like, or flow-like patterns in images.However, their first derivative is zero.We called them weak edges.These patterns play very important role as the strong edge did.Therefore, finding out the method for detecting and enhancing these oriented structure is also in great need.
Let us denote the intensity image by u(x, y).The directional derivative of U at point (x, y) in the direction, a = (cosθ, sinθ) T is denoted by u a (x, y).It is defined as: The second-order directional derivative of u along direction a is denoted by u α (x, y), and it follows that where H is the Hessian matrix of u. Figure 1 shows the Edge structure and its differential geometric property.Figure 1a, b shows the step edges and its first-and second-order derivatives.We could see that its first-order derivative can reach its extreme.Therefore, the strong edge can be detected by its first derivative.
In the following, we could see that the geometry property of the strong edge can by analyzed by the structure tensor: J 0 = ∇u σ ⊗ ∇u T σ , where u σ = G σ *u is the slightly smooth version image by convolving u with Gauss kernel G s .We call J 0 the first-order structure tensor.
From linear algebra, we can derive that the two eigenvalues of J 0 are μ 1 = |∇u σ | 2 , μ 2 = 0, and the correspond- , where ∇u ⊥ σ denotes the vector that is perpendicular to ∇u s .We know that the direction of ∇u s is perpendicular to the edge.Therefore, we can conclude that the coherent direction is e 2 ∇u ⊥ σ , because the desired diffusion direction is along the edge but not across ro it.
Figure 1c, d shows the weak edge, whose first-order directional differential is zero.However, we could see that the ridge-like structure has the extremum on its second-order directional differential.Let μ 1 , μ 2 (μ 1 ≥ μ 2 ) denote the eigenvalues of H and e 1 , e 2 the corresponding eigenvectors.From Rayleigh's quotient [17], we can derive that So, the eigenvalues of the Hessian matrix are exactly the two extreme of u α and the corresponding eigenvectors are the directions along which the second directional derivative reaches its extreme.
Since the Hessian matrix can describe the secondorder structure of the local intensity variations along the eigenvectors, we call it second-order structure tensor and denote it as J H = H s , where H s is the Hessian matrix of u s .
From Figure 1c, d, we can see that when there are dark (bright) narrow long structures in the image, u α reaches its maximum (minimum) along eigenvectors e 1 (e 2 ).As a result, the coherent direction is along e 2 (e 1 ), respectively.
From above, we can conclude that the first-and the second-order structure tensors play complementary role in detecting different kinds of edges.Let us use J r = G r *J 0 , where G r is the Gauss kernel, to replace J 0 .The first-order structure tensor J r is useful to analyze strong edges such as step edge, corners, and T-junctions, while fails in detecting weak edges precisely.However, the second-order structure tensor J H can capture weak edges as narrow peaks and ridge-like structures while fails in detecting strong edges.Therefore, the two structure tensors J r and J H can be complementary to each other and provide reliable coherence estimation on different structures.

Complementary diffusion tensor
A natural way to construct diffusion tensor D is from structure tensor J such that D has the same eigenvectors as J and its eigenvalues prefer the diffusion along the coherent direction than across to it.
Let μ 1 , μ 2 (μ1 ≥ μ 2 ) denote the eigenvalues of J and e 1 , e 2 the corresponding eigenvectors.The coherent direction estimated from J is denoted by e (co) and the orthogonal direction by e ⊥ (co) .When μ 1 >>μ 2 , it corresponds to anisotropic-oriented structure.When μ 1 ≈ μ 2 , it corresponds to isotropic structures.The coherence measurement is given by K 2 .To encourage coherent diffusion, the eigenvalues of D can be chosen as follows [10,11]: where l (co) and λ ⊥ (co) are the diffusivity along the direction e (co) and e ⊥ (co) , respectively.g > 0 serves as a threshold parameter.We can see that l (co) is an increasing function with respect to the coherence measurement K. when (μ 1 -μ 2 ) 2 > >g, l (co) ≈1.Otherwise, it leads to l (co) ≈c, where c (0, 1) is small positive parameter that guarantees that D is positive definite.It means that the more coherent the structure is, the more diffusion along the coherent direction e (co) is.Whereas the diffusion is not preferred at the direction of e ⊥ (co) as the diffusivity is very small there.
Therefore, D can be obtained by where P = e (co) , e ⊥ (co) is the eigenvector matrix.A switch parameter T is introduced to control the roles of the two structure tensors in detecting different coherent structures.When |∇u σ | ≥ T, J r is available to detect strong edges.Whereas |∇u σ | < T, J H is more reliable to detect weak edges.
Let D r , D H denote the diffusion tensor constructed from J r and J H , respectively.Therefore, the diffusion tensor is given by when |∇u σ | ≥ T, the coherent direction of D is always along ∇u ⊥ σ ; Otherwise, it is along e (H) (co) that is the coherent direction estimated from J H .It is noted that the coherent direction estimated from J H can be divided into the following two cases.When there are dark curvilinear structures in the bright background, the coherent direction is along e 2 .Otherwise there are bright curvilinear structures in the dark background, the coherent direction is along e 1 .
Substitute (8) into Equation 3, we can obtain the following partial differential diffusion equation: Equation 9 states that the diffusion tensor can steer a complementary coherence diffusion controlled by the parameter T, where the two diffusion process are both coherent and can strength the step edge, corners as well as narrow peaks, ridge-like structures while smoothing the interior of the image.

Numerical implementation by parallel AOS algorithm
The numerical solution of the diffusion equation ( 9) can be implemented by the semi-implicit discretization and AOS scheme [10,18].It is a parallel algorithm and has fast algorithm for inverse matrix.Therefore, it is 10 times faster than usual numerical method.
As the diffusion tensor D is a positive definite symmetric matrix, let D = d 11 d 12 Using central differential discretization, we obtain: Similarly, Using central differential discretization, we obtain Similarly, Let L ij denote a central difference approximation to the operator where m is the dimension of the image.In our case, m = 2.
The discretization of Equation 9 is given by the finite difference scheme: Represent the whole image of size [M,N] as a vector of size [MN,1].In this vector-matrix notation and use the semi-implicit discretization, we obtain Therefore, and denote It is noted that L ll is tri-diagonal and diagonally dominant, and use AOS scheme [18], we can obtain AOS scheme is based on the fact that Equation 21 has the same first-order Taylor expansion in ττ as that of (20).In this way, the complex matrix inverse problem in (20) can be simplified to the matrix inverse in (21).The central difference approximation of L ll , (l = 1, 2) guarantees that it is diagonally dominant tri-diagonal.Therefore, the diffusion equation (21) converts to solving a diagonally dominant tri-diagonal system of linear equation, which can be easily solved by Thomas algorithm [19].By the AOS algorithm, the numerical implementation is 10 times faster than the usual numerical method.
The AOS algorithm of the PDE ( 9) is given as follows Input U = U k 1) Calculate structure tensor J for each point and calculate the corresponding eigenvalues and eigenvectors; 2) Construct diffusion tensor D using ( 6) and ( 7);

Experiments
Selection of parameters is important for the diffusion process and the detailed discussion can be found in [15].In the following experiments, we choose b = 0.1, c = 0.001, and g the 90% quantile of the histogram for (μ 1 -μ 2 ) 2 .In the semi-implicit discretization case, the recommendation of time step τ is not more than 5.In the experiments, we set τ = 2.5.The iteration time is related to the spatial scale [15].In another word, the bigger the iteration time is, the more the diffusion is closer to its steady state.
Parameter T is crucial to switch the two diffusion processes.If T is chosen too small, some narrow long structures cannot be captured precisely.However if T is selected too large, some gradient information may lose and computational cost increases as more pixels are involved in second-order differential computing.Because setting of T is to recognize the weak edge where the gradient is very small.When T is setting to be x% quantile of the histogram for |∇u σ | 2 , it means that x% of the gradients are smaller than T. The small portion of x% has small gradients that are the potential weak edge and the second-order directional needs to be calculated there.Experiments show that setting T to be 5-10% quantile of the histogram for |∇u σ | 2 is for wide of images.In the experiments, we set T to be 10% quantile of |∇u σ | 2 .The noise scale s and integration scale r are given for each case.
We design three groups of experiments as follows: 1) Complementary diffusion for image coherenceenhancing.The diffusion equation is give by (9).
3) Isotropic diffusion by P-M equation for image enhancement [2].It corresponds to the case that D is a 2 where l is the contrast parameter that can be chosen as the 90% quantile of the histogram for | ∇f |.
The numerical solutions that are implemented by the AOS scheme are described in Section 5. When D is scalar diffusivity, it can be considered as the special case that D is single element matrix.Then where L ii is the central difference approximation to the operator The experimental results are shown in Figures 2, 3, and 4. The first columns are the original images.The second columns are the results by the proposed complementary diffusion tensor-driven approach.The third columns are the results by Weickert's coherence-diffusion method.The fourth columns are the results by P-M-diffusion equation.
Figure 2 is a noisy tree texture image, where the growth ring is corresponding to ridge-like peaks.From Figure 2b, we can see that the growth ring of tree is preserved precisely and some interrupted line is closed as well.The growth ring in Figure 2b is more straight and smooth than that of Figure 2c.Moreover, the noise is removed better by our approach.Figure 2c shows that the noise is magnified somewhere and some flow-like artifacts appear in the interior area.In Figure 2d, the edge is blurred and the growth ring could not be seen clearly.
Figure 3 is a noisy wood texture image, where texture lines are corresponding to narrow peaks.Figure 3b shows that the important image features as line structures are preserved perfectly whereas Figure 3c severely blurs and twists some lines, and some flow-like artifacts appear in the interior area in Figure 3c. Figure 3d blurs the edge and also could not remove the scattered noise.
Figure 4 is a zebra texture image, where the stripe corresponds to ridge-like structures.Figure 4b precisely preserve these structures while removing the noise and the enhanced stripe is smooth and natural.Figure 4c, however, shows some heterogeneous in the interior of the stripe and deforms the edge in some degree.Figure 4d could not obtain the oriented structure and fails in removing the noise.
Experiments show that by our approach the diffusion is preferred along the desired coherent direction for the image features that are precisely described by the proposed complementary structure tensor.Therefore, important features are preserved and enhanced while the noise is removed very well.However, Weickert's approach deforms some narrow peak and ridge-like-oriented structures because the local average of the first-order structure tensor cannot recognize these patterns precisely [10].As to the isotropic diffusion by P-M equation, it cannot close the interrupted line structures and also blurs the edge [2].The proposed switch diffusion is robust and reliable for enhancing-oriented structures.

Conclusions
In this article, complementary diffusion tensor-driven method for image coherence enhancement was proposed.A new structure tensor combing the first-and the secondorder directional differential information were proposed, which can capture not only strong edges but also narrow peak and ridge-like structures precisely.The two structure tensors play important roles in different diffusion stage controlled by a switch parameter, which can provide precise coherence estimation on different structures.A complementary diffusion was steered by the diffusion tensor constructed from the new structure tensor, which can preserve many important edges, corners, T-junctions as well as narrow peaks and ridge-like structures while removing the noise.Furthermore, fast parallel AOS algorithm was applied to numerical implementation that is very efficient.Experiments by our approach were provided and compared with that of other methods, which were very promising.Our approach is reliable and could be applied to oriented structure coherence enhancement.

Figure 1
Figure 1 Edge structures and its first-and second-order directional derivatives.(a) Step edge-I structure, (b) step edge-II structure, (c) narrow peak structure in bright background, and (d) narrow peak structure in dark background.