An improved method for color image editing
 Bekir Dizdaroğlu^{1}Email author and
 Cevat İkibaş^{1}
https://doi.org/10.1186/16876180201198
© Dizdaroğğlu and İkibaşş; licensee Springer. 2011
Received: 8 March 2011
Accepted: 10 November 2011
Published: 10 November 2011
Abstract
In this article, an improved approach for editing color images has been presented. In this approach, using the Poisson equation, a guided vector field is created by employing source and target images within a selected region at the first step. Next, the guided vector is used in generating the result image. Most of the existing techniques in the literature perform image editing without taking color information into account. However, without utilizing color information, image geometry cannot be created properly in some cases which may result in unsatisfactory results. Unlike the existing methods, the proposed study utilizes all the information contained in each of the color channels in computing gradient norm and performing inpainting process. The test results show that the suggested technique generates satisfactory results in editing color images.
Keywords
1. Introduction
Digital image processing operations are related with global changes including image correction, filtering, colorization, or local changes in a selected region where altering processes take place. Commercial or artistic photomontages take the local changes in images into account. Along with the technologic improvement, some softwares, such as Adobe^{©} Photoshop^{©}, have been released for image editing. However, professional experience is required to be able to use those kinds of softwares efficiently, and editing photos by the indicated software is tiresome. In addition, the regions altered using those tools may include some visible artifacts.
Digital image editing methods based on the Poisson equation have frequently been used in recent researches [1–12]. Perez et al. [1] presented an image editing approach based on the Poisson equation with Dirichlet boundary conditions. However, using this method, color inconsistency may occur in edited regions of images. Sun et al. [2] proposed an image matting algorithm with the Poisson equation. However, because of a long processing time, the method is not practically usable. Chuan et al. [3] improved the method that was suggested by Perez et al. to overcome the color inconsistency problem. But, the experiments show that the improved method is still too complex. Leventhal and Sibley [4] suggested an alpha interpolation technique to remove brightly colored artifacts caused by mixed seamless image cloning in edited images. Jia et al. [5] presented a method, called draganddrop pasting, which computes an optimized boundary condition automatically by employing a new objective function. However, the method is compared not with mixed seamless image cloning but with only seamless image cloning approach proposed in [1]. Georgiev [6–8] developed a new method that is invariant to relighting and handles seamlessly illumination change including adaptation and perceptual correctness of results. The method processes the image by considering its surrounding texture contrast as well. Fattal et al. [9] presented a method to render high dynamic range images on a monitor using a Poisson equation. Shen et al. [10] suggested a method whose outputs are generated from the gradient maps by employing a Poisson equation. Dizdaroğlu and İkibaş [11] introduced a color image editing method with the Poisson equation, on which this study was built. Yang et al. [12] proposed a variational model, a distanceenhanced random walks algorithm and a multiresolution framework based on the Poisson image editing method. Although many methods were summarized related with the research area, most of those methods are complex or may cause artifacts because of independent implementation of each color channel, or using only the lightness channel.
In this article, we present a digital image editing method that utilizes all the information contained in each of the color channels based on the Poisson equation. The test results show that the proposed method generates output images in a seamless manner with no blurring artifacts.
2. Image editing
Let f: Ω→R^{ n } and f: Ω→R be color (n = 3) and grayscale images, respectively, and they are defined on domain of Ω→R^{ 2 } . f_{ i } : Ω→R stands for the image channel i of f(1 ≤ i ≤ n): ∀p = (x, y)∈Ω. The proposed method will be explained in detail in the following sections.
where $F={\u2225\nabla f\u2225}^{2}={\left(\sqrt{{f}_{x}^{2}+{f}_{y}^{2}}\right)}^{2}$.
Accordingly, $2\left({f}_{xx}+{f}_{yy}\right)=0$ is found as a solution to Equation 5. Constant 2 may be removed from the equation for simplicity.
is found. Here, Δ is the Laplace operator, and also Taylor's formula is used in finding secondorder derivatives of f_{ xx } and f_{ yy } . Equation 7 is called as diffusion or heat equation.
Basically, Equation 7, at a particular time t, gives the convolution of f_{initial} with a normalized 2Dimensional Gaussian kernel G_{ σ } of variance $\sigma =\sqrt{2t}:{f}_{\mathsf{\text{initia}}l}*{G}_{\sigma}$ which means the linear smoothing, where ${G}_{\sigma}=\frac{1}{2\pi {\sigma}^{2}}exp\left(\frac{{x}^{2}+{y}^{2}}{2{\sigma}^{2}}\right)$[14–16].
where dt represents adapting time step.
Here, $\mathsf{\text{div}}\phantom{\rule{0.3em}{0ex}}\mathbf{v}=\frac{\partial u}{\partial x}+\frac{\partial v}{\partial y}$ is the divergence of v = (u,v). This is a fundamental equation used in digital image editing.
Therefore, the source image is seamlessly cloned to the target image without blurring artifacts within the userselected region as seen in Figure 3d. If the source image is directly copied to the destination image, some visual annoying effects may reveal in the result image as shown in Figure 3b.
with α = 0.2 times the average gradient norm of f* over Γ and, β = 0.2.
This approach, covariant Laplace equation, can be considered as a Poisson equation with a modified Δg term. Actually, the above equation can be seen as equal to $\Delta \frac{f}{g}=0$ by calculating simple differentiation. The equation is solved in three main steps for covariant inpainting which fills the image holes by propagating linear structures of the source region into the target region by means of diffusion process utilizing the Poisson equation approach [6, 8].
 (a)
Dividing the result image f by the source image g.
 (b)
Solving the PDE in the userselected region.
 (c)
Multiplying the result by the source image g.
3. The proposed method
More consistent geometry is obtained, provided that J_{σ} = J*G_{σ} is smoothed by a Gaussian filter. Here, J_{σ} is a good estimator of the local geometry of f at point p. Its spectral elements give the vectorvalued variations (by the eigenvalues λ^{+}, λ^{}, of J_{σ}) at the same time and the orientations (edges) of the local image structures (by the eigenvectors θ^{+}⊥θ^{} of J_{σ}).
Here, the goal of smoothing operation can be listed as follows:

The pixels on image edges are smoothed along θ^{} with a strength inversely relative to the vector edge strength (anisotropic smoothing).

The pixels on homogeneous regions are smoothed along all possible directions (isotropic smoothing).
In this study, H_{ i } is a symmetric matrix since the images are regular ones and $\frac{{\partial}^{\mathsf{\text{2}}}{f}_{i}}{\partial x\partial y}=\frac{{\partial}^{\mathsf{\text{2}}}{f}_{i}}{\partial y\partial x}$.
with the diffusion tensor $\mathbf{T}={s}^{}\left({\lambda}^{+},{\lambda}^{}\right){\theta}^{}{\theta}^{T}$.
In our method, the color image gradient norm is used to obtain the image structure more accurately as shown in Figure 10b.
where M is the binary image obtained by employing ∇f*(p).
Therefore, we calculate only the gradient norm by taking the interaction among the channels into account in order to edit color images using the proposed methods. However, diffusion process in these methods, except for our covariant inpainting approach, has been done separately for each color channel.
4. Experimental results
The proposed method is compared with the previous approaches suggested in [1, 5–8, 12, 15, 18–21] by utilizing seamless and mixed seamless image cloning, arrangement of local lightness variation, texture flatting, multiresolution cloning considering the color fidelity, covariant inpainting, image completion, curvaturepreserving PDEbased inpainting, combined PDE and texture synthesis, and variational inpainting approaches. The methods are tested on color images containing RGB color space and some images in referenced papers were utilized in these performed tests. The proposed editing operation is performed on only the selected region marked by the user.
The test results of seamless image cloning method suggested in [1] and our seamless image cloning approach are depicted in Figure 12. Both methods give almost the same results. But, the result generated by our method has slightly more vibrant colors.
The test results of seamless and mixed seamless image cloning suggested in [1] and our proposed method are given in Figure 4. Both approaches developed for seamless editing method cause blurring on some parts of the edited region of the target image. However, mixed seamless image cloning method does not cause any blurriness on the result image. As seen in Figure 4f, there is almost no color inconsistency on the result image generated by the proposed method since all the information contained in each of the color channels is used in editing the image. This situation can be clearly observed on variation of color of fume in the result image.
The results of texture flattening methods are given in Figure 6. It is seen that our method properly flattens the selected region on the test image.
The results for removing a scratch from a shadow region are depicted in Figure 8. Both covariant cloning methods in [6–8] and our covariant inpainting approach not only correct the lighting but also reconstruct texture in a seamless manner as shown in Figure 8g,h. However, constrained PDEbased method in [15] and Poisson cloning approach in [6, 8] generate visually annoying effects as seen in Figure 8e,f.
The results of other tests on defect removal are shown in Figure 13. Our covariant inpainting method, our seamless image cloning approach, and the image completion method in [20] produce better results compared with other methods as seen in Figure 13c,e,f. However, direct cloning, our mixed seamless image cloning methods give visually annoying effects. We are also able to compare our covariant inpainting method with the method presented in [6, 8] since it was applied to a grayscale form of the original image. Compared to the method in [6, 8], our covariant inpainting approach generates less blurring artifacts and linear structures of the given image is propagated better, as shown in Figure 14a,b.
The methods were implemented in Microsoft Visual C++ 2005 by employing CImg Library[23]. The program was run on a PC with Pentium 2.20 GHz processor and 2 GB RAM. The average required time depends typically on selected regions as well as image editing methods. The processing time for our covariant inpainting method shown in Figure 13f is about 5 s for seven iterations.
5. Conclusion
In this article, a method is presented for editing color images by effectively using all the information contained in each of the color channels. In this method, gradient norm and inpainting process are utilized by considering the effects of color channels on each other. This approach minimizes the color inconsistency and thus the selected region in the source image is cloned to the target image seamlessly.
As a future task, automatic region selection using moment invariants may be developed so that the proposed method is improved to be able to generate a faster output. In addition to this, the method will be extended to remove defects such as blotches from old motion pictures since there are proper patches in neighbor frames to retouch the current one [22].
Declarations
Authors’ Affiliations
References
 Perez P, Gangnet M, Blake A: Poisson image editing. ACM Trans Graph 2003,22(3):313318. 10.1145/882262.882269View ArticleGoogle Scholar
 Sun J, Jia J, Tang CK, Shum HY: Poisson matting. ACM Trans Graph 2004,23(3):315321. 10.1145/1015706.1015721View ArticleGoogle Scholar
 Chuan Q, Shuozhong W, Xinpeng Z: Image editing without color inconsistency using modified poisson equation. International Conference on Intelligent Information Hiding and Multimedia Signal Processing 2008.Google Scholar
 Leventhal D, Sibley PG: Poisson image editing extended. International Conference on Computer Graphics and Interactive Techniques, ACM SIGGRAPH 2006.Google Scholar
 Jia J, Sun J, Tang C, Shum H: Draganddrop pasting. International Conference on Computer Graphics and Interactive Techniques, ACM SIGGRAPH 2006.Google Scholar
 Georgiev T: Covariant derivatives and vision. ECCV 2006, 9th European Conference on Computer Vision, Graz, Austria 2006.Google Scholar
 Georgiev T: Photoshop healing brush: a tool for seamless cloning. Workshop on Applications of Computer Vision in conjunction with ECCV 2004, Prague 2004, 18.Google Scholar
 Georgiev T: Image reconstruction invariant to relighting. EUROGRAPHICS 2005.Google Scholar
 Fattal R, Lischinski D, Werman M: Gradient domain high dynamic range compression. Proceedings of ACM SIGGRAPH 2002 2002, 249256.Google Scholar
 Shen J, Jin X, Zhou C, Wang CCL: Gradient based image completion by solving the Poisson equation. Comput Graph 2007,31(1):119126. 10.1016/j.cag.2006.10.004View ArticleGoogle Scholar
 Dizdaroğlu B, İkibaş C: A seamless image editing technique using color information. IPCV'09: International Conf. on Image Processing, Computer Vision & Pattern Recognition, Las Vegas, USA 2009.Google Scholar
 Yang W, Zheng J, Cai J, Rahardja S, Chen CW: Natural and seamless image composition with color control. IEEE Trans Image Process 2009,18(11):25842592.MathSciNetView ArticleGoogle Scholar
 Tikhonov AN: Regularization of incorrectly posed problems. Soviet Math Dokl 1963, 4: 16241627.MATHGoogle Scholar
 Tschumperlé D, Deriche R: Vectorvalued image regularization with PDE's: a common framework for different applications. IEEE Trans Pattern Anal Mach Intell 2005,27(4):506517.View ArticleGoogle Scholar
 Tschumperlé D: Fast anisotropic smoothing of multivalued images using curvaturepreserving PDE's. Int J Comput Vis 2006,68(1):6582. 10.1007/s112630065631zView ArticleGoogle Scholar
 Dizdaroğlu B: Solving heat flow equation for image regularization. In The 1st International Symposium on Computing in Science & Engineering (ISCSE). Aydin, Turkey; 2010:372377.Google Scholar
 Tschumperlé D: PDE's based regularization of multivalued images and applications. PhD thesis. Université de NiceSophia Antipolis; 2002.Google Scholar
 Dizdaroğlu B: An image completion method using decomposition. EURASIP J Adv Signal Process 2011. Article ID 831724, 15Google Scholar
 Harald G: Combined PDE and texture synthesis approach to inpainting. Proc ECCV 2004, 2: 214224.MATHGoogle Scholar
 Barnes C, Shechtman E, Finkelstein A, Goldman DB: PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans Graph 2009,28(3):24:124:11.View ArticleGoogle Scholar
 Bugeau A, Bertalmío M, Caselles V, Sapiro G: A comprehensive framework for image inpainting. IEEE Trans Image Process 2010,19(10):26342645.MathSciNetView ArticleGoogle Scholar
 Dizdaroğlu B, Gangal A: A spatiotemporal algorithm for detection and restoration of defects in old color films. Lecturer Notes in Computer Sciences 2007, 4678: 509520. 10.1007/9783540746072_46View ArticleGoogle Scholar
 Tschumperlé D: The C++ template image processing library, The CImg Library.[http://cimg.sourceforge.net/]
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.