An improved method for color image editing
© Dizdaroğğlu and İkibaşş; licensee Springer. 2011
Received: 8 March 2011
Accepted: 10 November 2011
Published: 10 November 2011
In this article, an improved approach for editing color images has been presented. In this approach, using the Poisson equation, a guided vector field is created by employing source and target images within a selected region at the first step. Next, the guided vector is used in generating the result image. Most of the existing techniques in the literature perform image editing without taking color information into account. However, without utilizing color information, image geometry cannot be created properly in some cases which may result in unsatisfactory results. Unlike the existing methods, the proposed study utilizes all the information contained in each of the color channels in computing gradient norm and performing inpainting process. The test results show that the suggested technique generates satisfactory results in editing color images.
Digital image processing operations are related with global changes including image correction, filtering, colorization, or local changes in a selected region where altering processes take place. Commercial or artistic photomontages take the local changes in images into account. Along with the technologic improvement, some softwares, such as Adobe© Photoshop©, have been released for image editing. However, professional experience is required to be able to use those kinds of softwares efficiently, and editing photos by the indicated software is tiresome. In addition, the regions altered using those tools may include some visible artifacts.
Digital image editing methods based on the Poisson equation have frequently been used in recent researches [1–12]. Perez et al.  presented an image editing approach based on the Poisson equation with Dirichlet boundary conditions. However, using this method, color inconsistency may occur in edited regions of images. Sun et al.  proposed an image matting algorithm with the Poisson equation. However, because of a long processing time, the method is not practically usable. Chuan et al.  improved the method that was suggested by Perez et al. to overcome the color inconsistency problem. But, the experiments show that the improved method is still too complex. Leventhal and Sibley  suggested an alpha interpolation technique to remove brightly colored artifacts caused by mixed seamless image cloning in edited images. Jia et al.  presented a method, called drag-and-drop pasting, which computes an optimized boundary condition automatically by employing a new objective function. However, the method is compared not with mixed seamless image cloning but with only seamless image cloning approach proposed in . Georgiev [6–8] developed a new method that is invariant to relighting and handles seamlessly illumination change including adaptation and perceptual correctness of results. The method processes the image by considering its surrounding texture contrast as well. Fattal et al.  presented a method to render high dynamic range images on a monitor using a Poisson equation. Shen et al.  suggested a method whose outputs are generated from the gradient maps by employing a Poisson equation. Dizdaroğlu and İkibaş  introduced a color image editing method with the Poisson equation, on which this study was built. Yang et al.  proposed a variational model, a distance-enhanced random walks algorithm and a multi-resolution framework based on the Poisson image editing method. Although many methods were summarized related with the research area, most of those methods are complex or may cause artifacts because of independent implementation of each color channel, or using only the lightness channel.
In this article, we present a digital image editing method that utilizes all the information contained in each of the color channels based on the Poisson equation. The test results show that the proposed method generates output images in a seamless manner with no blurring artifacts.
2. Image editing
Let f: Ω→R n and f: Ω→R be color (n = 3) and grayscale images, respectively, and they are defined on domain of Ω→R 2 . f i : Ω→R stands for the image channel i of f(1 ≤ i ≤ n): ∀p = (x, y)∈Ω. The proposed method will be explained in detail in the following sections.
Accordingly, is found as a solution to Equation 5. Constant 2 may be removed from the equation for simplicity.
is found. Here, Δ is the Laplace operator, and also Taylor's formula is used in finding second-order derivatives of f xx and f yy . Equation 7 is called as diffusion or heat equation.
where dt represents adapting time step.
Here, is the divergence of v = (u,v). This is a fundamental equation used in digital image editing.
Therefore, the source image is seamlessly cloned to the target image without blurring artifacts within the user-selected region as seen in Figure 3d. If the source image is directly copied to the destination image, some visual annoying effects may reveal in the result image as shown in Figure 3b.
with α = 0.2 times the average gradient norm of f* over Γ and, β = 0.2.
This approach, covariant Laplace equation, can be considered as a Poisson equation with a modified Δg term. Actually, the above equation can be seen as equal to by calculating simple differentiation. The equation is solved in three main steps for covariant inpainting which fills the image holes by propagating linear structures of the source region into the target region by means of diffusion process utilizing the Poisson equation approach [6, 8].
Dividing the result image f by the source image g.
Solving the PDE in the user-selected region.
Multiplying the result by the source image g.
3. The proposed method
More consistent geometry is obtained, provided that Jσ = J*Gσ is smoothed by a Gaussian filter. Here, Jσ is a good estimator of the local geometry of f at point p. Its spectral elements give the vector-valued variations (by the eigenvalues λ+, λ-, of Jσ) at the same time and the orientations (edges) of the local image structures (by the eigenvectors θ+⊥θ- of Jσ).
Here, the goal of smoothing operation can be listed as follows:
The pixels on image edges are smoothed along θ- with a strength inversely relative to the vector edge strength (anisotropic smoothing).
The pixels on homogeneous regions are smoothed along all possible directions (isotropic smoothing).
In this study, H i is a symmetric matrix since the images are regular ones and .
with the diffusion tensor .
In our method, the color image gradient norm is used to obtain the image structure more accurately as shown in Figure 10b.
where M is the binary image obtained by employing ||∇f*(p)||.
Therefore, we calculate only the gradient norm by taking the interaction among the channels into account in order to edit color images using the proposed methods. However, diffusion process in these methods, except for our covariant inpainting approach, has been done separately for each color channel.
4. Experimental results
The proposed method is compared with the previous approaches suggested in [1, 5–8, 12, 15, 18–21] by utilizing seamless and mixed seamless image cloning, arrangement of local lightness variation, texture flatting, multi-resolution cloning considering the color fidelity, covariant inpainting, image completion, curvature-preserving PDE-based inpainting, combined PDE and texture synthesis, and variational inpainting approaches. The methods are tested on color images containing RGB color space and some images in referenced papers were utilized in these performed tests. The proposed editing operation is performed on only the selected region marked by the user.
The test results of seamless image cloning method suggested in  and our seamless image cloning approach are depicted in Figure 12. Both methods give almost the same results. But, the result generated by our method has slightly more vibrant colors.
The test results of seamless and mixed seamless image cloning suggested in  and our proposed method are given in Figure 4. Both approaches developed for seamless editing method cause blurring on some parts of the edited region of the target image. However, mixed seamless image cloning method does not cause any blurriness on the result image. As seen in Figure 4f, there is almost no color inconsistency on the result image generated by the proposed method since all the information contained in each of the color channels is used in editing the image. This situation can be clearly observed on variation of color of fume in the result image.
The results of texture flattening methods are given in Figure 6. It is seen that our method properly flattens the selected region on the test image.
The results for removing a scratch from a shadow region are depicted in Figure 8. Both covariant cloning methods in [6–8] and our covariant inpainting approach not only correct the lighting but also reconstruct texture in a seamless manner as shown in Figure 8g,h. However, constrained PDE-based method in  and Poisson cloning approach in [6, 8] generate visually annoying effects as seen in Figure 8e,f.
The results of other tests on defect removal are shown in Figure 13. Our covariant inpainting method, our seamless image cloning approach, and the image completion method in  produce better results compared with other methods as seen in Figure 13c,e,f. However, direct cloning, our mixed seamless image cloning methods give visually annoying effects. We are also able to compare our covariant inpainting method with the method presented in [6, 8] since it was applied to a grayscale form of the original image. Compared to the method in [6, 8], our covariant inpainting approach generates less blurring artifacts and linear structures of the given image is propagated better, as shown in Figure 14a,b.
The methods were implemented in Microsoft Visual C++ 2005 by employing CImg Library. The program was run on a PC with Pentium 2.20 GHz processor and 2 GB RAM. The average required time depends typically on selected regions as well as image editing methods. The processing time for our covariant inpainting method shown in Figure 13f is about 5 s for seven iterations.
In this article, a method is presented for editing color images by effectively using all the information contained in each of the color channels. In this method, gradient norm and inpainting process are utilized by considering the effects of color channels on each other. This approach minimizes the color inconsistency and thus the selected region in the source image is cloned to the target image seamlessly.
As a future task, automatic region selection using moment invariants may be developed so that the proposed method is improved to be able to generate a faster output. In addition to this, the method will be extended to remove defects such as blotches from old motion pictures since there are proper patches in neighbor frames to retouch the current one .
- Perez P, Gangnet M, Blake A: Poisson image editing. ACM Trans Graph 2003,22(3):313-318. 10.1145/882262.882269View ArticleGoogle Scholar
- Sun J, Jia J, Tang CK, Shum HY: Poisson matting. ACM Trans Graph 2004,23(3):315-321. 10.1145/1015706.1015721View ArticleGoogle Scholar
- Chuan Q, Shuozhong W, Xinpeng Z: Image editing without color inconsistency using modified poisson equation. International Conference on Intelligent Information Hiding and Multimedia Signal Processing 2008.Google Scholar
- Leventhal D, Sibley PG: Poisson image editing extended. International Conference on Computer Graphics and Interactive Techniques, ACM SIGGRAPH 2006.Google Scholar
- Jia J, Sun J, Tang C, Shum H: Drag-and-drop pasting. International Conference on Computer Graphics and Interactive Techniques, ACM SIGGRAPH 2006.Google Scholar
- Georgiev T: Covariant derivatives and vision. ECCV 2006, 9th European Conference on Computer Vision, Graz, Austria 2006.Google Scholar
- Georgiev T: Photoshop healing brush: a tool for seamless cloning. Workshop on Applications of Computer Vision in conjunction with ECCV 2004, Prague 2004, 1-8.Google Scholar
- Georgiev T: Image reconstruction invariant to relighting. EUROGRAPHICS 2005.Google Scholar
- Fattal R, Lischinski D, Werman M: Gradient domain high dynamic range compression. Proceedings of ACM SIGGRAPH 2002 2002, 249-256.Google Scholar
- Shen J, Jin X, Zhou C, Wang CCL: Gradient based image completion by solving the Poisson equation. Comput Graph 2007,31(1):119-126. 10.1016/j.cag.2006.10.004View ArticleGoogle Scholar
- Dizdaroğlu B, İkibaş C: A seamless image editing technique using color information. IPCV'09: International Conf. on Image Processing, Computer Vision & Pattern Recognition, Las Vegas, USA 2009.Google Scholar
- Yang W, Zheng J, Cai J, Rahardja S, Chen CW: Natural and seamless image composition with color control. IEEE Trans Image Process 2009,18(11):2584-2592.MathSciNetView ArticleGoogle Scholar
- Tikhonov AN: Regularization of incorrectly posed problems. Soviet Math Dokl 1963, 4: 1624-1627.MATHGoogle Scholar
- Tschumperlé D, Deriche R: Vector-valued image regularization with PDE's: a common framework for different applications. IEEE Trans Pattern Anal Mach Intell 2005,27(4):506-517.View ArticleGoogle Scholar
- Tschumperlé D: Fast anisotropic smoothing of multi-valued images using curvature-preserving PDE's. Int J Comput Vis 2006,68(1):65-82. 10.1007/s11263-006-5631-zView ArticleGoogle Scholar
- Dizdaroğlu B: Solving heat flow equation for image regularization. In The 1st International Symposium on Computing in Science & Engineering (ISCSE). Aydin, Turkey; 2010:372-377.Google Scholar
- Tschumperlé D: PDE's based regularization of multivalued images and applications. PhD thesis. Université de Nice-Sophia Antipolis; 2002.Google Scholar
- Dizdaroğlu B: An image completion method using decomposition. EURASIP J Adv Signal Process 2011. Article ID 831724, 15Google Scholar
- Harald G: Combined PDE and texture synthesis approach to inpainting. Proc ECCV 2004, 2: 214-224.MATHGoogle Scholar
- Barnes C, Shechtman E, Finkelstein A, Goldman DB: PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans Graph 2009,28(3):24:1-24:11.View ArticleGoogle Scholar
- Bugeau A, Bertalmío M, Caselles V, Sapiro G: A comprehensive framework for image inpainting. IEEE Trans Image Process 2010,19(10):2634-2645.MathSciNetView ArticleGoogle Scholar
- Dizdaroğlu B, Gangal A: A spatiotemporal algorithm for detection and restoration of defects in old color films. Lecturer Notes in Computer Sciences 2007, 4678: 509-520. 10.1007/978-3-540-74607-2_46View ArticleGoogle Scholar
- Tschumperlé D: The C++ template image processing library, The CImg Library.[http://cimg.sourceforge.net/]
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.