 Research
 Open Access
 Published:
Efficient nonuniform deblurring based on generalized additive convolution model
EURASIP Journal on Advances in Signal Processing volume 2016, Article number: 22 (2016)
Abstract
Image with nonuniform blurring caused by camera shake can be modeled as a linear combination of the homographically transformed versions of the latent sharp image during exposure. Although such a geometrically motivated model can well approximate camera motion poses, deblurring methods in this line usually suffer from the problems of heavy computational demanding or extensive memory cost. In this paper, we develop generalized additive convolution (GAC) model to address these issues. In GAC model, a camera motion trajectory can be decomposed into a set of camera poses, i.e., inplane translations (slice) or roll rotations (fiber), which can both be formulated as convolution operation. Moreover, we suggest a greedy algorithm to decompose a camera motion trajectory into a more compact set of slices and fibers, and together with the efficient convolution computation via fast Fourier transform (FFT), the proposed GAC models concurrently overcome the difficulties of computational cost and memory burden, leading to efficient GACbased deblurring methods. Besides, by incorporating group sparsity of the pose weight matrix into slice GAC, the nonuniform deblurring method naturally approaches toward the uniform blind deconvolution. Experimental results show that GACbased deblurring methods can obtain satisfactory deblurring results compared to both stateoftheart uniform and nonuniform deblurring methods and are much more efficient than nonuniform deblurring methods.
Introduction
Image blur is generally inevitable due to various factors such as defocus and camera shake. Blind deblurring from a realworld blurry image that needs to estimate the blur procedure and latent sharp image is a very illposed problem. Recently, several hardwareassisted approaches had been developed [1–6], by which additional information can be acquired to reduce the illposedness of the blind deblurring. These hardwareassisted approaches can provide much easier deviation of the blur or sharp image, but require complex camera configurations [1, 2] or dedicatedly designed hardware support [3–6], far from taking the place of traditional imaging devices. At the same time, in the era of ubiquitous acquisition of digital images using portable imaging devices, e.g., digital camera, mobile phone, camera shake is often unavoidable during exposure procedure, a major cause that ruins a photograph. For effective and efficient deblurring, it is crucial to develop appropriate forward blur model that can well explain the image degradation process of camera shake and then to design proper regularizers and efficient optimization algorithms.
Earlier approaches on camera shake deblurring usually assume that the blur is spatially invariant, due to which the blurry observation can be simply modeled as convolution of sharp image and blur kernel, and deblurring can thus be modeled as a blind deconvolution problem [7–12], where much attention had been devoted to design effective optimization algorithms. Since the convolution operation can be computed efficiently via fast Fourier transform (FFT), these blind deconvolution algorithms are commonly efficient. On one hand, Fergus et al. [7] adopted a mixture of Gaussians for representing the distribution of gradients and introduced a variational Bayesian (VB) approach [13] to estimate the blur kernel. The theoretical and experimental analysis by Levin et al. [8] demonstrated the advantages of VB over maximum a posterior (MAP) and inspired a number of VBbased blind deconvolution algorithms [7, 8, 14, 15]. On the other hand, by enforcing sparser priors [10, 16, 17] or exploiting edge prediction step [9, 18–21] to select salient edges in latent image, several carefully designed MAP algorithms can also exhibit promising performance. Most recently, discriminative learning approaches had been developed to learn proper priors for better estimating blur kernel [22–25].
However, camera shake blurring is often spatially varying and consequently cannot be simply modeled as a single blur kernel. Thus, how to reasonably model camera shake blurring plays a central role in nonuniform deblurring problem that can be categorized from two aspects. One direct strategy is to approximate the nonuniform blur as multiple blur kernels, where the blurry image is divided into several regions, and each region takes uniform blur. The multiple blur kernels can then be estimated from each region by performing uniform blind deconvolution method locally [26, 27]. The method proposed by Cao et al. [27] incorporated natural and textspecific dictionaries for the blind deblurring of natural scene text image. For nonuniform deblurring, they simply adopted the strategy of estimating the dedicated blur kernel for each text field. However, this kind of approach does not consider the global constraints on local blur kernels based on camera motion trajectory. The other geometrical strategy is based on the projective motion path blur model (PMPB) [28], where a camera shake trajectory can be decomposed into a sequence of camera poses lying in the 6D camera pose space (x, y, zaxes rotation and x, y, zaxes translation), and consequently, the sharp image projected by each pose is a homography, which is then weighted according to its exposure time, resulting in the blurry image. By far, there are mainly two simplified 3D geometrical models to approximate the 6D subspace, i.e., Whyte et al. [29] suggested to employ x(pitch), y(yaw), and z(roll) axes rotation, while Gupta et al. [30] proposed to adopt x and yaxes (inplane) translation and zaxis rotation. By combining the global camera motion constraint and the efficient filter flow (EFF) framework [31], Hirsch et al. suggested to construct local uniform blur models guided by the geometrical constraint [32].
Even in the simplified 3D camera pose subspace [29, 30], nonuniform deblurring methods still suffer the problem of high computational cost or extensive memory burden, placing prominent restriction on its wide applications. To speed up the blur estimation step, Gupta et al. [30] precomputed a sparse matrix for each homography transform, and thus, the forward nonuniform blur operator can be equivalently defined as the weighted sum of the homography transform matrices, and Hu and Yang [33] further restricted the possible camera pose in a low dimensional subspace. Although the precomputation techniques can relatively relax the computational inefficiency, these accelerated methods [30, 33] take increasing memory burden to store the huge homography transform matices. Even so, subsequent computation of homography transform matrices is still computationally costly, and thus, it is interesting to ask that is it possible to design forward blur model to benefit from FFT for efficient nonuniform deblurring?
In this paper, we propose generalized additive convolution (GAC) model, by which a camera motion trajectory can be decomposed into inplane translations (slice) or roll rotations (fiber), resulting in slice GAC model or fiber GAC model. In slice GAC model, a homography is formulated as the rotation of the convolution image of slice and sharp image, and in fiber GAC model, a homography is the inverse polar transform of the convolution of a kernel with the transformed sharp image (please refer to Section 3.2 for detailed proof), in which the convolution operation can be efficiently computed with the help of FFT. By this way, the GACbased forward blur models only require several FFTs and a number of pixelwise operations, significantly reducing computational complexity, and concurrently only the basis kernels and a number of rotation matrices are required to be stored so that the problem of memory burden is also relaxed. Furthermore, a greedy algorithm is proposed to generate a more compact set of slices and fibers from any camera motion trajectory, resulting in hybrid GAC model, a more promising way to concurrently solve the problems of computational inefficiency and memory burden. As for the optimization algorithm, we adopt the MAP framework to alternatively estimate nonuniform blur and latent image, in which generalized accelerated proximal gradient (GAPG) algorithm [34], a much efficient optimization algorithm for nonblind deconvolution, is employed for GACbased nonuniform deblurring.
Moreover, in slice GAC, we introduce group sparsity, i.e., l _{2,1}norm, on the pose weight matrix along the rotation angle dimension, interestingly providing a way to make nonuniform deblurring approach toward uniform blind deconvolution. Under alternative minimization framework, we also propose an effective solution to solve this problem. With the group sparsity constraint, more slices with rotation angles around 0 can be activated, and extremely when all the slices except angle 0 are inactivated, the nonuniform delburring method can degrade to uniform blind deconvolution.
Experimental results show that the proposed GAC method can obtain comparable or better deblurring results than the competing uniform and nonuniform deblurring methods. Compared to nonuniform deblurring method, GAC has a much lower peak memory usage than [16, 30, 33] and is much more efficient than the stateoftheart camera shake removal methods [16, 29, 30, 32, 33].
We summarize our contributions as follows:

We develop GAC framework together with two GAC models, i.e., slice GAC and fiber GAC, for forward modeling of camera shake blurring, by which FFT can be employed for efficient computation.

To further reduce the computational complexity, a greedy algorithm is proposed to generate hybrid GAC from any camera motion trajectory, resulting in a more compact GAC model.

As for the optimization algorithm, we adopt a fast gradient method, i.e., GAPG [34], to estimate latent image, contributing to the faster convergence of the deblurring algorithm.

By imposing group sparsity of pose weight matrix in slice GAC, we interestingly provide a way to connect uniform and nonuniform deblurring.
The rest of the paper is organized as follows. Section 2 provides a brief review on camera shake models and optimization algorithms. Section 3 presents the proposed GAC models, which in Section 4 are embedded in nonuniform deblurring, and can be efficiently solved via GAPG. Section 5 presents experimental results, and finally, Section 6 concludes this paper.
Prerequisites and related work
For efficient nonuniform deblurring, it is crucial to design forward blur models and efficient optimization algorithms. In this section, we briefly review the original projective motion path blur (PMPB) model and its simplified 3D approximation, and the optimization algorithms in MAPbased deblurring.
Homographybased model
In the PMPB model, a camera shaken image is geometrically the integration of what the camera “sees” along the motion trajectory that can be modeled in a 6D camera pose space including x, y, and zaxes rotation and x, y, and zaxes translation. With the discretion of the time t and camera pose space, the integration can be rewritten as the weighted summation of all homographies in the discrete camera pose space, where a homography is the transformed sharp image projected by corresponding camera pose.
For each pose j along the camera motion trajectory, the corresponding homography can be represented as:
where C is the matrix of camera intrinsic parameters, d is the depth of the scene, and R _{ j } and t _{ j } are the rotation matrix and translation vector, respectively.
Given H _{ j }, we can construct the corresponding warping matrix K _{ j }. Figure 1 shows a flowchart of PMPB model, and the PMPB model treats the process of nonuniform blur as a summation over the images transformed by the warping matrix
where w _{ j } is the fraction of time the camera spent at pose j, \(\boldsymbol {b}\in {{\mathbb {R}}^{n\times n}}\) is the blurry image, \(\boldsymbol {x}\in {{\mathbb {R}}^{n\times n}}\) is the latent sharp image, K is the sparse warping matrix with size n ^{2}×n ^{2}, and v denotes the additive Gaussian white noise.
PMPB is faithful to realworld camera motion, but has too many unknowns to be estimated. Fortunately, recent studies have shown that the full 6D camera pose space can be well approximated by the discrete 3D subspace of pitch, yaw, and roll rotations [29], or the 3D subspace of inplane translations and roll rotation in wide focal lengths range [30], which are both effective [35]. In [33], constrained camera pose subspace was further introduced to refine the set of camera poses. Interestingly, the global camera motion constraint can be adopted to guide the construction of local blur kernels [32]. However, the method in [29] suffers from the problem of heavy computational load to compute huge warping matrices, while the methods in [30, 33] suffer from the extensive memory burden problem to store huge warping matrices.
Additive convolution model
Deng et al. [36] suggested an additive convolution (AC) model for nonblind deblurring, where nonuniform blur is modeled as the weighted summation of the convolution of the sharp image with a set of basis convolution kernels
where k _{ i } is the ith basis convolution kernel, α _{ i } denotes the ith weight matrix, ⊗ denotes the convolution operation, ∘ denotes the pixelwise multiplication operator, and C is the number of basis convolution kernels. With FFT, the computational complexity of AC model is only \(O(Cn^{2}\mathop {\log }n)\). Furthermore, principal component analysis (PCA) is adopted to learn the basis convolution kernels in advance. However, when applied to blind nonuniform deblurring, both the basis kernels and the weighted maps should be updated in each iteration, making it unpractical in blind deblurring of camera shaken images.
In this paper, we propose a fast forward blur model, i.e., GAC, to represent nonuniform blur caused by camera shake, which is distinctly different with the existing models. Instead of utilizing sparse warping matrix [29, 30, 33], GAC models reformulate homography as convolution related to slice and fiber and can achieve better tradeoff between computational cost and memory complexity. Different from [32], GAC is an efficient implementation of the global geometric model rather than the locally uniform approximation. Compared with [36], the basis convolution kernels and the weighted maps can be efficiently constructed based on the camera motion trajectory, and thus is easy to update.
MAPbased blind deblurring
Given the blurry image b, MAP strategy is usually adopted to estimate the blur K and the latent sharp image x by minimizing the negative logarithm of posterior probability with respect to K and x,
where Φ _{1}(x) denotes the regularizer on the latent sharp image and Φ _{2}(K) denotes the regularizer on the blur operator. For nonuniform blur, \(\boldsymbol {K} \boldsymbol {x} = {\sum \nolimits }_{j} {{w_{j}}} {{\boldsymbol {K}}_{j}}{\boldsymbol {x}}\phantom {\dot {i}\!}\), where the blur operator is parameterized by the weights w _{ j }. Like the alternative updating strategy commonly adopted in uniform blind deconvolution [16, 18], the estimation of weight matrix W and latent image x should be alternatively performed.
To constrain the illposed problem for better deblurring quality, proper regularizers should be imposed. In 3D camera pose subspace, only a few poses are active given a camera motion trajectory so that it is reasonable to introduce sparsity on W. In [29, 33], l _{2}norm of weight matrix or its gradient is imposed, where the estimation of W is a linear least square problem, which can be solved by gradientbased optimization method, while in [30], nonconvex regularizer is introduced, which is optimized by an iterative reweighted least squares (IRLS).
As for updating the latent clear image, the edge prediction step is usually necessary to guarantee the algorithm converge to the desired solution, so that the sparsity, e.g., total variation [29, 33], is also imposed. In earlier researches, simple but efficient optimization method, e.g., RachardsonLucy (RL) algorithm, was adopted to solve the problem [28]; however, the deblurring results often suffer from ringing artifacts. As a nonblind deconvolution problem, fast gradientbased optimization method has been intensively studied [37, 38]. The iterative shrinkage thresholding algorithm (IST) [37] was first proposed, and due to its simplicity and efficiency, two accelerated ISTbased algorithms had been developed, i.e., FISTA [39] and TwIST [38], which both possess higher convergence rate. Furthermore, Zuo and Lin proposed the GAPG algorithm [34], which further accelerated the convergence rate of APG algorithm.
In this paper, the GACbased deblurring problem is solved in the alternative minimization framework, where the updating of pose weight matrix with both sparsity and group sparsity regularizers can be solved, and the updating of latent clear image can be efficiently solved by GAPG algorithm.
Generalized additive convolution model for camera shake
In this section, we first propose the general form of the GAC model and then decompose camera motion trajectory into slices and fibers, which provides a solution to specify GAC model for efficient modeling of camera shake blurring. Finally, we propose a greedy algorithm to generate hybrid GAC model and discuss the memory and computational complexity of the GAC model.
Generalized additive convolution model
The form of the GAC model is defined as
where f _{ i } and g _{ i } are two pixelwise operators. GAC is the generalization of the AC model [36], by defining f _{ i }(x)=α _{ i }∘x and g _{ i }(x)=x. Moreover, since both f _{ i } and g _{ i } are pixelwise operators, the computational complexity of GAC is the same as that of the AC model, i.e., O(C n ^{2}log_{2} n), where n ^{2} is the number of pixels of the image x. The smaller the C value indicates that GAC model is more efficient. Thus, the key issue of GAC model is to better specify k _{ i }, f _{ i }, and g _{ i } to reduce the C value.
Decomposition of slice and fiber
In this subsection, we show that camera poses in the 3D camera pose subspace [30] can be decomposed into slices and fibers, which then provides proper choices to design k _{ i }, f _{ i }, and g _{ i } in GAC model to reduce C value.
In the 3D camera pose subspace [30], pose in a camera motion trajectory is parameterized as
where θ _{ z } is the roll angle and t _{ x } and t _{ y } are the translations along x and y axes, respectively. For each pose in the 3D camera subspace, θ _{ j }=(θ _{ z,j },t _{ x,j },t _{ y,j }), by defining
the homography [40] can be defined as:
where t _{ j }=[t _{ x,j },t _{ y,j },1]^{T} is the translation vector, d is the depth of the scene.
As in [30, 33], we also assume that the camera intrinsic parameters are known in advance, and the calibration matrix has the standard form:
where \(\alpha _{x}=f*\frac {{{m}_{x\_{\text {ccd}}}}}{{{m}_{\textit {x\_im}}}}\phantom {\dot {i}\!}\) is the scale factors relating pixels to distance, \({{m}_{x\_{\text {ccd}}}}\) is the maximum width of the CCD, \({{m}_{x\_{\text {im}}}}\) is the maximum width of the related image, \(\alpha _{y}=f*\frac {{{m}_{y\_{\text {ccd}}}}}{{{m}_{y\_{\text {im}}}}}\phantom {\dot {i}\!}\) is quantified in the same manner, f is the focal length, and (x _{0},y _{0}) is the center coordinate.
Given \({{\boldsymbol {H}}_{\boldsymbol {\theta }}}_{_{j}}\phantom {\dot {i}\!}\), we can construct the corresponding warping matrix \({{\boldsymbol {K}}_{\boldsymbol {\theta }}}_{_{j}}\phantom {\dot {i}\!}\) (more detailed explanations can be found in [30, 33, 36]). We then have that the homography transform can be decomposed into inplane translation followed by roll rotation, yielding the following proposition.
Proposition 1.
Proof.
By defining
and according to the definition of \({{\boldsymbol {M}}_{\boldsymbol {\theta }}}_{_{j}}\phantom {\dot {i}\!}\) in Eq. (7), it is obvious to see that \({{\boldsymbol {M}}_{\boldsymbol {\theta }}}_{_{j}} = {{\boldsymbol {T}}_{\boldsymbol {\theta }}}_{_{j,t}}{{\boldsymbol {R}}_{\boldsymbol {\theta }}}_{_{j,r}}\phantom {\dot {i}\!}\). Based on the definition of \({{\boldsymbol {H}}_{\boldsymbol {\theta }}}_{_{j}}\phantom {\dot {i}\!}\) in Eq. (8), we can define \({{\boldsymbol {H}}_{\boldsymbol {\theta }}}_{_{j,r}}\phantom {\dot {i}\!}\) and \({{\boldsymbol {H}}_{\boldsymbol {\theta }}}_{_{j,t}}\phantom {\dot {i}\!}\) as \({{\boldsymbol {H}}_{\boldsymbol {\theta }}}_{_{j,r}} = {\boldsymbol {C}}{{\boldsymbol {R}}_{\boldsymbol {\theta }}}_{_{j,r}}{{\boldsymbol {C}}^{1}}\phantom {\dot {i}\!}\) and \({{\boldsymbol {H}}_{\boldsymbol {\theta }}}_{_{j,t}} = {\boldsymbol {C}}{{\boldsymbol {T}}_{\boldsymbol {\theta }}}_{_{j,t}}{{\boldsymbol {C}}^{1}}\phantom {\dot {i}\!}\). One can easily see that
and we then have
Based on the definition of \({{\boldsymbol {K}}_{\boldsymbol {\theta }}}_{_{j}}{\boldsymbol {x}}\phantom {\dot {i}\!}\), for the pixel at the location [l _{ x1},l _{ x2},1]^{T}, \({{\boldsymbol {K}}_{\boldsymbol {\theta }}}_{_{j}}{\boldsymbol {x}}\phantom {\dot {i}\!}\) assigns the value of the pixel located at \({\boldsymbol {H}}_{{{\boldsymbol {\theta }}_{j}}}^{ 1}\left ({\left [l_{x1}, l_{x2}, 1\right ]^{T}}\right)\) of x to the pixel located at [l _{ x1},l _{ x2},1] of \({{\boldsymbol {K}}_{\boldsymbol {\theta }}}_{_{j}}{\boldsymbol {x}}\phantom {\dot {i}\!}\). Based on Eq. (10), \({\boldsymbol {H}}_{{{\boldsymbol {\theta }}_{j}}}^{ 1}\left ({\left [l_{x1}, l_{x2}, 1\right ]^{T}}\right)\) can also be explained as \({\boldsymbol {H}}_{{{\boldsymbol {\theta }}_{j,r}}}^{ 1}{\boldsymbol {H}}_{{{\boldsymbol {\theta }}_{j,t}}}^{ 1}\left ({\left [l_{x1}, l_{x2}, 1\right ]^{T}}\right)\), which means that \({{\boldsymbol {K}}_{{{\boldsymbol {\theta }}_{j}}}}{\boldsymbol {x}} = {{\boldsymbol {K}}_{{{\boldsymbol {\theta }}_{j,r}}}}{{\boldsymbol {K}}_{{{\boldsymbol {\theta }}_{j,t}}}}{\boldsymbol {x}}\phantom {\dot {i}\!}\).
Proposition 1 indicates that \({{\boldsymbol {K}}_{{{\boldsymbol {\theta }}_{j}}}}{\boldsymbol {x}}\phantom {\dot {i}\!}\) is the combination of two atom operations, i.e., first translating the image x by t _{ x,j } and t _{ y,j } along xaxis and yaxis respectively, and then rotating the translated image by roll angle θ _{ z,j }. So we can rewrite \({{\boldsymbol {K}}_{{{\boldsymbol {\theta }}_{j,r}}}}{\boldsymbol {x}}\phantom {\dot {i}\!}\) and \({{\boldsymbol {K}}_{{{\boldsymbol {\theta }}_{j,t}}}}{\boldsymbol {x}}\phantom {\dot {i}\!}\) as
where \({{\boldsymbol {R}}_{{{{\theta }}_{z,j}}}}\phantom {\dot {i}\!}\) denotes the pixelwise image rotation operation, and the translation convolution kernel is defined as
The nonuniform forward blur model Eq. (2) can be further formulated as
where \({w_{{{\boldsymbol {\theta }}_{j}}}}\phantom {\dot {i}\!}\) denotes the contribution (weight) of pose θ _{ j }. It should be noted that the poses along a real camera motion trajectory form a connected 1D path, and thus, the weights of most poses are zeros. So in Eq. (14), we can only consider the subset of poses with positive weights, \({\mathcal {P}} = \left \{ {{{\boldsymbol {\theta }}_{j}}:{w_{{{\boldsymbol {\theta }}_{j}}}} > 0} \right \}\phantom {\dot {i}\!}\).
Finally, we define two special classes of subsets of \({\mathcal {P}}\): slice and fiber. A slice \({{\mathcal {S}}_{\theta } }\) is defined as \({{\mathcal {S}}_{\theta }} = \{ {{\boldsymbol {\theta }}_{j}} = \{ {\theta _{z,j}},{t_{x,j}},{t_{y,j}}\} : {{\boldsymbol {\theta }}_{j}} \in {\mathcal {P}} {\; \text {and} \;} {\theta _{z,j}} = \theta \}\phantom {\dot {i}\!}\), while a fiber \({{\mathcal {F}}_{\boldsymbol {t}}}\) is defined as \({{\mathcal {F}}_{\boldsymbol {t}}} = \{ {{\boldsymbol {\theta }}_{j}} = \{ {\theta _{z,j}}, {t_{x,j}},{t_{y,j}}\} :{{\boldsymbol {\theta }}_{j}} \in {\mathcal {P}} {\; \text {and} \;}({t_{x,j}},{t_{y,j}}) = {\boldsymbol {t}}\}\phantom {\dot {i}\!}\). Actually, \({\mathcal {P}}\) can be decomposed into a number of nonintersected slices and fibers. In the following, we will introduce how to construct slices and fibers from \({\mathcal {P}}\) and how to reformulate camera shake as the GAC model.
Slicebased GAC
Figure 2 shows an example of a slice of camera poses which have the same roll angle. Given a slice \({{\mathcal {S}}_{\theta } }\), the blur caused by camera motion within \({{\mathcal {S}}_{\theta } }\) can be formulated as
where \({{\boldsymbol {k}}_{{{\mathcal {S}}_{\theta } }}}\) denotes the slice kernel with respect to the roll angle θ.
For a general camera motion trajectory \({\mathcal {P}}\), we first classify the poses in \({\mathcal {P}}\) into a number of nonintersected slices with \({\mathcal {P}} = {\cup _{\theta }}\{{{\mathcal {S}}_{\theta }}\}\), and then, the nonuniform blur in Eq. (14) can be equivalently reformulated as
It is obvious that this is a GAC model with f _{ θ }(x)=R _{ θ }(x) and g _{ θ }(x)=x. If the range of the roll angles is discretized into n _{ z } intervals, we can see that the number C in slice GAC should be not higher than n _{ z }.
Similarly, we can define the adjoint operator K ^{T} y as
where \({{\boldsymbol {\tilde k}}_{{{\mathcal {S}}_{\theta } }}}\) is the adjoint operator of \({{\boldsymbol {k}}_{{{\mathcal {S}}_{\theta } }}}\) constructed by flipping the \({{\boldsymbol {k}}_{{{\mathcal {S}}_{\theta } }}}\) upsidedown and lefttoright, and \({\boldsymbol {R}}_{\theta }^{T}\) is the strict adjoint operator of the discrete version of R _{ θ }. It is obvious that the adjoint operator in Eq. (17) is also a GAC model and can be efficiently computed.
Finally, we discuss some implementation issues of slice GAC. To implement R _{ θ }, one can simply adopt the Matlab command imrotate. To enhance the efficiency, we maintain a lookup table (LUT) for each discrete roll angle to record the correspondence of the coordinates before and after rotation. By discretizing the range of roll angles into n _{ z } intervals, we precompute and store n _{ z } LUTs. In the continuous case, R _{−θ } is the adjoint operator of R _{ θ }, but in the discrete case, the error caused by discretization and interpolation cannot be overlooked. Thus, instead of using R _{−θ }, we adopt the strictly adjoint operator \({\boldsymbol {R}}_{\theta }^{T}\).
Fiberbased GAC
Figure 3 shows an example of a fiber of camera poses which have the same translation t=[t _{ x },t _{ y }]^{T}, which applies a visual validation of our fiberbased GAC model. Given a fiber \({{\mathcal {F}}_{\boldsymbol {t}}}\), the nonuniform blur caused by the camera motion along \({{\mathcal {F}}_{\boldsymbol {t}}}\) can be formulated as
where θ _{ t }=[0,t _{ x },t _{ y }]^{T}, \(\boldsymbol {K}_{{\boldsymbol {\theta }}_{\boldsymbol {t}}} \boldsymbol {x}\phantom {\dot {i}\!}\) denotes the inplane translation operation, and PT(·) and IPT(·) stand for the polar transform and inverse polar transform [41], respectively. In the polar transform, we use the same interval to discretize angular and roll angles, and thus, the basis filter w _{ t } can be defined as
where θ _{1} is the minimal roll angle and θ _{ nz } is the maximal roll angle.
We compare the model in Eq. (18) with the 3D camera pose subspace model in [30] for the simulation of blur caused by camera shake along a fiber. Figure 3 shows the blurry images obtained by using these two methods. One can easily see that the difference between Fig. 3 c, d is insignificant, which demonstrates that the model in Eq. (18) can be used for modeling camera shaken image with a series of roll rotations.
For a general camera motion trajectory \({\mathcal {P}}\), we first classify the poses in \({\mathcal {P}}\) into a number of nonintersected fibers with \({\mathcal {P}} = { \cup _{\boldsymbol {t}}}\{ {{\mathcal {F}}_{\boldsymbol {t}}}\}\), and then, the nonuniform blur in Eq. (14) can be equivalently reformulated as
It is obvious that this is a GAC model with f _{ t }(x)=IPT(x) and \({{\boldsymbol {g}}_{\boldsymbol {t}}}({\boldsymbol {x}}) = \text {PT}({{\boldsymbol {K}}_{\boldsymbol {\theta }_{\boldsymbol {t}}}}{\boldsymbol {x}})\phantom {\dot {i}\!}\). If the range of the inplane translations is discretized into n _{ x }×n _{ y } intervals, then the number C in fiber GAC should be not higher than n _{ x } n _{ y }.
We can then define the adjoint operator K ^{T} y as
where \({{\boldsymbol {\tilde w}}_{\boldsymbol {t}}}\) is the adjoint operator of w _{ t }, and PT^{T} and IPT^{T} are the adjoint operators of PT and IPT, respectively. To enhance the computational efficiency, two extra LUTs are precomputed to record the correspondence of polar and inverse polar transform, respectively. Moreover, we use the strict adjoint operators of PT and IPT, i.e., PT^{T} and IPT^{T}, to avoid the inconsistence caused by discretization and interpolation.
Hybrid GAC for modeling
For GAC, the key to save computational cost is to reduce C, the number of basis filters. Given a general camera motion trajectory as shown in Fig. 4, neither pure slicebased nor pure fiberbased GAC can guarantee sufficiently small value of C. However, in a hybrid (slice and fiber mixed) decomposition, only two slices and two fibers are required to model the camera motion trajectory, so that the computational complexity is significantly reduced. Thus, we propose a greedy method to decompose a camera motion trajectory into a hybrid set of slices and fibers to reduce the C value.
Given the pose subset \({\mathcal {P}}\) and the 3D weight matrix W with W(θ _{ z },t _{ x },t _{ y }), the weight of the pose θ=(θ _{ z },t _{ x },t _{ y }), in each iteration, the proposed method first finds a candidate slice \({{\mathcal {S}}_{{{\hat \theta }_{z}}}}\), and a candidate fiber \(\phantom {\dot {i}\!}{\mathcal {F}}_{{{\hat t}_{x}},{{\hat t}_{y}}}\), compare their relative contributions, and then choose the slice or fiber with higher weights. By this way, we can obtain a slice set \(\phantom {\dot {i}\!}\left \{ {\left ({{\theta _{j}},{{\mathcal {S}}_{j}}, {{\boldsymbol {k}}_{j}}} \right):j = 1,...,{n_{s}}} \right \}\) and a fiber set \(\phantom {\dot {i}\!}\left \{ {\left ({{\boldsymbol {t}_{i}},{{\mathcal {F}}_{i}},{{\boldsymbol {w}}_{j}}} \right):i = 1,...,{n_{f}}} \right \}\). As shown in Fig. 4, the proposed greedy algorithm can successfully decompose the camera motion trajectory into two slices and two fibers. The detailed algorithm is summarized in Algorithm 1.
Based on the slice set and the fiber set, the nonuniform caused by camera shake can be reformulated as
The adjoint operator K ^{T} y is then defined as
Discussions
With FFT, the computational complexity of the model in Eq. (22) is O((n _{ s }+n _{ f })n ^{2} log2n). If n _{ s } and n _{ f } are small, GAC would be more efficient than the other methods. Let n _{ z } be the number of intervals for the roll angle. It is reasonable to assume that (n _{ s }+n _{ f })<n _{ z }; otherwise, we can use the pure slicebased GAC model.
To further improve the computational efficiency, the LUT method can be adopted for fast image rotation, polar and inverse polar transform, and n _{ z }+2 LUTs should be precomputed and stored in memory. In [30, 33], a sparse n ^{2}×n ^{2} matrix was constructed for each pose in the 3D camera pose subspace. Compared with the model in [30, 33], GAC can achieve much better tradeoff between memory and computational complexity.
In [32], camera shake blurring is approximated as the sum of R ^{2} uniformly blurry patches, and the computational complexity of the model in [32] is O(R ^{2}(n/R+w)^{2} log2(n/R+w)), where w×w is the size of the blur kernel. Compared with [32], when R and w are higher, GAC would be computationally more efficient, and our experimental results also validates the efficiency of GAC against the model by Hirsch et al. [32]. Moreover, the model in [32] is a locally uniform approximation of the camera shake blurring model, while GAC can be strictly equivalent with the geometric model in [30].
GACbased nonuniform deblurring
In this section, by incorporating the proposed GAC forward blur model into existing deblurring model, the GACbased nonuniform deblurring can be efficiently solved. Then, by imposing l _{2,1}norm regularizer on pose weight matrix W, the slice GAC deblurring can approach toward uniform blind deconvolution.
GACbased deblurring via GAPG
In typical nonuniform deblurring methods, TV regularizer on latent image and l _{2}norm regularizer on pose weight matrix are imposed, yielding the deblurring formulation
where λ and τ are tradeoff parameters. By substituting the forward blur model with any GAC model, we come to the GACbased nonuniform deblurring method, which can be solved by alternatively updating pose weight matrix W and latent clear image x. By fixing x, we use the method in [33] to update W, while by fixing W, we develop an efficient solution to x based on GAPG algorithm.
In [34], GAPG was developed to solve uniform nonblind deblurring, where the Lipschitz constant is generalized to diagonal matrix that can guarantee faster convergence rate. By introducing two auxiliary variables d _{ h } and d _{ v }, we reformulate the TVbased model into the following equivalent problem
where μ is the relaxation parameter and D _{ h } and D _{ v } are the horizontal and vertical gradient operators, respectively. According to [34], in each iteration, several subproblems should be solved for the updating of x, D _{ h }, and D _{ v }, respectively. We use the same method as [34] for updating D _{ h } and D _{ v } and choose the proper Lipschitz matrix for updating x. For the updating of x, the subproblem is formulated as
where \({\boldsymbol {y}}_{\boldsymbol {x}},{\boldsymbol {y}}_{{{\boldsymbol {d}}_{v}}},\phantom {\dot {i}\!}\) and \({\boldsymbol {y}}_{{{\boldsymbol {d}}_{h}}}\phantom {\dot {i}\!}\) have the same definition as in [34]. Here, K ^{T}(Ky _{ x }−b) is computed based on Eq. 22 and Eq. 23. In [34], λ _{max} is set based on the inequality \({\lambda _{\max }} \le {\left ({\sqrt \mu {{\left \ {\boldsymbol {K}} \right \}_{2}} + \sqrt \eta {{\left \ {{{\boldsymbol {D}}_{h}}} \right \}_{2}} + \sqrt \eta {{\left \ {{{\boldsymbol {D}}_{v}}} \right \}_{2}}} \right)^{2}}\phantom {\dot {i}\!}\) with η=1, ∥D _{ h }∥_{2}≤2, and ∥D _{ v }∥_{2}≤2 [34]. The diagonal Lipschitz matrix is
By using smaller λ _{max} and η, GAPG algorithm will converge faster. For nonuniform blur, we choose ∥K∥_{2}=1 based on the following proposition.
Proposition 2.
Proof.
In matrix analysis, we have
where ∥·∥_{1} and ∥·∥_{ ∞ } denote the largest l _{1}norm of the columns and rows, respectively. For the uniform blur procedure b=Kx, where each row of K is a shifted version of blur kernel K, Zuo and Lin had proved that the l _{1}norm of the row of matrix K corresponding to the (i,j)th entry of b is k_{1} [34]. For the nonuniform blur matrix K, (i,j)th entry of b has its own blur kernel k ^{(i,j)}, and consequently, K_{ ∞ }=max_{(i,j)}k ^{(i,j)}_{1}. By the nonnegative constraint k _{ i }≥0,∀i and the normalization constraint \({\sum \nolimits }_{i}{{{k}_{i}}=1}\), l _{1}norm of any blur kernel is 1, i.e., k ^{(i,j)}_{1}=1. Thus, we have K_{ ∞ }=1. Similarly, K_{1}=1 can be derivated from [34]. Thus, the inequality is proved.
The updating of subproblems should be performed several iterations. With the properly chosen Lipschitz matrix, each variable has it own Lipschitz constant, rather than the largest one adopted in APG algorithm, so that the GAPGbased deblurring method has a faster convergence rate.
Connection with uniform blind deconvolution
We impose group sparsity, i.e., l _{2,1}norm, on weight matrix W along roll angle dimension to connect GACbased nonuniform deblurring and uniform blind deconvolution
where \(\tau _{\theta _{j}}\) is the tradeoff parameter controlling the weight of all slices with angle θ _{ j }, and by setting \(\tau _{\theta _{j}=0}<\tau _{\theta _{i}},\forall i> j\), the slices with larger rotation angle will be gradually inactivated, and naturally, the nonuniform deblurring will approach toward uniform blind deconvolution.
To solve the this problem, we also adopt the alternative minimization strategy, where the updating of latent clear image shares the same solution via GAPG as Section 4.1. Due to the nonsmoothness of l _{2,1}norm regularizer, we propose an effective solution to solve it. By introducing auxiliary variable W ^{′}, the problem can be reformulated as
where δ is the positive penalty parameter. For the Wsubproblem, it is also a linear least square problem, which can be efficiently solved [33], and for the W ^{′}subproblem, it is a standard l _{2,1}norm optimization problem, which can be solved by group shrinkage operator [42].
The slice GAC with group sparsity provides a natural connection of nonuniform deblurring and uniform deconvolution. When there are only camera inplane translations, i.e., no roll rotation, the camera shake blurring will be uniform, and the slice GAC can play like the traditional uniform blind deconvolution method. Otherwise, roll rotations will cause the nonuniform blur, where slice GAC can play like nonuniform deblurring method.
Figure 5 shows a deblurring example of the standard slice GAC (SGAC) and slice GAC with group sparsity (GSGAC) on a real camerashaken image. From the distribution of pose weights shown as Fig. 5 d, the group sparsity constrains the slices gathering at small rotation angles, where especially slices with angle 0 dominate, while in slice GAC, the activated poses are more randomly distributed. Thus, for the blurry image with slight roll rotation shown as Fig. 5 a, GSGAC performs much better than SGAC, significantly suffering less artifacts.
Experimental results
In this section, we evaluate the performance of proposed GAC deblurring methods. First, three GAC variants, i.e., slicebased GAC (SGAC), fiberbased GAC (FGAC), and hybrid GAC (HGAC), are compared in terms of computational time, memory usage, and visual quality. Due to its superiority over SGAC and FGAC, HGAC is then compared with the stateoftheart geometrical methods, i.e., Whyte et al. [29], Gupta et al. [30], and Hu and Zhang [33], and the stateoftheart EFFbased methods, i.e., Hirsch et al. [32], Xu et al. [16], and Cao et al. [27]. Finally, in the comparison with the stateoftheart uniform deblurring methods [9, 21, 24, 43], slice GAC with GSGAC is adopted. All the experiments are conducted on a PC with Intel Core i52400 3.10 GHz CPU and 16 G RAM memory, and the proposed method is implemented in Matlab.
Comparison of three GAC variants
On three realworld camerashaken images shown as Fig. 6, three GAC variants are evaluated in terms of running time and memory usage. As shown in Table 1, the three GAC methods have comparable peak memory cost to each other, while in terms of running time, HGAC is more efficient than both SGAC and FGAC, since the set of slices and fibers decomposed by HGAC is usually more compact, thus having lower computational complexity.
From the restoration quality shown as Fig. 6, HGAC and SGAC perform better than FGAC in recovering more details and achieving more visual plausible deblurring quality. Due to its superiority over both SGAC and FGAC, HGAC is adopted in the following comparison with nonuniform deblurring methods.
Comparison with geometrical methods
We use five realworld camerashaken images, shown in the top row of Fig. 7, to compare GAC with three geometrical methods, i.e., Whyte et al. [29], Gupta et al. [30], and Hu et al. [33], where their deblurring results are obtained by running source codes or executable programs provided by authors. Although GAC costs more memory than that of Whyte et al.’s [29] shown as Table 2, it is at least 100 × faster, and in terms of deblurring quality shown as Fig. 7, GAC performs much better than that of Whyte et al.’s [29] in achieving clearer and plausible texture details. At the same time, Gupta et al.’s [30], Hu and Yang’s [33], and the proposed GAC model actually adopt the same 3D subspace to approximate the full 6D camera pose space and consequently can obtain the similar deblurring results. Thus, it is more critical to evaluate their performance in terms of computational efficiency and memory usage. Tables 2 and 3 show that GAC not only is at least 2.5 × faster but also significantly relaxes memory burden than Gupta et al.’s [30] and Hu and Yang’s [33].
Comparison with nongeometrical methods
Using the images in Fig. 8, we further compare GAC with four nongeometrical methods proposed by Harmeling et al. [44], Hirsch et al. [32], Xu et al. [16], and Cao et al. [27].
Since neither source code nor executable program of Harmeling et al. [44] and Hirsch et al. [32] is available, we collected the deblurring results from their papers or websites. The nongeometrical methods greatly rely on the reasonability of region division and often sacrifice image details to smooth out possible artifacts at region boundaries. As shown in Fig. 9 a, b, GAC can achieve more visually plausible deblurring results, while the results by Harmeling et al. [44] and Hirsch et al. [32] are visually oversmoothed. Although the method by Cao et al. [27] is designed for better recovering clear text, the camera shake blurring in text field is not fully removed and significant artifacts are included in the results, like the first two images in Fig. 10. Moreover, GAC also performs better than the method by Cao et al. [27] in textless fields. Xu et al. [16] and Cao et al. [27] respectively provided the executable program and source codes, so in Table 4, we report the CPU running time and memory usage comparison with Xu et al. [16] and Cao et al. [27] on several camerashaken images shown in Fig. 8, from which one can see that GAC also performs better than Xu et al. or Cao et al. [27]. As for deblurring quality shown as Fig. 9 c, GAC can obtain comparable if not superior deblurring result than Xu et al. [16].
Comparison with uniform deblurring methods
In the comparison experiments with the stateoftheart uniform deblurring methods, i.e., Shan et al. [9], Xu et al. [21], Zuo et al. [24], and Pan et al. [43], we adopt GSGAC, which performs better in handling spatially invariant blurry images. In Fig. 11, the first image is of nonuniform blur with slight roll rotations, from which one can see that GAC can achieve more visually plausible deblurring results, such as the statue’s face is recovered more clearly, while the uniform deblurring methods usually suffer from ringing artifacts. Then, on two spatially invariant blurred images, GAC can also achieve satisfactory deblurring results, since the imposed group sparsity enforces the GAC method to play like uniform deblurring method.
Conclusions
In this paper, by designing GAC model to geometrically represent camera shake blurring, nonuniform deblurring is efficiently addressed. Since the slices and fibers decomposed from the camera motion trajectory can be formulated as convolution, all the GAC methods can exploit FFT for efficient optimization. Compared with the methods in [30, 33], the proposed method only needs several FFTs and to store several basis convolution kernels and lookuptables in memory. By incorporating group sparsity into pose weight matrix, the GACbased deblurring methods can also work like uniform blind deconvolution, better handling uniform blurry images. Compared with nonuniform deblurring methods, GAC method has a much lower peak memory usage and is much more efficient. Compared with uniform deblurring methods, GAC method also can achieve satisfactory deblurring results.
References
 1
S McCloskey, Y Ding, J Yu, Design and estimation of coded exposure point spread functions. IEEE Trans. Pattern Anal. Mach. Intell.34(10), 2071–2077 (2012).
 2
S McCloskey, in ECCV. Velocitydependent shutter sequences for motion deblurring (SpringerCrete, Greece, 2010).
 3
A Levin, R Fergus, F Durand, WT Freeman, Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph.26(3), 70 (2007).
 4
N Joshi, SB Kang, CL Zitnick, R Szeliski, Image deblurring using inertial measurement sensors. ACM Trans. Graph.29(4), 30 (2010).
 5
YW Tai, H Du, MS Brown, S Lin, Correction of spatially varying image and video motion blur using a hybrid camera. IEEE Trans. Pattern Anal. Mach. Intell.32(6), 1012–1028 (2010).
 6
YW Tai, N Kong, S Lin, SY Shin, in CVPR. Coded exposure imaging for projective motion deblurring (IEEESan Francisco, USA, 2010), pp. 2408–2415.
 7
R Fergus, B Singh, A Hertzmann, ST Roweis, WT Freeman, Removing camera shake from a single photograph. ACM Trans. Graph.25(3), 787–794 (2006).
 8
A Levin, Y Weiss, F Durand, WT Freeman, in CVPR. Understanding and evaluating blind deconvolution algorithms (IEEEMiami, USA, 2009), pp. 1964–1971.
 9
Q Shan, J Jia, A Agarwala, Highquality motion deblurring from a single image. ACM Trans. Graph.27(3), 73 (2008).
 10
D Krishnan, T Tay, R Fergus, in CVPR. Blind deconvolution using a normalized sparsity measure (IEEEColorado Springs, USA, 2011), pp. 233–240.
 11
TS Cho, S Paris, BKP Horn, WT Freeman, in CVPR. Blur kernel estimation using the radon transform (IEEEColorado Springs, USA, 2011), pp. 241–248.
 12
Z Hu, MH Yang, in ECCV, 7576. Good regions to deblur (SpringerFirenze, Italy, 2012), pp. 59–72.
 13
J Miskin, DJ MacKay, in Adv. Ind. Compon. Anal. Ensemble learning for blind image separation and deconvolution (SpringerEdinburgh, UK, 2000), pp. 123–141.
 14
A Levin, Y Weiss, F Durand, WT Freeman, in CVPR. Efficient marginal likelihood optimization in blind deconvolution (IEEEColorado Springs, USA, 2011), pp. 2657–2664.
 15
SD Babacan, R Molina, MN Do, AK Katsaggelos, in ECCV, 7577. Bayesian blind deconvolution with general sparse image priors (SpringerFirenze, Italy, 2012), pp. 341–355.
 16
L Xu, S Zheng, J Jia, in CVPR. Unnatural l0 sparse representation for natural image deblurring (IEEEColumbus, USA, 2013), pp. 1107–1114.
 17
J Kotera, F Šroubek, P Milanfar, in Compt. Anal. Images Patterns, 8048. Blind deconvolution using alternating maximum a posteriori estimation with heavytailed priors (SpringerYork, UK, 2013), pp. 59–66.
 18
S Cho, S Lee, Fast motion deblurring. ACM Trans. Graph.28(5), 145 (2009).
 19
N Joshi, R Szeliski, D Kriegman, in CVPR. PSF estimation using sharp edge prediction (IEEEAnchorage, USA, 2008), pp. 1–8.
 20
L Sun, S Cho, J Wang, J Hays, in ICCP. Edgebased blur kernel estimation using patch priors (IEEEClujNapoca, Romania, 2013), pp. 1–8.
 21
L Xu, J Jia, in ECCV. Twophase kernel estimation for robust motion deblurring (SpringerCrete, Greece, 2010), pp. 157–170.
 22
K Schelten, S Nowozin, J Jancsary, C Rother, S Roth, in WACV. Interleaved regression tree field cascades for blind image deconvolution (IEEEHawaii, USA, 2015).
 23
CJ Schuler, M Hirsch, S Harmeling, B Schölkopf, in NIPS. Learning to deblur (NIPSMontréal, Canada, 2014).
 24
W Zuo, D Ren, S Gu, L Lin, L Zhang, in CVPR. Discriminative learning of iterationwise priors for blind deconvolution (IEEEBoston, USA, 2015), pp. 3232–3240.
 25
Z Hu, MH Yang, Learning good regions to deblur images. Int. J. Comp. Vis.115(3), 345–362 (2015).
 26
H Ji, K Wang, in CVPR. A twostage approach to blind spatiallyvarying motion deblurring (IEEERhode Island, USA, 2012), pp. 73–80.
 27
X Cao, W Ren, W Zuo, X Guo, H Foroosh, Scene text deblurring using textspecific multiscale dictionaries. IEEE Trans. Image Process.24(4), 1302–1314 (2015).
 28
YW Tai, P Tan, MS Brown, RichardsonLucy deblurring for scenes under a projective motion path. IEEE Trans. Pattern Anal. Mach. Intell.33(8), 1603–1618 (2011).
 29
O Whyte, J Sivic, A Zisserman, J Ponce, Nonuniform deblurring for shaken images. Int. J. Comp. Vis.98(2), 168–186 (2012).
 30
A Gupta, N Joshi, CL Zitnick, M Cohen, B Curless, in ECCV, 6311. Single image deblurring using motion density functions (SpringerCrete, Greece, 2010), pp. 171–184.
 31
M Hirsch, S Sra, B Schölkopf, S Harmeling, in CVPR. Efficient filter flow for spacevariant multiframe blind deconvolution (IEEESan Francisco, USA, 2010), pp. 607–614.
 32
M Hirsch, CJ Schuler, S Harmeling, B Schölkopf, in ICCV. Fast removal of nonuniform camera shake (IEEEBarcelona, Spain, 2011), pp. 463–470.
 33
Z Hu, MH Yang, in BMVC. Fast nonuniform deblurring using constrained camera pose subspace (BMVAGuildford, UK, 2012).
 34
W Zuo, Z Lin, A generalized accelerated proximal gradient approach for totalvariationbased image restoration. IEEE Trans. Image Process.20(10), 2748–2759 (2011).
 35
R Köhler, M Hirsch, B Mohler, B Schölkopf, S Harmeling, in ECCV. Recording and playback of camera shake: benchmarking blind deconvolution with a realworld database (SpringerFirenze, Italy, 2012), pp. 27–40.
 36
H Deng, W Zuo, H Zhang, D Zhang, An additive convolution model for fast restoration of nonuniform blurred images. Int. J. Comp. Math.91(11), 2446–2466 (2014).
 37
I Daubechies, M Defrise, C De Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm. Pure. Appl. Math.57(11), 1413–1457 (2004).
 38
JM BioucasDias, MA Figueiredo, A new twist: twostep iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process.16(12), 2992–3004 (2007).
 39
A Beck, M Teboulle, Fast gradientbased algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process.18(11), 2419–2434 (2009).
 40
S Stančin, S Tomažič, Angle estimation of simultaneous orthogonal rotations from 3d gyroscope measurements. Sensors. 11(9), 8536–8549 (2011).
 41
R Matungka, YF Zheng, RL Ewing, Image registration using adaptive polar transform. IEEE Trans. Image Process.18(10), 2340–2354 (2009).
 42
J Yang, W Yin, Y Zhang, Y Wang, A fast algorithm for edgepreserving variational multichannel image restoration. SIAM J. Imag. Sci.2(2), 569–592 (2009).
 43
J Pan, Z Hu, Z Su, MH Yang, in CVPR. Deblurring text images via l0regularized intensity and gradient prior (IEEEColumbus, USA, 2014), pp. 2901–2908.
 44
S Harmeling, M Hirsch, B Schölkopf, in NIPS. Spacevariant singleimage blind deconvolution for removing camera shake (NIPSVancouver, Canada, 2010), pp. 829–837.
Acknowledgements
This work is supported in part by NSFC grants (61271093, 61471146, 61173086) and the program of ministry of education for new century excellent talents (NCET120150).
Author information
Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Deng, H., Ren, D., Zhang, D. et al. Efficient nonuniform deblurring based on generalized additive convolution model. EURASIP J. Adv. Signal Process. 2016, 22 (2016). https://doi.org/10.1186/s1363401603182
Received:
Accepted:
Published:
Keywords
 Camera shake
 Image deblurring
 Nonuniform deblurring
 Blind deconvolution
 Fast Fourier transform