 Research
 Open Access
 Published:
Impulsive noise rejection method for compressed measurement signal in compressed sensing
EURASIP Journal on Advances in Signal Processing volume 2012, Article number: 68 (2012)
Abstract
The Lorentzian norm of robust statistics is often applied in the reconstruction of the sparse signal from a compressed measurement signal in an impulsive noise environment. The optimization of the robust statistic function is iterative and usually requires complex parameter adjustments. In this article, the impulsive noise rejection for the compressed measurement signal with the design for image reconstruction is proposed. It is used as the preprocessing for any compressed sensing reconstruction given that the sparsified version of the signal is obtained by utilizing octavetree discrete wavelet transform with db8 as the mother wavelet. The presence of impulsive noise is detected from the energy distribution of the reconstructed sparse signal. After the noise removal, the noise corrupted coefficients are estimated. The proposed method requires neither complex optimization nor complex parameter adjustments. The performance of the proposed method was evaluated on 60 images. The experimental results indicated that the proposed method effectively rejected the impulsive noise. Furthermore, at the same impulsive noise corruption level, the reconstruction with the proposed method as the preprocessing required much lower measurement rate than the modelbased Lorentzian iterative hard thresholding.
1. Introduction
Compressed sensing (CS) is a sampling paradigm that provides compressible signals at a rate significantly below the Nyquist rate. It reveals that a compressible or sparse signal can be recovered by a small amount of measurements [1–3]. The connection between sampling and reconstruction methods of CS and those of other sparse signal processing is presented in [4]. The description of commonly used reconstruction algorithms is also given. Consider a measurement process in CS that is modeled as
where y and Φ are an M × 1 compressed measurement signal and an M × N random measurement matrix, respectively; x is an N × 1 compressible signal. In CS, it is considered that M < N. A signal is compressible if it is sparse in some domain; thus, x can be written as follows
where s and Ψ are a ksparse signal and an N × N orthogonal basis matrix, respectively. k is the number of nonzero elements or a sparsity level. Without loss of generality, Ψ is defined as an identity matrix in this article and x is equivalent to s.
In practice, y can be corrupted by noise during transmission in a noisy channel. The measurement process in the noisy channel is modeled as
where e is the additive noise.
CS reconstruction methods aim to find the sparsest s that creates y. The reconstruction of s in the noisy channel can be written as the following optimization problems.
where ε and u_{ p }are the error bound and the L_{ P } norm of u, respectively. The error bound is set based on the noise characteristics, such as bounded noise, Gaussian noise, finite variance noise, etc. [5–14]. L_{0} norm in Equation (4) is relaxed to L_{1} norm in the reconstruction by Basis Pursuit Denoising (BPDN), whereas it is replaced by heuristic rules in the reconstruction by greedy algorithms.
The optimization problems in BPDN [6] is
which is equivalent to
where τ is a regularization parameter.
When the noise is impulsive noise, e is considered to be very large. It is well known that the optimization of L_{2} norm is not robust to outliers in y; thus, the optimization leads to the incorrect result of s. In [15], the reconstruction from the signal corrupted by the impulsive noise is performed by solving one of the following two optimization problems.
where e_{ δ }and α are a sparse vector with large nonzero coefficients (impulsive noise) and a predefined threshold, respectively; u_{TV} is a total variation norm of u. This method first estimates s and then estimates e_{ δ }. The estimation is performed iteratively. However, the unique solution is guaranteed only when the cost function is convex. The effect of impulsive noise can be suppressed by applying robust statistics [16–22]. The Generalized Cauchy Distribution (GCD)based maximum likelihood has been proposed as the optimization algorithm that is robust to impulsive noise [16–22]. The Lorentzian norm, which is the special case of GCD, is utilized in a number of robust CS reconstructions [18–20, 22]. The Lorentzian norm is used in the place of L_{2} norm in Equation (5) for the Lorentzianbased Basis Pursuit (LBP) [18]. Similar to Basis Pursuit (BP), the LBP is slow to solve. Furthermore, it requires complex parameter adjustments for the effective optimization of the Lorentzian norm. The reconstruction in [19, 20] applies the iterative algorithm and the weighted myriad operator to solve the following problem.
where u_{ LL }, H and R are the Lorentzian norm of u, a Cauchy random projection signal and a Cauchy random projection matrix, respectively. The reconstruction in [21] applies the weighted median operator and the iterative thresholding to solve the following L_{0}regularized Least Absolute Deviation regression problem.
where y_{ i } and ${\phi}_{i}^{T}$ are the i th element of y and the i th row of Φ, respectively. The Lorentzianbased Iterative Hard Thresholding (LIHT) approach is proposed as the fast reconstruction method in [22]. Iterative Hard Thresholding (IHT) is used in the place of BP to increase the speed of LBP. However, it faces the same problem as IHT in that it requires high measurement rate in order to acquire successful reconstruction [13]. Consequently, LIHT is suitable for very sparse signals.
The noise tolerance can be increased by including prior knowledge. One of the popular knowledge is the model of a sparse signal [23–29], such as the wavelettree structure. Modelbased reconstruction methods have three benefits: (1) the reduction of the number of measurements, (2) the increase in robustness, and (3) the faster reconstruction.
Even though robust statistic provides the tolerance against impulsive noise, its optimization problem is often difficult. In this article, the impulsive noise rejection method for image data is proposed. It is used as the preprocessing to estimate the noisefree y. It iteratively applies the heuristic rule that is based on the energy distribution of the image data in wavelet domain to detect the existence of the impulsive noise. Octavetree discrete wavelet transform (DWT) is used to transform signals to sparse domain in this article. In an image, most energy should be contained in the thirdlevel subband. The existence of the impulsive noise leads to the high ratio of the energy outside the thirdlevel subband to the total energy. The rejection and the estimation of the noise corrupted elements are made possible by the following fact. In most images, the ksparse signal s can successfully be reconstructed even though some elements in y are removed, because the image data are redundant. The proposed rejection method requires only two parameters: the energyratio threshold and the rejectionratio threshold. These two parameters are easily adjusted and are evaluated for the optimal values as presented in the "Experimental" section.
The proposed method and the impulsive noise cancellation method in [30] are similar as they have two stages: the noise detection and the signal estimation stages. Both methods detect impulsive noise iteratively. However, they are different in a number of aspects. Only a few are mentioned here. The proposed method detects the impulsive noise via the energy distribution of the projected sparse signal. Its estimation stage is separated from its detection stage. The estimation is performed only once after the noise detection has been completed. On the other hand, the method in [30] detects the noise via the difference between the original noisy and the estimated signals; consequently, its estimation stage is integrated into the same loop as its detection stage. The estimation is performed iteratively.
The remainder of this article is organized as follows. Section 2 addresses a brief review of CS, the reconstruction by Orthogonal Matching Pursuit (OMP) and OMP with Partially Known Support (OMPPKS). Section 3 describes the proposed impulsive noise rejection method. The block processing and the vectorization are also given. In Section 4, the proposed method is evaluated. The conclusion is given in Section 5.
2. Background
2.1. Compressed sensing
CS is based on the assumption of the sparse property of signals and incoherency between the basis of the sparse domain and the basis of measurement vectors [1–3]. CS has three major steps: the construction of ksparse representation, the measurement, and the reconstruction. The first step is the construction of the ksparse representation, where k is the sparsity level of the sparse signal. Most natural signals can be made sparse by applying orthogonal transforms such as wavelet transform, Fast Fourier Transform, or Discrete Cosine Transform. This step is represented as Equation (2).
The random measurement matrix is applied to measure the signal by the following equation.
Since Ψ is an identity matrix in this article, s is equivalent to x. The sufficient condition for the high probability of successful reconstruction is as follows.
for some positive constant C. μ(Φ, Ψ) is the coherence between Φ and Ψ, and defined by
where φ_{ i } and ψ_{ j } are the i th and the j th column in Φ and Ψ, respectively. If the elements in Φ and Ψ are correlated, the coherence is large. Otherwise, it is small. From linear algebra, it is known that $\mu \left(\mathbf{\Phi},\mathbf{\Psi}\right)\phantom{\rule{0.3em}{0ex}}\in \phantom{\rule{2.77695pt}{0ex}}\left[1,\sqrt{N}\right]$[2]. In the measurement process, the error (due to hardware noise, transmission error, etc.) may occur. The error is added into the compressed measurement signal as described in Equation (3).
The final step is the reconstruction. There are two major reconstruction approaches: L_{1}minimization [5–8] and greedy algorithm [10–14, 31]. Convex optimization is applied in the reconstruction by L_{1}minimization approach. The successful reconstruction depends on the degree that Φ complies with the Restricted Isometry Property (RIP). RIP is defined as follows.
where δ_{ k } is the krestricted isometry constant of Φ. RIP is used to ensure that all the subsets of k columns taken from Φ are nearly orthogonal. It should be noted that Φ has more columns than rows; thus, Φ cannot exactly be orthogonal [2].
The reconstruction by L_{1}minimization as in BP is stable but slow. Greedy algorithms increase the reconstruction speed by applying heuristic rules. In OMP [31], the heuristic rule is created based on the assumption that y has the large correlation to the bases corresponding to the nonzero elements (or the elements with large magnitude) of s. OMP selects the bases of the nonzero elements according to the correlation and estimates the values of the nonzero elements by the least squared method. The selection is iterated until the certain condition is reached. The reconstruction by greedy algorithms has a fast runtime, but lacks stability and uniform guarantee. RIP is not seriously considered in the greedy algorithms [12].
2.2 Orthogonal matching pursuit
OMP is a wellknown reconstruction algorithm [31]. It was developed from matching pursuit [32] using different method to estimate the magnitude of the nonzero elements in s. Instead of projecting the residual signal onto the selected basis, it estimates the magnitude of the nonzero elements by solving the least squared error between the projection of the reconstructed s and y. OMP has the advantage of simple and fast implementations. The algorithm is as follows.
Input:

The M × N measurement matrix, $\mathbf{\Phi}=\left[\begin{array}{cccc}\hfill {\phi}_{1}\hfill & \hfill {\phi}_{\mathsf{\text{2}}}\hfill & \hfill ...\hfill & \hfill {\phi}_{N}\hfill \end{array}\right]$

The Mdimension compressed measurement signal, y

The sparsity level of the sparse signal, k
Output:

The reconstructed signal, ŝ

The set containing k indexes of nonzero elements in ŝ, Λ_{ k }= {λ_{1}, λ_{2}, ..., λ_{ k }}
Procedure:

(a)
Initialize the residual (r _{0}), the index set (Λ_{0}) and the iteration counter (t) as follows.
$${\mathbf{r}}_{0}=\mathbf{y},\phantom{\rule{0.3em}{0ex}}{\mathrm{\Lambda}}_{0}=\varnothing ,\phantom{\rule{0.3em}{0ex}}t=1$$ 
(b)
Find the index λ_{ t } of the measurement basis that has the highest correlation to the residual in the previous iteration, r _{t1}.
$${\lambda}_{t}=arg\underset{j=1,...,N}{max}\left\u3008{\mathbf{r}}_{t1},{\phi}_{j}\u3009\right\phantom{\rule{0.3em}{0ex}}$$
If the maximum occurs in multiple bases, select one deterministically.

(c)
Augment the index set and the matrix of chosen bases: Λ_{ t }= Λ_{t1}∪{λ_{ t } } and ${\mathbf{\Phi}}_{t}=\left[{\mathbf{\Phi}}_{t1}\phantom{\rule{1em}{0ex}}{\phi}_{{\lambda}_{t}}\right]\phantom{\rule{0.3em}{0ex}}$, where Φ _{0} is an empty matrix.

(d)
Solve the following least squared problem to obtain the new reconstructed signal, z _{ t }.
$${\mathbf{z}}_{t}=arg\phantom{\rule{0.2em}{0ex}}\underset{z}{min}{\u2225\mathbf{y}{\mathbf{\Phi}}_{t}\mathbf{z}\u2225}_{2}$$ 
(e)
Calculate the new approximation, a _{ t }, that best describes y. Then, calculate the residual of the tth iteration, r _{ t }.
$$\begin{array}{c}{\mathbf{a}}_{t}={\mathbf{\Phi}}_{t}{\mathbf{z}}_{t}\\ {\mathbf{r}}_{t}=\mathbf{y}{\mathbf{a}}_{t}\end{array}$$ 
(f)
Increment t by one.

(g)
If t > k, terminate; otherwise, go to step (b).
The reconstructed signal, ŝ, has nonzero elements at the indexes listed in Λ_{ k }. The value of the λ_{ j } th elements in ŝ equals to the j th element of z_{ k }(j = 1,2,...,k). The termination criterion can be changed from t > k to that r_{t1}is less than the predefined threshold.
2.3. OMP with partially known support
OMPPKS [28] is adapted from the classical OMP [31]. The partially known support gives a priori information to determine which subbands in the sparse signal structure are more important than the others and should be selected as nonzero elements. It has the characteristic of OMP that the requirement of RIP is not as severe as BP's [6]. It has a fast implementation but may fail to reconstruct the signal (lacks stability). It requires very low measurement rate. It is different from Treebased OMP (TOMP) [24] in that the subsequent basis selection of OMPPKS does not consider the previously selected bases, while TOMP sequentially compares and selects the next good wavelet subtree and the group of related atoms in the wavelet tree.
The wavelet transform of an image is realized using filter banks as shown in Figure 1. The image is decomposed into four subbands: HH, HL, LH, and LL. These four subbands contain diagonal details, vertical details, horizontal details, and approximation coefficients, respectively. In this article, octavetree DWT is used to obtain the sparse representation of images. The second and the thirdlevel subbands are constructed by applying the filter bank analysis to the LL subband in the first and the second levels, respectively. The example of octavetree DWT is shown in Figure 2. The original and the wavelet transformed images are shown in Figure 2a, b, respectively. Since the LL subband in the third level (LL_{3} subband) contains most information in the image, the signal in the LL_{3} subband must be included for successful reconstruction. All elements in the LL_{3} subband are selected as nonzero elements without testing for the correlation. The algorithm for OMPPKS when the data are represented in wavelet domain is as follows.
Input:

The M × N measurement matrix, $\mathbf{\Phi}=\left[\begin{array}{cccc}\hfill {\phi}_{1}\hfill & \hfill {\phi}_{\mathsf{\text{2}}}\hfill & \hfill ...\hfill & \hfill {\phi}_{N}\hfill \end{array}\right]$

The Mdimension compressed measurement signal, y

The set containing the indexes of the bases in LL_{3} subbands, Γ = {γ_{1}, γ_{2}, ..., γ_{Γ}}

The sparsity level of the sparse signal, k
Output:

The reconstructed signal, ŝ

The set containing k indexes of the nonzero element in ŝ, Λ_{ k }= {λ_{1}, λ_{2}, ..., λ_{ k }}
Procedure:
Phase 1: Selection without correlation test

(a)
Select every basis in the LL_{3} subband.
$$t=\left\mathrm{\Gamma}\right$$$${\mathrm{\Lambda}}_{t}=\mathrm{\Gamma}$$$${\mathbf{\Phi}}_{t}=\left[\begin{array}{cccc}\hfill {\phi}_{{\gamma}_{1}}\hfill & \hfill {\phi}_{{\gamma}_{2}}\hfill & \hfill ...\hfill & \hfill {\phi}_{{\gamma}_{t}}\hfill \end{array}\right]$$ 
(b)
Solve the least squared problem to obtain the new reconstructed signal, z _{ t }.
$${\mathbf{z}}_{t}=arg\phantom{\rule{0.2em}{0ex}}\underset{z}{min}{\u2225\mathbf{y}{\mathbf{\Phi}}_{t}\mathbf{z}\u2225}_{2}$$ 
(c)
Calculate the new approximation, a _{ t }, and find the residual (error, r _{ t }). a _{ t }is the projection of y on the space spanned by Φ _{ t }.
$$\begin{array}{c}{\mathbf{a}}_{t}={\mathbf{\Phi}}_{t}{\mathbf{z}}_{t}\\ {\mathbf{r}}_{t}=\mathbf{y}{\mathbf{a}}_{t}\end{array}$$
Phase 2: Reconstruction by OMP

(a)
Increment t by one, and terminate if t > k.

(b)
Apply steps (b)(g) of OMP described in Section 2.2 to find the remaining kΓ nonzero elements of ŝ.
The reconstructed sparse signal, ŝ, has the indexes of nonzero elements listed in Λ_{ k }. The value of the λ_{ j } th element of ŝ equals to the j th element of z_{ k }.
3. Proposed method
The proposed impulsive noise rejection method is described in this section. Block processing and the vectorization of the wavelet coefficients are addressed before a description of the noise rejection method. The block processing is applied to reduce the computation cost. The proposed noise rejection method is applied before the reconstruction and divided into two stages. In the first stage, the algorithm to detect impulsive noise is applied. Then, OMPPKS is applied to estimate the information that is lost due to the impulsive noise. The algorithm to detect the impulsive noise and the estimation of the missing information are described in Sections 3.2 and 3.3, respectively.
3.1. Block processing and the vectorization of the wavelet coefficients
In this article, the DWT is used to obtain the sparsified version of an image. Figure 3 shows an example of block processing and the vectorization of the wavelet coefficients. Figure 3a shows the structure of a wavelet transformed image. The LL_{3} subband is presented in red. Other subbands (LH, HL, and HH) in the third, the second, and the first levels are presented in green, orange, and blue, respectively. The LL_{3} subband is the most important subband, because it contains most of the energy in the image. Figure 3b shows the reordering of the wavelet coefficients. The coefficients are ordered such that the LL_{3} subband is located at the beginning of each row. The LL_{3} subband is followed by the other subbands in the third, the second, and the first levels.
The waveletdomain image in Figure 3b is divided into blocks along its rows as shown in Figure 3c. In Figure 3c, the image has eight rows; consequently, it is divided into eight blocks. Each row in Figure 3c is considered as a sparse signal in this article.
The signal can be made more sparse by the wavelet shrinkage thresholding [33]. In the wavelet shrinkage thresholding, all the coefficients in the LL_{3} subband are preserved, while coefficients outside the LL_{3} subband with magnitude less than the wavelet shrinkage threshold are set to zero. Note that not all coefficients outside the LL_{3} subband are set to zero. Since only the small coefficients in highfrequency subband are set to zero, most distinct edges in the image are preserved. The sparsifying transformation by the wavelet shrinkage thresholding has little distinct visual degradation if the wavelet shrinkage threshold is selected properly.
In the experiments, it is found that the vectorization according to the structure of Figure 3c is better than the one by the lexicographic order of Figure 3a. Figure 4 shows some reconstruction examples when these two vectorization methods were used. The sparsity rate (k/N) and the measurement rate (M/N) were set to 0.1 and 0.3, respectively. All images were reconstructed by OMPPKS. The top row of each image shows the reconstruction when the vectorization in each block was done such that it had the structure as shown in Figure 3c. The bottom row of each image shows the reconstruction when the vectorization in each block was done by the lexicographic order of the structure shown in Figure 3a. There was no fail reconstruction (dark spot) in the top row, whereas there were some in the bottom row.
3.2. The detection of the impulsive noise
Figures 5 and 6 show the examples of the reconstruction from y corrupted by impulsive noise. Figures 5ac and 6ac show the original blue y corrupted by the red impulsive noise, the original s, and the reconstructed ŝ from Figures 5a and 6a, respectively. The figures clearly indicate that the energy distribution was different. The energy of the signals in Figures 5c and 6c was spread out, while most energy of the signals in Figures 5b and 6b was contained in the thirdlevel subbands.
Even though there is no definite structure of y, Figures 5 and 6 indicate that the energy distribution of s can be exploited to detect the existence of impulsive noise. The large impulsive noise leads to a bad approximation of ŝ whose energy leaks out of the thirdlevel subband. The ratio of the energy outside the thirdlevel subband to the total energy is used to determine the existence of the impulsive noise in y. The high ratio indicates that the energy is spread out; thus, the existence of the impulsive noise. The impulsive noise has very large magnitude in comparison to y. Consequently, if the impulsive noise exists, it has the largest magnitude. The removal of the impulsive noise is simply the removal of the elements with the largest magnitude. The size of the impulsive noise may vary, so the removal is performed iteratively until either of the following two stopping criteria is satisfied.

(1)
The reconstructed ŝ has most of its energy inside the thirdlevel subband.

(2)
The reconstruction is unlikely to be successful because too many elements in y have been removed.
According to the stopping criteria, there are two thresholds that need to be defined. The threshold in the first criterion is used to indicate the amount of the energy that is allowed to be leaked out of the thirdlevel subband. The amount of the energy is measured as the ratio to the total energy. The threshold is defined as the energyratio threshold, η. The threshold in the second criterion is required to ensure that there is sufficient information left for the reconstruction. This threshold is called rejectionratio threshold, T, which is defined as the ratio between the numbers of the removed elements to the size of y(M). Thus, the maximum number of the elements that can be removed is TM. The optimum values of η and T are investigated in Section 4.2.
At each iteration, the noisecorrupted elements are removed and the size of the available measurement signal becomes smaller. Hence, it is required that the reconstruction algorithm is still effective at low measurement rate. OMPPKS is adopted by including the algorithm for the detection and the removal of impulsive noise as follows.
Input:

The M × N measurement matrix, $\mathbf{\Phi}=\left[\begin{array}{cccc}\hfill {\phi}_{1}\hfill & \hfill {\phi}_{\mathsf{\text{2}}}\hfill & \hfill ...\hfill & \hfill {\phi}_{N}\hfill \end{array}\right]$

The Mdimension compressed measurement signal, y

The sparsity level of the sparse signal, k

The number of wavelet coefficients in the thirdlevel subband, l_{3}

The energyratio threshold, η

The rejectionratio threshold, T
Output:

The number of impulsive noise corrupted elements, n_{ δ }

The set containing the n_{ δ }indexes of the impulsive noisecorrupted elements, ${\varsigma}_{\delta}=\left\{{\varpi}_{1},\phantom{\rule{2.77695pt}{0ex}}\phantom{\rule{0.3em}{0ex}}{\varpi}_{2},\phantom{\rule{0.3em}{0ex}}\phantom{\rule{2.77695pt}{0ex}}...,\phantom{\rule{2.77695pt}{0ex}}{\varpi}_{{n}_{\delta}}\right\}$
Procedure:

(a)
Initialize t = 0, n_{ δ } = 0, ς_{ δ } = ∅, y _{ t }= y, Φ_{ t }= Φ.

(b)
Apply OMPPKS to reconstruct ŝ from y _{ t }and Φ _{ t }.

(c)
Calculate the energyratio (ER).
$$\mathsf{\text{ER}}=\frac{\sum _{i={l}_{3}+1}^{N}{{\mathit{\u015d}}_{i}}^{2}}{\sum _{j=1}^{N}{{\mathit{\u015d}}_{j}}^{2}},$$
where ŝ_{ i } is the i th element of ŝ.

(d)
Terminate if ER < η.

(e)
Assign the elements in y _{ t }having the maximum magnitude as the impulsive noise. α_{ m } ($m=1,2,\dots ,{n}_{{\delta}_{t}};\phantom{\rule{2.77695pt}{0ex}}{n}_{{\delta}_{t}}$is the number of the elements having the maximum magnitude in y _{ t }) are defined as the indexes of the recently assigned impulsive noise elements. Note that α_{ m } are the indexes of y. In case that there are more than one element having the maximum magnitude (${n}_{{\delta}_{t}}>1$), all of them are to be removed in step (i).

(f)
Increment n_{ δ } by ${n}_{{\delta}_{t}}$ and add α_{ m } to ς_{ δ } .

(g)
Terminate if n_{ δ } ≥ TM.

(h)
Set t = t+1.

(i)
y _{ t }is assigned the value of y after the noise elements (the elements with the indexes in ς_{ δ } ) are removed from y. Φ _{ t }is assigned the value of Φ after the rows corresponding to the noise elements are removed from Φ.

(j)
Go to step (b).
If the algorithm is terminated in step (g), then the removal of impulsive noise is unsuccessful. Too many elements have been removed and it is unlikely that there is sufficient information to reconstruct ŝ and estimate the missing information in the next stage.
It should be noted that the proposed algorithm is applicable to images because image data have some degree of redundancy. The rejectionratio threshold, T, can be set quite large. For the signal data that have low degree of redundancy, the value of T has to be very small. In this case, the reconstruction is unlikely to succeed if every information in y is not used.
3.3. Estimation of the missing information
The outputs from the detection stage and y are used as the inputs of this stage. The noisecorrupted elements, specified in ς_{ δ } , are removed. After the noise removal, the size of the compressed measurement signal y is smaller than the size of the original y; consequently, the reconstruction methods requiring high measurement rate may fail to reconstruct ŝ. It is necessary to estimate the values of the removed elements to preserve the measurement rate. In the proposed method, the values are estimated such that they comply with other noiseless elements. The estimation algorithm is as follows.
Input:

The Mdimension compressed measurement signal, y

The number of impulsive noisecorrupted elements, n_{ δ }

The set containing the n_{ δ }indexes of the impulsive noisecorrupted elements, ${\varsigma}_{\delta}=\left\{{\varpi}_{1},\phantom{\rule{2.77695pt}{0ex}}\phantom{\rule{0.3em}{0ex}}{\varpi}_{2},\phantom{\rule{0.3em}{0ex}}\phantom{\rule{2.77695pt}{0ex}}...,\phantom{\rule{2.77695pt}{0ex}}{\varpi}_{{n}_{\delta}}\right\}$
Output:

The estimated noisefree y, ŷ
Procedure:

(a)
Define y_{ s } as y with its ϖ_{ i } th (i = 1, 2, ..., n_{ δ } ) elements removed. Define Φ_{ s } as Φ with its ϖ_{ i } th (i = 1, 2, ..., n_{ δ } ) rows removed.

(b)
Apply OMPPKS to reconstruct ŝ _{s} from y_{ s } and Φ _{ s }.

(c)
Define ỹ = Φŝ _{s} and estimate the i th elements in ŷ as follows.
$${\mathit{\u0177}}_{i}=\left\{\begin{array}{cc}\hfill {y}_{i}\hfill & \hfill ;i\notin {\varsigma}_{\delta}\hfill \\ \hfill {\mathit{\u1ef9}}_{i}\hfill & \hfill ;i\in {\varsigma}_{\delta}\hfill \end{array}\right.,$$
where the subscript i indicates the i th elements of the signal and i = 1, 2, ..., M.
After this process, the impulsive noisecorrupted elements in y are replaced by values complying with noisefree elements. Conventional CS reconstruction methods can be applied to reconstruct ŝ from the impulsive noisefree ŷ.
4. Experiment and discussion
4.1. Experimental environment
The experiment was conducted on a PC with 2.83GHz Intel Core 2 Quad CPU and 4GB RAM. All methods were implemented by 64bit MATLAB R2011a. The proposed method was tested on 60 images. All the test images were resized to 256 × 256. Figure 7 shows the test images that consist of 10 standard test images, 12 artificial images, and 38 natural images. The artificial and the natural images are available at http://sourceforge.net/projects/testimages/files/.
Octavetree DWT was used to transform test images to sparse domain. The mother wavelet used in the implementation was Daubechies 8 (db8). The wavelet shrinkage thresholding [33] was applied to make the signal more sparse. The probability of impulsive noise is denoted as p; p ∈ {0, 0.05, 0.10, 0.15, 0.20}. The magnitude of impulsive noise was set relative to the maximum magnitude in y(y_{max}). The measurement matrix was Hadamard matrix. Each wavelet image was divided into 256 blocks of 1 × 256. The sparsity rates (k/N) of blocks in an image were intentionally varied to demonstrate that one set of thresholds was applicable for various sparsity rates. The average sparsity rate in each test image was set to 0.1. The measurement rate (M/N) of an image was the rate averaged over every block in the image. The average measurement rates used in the experiment were 0.2, 0.3, 0.4, 0.5, and 0.6.
The experiment consists of two parts: (1) the evaluation of the two thresholds (η and T) and the minimum size of the detectable impulsive noise given in Section 4.2 and (2) the performance evaluation of the proposed method given in Section 4.3.
4.2. Evaluation of the two thresholds and the minimum size of the detectable impulsive noise
In this section, 500 blocks were randomly selected from blocks in 60 test images. The sparsity rate was fixed at 0.1. Table 1 lists the percentage that the proposed method was unable to correctly reject the impulsive noisecorrupted elements. The threshold value η and the magnitude of impulsive noise were varied. The value in the table was the value averaged over five values of p and five values of measurement rates. The values in the table indicated that the proposed method was unable to keep the percentage of inaccurate rejection to less than 1% if the magnitude of the impulsive noise was less than 2.5 y_{max}.
Table 1 also indicated the relationship of η to the percentage of inaccurate rejection. The inaccurate rejection was the result of (1) the rejection of the noisefree elements and (2) the failure to reject the noisecorrupted elements. When η was too small, the energyratio criterion was too strict and the proposed method did not accept even the correct energy distribution of ŝ; consequently, it started to remove the elements uncorrupted by noise. In the opposite case, when η was too large, the energyratio criterion became too lax and the proposed method accepted even the incorrect energy distribution of ŝ; consequently, it failed to remove the noisecorrupted elements. The range of η giving less than 1% of inaccurate rejection was larger, when the magnitude of the impulsive noise was larger. This was because the effect of the impulsive noise to the energy distribution became more distinct and easier to detect when the size of the noise was larger. When the magnitude of the impulsive noise was at least 2.5 y_{max}, the values of η giving less than 1% inaccurate rejection were 0.03, 0.04, and 0.05. Among the three values, the values of η = 0.05 gave the most accurate rejection.
The evaluation for the optimum rejectionratio threshold, T, was performed by investigating for the maximum number of the elements in y that can be removed without causing the high error between ŝ and s. Figure 8 shows the MSE of the signals reconstructed by OMPPKS when TM elements in y were removed. Different measurement rates were presented with different colors. The figure indicated that when the measurement rate increased, more elements could be removed without causing a drastic change in MSE. At the measurement rate of 0.2, MSE approximately increased at the exponential rate, when T was larger or equal to 0.45. At the higher measurement rates, the effect of T was not distinct, even when more than half of y was removed.
Because the benefit of CS is the capability of compressing the signal to very small size, the measurement rate should be kept low. It is recommended that T be selected such that it is applicable even at low measurement rate. In the following section, T was set to 0.4 to ensure the high probability of successful reconstruction. The value of η was set to 0.05 as it was the optimal value (Table 1).
4.3. Performance evaluation
In this section, the following four reconstruction methods were investigated.

(1)
OMPPKS

(2)
OMPPKS with the proposed rejection method as the preprocessing (OMPPKS+R)

(3)
Modelbased LIHT (MLIHT) which is the LIHT that is forced to consider the elements in LL_{3} subband as nonzero elements.

(4)
MLIHT with the proposed rejection method as the preprocessing (MLIHT+R)
The Lorentzian parameter and the number of iteration for MLIHT were 0.25 and 100, respectively. The values of η and T were 0.05 and 0.4, respectively. There were 256 y's in an image and y_{max} was chosen as the maximum magnitude among 256 y's in the image. The magnitude of impulsive noise varied according to the Gaussian pdf with the mean of 7 y_{max} and the standard deviation of y_{max}.
The performance is evaluated based on the PSNR of the reconstructed images, the computation time and the visual quality of the reconstructed images.
Figure 9 shows the experimental results of the standard test images. Figure 9ae shows the PSNR (the left column) and the computation time (the right column) at different p (noise probability). At p = 0 (noiseless), the addition of the proposed method to OMPPKS and MLIHT did not reduce the PSNR of the reconstructed images. It indicated that the proposed method preserved y when there was no impulsive noise. When y was corrupted by impulsive noise (p > 0), the reconstruction based on OMPPKS (the blue line) gave very low PSNR, because OMPPKS is designed with the assumption of bounded noise. The reconstruction based on OMPPKS could not be improved by increasing the measurement rate. However, when the noisy y was preprocessed by the proposed method, the reconstruction based on OMPPKS (the dashed blue line) was very effective. At the measurement rate of 0.4 and higher, the reconstruction from the noisy y by OMPPKS+R had the comparable PSNR to the reconstruction from the noiseless y by OMPPKS.
At p = 0.05, the effect of adding the proposed method as the preprocessing to MLIHT was minimal; however, at higher p, the addition of the proposed method (the dashed red line) resulted in higher PSNR than the reconstruction by MLIHT alone (the red line). When p was 0.15 or higher, MLIHT was no longer an effective reconstruction method, but MLIHT+R was still effective. It indicated that the addition of the proposed method increased the robustness against p to MLIHT.
It should be noted that even though MLIHT was based on LIHT which was designed to be robust against impulsive noise. MLIHT+R provided less PSNR than OMPPKS+R, because MLIHT required the higher measurement rate. Figure 9 indicated that MLIHT+R was as effective as OMPPKS when the measurement rate was 0.6 and it should become better at the higher measurement rate. However, the improvement by increasing the measurement rate is not recommended because it leads to the large size of y and eliminates the benefit of CS.
Figure 9 also indicates the relationship between measurement rate and p (noise probability). When p was higher, the measurement rate should be set higher. This was because the number of the noisecorrupted elements was larger at higher p. Consequently, the larger size of y was required to cope with the removal of more elements. The figure shows that in OMPPKS+R, the measurement rate of 0.4 gave the good reconstruction for all p in this experiment. The right column of Figure 9 shows the computation time of OMPPKS, OMPPKS+R, MLIHT, and MLIHT+R. Since at least one reconstruction is required in the proposed method, the computation time will be at least doubled. The computation time for reconstructing 256 blocks in an image could be reduced as follows.

(a)
Apply the proposed rejection method to the first block. Define β as the smallest magnitude of the noise corrupted elements in the first block.

(b)
Move to the next block. Define the compressed measurement of the new block as y _{ curr }

(c)
Assign the elements in y _{ curr }having the magnitude not less than β as the impulsive noise. Initialize variables in step (a) of Section 3.2 such that they reflect the removal of the elements with the magnitude not less than β.

(d)
Apply the proposed rejection method to y _{ curr }. If β is larger than the smallest magnitude of the noise corrupted elements in y _{ curr }, set β to this value.

(e)
If the current block is the last block in the image, terminate. Otherwise, go to step (b).
The assumption of the above algorithm is that the magnitude of impulsive noise in every block is approximately the same (or share the same distribution). The graphs indicated that the computation time of the reconstruction with the proposed rejection method was no more than four times the computation time of the reconstruction without the proposed rejection method.
Figures 10 and 11 show the results from the artificial and the natural images, respectively. The trends of the PSNR and the computation time were similar to Figure 9. From the three figures, it could be concluded that the proposed method should be included in the reconstruction from the impulsive noise corrupted y. The addition of the proposed method increased the computation time no more than four times the original computation time. Finally, OMPPKS+R was more optimal than MLIHT+R.
Figures 12, 13, 14, 15, 16, and 17 show the examples of the reconstruction results when the measurement rate was 0.5. The original image is shown in the first column. The reconstruction results based on MLIHT, MLIHT+R, OMPPKS, and OMPPKS+R are shown in the second, the third, the fourth, and the fifth columns, respectively. When the impulsive noise was added to y, the reconstruction based on OMPPKS failed in every case. The reconstruction based on MLIHT failed in some cases at p = 0.1, and failed in every case at p ≥ 0.15. The addition of the proposed algorithm to OMPPKS and MLIHT, namely, OMPPKS+R and MLIHT+R, led to the successful reconstruction in every case. Furthermore, the reconstruction based on OMPPKS+R provided the reconstruction results that were more similar to the original images than the ones based on MLIHT+R. These results complied with the conclusion that was drawn from the PSNR graphs in Figures 9, 10, and 11.
It is possible that more than one kind of noise exist in the system. The proposed method was applied to the reconstruction from y corrupted by both Gaussian and impulsive noises. The examples of the reconstruction results are shown in Figure 18. The Gaussian noise was applied such that the SNR of y was 20 dB. The noise probability (p) and the measurement rate were set to 0.1 and 0.5, respectively. The first column shows the reconstruction results when the impulsive noise was correctly removed. The second column shows the reconstruction results when OMPPKS+R was applied to the reconstruction from y corrupted by both Gaussian and impulsive noises. In order to cope with the higher error from the Gaussian noise, more energy was allowed outside the thirdlevel subband and more data were required for the reconstruction. The values of η and T were set to 0.1 and 0.3, respectively. The images in the first and the second columns were quite similar. The artifacts in the reconstruction based on OMPPKS+R were mostly the result of the Gaussian noise. Figure 18 demonstrated the prospect of using the proposed method to remove the impulsive noise in the environment corrupted by more than one type of noise. However, further test for y corrupted by more than one type of noise is necessary and is the part of our future research.
5. Conclusion
The impulsive noise rejection for CS reconstruction of image data is proposed. The sparsified version of an image is obtained by applying octavetree DWT using db8 as the mother wavelet. The structure of energy distribution in wavelet domain and the capability to reconstruct the signal from an incomplete y are exploited in order to detect the presence of the impulsive noise. After the noisecorrupted elements are removed, the values of the removed elements are estimated. The experimental results of 60 test images indicated that the proposed rejection method improved the robustness against the impulsive noise of the conventional CS reconstruction methods. The robustness of the reconstruction method against both Gaussian and impulsive noises was also investigated.
References
 1.
Donoho DL: Compressive sensing. IEEE Trans Inf Theory 2006, 52(4):12891306.
 2.
Candes EJ, Romberg J: Sparsity and incoherence in compressive sampling. Inverse Problem 2007, 23(3):969985. 10.1088/02665611/23/3/008
 3.
Candes EJ, Wakin MB: An introduction to compressive sampling. IEEE Signal Process Mag 2008, 25(2):2130.
 4.
Marvasti F, Amini A, Haddadi F, Soltanolkotabi M, Khalaj BH, Aldroubi A, Sanei S, Chambers J: A unified approach to sparse signal processing. EURASIP J Adv Signal Process 2012. doi:10.1186/PREACCEPT1686979482577015
 5.
Tropp JA: Just relax: convex programming methods for identifying sparse signals in noise. IEEE Trans Inf Theory 2006, 52(3):10301051.
 6.
Candes EJ, Romberg J, Tao T: Stable signal recovery from incomplete and inaccurate measurements. Commun Pure Appl Math 2006, 59(8):12071223. 10.1002/cpa.20124
 7.
Needell D, Vershynin R: Uniform uncertaintity principle and signal reconstruction via regularized orthogonal matching pursuit. Found Comput Math 2008, 9(3):317334.
 8.
Omidiran D, Wainwright MJ: Highdimensional subset recovery in noise: sparsified measurements without loss of statistical efficiency. Department of Statistics, UC Berkeley, USA; 2008.
 9.
Candès EJ, Tao T: The dantzig selector: Statistical estimation when p is much larger than n . Ann Statist 2007, 35(6):23132351. 10.1214/009053606000001523
 10.
Needell D, Tropp JA: CoSaMP: iterative signal recovery from incomplete and inaccurate samples. Appl Comput Harmonic Anal 2008, 26(3):301321.
 11.
BenHaim Z, Eldar YC, Elad M: Coherencebased nearoracle performance guarantees for sparse estimation under Gaussian noise. In Proc ICASSP. Texas, USA; 2010:3590.
 12.
Needell D: Topics in compressed sensing. In Ph.D. dissertation. Math. Univ. of California, Davis; 2009.
 13.
Blumensath T, Davies ME: Iterative hard thresholding for compressed sensing. Appl Comput Harmonic Anal 2009, 27(3):265274. 10.1016/j.acha.2009.04.002
 14.
Blumensath T, Davies ME: Normalized iterative hard thresholding: guaranteed stability and performance. IEEE J Sel Topics Signal Process 2010, 4(2):298309.
 15.
Popilka B, Setzer S, Steidl G: Signal recovery from incomplete mesurements in the presence of outliers. Inverse Problems Imag 2007, 1(4):661672.
 16.
Carrillo RE, Aysal TC, Barner KE: A theoretical framework for problem requiring robust behavior. In Proc IEEE CAMSAP. Aruba, Dutch Antilles; 2009:25.
 17.
Carrillo RE, Aysal TC, Barner KE: A generalized cauchy distribution framework for problems requiring robust behavior. EURASIP J Adv Signal Process 2010, 2010: 19. Article ID 312989
 18.
Carrillo RE, Barner KE, Aysal TC: Robust sampling and reconstruction methods for sparse signals in the presence of impulsive noise. IEEE J Sel Topics Signal Process 2010, 4(2):392408.
 19.
Arce GR, Otero D, Ramirez AB, Paredes J: Reconstruction of sparse signal from l_{1}dimensionality reduced cauchy randomprojections. In Proc IEEE ICASSP. Texas, USA; 2010:4014.
 20.
Ramirez AB, Arce GR, Sadler BM: Fast algorithm for reconstruction of sparse signal from cauchy random projections. In Proc EUSIPCO. Aalborg, Denmark; 2010:432.
 21.
Paredes JL, Arce GR: Compressive sensing signal reconstruction by weighted median regression estimates. IEEE Trans Signal Proces 2011, 59(6):25852601.
 22.
Carrillo RE, Barner KE: Lorentzian based iterative hard thresholding for compressed sensing. In Proc IEEE ICASSP. Prague, Czech Republic; 2011:3664.
 23.
La C, Do MN: Signal reconstruction using sparse tree representation. In Proc of SPIE Conf on Wavelet App in Sig and Image Proc. Volume 5914. San Diego, USA; 2005:273.
 24.
La C, Do MN: Treebased orthogonal matching pursuit algorithm for signal reconstruction. In Proc IEEE ICIP. Georgia, USA; 2006:1277.
 25.
Duarte MF: Compressed sensing for signal ensembles. In Ph.D. dissertation. Dept. Elect. Eng., Rice Univ., Houston, Texas; 2009.
 26.
He L, Carin L: Exploiting structure in waveletbased Baysian compressive sensing. IEEE Trans Signal Process 2009, 57: 34883497.
 27.
Baron D, Wakin MB, Duarte MF, Sarvotham S, Baraniuk RG: Distributed compressed sensing. Rice Univ., Dept. Elect. and Com. Eng., Houston, TX; 2006.
 28.
Carrillo RE, Polania LF, Barner KE: Iterative algorithm for compressed sensing with partially known support. In Proc IEEE ICASSP. Texas, USA; 2010:3654.
 29.
Xu M, Lu J: Kclustervalues compressive sensing for imaging. Eurasip J Adv Signal Process 2011, 2011: 75. 10.1186/16876180201175
 30.
Zahedpour S, Feizi S, Amini A, Ferdosizadeh M, Marvasti F: Impulsive noise cancellation based on soft decision and recursion. IEEE Trans Instrum Meas 2009, 58(8):27802790.
 31.
Tropp JA, Gilbert AC: Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans Inf Theory 2007, 53(12):46554666.
 32.
Mallat SG, Zhang Z: Matching pursuits with timefrequency dictionaries. IEEE Trans Signal Process 1993, 41(12):33973415. 10.1109/78.258082
 33.
Donoho DL: Denoising by soft thresholding. IEEE Trans Inf Theory 1995, 38(2):613627.
Acknowledgements
The authors would like to thank the reviewers for their valuable comments and suggestions. This research was financially supported by the National Telecommunications Commission Fund (Grant No. PHD/006/2551 to P. Sermwuthisarn and S. Auethavekiat) and the Telecommunications Research Industrial and Development Institute (TRIDI).
Author information
Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Sermwuthisarn, P., Gansawat, D., Patanavijit, V. et al. Impulsive noise rejection method for compressed measurement signal in compressed sensing. EURASIP J. Adv. Signal Process. 2012, 68 (2012). https://doi.org/10.1186/16876180201268
Received:
Accepted:
Published:
Keywords
 compressed sensing (CS)
 modelbased method
 impulsive noise
 orthogonal matching pursuit with partially known support (OMPPKS)