Cosparsity-based Stagewise Matching Pursuit algorithm for reconstruction of the cosparse signals

3.1 Algorithm description

To address the above issue, we propose a novel greedy algorithm for blind cosparse reconstruction, where the cosupports are refined iteratively and the information on cosparsity is extracted automatically. The proposed CSMP algorithm, as shown in Algorithm 1, is composed of two processes, namely, the forward and backward processes. The forward process estimates the cosparsity, constructs the cosupport starting from a cosupport with all rows in Ω, and updates the measurement residual simultaneously. The procedure ends with a backward process that tries to add the rows in Ω with a smaller correlation until the terminating condition is reached. The terminating condition for CSMP is controlled by a threshold, which ensures that the estimated cosparsity is fairly close to the actual one and the target signal has been well reconstructed. The main steps of CSMP are summarized in Algorithm 1.

In Algorithm 1, cosupp(x) = {i: Ω _i x = 0}, cosupp(x, l _est) returns the index set of l _est smallest (in absolute value) elements in Ω x, 2l _est − p is the size of |Γ| and for a vector α, function Min(α, 2l _est − p) returns the 2l _est − p indices corresponding to the 2l _est − p smallest values of α, index(Ω x, q) returns q elements from the (l _est + 1)-th to (l _est + q)-th smallest (in absolute value) rows in Ω x, and Γ, Λ ^Δ , \( {\tilde{\varLambda}}^k \) and Λ ^k are subsets of {1, 2, ⋯, p}, and ⌈s/q⌉ returns the smallest integer which is larger than s / q.

The CSMP algorithm adopts a stagewise approach [6] to estimate the real cosparsity in each stage in the forward process, which only requires the step size s to be set in initialization. Here, l is defined as the real cosparsity of the original signal, and s should not be larger than d − l normally [11]. The initial cosupport Λ ⁰ has the maximum number of rows in Ω that enables CSMP to recover the signal. To avoid under-estimation of the cosparsity, a safe choice is s = 1 if l is unknown. Nevertheless, there is a tradeoff between s and the recovery speed as a smaller s requires more stages. We can see that GAP and ASP/ACoSaMP are the special cases of the proposed algorithm when CSMP has a step size of s = 1 and s = d − l, respectively.

Suppose CSMP has a cosparsity of l _est in the forward process. With this cosparsity, the candidate set is constructed by selecting some rows in Ω with smaller correlations. Here, we should explain the relation between greedy synthesis algorithms and their analysis counterparts. Given two vectors v ₁, v ₂ ∈ ℜ ⁿ such that Λ ₁ = cosupp(Ω v ₁) and Λ ₂ = cosupp(Ω v ₂). Assuming that ‖Λ ₁‖₀ ≤ (l _est)₁ and ‖Λ ₂‖₀ ≤ (l _est)₂, it holds that ‖Λ ₁ ∩ Λ ₂‖₀ ≥ (l _est)₁ + (l _est)₂ − p. For the case ‖Λ ₁‖₀ = ‖Λ ₂‖₀ = l _est, we have 2l _est − p ≤ ‖Λ ₁ ∩ Λ ₂‖₀ ≤ l _est. So 2l _est − p is a reasonable size of the candidate set for CSMP, which corresponds to the candidate set size of 2k of CoSaMP in the synthesis model. Denoting T ₁ = supp(Ω v ₁) and T ₂ = supp(Ω v ₂), it is clear that supp(Ω(v ₁ + v ₂)) ⊆ T ₁ ∪ T ₂. Noticing that supp(⋅) = cosupp(⋅) ^C , we get cosupp(Ω(v ₁ + v ₂)) ⊇ (T ₁ ∪ T ₂) ^C  = T ₁ ^C  ∩ T ₂ ^C  = Λ ₁ ∩ Λ ₂, where the superscript C denotes the complementary set. This implies that the union of the supports in the synthesis case is parallel to the intersection of the cosupports in the analysis case.

Now, with the candidate set, we begin to construct the cosupport and update the measurement residual. The rows in the analysis operator, which correspond to the smallest l _est components in the temporarily estimated signal (calculated in Step 4 in Algorithm 1), are used to form the cosupport. Then, we approximate the signal and update the measurement residual of the current iteration with this cosupport. The backtracking strategy [12, 19], which can be described as not only selecting rows that match the current residual signal better from the candidate set in each iteration but also excluding the other rows from the cosupport, provides the basis for constructing a more accurate cosupport and obtaining a smaller measurement residual. Here, \( \mathrm{index}\left(\boldsymbol{\Omega} {\widehat{\boldsymbol{x}}}_{\mathrm{temp}},q\right) \) is reserved for sharing in the backward process. An efficient mechanism is required for stage switching when each stage finishes till l _est < l. This can be performed if the current measurement residual energy no longer decreases when compared with that in the last iteration. From Step 9 in Algorithm 1, we can see that the algorithm will run for some iterations with the same cosparsity l _est until it reaches the stage switching condition. The proposed algorithm will perform with a cosparsity of l _est − s in the next stage. The forward process will not stop until the measurement residual reaches a pre-set threshold such as ‖y _r ‖₂/‖y‖₂ ≤ 10^‐ 6 for the noiseless case or ‖y _r ‖₂ ≤ ε for the noisy case.

In the backward process, the algorithm tries to further increase the cosparsity by adding the less used rows into the cosupport. These q rows have been chosen in the last iteration of the forward process. The choice of the value of q can be made in terms of the value of s. As a rule of thumb, a small q is chosen when s is relatively small, and likewise, a greater q should be chosen if s is large. Typically, in our experiments, we choose q = 1 when s = 10, but we select q = 50 when s is in the order of thousands. With this strategy, we can get a more accurate cosupport for signal approximation. The iterations stop as soon as the measurement residual reaches the terminating threshold used in the forward process. The backward process needs to repeat less than ⌈s/q⌉ times since it is enough for the cosparsity to change from l _est to l _est + s in this process.

3.2 Relaxed versions for high-dimensional problems

3.3 Theoretical performance analysis

This section describes our theoretical analysis of the behavior of CSMP for the cosparse model in both noiseless and noisy cases. Because the proposed algorithm has a similar strategy of backtracking which is used in ASP, the proofs are mainly based on the proof framework of ASP/ACoSaMP. The following theorems are formed in parallel with those in [12], except for the unknown cosparsity and the initial cosupport and measurement vectors.

To show the ability of exact and stable recovery of cosparse signals by the CSMP algorithm, we define all variables involved in the process of signal reconstruction as follows:

Definition 3.2 (Problem p) [12] Consider a measurement vector y ∈ ℜ ^m such that y = Mx + e, where x ∈ ℜ ^d is l-cosparse, M ∈ ℜ ^m × d is a measurement matrix, and e ∈ ℜ ^m is a bounded additive noise. The largest singular value of M is σ _M and its Ω -RIP constant is δ _l . The analysis operator Ω ∈ ℜ ^p × d  is given and fixed. Define C _l to be the fraction of the largest and the smallest eigenvalues (which are not zero) of the sub-matrix composed of l rows from Ω. Assume that Ŝ _l  = cosupp(Ω x,l). According to Definition 3.1, the cosupport of x is a near-optimal projection. Our task is to recover x from y. The recovery result is denoted by \( \widehat{\boldsymbol{x}} \).

Now, we give the guarantees on exact recovery and stability of CSMP for recovering the cosparse signals.

Moreover, when \( {\rho}_1^2{\rho}_2^2<1, \) the iteration converges. The constant γ gives a tradeoff between satisfying the theorem conditions and the noise level, and the conditions for the noiseless case are achieved when γ tends to zero.

Theorem 3.4 (Exact recovery for cosparse signals) Consider the problem p when ‖e‖₂ = 0. Let l _s  = d − s⌈(d − l)/s⌉ and l _q  = l _s  + q⌊(l − l _s )/q⌋. If the measurement matrix M satisfies the Ω-RIP with parameter \( {\delta}_{4{l}_q-3p}\le \delta \left({C}_{2{l}_q-p},{\sigma}_{\boldsymbol{M}}^2,\gamma \right), \) where \( {C}_{2{l}_q-p} \) and γ are as in Theorem 3.3 and \( \delta \left({C}_{2{l}_q-p},{\sigma}_{\boldsymbol{M}}^2,\gamma \right) \) is a constant guaranteed to be greater than zero whenever ( 10) is satisfied, the CSMP algorithm guarantees an exact recovery of x from y via a finite number of iterations.

The proof is mainly based on the following lemma:

In the Appendix, Lemma 3.5 is proved in detail.

Lemma 3.5 describes that CSMP has a process of signal reconstruction that is equivalent to ASP, and could complete the exact recovery of the cosparse signals in a finite number of stages. To complete the proof, it is sufficient to show that the CSMP algorithm never gets stuck at any iteration of either stage, i.e., it takes a finite number of iterations up to ⌈(d − l)/s⌉ + ⌊(l − l _s )/q⌋ stages. At each stage, the cosupport (whose size is assumed to be l _est) adds and discards some rows of Ω, and the number of rows is fixed and finite. Hence, there are a finite number of combinations, at most, \( \left(\begin{array}{c}\hfill d\hfill \\ {}\hfill {l}_{\mathrm{est}}\hfill \end{array}\right), \) where d is the length of the signal. Thus, if CSMP takes an infinite number of iterations in this stage, the construction of cosupport would be repeated after at most \( \left(\begin{array}{c}\hfill d\hfill \\ {}\hfill {l}_{\mathrm{est}}\hfill \end{array}\right) \) iterations. Hence, Theorem 3.6 follows.

Similarly, the proof of Lemma 3.6 is based on Lemma 3.5 and the corresponding theorems of ASP algorithm in [12], and we omit the detailed proof here.

The above theorems are sufficient conditions of CSMP for exact recovery and stability. They are slightly more restrictive than the corresponding results of ASP algorithms because the true cosparsity level l is always larger than or equal to the estimated one l _q . This may be regarded as an additional cost for not having precise information of cosparsity. On the other hand, the proofs also show that these sufficient conditions may not be optimal or tight enough because they only consider the final stage and ignore the influence of previous stages on the performance of the algorithm.

4.1 Phase transition diagrams for synthetic signals in the noiseless case

We show the performance of the proposed algorithm as compared with six baseline methods, namely, AIHT, AHTP, ASP, ACoSaMP, l ₁-relaxation, and GAP, using the same experiments as performed in [12] for the noiseless case. We begin with synthetic signals and test the performance of CSMP with s = 1, s = 5, and s = 10, respectively. The results of the proposed algorithm are compared with those of AIHT and AHTP with an adaptively changing step size, ASP and ACoSaMP with a = 1 and \( a={\scriptscriptstyle \frac{2l-p}{l}} \), l₁-relaxation, and GAP. We use a random matrix M, where each entry is drawn independently from a Gaussian distribution, and a random tight frame Ω of size d = 120 and p = 144.

The reconstruction results of the proposed algorithm and the baseline algorithms are shown in Fig. 2.

In Fig. 2, experiments with s = 1, s = 5, and s = 10 are performed for CSMP, respectively. From Fig. 2, it can be seen that CSMP has better results than those of AIHT and AHTP with an adaptively changing step size, and ASP and ACoSaMP with \( a={\scriptscriptstyle \frac{2l-p}{l}} \) when l _est is far from d. In addition, the proposed algorithm with s = 5 and s = 10 provides comparable performance to ASP and ACoSaMP with a = 1 and the l₁-relaxation when the knowledge of cosparsity is unknown. Although the accurate recovery rates of GAP for experiments of all pairs of δ and ρ are higher than those of CSMP, the number of white cells is 59 in Fig. 2k, which is less than 67 in Fig. 2b and 63 in Fig. 2c.

4.2 Reconstruction of high-dimensional images in the noiseless and noisy cases

We now test the methods for high-dimensional signals. We use RCSMP (relaxed versions of CSMP defined in Section 3.2) for the reconstruction of the Shepp-Logan phantom from a few number of measurements. RCSMP is computationally more efficient than CSMP for high-dimensional signals and is thus chosen in this experiment. The sampling operator M is a two-dimensional Fourier transform that measures only a certain number of radial lines from the Fourier transform. The cosparse operator is a two-dimensional finite difference operator Ω _2D-DIF, of which the number of rows is p = 130, 560 and the real cosparsity of the signal under this operator is l = 128, 014. The original phantom image is presented in Fig. 3a. Using the RCSMP with s = 4000, q = 50, we get perfect reconstructions using only 12 radial lines just as RASP (relaxed Analysis Subspace Pursuit), i.e., only m = 3032 measurements out of d = 65,536 which is less than 4.63 % of the data in the original image. The algorithm requires less than 15 iterations which are less than those required by RASP for achieving this recovery percentage. The reconstruction results of CSMP are shown in Fig. 3c.

We now turn to test the method for the noisy case; we perform reconstruction using RCSMP with s = 4000, q = 50 of a noisy measurement of the phantom with 22 radial lines (in Fig. 4b) and signal-to-noise ratio (SNR) at 20 dB. Figure 4c presents the noisy image, which is the result of applying inverse Fourier transform on the measurements. We get the reconstruction results by the proposed method with a peak SNR (PSNR) of 37.11 dB in Fig. 4e. The recovery image using GAP is shown in Fig. 4f, with a little worse PSNR of 34.34 dB. Note that for the minimization process, we solve conjugate gradients in Steps 4, 6, 12, and 14 in each iteration in Algorithm 1, take only the real part of the result, and crop the values of the resulting image to be in the range of [0, 1] [12].