On recovery of block-sparse signals via mixed l ₂ /l _q (0 < q ≤ 1) norm minimization

Abstract

Compressed sensing (CS) states that a sparse signal can exactly be recovered from very few linear measurements. While in many applications, real-world signals also exhibit additional structures aside from standard sparsity. The typical example is the so-called block-sparse signals whose non-zero coefficients occur in a few blocks. In this article, we investigate the mixed l ₂/l _q (0 < q ≤ 1) norm minimization method for the exact and robust recovery of such block-sparse signals. We mainly show that the non-convex l ₂/l _q (0 < q < 1) minimization method has stronger sparsity promoting ability than the commonly used l ₂/l ₁ minimization method both practically and theoretically. In terms of a block variant of the restricted isometry property of measurement matrix, we present weaker sufficient conditions for exact and robust block-sparse signal recovery than those known for l ₂/l ₁ minimization. We also propose an efficient Iteratively Reweighted Least-Squares (IRLS) algorithm for the induced non-convex optimization problem. The obtained weaker conditions and the proposed IRLS algorithm are tested and compared with the mixed l ₂/l ₁ minimization method and the standard l _q  minimization method on a series of noiseless and noisy block-sparse signals. All the comparisons demonstrate the outperformance of the mixed l ₂/l _q (0 < q < 1) method for block-sparse signal recovery applications, and meaningfulness in the development of new CS technology.

1 Introduction

According to the Shannon/Nyquist sampling theorem [1, 2], if we would like to avoid lose of information when capturing a signal, we must sample the signal at the so-called Nyquist rate, which means twice the highest frequency of the signal. Since the theorem only exploits the bandlimitedness of a signal and most real-world signals are sparse or compressible, the process of massive data acquisition based on Shannon/Nyquist sampling theorem usually samples too many useless information and eventually we have to compress to store or encode a very few essential information of the signal. Obviously, this process is extremely wasteful and therefore, a more effective sampling way to directly acquire the essential information of a signal has been expected.

Compressed sensing (CS) [3–5] was motivated by this purpose and it can completely acquire the essential information of a signal by exploiting its compressibility. In a word, the main contribution of CS is that it presents a new efficient scheme to capture and recover the compressible or sparse signals at a reduced sampling rate far below the Nyquist rate. The basic principle of CS is that it first employs non-adaptive linear projections to preserve the structure of the signal; then one can exactly recover these signals from a surprisingly small number of random linear measurements through a nonlinear optimization procedure (such as l ₁-minimization) if the measurement matrix satisfies some suitable sufficient conditions in terms of restricted isometry property (RIP, [6]). Consequently, CS implies that it is indeed possible to acquire the data in already compressed form. Nowadays, driven by a wide range of applications, CS and other related problems have attracted much interest in various communities, such as in signal processing, machine learning, and statistics.

Different from general sparse signals in conventional sense, some real-world signals may exhibit some additional structures, i.e., the non-zero coefficients appear in a few fixed blocks, we refer to these signals as block-sparse signals in this article. Such block-sparse signals arise in various application problems, say, DNA microarrays [7, 8], equalization of sparse communication channels [9], source localization [10], wideband spectrum sensing [11], and color imaging [12].

Using standard convex relaxation ( l ₁-minimization) as conventional CS framework for recovering the block-sparse signal does not exploit the fact that the non-zero elements of the signal appear in consecutive positions. Therefore, one natural idea is considering the block version of l ₁-minimization, i.e., the mixed l ₂/l ₁-minimization, to exploit the block-sparsity. Many previous works have shown that the mixed l ₂/l ₁-minimization is superior to standard l ₁-minimization when dealing with such block-sparse signals [13–15]. Huang and Zhang [13] developed a theory for the mixed l ₂/l ₁-minimization by using a concept called strong group sparsity and they demonstrated that the mixed norm minimization is very efficient for recovering strongly group-sparse signals. Stojnic et al. [15] obtained an optimal number of Gaussian measurements for uniquely recovering a block-sparse signal through the mixed l ₂/l ₁ norm minimization. By generalizing the conventional RIP notion to the block-sparse case, Eldar and Mishali [14] showed that if the measurement matrix D has the same restricted isometry constant as that in the l ₁ case, then the mixed norm method is guaranteed to exactly recover any block-sparse signals in the noiseless case. Furthermore, they also showed that the block-sparse signal recovery would be robust in the noisy case under the same recovery condition. Another common approach to deal with the block-sparsity problem is by suitably extending the standard greedy methods, such as orthogonal matching pursuit, iterative hard thresholding (IHT), and compressive sampling matching pursuit (CoSaMP), to the block-sparse case. In [16], the CoSaMP algorithm and the IHT algorithm were extended to the model-based setting, treating the block-sparsity as a special case. It was also shown that the new recovery algorithms demonstrate provable performance guarantees and robustness properties. Eldar et al. [17] generalized the notion of coherence to the block-sparse setting and proved that a block-version of the orthogonal matching pursuit (BOMP) algorithm can exactly recover any block-sparse signal if the block-coherence is sufficiently small. In addition, the mixed l ₂/l ₁-minimization approach was certified to guarantee successful recovery with the same condition on block-coherence. Ben-Haim and Eldar [18] examined the ability of greedy algorithms to estimate a block-sparse signal from noisy measurements. They derived some near-oracle results for the block-sparse version of greedy pursuit algorithms both in the adversarial noise and the Gaussian noise cases. Majumdar and Ward [19] used BOMP to deal with the block-sparse representation-based classification problem. The validity and robustness of these new methods were theoretically proved.

In recent years, several studies [20–27] have showed that the non-convex l _q (0 < q < 1) minimization allows the exact recovery of sparse signals from fewer linear measurements than that by l ₁-minimization. Chartrand and Staneva [21] provided a weaker condition to guarantee perfect recovery for the non-convex l _q (0 < q < 1) minimization method using a l _q  variant of the RIP. They obtained the number of random Gaussian measurements necessary for the successful recovery of sparse signals via l _q  (0 < q < 1) minimization with high probability. Sun [27] used the conventional RIP as in the l ₁ case to prove that whenever q is chosen to be about 0.6796 × (1 - δ _2k ), every k-sparse signal can exactly be recovered via l _q minimization, here δ _2k  is the restricted isometry constant for the measurement matrix. Xu et al. [24–26] considered a specially important case ( q = 1/2) of l _q minimization. They developed a thresholding representation theory for l _1/2 minimization and conducted a phase diagram study to demonstrate the merits of l _1/2 minimization.

This article presents an ongoing effort to extend the non-convex l _q (0 < q < 1) minimization methodology to the setting of block-sparsity. Specifically, we will study the performance of the block-sparse signal recovery via the mixed l ₂/l _q  (0 < q < 1) norm minimization by means of the block RIP (block-RIP). We first exhibit that under similar RIP conditions with that in the standard l _q  case, the mixed l ₂/l _q  recovery method can assuredly to recover any block-sparse signal, irrespective to the locations of non-zero blocks. In addition, the method is robust in the presence of noise. Our formulated recovery conditions will show that the non-convex l ₂/l _q (0 < q < 1) minimization is superior to the convex l ₂/l ₁ minimization within block-RIP framework. Furthermore, we will compare the sparse signal recovery ability of the non-convex l ₂/l _q (0 < q < 1) method to the convex l ₂/l ₁ method and the standard l _q  method by conducting a series of simulation studies. To the best of the authors’ knowledge, although Majumdar and Ward [12] first proposed the non-convex l ₂/l _q (0 < q < 1) method in CS literature for color imaging and showed that the l ₂/l _0.4 minimization has the best performance on some imaging experiments, their works were experimental in feature and lack of convincing theoretical assessment. As compared, our work not only highlights theoretical merits of the non-convex block optimization method, but also makes a more intensive study on the block-sparse signal recovery capabilities for some different values of q via numerical experiments.

We begin with our study in Section 2,3 and 4 by presenting the problem setting. In Section 5, we establish the sufficient conditions for the mixed l ₂/l _q (0 < q < 1) optimization approach to guarantee exact and robust recovery of block-sparse signals in terms of block-RIP. We also develop an efficient Iteratively Reweighted Least-Squares (IRLS) algorithm to recover block-sparse signals from given fewer measurements, which generalizes the algorithm of [28] to the unconstrained l ₂/l _q (0 < q ≤ 1) norm minimization case in Section 6. In Section 7, we show that the non-convex l ₂/l _q (0 < q < 1) method has stronger block-sparsity promoting ability than the convex l ₂/l ₁ method and the standard l _q  method through a series of simulations. Finally, we conclude the article in Section 8 with some useful remarks.

2 Block-sparsity

The conventional CS only consider the sparsity that the signal x has at most k non-zero elements, and it does not take into account any further structure. However, in many practical scenarios, the non-zero elements are aligned to blocks, meaning that they appear in regions. These signals are referred to the block-sparse signals. Mathematically speaking, a block-sparse signal $\mathbf{x}\in {\mathbb{R}}^N$ over block index set $\mathcal{I}=\{d_1,\dots ,d_m\}$ can be modeled as follows:

$$\mathbf{x}={[\underset{\mathbf{x}\left[1\right]}{\underbrace{x_1\cdots x_{d_1}}}\underset{\mathbf{x}\left[2\right]}{\underbrace{x_{d_1+1}\cdots x_{d_1+d_2}}}\cdots \underset{\mathbf{x}\left[m\right]}{\underbrace{x_{N-d_m+1}\cdots x_N}}]}^T.$$

(1)

Here, x[i] denotes the i th block of x and d _i  is the block size for the i th block. The block-sparsity we consider in this article means that there are at most k < m non-zero blocks. Obviously, if d ₁ = ⋯ = d _m  = 1, the block-sparse signals degenerate to the conventional sparse signals well studied in CS.

Definition 1.

 ([14]). A block k-sparse signal over index set $\mathcal{I}=\{d_1,\dots ,d_m\}$ is a signal of the form (1) in which x[i] is non-zero for at most k indices i, i ∈ {1,2,…,m}.

The main focus of this study is to recover a block-sparse signal x from random linear measurement y = Φ x (noiseless case) or y = Φ x + z(noisy case). Here, $\mathbf{y}\in {\mathbb{R}}^M$ is a vector, $\Phi \in {\mathbb{R}}^{M\times N}$ is a measurement matrix, whose entries are usually randomly drawn from a Gaussian or a Bernoulli distribution, and z is an unknown bounded noise. We represent Φ as a concatenation of column-blocks Φ[i] of size M × d _i , that is,

$$\mathbf{\Phi }=[\underset{\Phi \left[1\right]}{\underbrace{{\varphi }_1\cdots {\varphi }_{d_1}}}\underset{\Phi \left[2\right]}{\underbrace{{\varphi }_{d_1+1}\cdots {\varphi }_{d_1+d_2}}}\cdots \underset{\Phi \left[m\right]}{\underbrace{{\varphi }_{N-d_m+1}\cdots {\varphi }_N}}].$$

(2)

Then we are interested in formulating sufficient conditions on the measurement matrix Φ under which a block-sparse signal x can assuredly be and stably recovered from its fewer noiseless measurements $\mathbf{y}=\sum _{i=1}^m\Phi \left[i\right]\mathbf{x}\left[i\right]$ or noisy measurements $\mathbf{y}=\sum _{i=1}^m\Phi \left[i\right]\mathbf{x}\left[i\right]+\mathbf{z}$. Denote

$$\parallel \mathbf{x}{\parallel }_{2,0}=\sum _{i=1}^{m}I(\parallel \mathbf{x}[i]{\parallel }_2>0),$$

(3)

where I(∥x[i]∥₂ > 0) is an indicator function, we then notice that a block k-sparse signal x can be defined as a vector that satisfies ∥x∥_2,0 ≤ k. In the remainder of the article, we will restrict our attention to how and in what conditions these block-sparse signals can be recovered exactly and stably in noiseless and noisy scenarios respectively.

3 Block-RIP

Candes and Tao [6] first introduced the notion of RIP of a matrix to characterize the condition under which the sparsest solution of an underdetermined linear system exists and can be found. And then the RIP was used as a powerful tool to study CS in several previous works [4, 5, 21, 29]. Let Φ be a matrix of size M × N, where M < N, we say that matrix Φ vsatisfies RIP of order k if there exists a constant δ _k  ∈ [0,1) such that for every $\mathbf{x}\in {\mathbb{R}}^N(\parallel \mathbf{x}{\parallel }_0\le k),$

$$(1-{\delta }_k)\parallel \mathbf{x}{\parallel }_2^2\le \parallel \Phi \mathbf{x}{\parallel }_2^2\le (1+{\delta }_k)\parallel \mathbf{x}{\parallel }_2^2$$

(4)

Obviously, δ _k  quantifies how close to isometric the all M × k submatrices of Φ should be. Since the block-sparse signals exhibit additional structure, Eldar and Mishali [14] extended the standard RIP to the block-sparse setting and showed that the new block-RIP constant is typically smaller than the standard RIP constant. Now we state the new definition in block-sparse setting.

Definition 2.

 ([14]). Let $\Phi :{\mathbb{R}}^N\to {\mathbb{R}}^M$ be a M × N measurement matrix. Then Φ is said to have the block-RIP over $\mathcal{I}=\{d_1,\dots ,d_m\}$ with constant ${\delta }_{k|\mathcal{I}}$ if for every vector x ∈ R ^N that is block k-sparse over $\mathcal{I}$, it satisfies

$$(1-{\delta }_{k|\mathcal{I}})\parallel \mathbf{x}{\parallel }_2^2\le \parallel \Phi \mathbf{x}{\parallel }_2^2\le (1+{\delta }_{k|\mathcal{I}})\parallel \mathbf{x}{\parallel }_2^2.$$

(5)

For convenience, in the remainder of the article, we still use δ _k , instead of ${\delta }_{k|\mathcal{I}}$, to represent the block-RIP constant whenever the confusion is not caused.

With the new notion, Eldar and Mishali [29] generalized the sufficient recovery conditions to the block-sparse signals both in noiseless and noisy settings. They showed that if Φ is taken random as conventional CS, it satisfies the block-RIP with overwhelming probability. All these results illustrated that one can recover a block-sparse signal exactly and stably via the convex mixed l ₁/l ₂ minimization method whenever the measurement matrix Φ is constructed from a random ensemble (i.e., Gaussian ensemble).

4 Non-convex recovery method

It is known from [14] that whenever Φ satisfies the block-RIP with δ _2k  < 1, there is a unique block-sparse signal x which can be recovered by solving the following problem:

$$\begin{array}{l}\begin{array}{l}\underset{\mathbf{x}}{\text{min}}\parallel \mathbf{x}{\parallel }_{2,0}\\ \text{s.t.}\mathbf{y}=\Phi \mathbf{x}.\end{array}\end{array}$$

(6)

Unfortunately, the problem (6) is an NP-hard problem and finding the optimal solution of (6) has exponential complexity. In principle, one only can solve the problem exactly by searching over all possible sets of k blocks whether there exists a vector consistent with the measurements. Obviously, this approach is unable to deal with high-dimensional signals.

One natural idea to find x more efficiently is to employ a convex relaxation technique, namely, to replace the l ₂/l ₀ norm by its closest convex surrogate l ₂/l ₁ norm, thus resulting in the following model:

$$\begin{array}{l}\begin{array}{l}\underset{\mathbf{x}}{\text{min}}\parallel \mathbf{x}{\parallel }_{2,1}\\ \text{s.t.}\mathbf{y}=\Phi \mathbf{x},\end{array}\end{array}$$

(7)

where $\parallel \mathbf{x}{\parallel }_{2,1}=\sum _{i=1}^m\parallel \mathbf{x}\left[i\right]{\parallel }_2$. This model can be treated as a second-order cone program (SOCP) problem and many standard software packages can be used for the solutions very efficiently. In many practical cases, the measurements y are corrupted by bounded noise, then we can apply the modified SOCP or the group version of basis pursuit denoising [30] program as the following:

$$\underset{\mathbf{x}}{\text{min}}\parallel \mathbf{y}-\Phi \mathbf{x}{\parallel }_2^2+\lambda \parallel \mathbf{x}{\parallel }_{2,1},$$

(8)

where λ is a tuning parameter, which controls the tolerance of the noise term. There are also many methods to solve this optimization problem efficiently, such as the block-coordinate descent technique [31] and the Landweber iterations technique [32].

As mentioned before, recent studies on non-convex CS have indicated that one can reduce the number of required linear measurements for successful recovery of a general sparse signal by replacing the l ₀ norm by a non-convex surrogate l _q (0 < q < 1) quasi-norm, which motivates us to generalize the better recovery capability of the non-convex CS to the block-sparse setting. Therefore, we suggest the use of the following non-convex optimization model for recovery of block-sparse signals, that is,

$$\begin{array}{l}\begin{array}{l}\underset{\mathbf{x}}{\text{min}}\parallel \mathbf{x}{\parallel }_{2,q}^q\\ \text{s.t.}\parallel \mathbf{y}-\Phi \mathbf{x}{\parallel }_2^2\le ϵ,\end{array}\end{array}$$

(9)

where ϵ ≥ 0 controls the noise error term ( ϵ = 0 means noiseless case) and $\parallel \mathbf{x}{\parallel }_{2,q}={(\sum _{i=1}^m\parallel \mathbf{x}[i\left]{\parallel }_2^q\right)}^{1/q}$ is a generalization of standard l _q  quasi-norm for 0 < q < 1. We will show that this new non-convex recovery approach can achieve better block-sparse recovery performance both practically and theoretically when compared with the commonly used convex l ₂/l ₁ minimization approach. In the following section, we will provide some sufficient conditions for exact and stable recovery of block-sparse signals through the mixed l ₂/l _q (0 < q < 1) norm minimization, and further develop a similar IRLS algorithm as in [28, 33] for solutions of such non-convex optimization problem.

5 Sufficient block-sparse recovery conditions

In this section, we first consider the recovery problem of a high-dimensional signal $\mathbf{x}\in {\mathbb{R}}^N$ in the noiseless setting. Thus, we propose the constrained mixed l ₂/l _q  norm minimization with 0 < q < 1:

$$\begin{array}{l}\begin{array}{l}\underset{\mathbf{x}}{\text{min}}\parallel \mathbf{x}{\parallel }_{2,q}^q\\ \text{s.t.}\mathbf{y}=\Phi \mathbf{x},\end{array}\end{array}$$

(10)

where $\mathbf{y}\in {\mathbb{R}}^M$ are available measurements, Φ is a known M × N measurement matrix.

To state our main results, we need more notations. We first consider the case where x is exactly block k-sparse. We use Null( Φ) to denote the null space of Φ and

T to denote the block index set of non-zero blocks of the signal x. Let x ^∗ be a solution of the minimization problem (10). From [15], it is known that x ^∗ is the unique sparse solution of (10) being equal to x if and only if

$$\parallel {\mathbf{h}}_T{\parallel }_{2,q}<\parallel {\mathbf{h}}_{T^c}{\parallel }_{2,q}$$

for all non-zero vector h in the null space of Φ. This is called the null space property (NSP). In order to characterize more accurately the NSP, one can consider the following equivalent form: There exists a smallest constant ρ satisfying 0 < ρ < 1 such that

$$\parallel {\mathbf{h}}_T{\parallel }_{2,q}\le \rho \parallel {\mathbf{h}}_{T^c}{\parallel }_{2,q}.$$

(11)

When x is not exactly block sparse, Aldroubi et al. [34] also showed that NSP actually guarantees stability. Precisely, if we use T to denote the block index set over the k blocks with largest l ₂ norm of x, then the NSP (11) gives

$$\parallel \mathbf{x}-{\mathbf{x}}^{\ast }{\parallel }_2\le \frac{C\parallel {\mathbf{x}}_{T^c}{\parallel }_{2,q}}{k^{1/q-1/2}}$$

(12)

here C is a constant. Indeed, from (11), it is easy to see that the following equality holds

$$\begin{array}{ll}\underset{\mathbf{h}\in \text{Null}\left(\Phi \right),\mathbf{h}\ne 0}{\text{sup}}& \frac{{(\sum _{i\in T}\parallel \mathbf{h}[i\left]{\parallel }_2^q\right)}^{1/q}}{{(\sum _{i\in T^c}\parallel \mathbf{h}[i\left]{\parallel }_2^q\right)}^{1/q}}\\ =\underset{\mathbf{h}\in \text{Null}\left(\Phi \right),\parallel \mathbf{h}{\parallel }_2=1}{\text{max}}\frac{{(\sum _{i\in T}\parallel \mathbf{h}[i\left]{\parallel }_2^q\right)}^{1/q}}{{(\sum _{i\in T^c}\parallel \mathbf{h}[i\left]{\parallel }_2^q\right)}^{1/q}}\end{array}$$

which is denoted by ρ. In general, for h = x ^∗-x, we let

$$\parallel {\mathbf{h}}_T{\parallel }_{2,q}=\gamma (\mathbf{h},q)\parallel {\mathbf{h}}_{T^c}{\parallel }_{2,q}.$$

Therefore, Our main point of the study is to show how to make γ(h,q) < 1 for all non-zero vector h in the null space of Φ. Our first conclusion is the following theorem.

Theorem 1.

 (Noiseless recovery). Let y = Φ x be measurements of a signal x. If the matrix Φ satisfies the block RIP (5) with

$${\delta }_{2k}<1/2,$$

then there exists a number q ₀(δ _2k ) ∈ (0,1] such that for any q < q ₀, the solution x ^∗ o the mixed l ₂/l _q  problem (10) obeys to

$$\begin{array}{l}\parallel \mathbf{x}-{\mathbf{x}}^{\ast }{\parallel }_{2,q}\le C_0(q,{\delta }_{2k})\parallel \mathbf{x}-{\mathbf{x}}_{T_0}{\parallel }_{2,q}\\ \parallel \mathbf{x}-{\mathbf{x}}^{\ast }{\parallel }_2\le C_1(q,{\delta }_{2k})k^{1/2-1/q}\parallel \mathbf{x}-{\mathbf{x}}_{T_0}{\parallel }_{2,q}\end{array}$$

(13)

where C ₀(q,δ _2k ) and C ₁(q,δ _2k ) are positive constants dependent on q and δ _2k , T ₀ is the block index set over the k blocks with largest l ₂ norm of the original signal x. In particular, if x is block k-sparse, the recovery is exact.

Remark 1.

 Theorem 1 provides a sufficient condition for the recovery of a signal x via l ₂ / l _q  minimization with 0 < q < 1 in the noiseless setting. Focusing on the case where x is block k-sparse, it is known [14] that when the l ₂/l ₁ minimization scheme (7) is employed to recover x, the sufficient condition on the block-RIP is that δ _2k  < 0.414. As compared, Theorem 1 says that, whenever the non-convex l ₂/l _q minimization scheme (10) is used, this constant can be relaxed to δ _2k  < 0.5 for some q < 1. This shows that similar as the standard sparse signal recovery, when compared with the convex minimization method, the non-convex minimization method can enhance performance of block-sparse signal recovery.

To prove Theorem 1, we need the following Lemmas:

Lemma 1.

 ([14]).

$$|〈\Phi \mathbf{x},\Phi x^{\prime }〉|\le {\delta }_{k+k^{\prime }}\parallel \mathbf{x}{\parallel }_2\parallel x^{\prime }{\parallel }_2$$

(14)

for all x,x ^′ supported on disjoint subsets T,T ^′ ⊆ {1,2,…,N} with |T| < k and |T ^′| < k ^′.

Lemma 2.

 ([35]). For any fixed q ∈ (0,1) and $\mathbf{x}\in {\mathbb{R}}^N$,

$$\parallel \mathbf{x}{\parallel }_2\le \frac{\parallel \mathbf{x}{\parallel }_q}{N^{1/q-1/2}}+\sqrt{N}\left(\underset{1\le i\le N}{\text{max}}\right|{\mathbf{x}}_i|-\underset{1\le i\le N}{\text{min}}|{\mathbf{x}}_i\left|\right).$$

(15)

Proof of Theorem 1.

 Set x ^∗ = x + h be a solution of (10), where x is the original signal we need to reconstruct. Throughout the article, x _T  will denote the vector equal to x on an index set T and zero elsewhere. Let T ₀ be the block index set over the k blocks with largest l ₂ norm of x. And we decompose h into a series of vectors ${\mathbf{h}}_{T_0},{\mathbf{h}}_{T_1},{\mathbf{h}}_{T_2},\dots ,{\mathbf{h}}_{T_J}$, such that

$$\mathbf{h}=\sum _i^J{\mathbf{h}}_{T_i}.$$

Here ${\mathbf{h}}_{T_i}$ is the restriction of h to the set T _i  and each T _i  consists of k blocks (except possibly T _J ). Rearranging the block indices such that $\parallel {\mathbf{h}}_{T_j}\left[1\right]{\parallel }_2\ge \parallel {\mathbf{h}}_{T_j}\left[2\right]{\parallel }_2\ge \cdots \ge \parallel {\mathbf{h}}_{T_j}\left[k\right]{\parallel }_2\ge \parallel {\mathbf{h}}_{T_{j+1}}\left[1\right]{\parallel }_2\ge \parallel {\mathbf{h}}_{T_{j+1}}\left[2\right]{\parallel }_2\ge \cdots$, for any j ≥ 1.

Note that

$$\begin{array}{l}\parallel \mathbf{h}{\parallel }_2=\parallel {\mathbf{h}}_{T_0\bigcup T_1}+{\mathbf{h}}_{{(T_0\bigcup T_1)}^c}{\parallel }_2\le \parallel {\mathbf{h}}_{T_0\bigcup T_1}{\parallel }_2+\parallel {\mathbf{h}}_{{(T_0\bigcup T_1)}^c}{\parallel }_2.\end{array}$$

(16)

For any j ≥ 2, if we let ${\mathbf{d}}_j=(\parallel {\mathbf{h}}_{T_j}[1]{\parallel }_2,\parallel {\mathbf{h}}_{T_j}[2]{\parallel }_2,\dots ,$ $\parallel {\mathbf{h}}_{T_j}\left[k\right]{\parallel }_2)$, then we have

$$\begin{array}{l}\parallel {\mathbf{d}}_j{\parallel }_2={\left(\sum _{i=1}^k\parallel {\mathbf{h}}_{T_j}\left[i\right]{\parallel }_2^2\right)}^{1/2}=\parallel {\mathbf{h}}_{T_j}{\parallel }_2,\\ \parallel {\mathbf{d}}_j{\parallel }_q={\left(\sum _{i=1}^k\parallel {\mathbf{h}}_{T_j}\left[i\right]{\parallel }_2^q\right)}^{1/q}=\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q},\\ \underset{1\le i\le k}{\text{max}}\left|{\mathbf{d}}_j\right(i\left)\right|=\underset{1\le i\le k}{\text{max}}\parallel {\mathbf{h}}_{T_j}\left[i\right]{\parallel }_2=\parallel {\mathbf{h}}_{T_j}{\parallel }_{\infty ,T},\\ \underset{1\le i\le k}{\text{min}}\left|{\mathbf{d}}_j\right(i\left)\right|=\underset{1\le i\le k}{\text{min}}\parallel {\mathbf{h}}_{T_j}\left[i\right]{\parallel }_2.\end{array}$$

(17)

From Lemma 2, it follows that

$$\parallel {\mathbf{d}}_j{\parallel }_2\le \frac{\parallel {\mathbf{d}}_j{\parallel }_q}{k^{1/q-1/2}}+k^{1/2}\left(\underset{1\le i\le k}{\text{max}}\right|{\mathbf{d}}_j\left(i\right)|-\underset{1\le i\le k}{\text{min}}|{\mathbf{d}}_j\left(i\right)\left|\right),$$

that is,

$$\parallel {\mathbf{h}}_{T_j}{\parallel }_2\le \frac{\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}}{k^{1/q-1/2}}+k^{1/2}(\parallel {\mathbf{h}}_{T_j}{\parallel }_{\infty ,T}-\underset{1\le i\le k}{\text{min}}\parallel {\mathbf{h}}_{T_j}[i\left]{\parallel }_2\right).$$

(18)

From Equation (18), we obtain

$$\begin{array}{l}\begin{array}{ll}{k}^{1/q-1/2}\sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_2& \le \sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}+k^{1/q}(\parallel {\mathbf{h}}_{T_j}{\parallel }_{\infty ,T}\\ -\underset{1\le i\le k}{\text{min}}\parallel {\mathbf{h}}_{T_j}\left[i\right]{\parallel }_2)\\ \le \sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}+k^{1/q}\parallel {\mathbf{h}}_{T_2}{\parallel }_{\infty ,T}\\ \le \sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}+k^{1/q}\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}/k^{1/q}\\ =\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}\\ \le {\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q},\end{array}\end{array}$$

(19)

where we have used the fact that (a + b)^q ≤ a ^q + b ^q for non-negative a and b. Therefore, we have

$$\sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_2\le \frac{1}{k^{1/q-1/2}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}.$$

(20)

On the other hand, let

$$\begin{array}{l}\parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}=\gamma (\mathbf{h},q)\parallel {\mathbf{h}}_{T_0^c}{\parallel }_{2,q},\\ \parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^q=t\sum _{i\ge 1}\parallel {\mathbf{h}}_{T_i}{\parallel }_{2,q}^q,t\in [0,1].\end{array}$$

(21)

Since $\parallel {\mathbf{h}}_{T_2}\left[1\right]{\parallel }_2\ge \parallel {\mathbf{h}}_{T_2}\left[2\right]{\parallel }_2\ge \cdots \ge \parallel {\mathbf{h}}_{T_2}\left[k\right]{\parallel }_2\ge \parallel {\mathbf{h}}_{T_3}\left[1\right]{\parallel }_2\ge \parallel {\mathbf{h}}_{T_3}\left[2\right]{\parallel }_2\ge \cdots$, it is easy to see that

$$\begin{array}{ll}\sum _{i\ge 2}\parallel {\mathbf{h}}_{T_i}{\parallel }_2^2& \le \parallel {\mathbf{h}}_{T_2}\left[1\right]{\parallel }_2^{2-q}\sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\\ \le \left(\right(\parallel {\mathbf{h}}_{T_1}\left[1\right]{\parallel }_2^q+\parallel {\mathbf{h}}_{T_1}\left[2\right]{\parallel }_2^q+\cdots \\ +\parallel {\mathbf{h}}_{T_1}\left[k\right]{\parallel }_2^q)/k{)}^{(2-q)/q}\sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\\ ={(\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^q/k)}^{\frac{2-q}{q}}\sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\\ ={(\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^q/k)}^{\frac{2-q}{q}}\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q-\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^q\right)\\ ={(\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^q/k)}^{\frac{2-q}{q}}\left(\frac{1}{t}\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^q-\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^q\right)\\ =\frac{1-t}{t{k}^{\frac{2-q}{q}}}\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^2\\ =\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{2/q}.\end{array}$$

(22)

By the definition of the block-RIP, Lemma 1, (20) and (22), it then implies that

$$\begin{array}{l}\parallel \Phi \left(\sum _{j\ge 2}{\mathbf{h}}_{T_j}\right){\parallel }_2^2\\ =\sum _{i,j\ge 2}〈\Phi \left({\mathbf{h}}_{T_i}\right),\Phi \left({\mathbf{h}}_{T_j}\right)〉\\ =\sum _{j\ge 2}〈\Phi \left({\mathbf{h}}_{T_j}\right),\Phi \left({\mathbf{h}}_{T_j}\right)〉+2\sum _{i,j\ge 2,i<j}〈\Phi \left(h_{T_i}\right),\Phi \left({\mathbf{h}}_{T_j}\right)〉\\ \le (1+{\delta }_k)\sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_2^2+2{\delta }_{2k}\sum _{j>i\ge 2}\parallel {\mathbf{h}}_{T_i}{\parallel }_2\parallel {\mathbf{h}}_{T_j}{\parallel }_2\\ \le \sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_2^2+{\delta }_{2k}{\left(\sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_2\right)}^2\\ \le \left(\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}+{\delta }_{2k}{k}^{1-2/q}\right){\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{2/q}.\end{array}$$

(23)

Since Φ x = Φ x ^∗, we have Φ h = 0, thus $\Phi ({\mathbf{h}}_{T_0}+{\mathbf{h}}_{T_1})=-$ $\Phi \left(\sum _{j\ge 2}{\mathbf{h}}_{T_j}\right)$. Therefore,

$$\begin{array}{ll}\parallel \Phi ({\mathbf{h}}_{T_0}+{\mathbf{h}}_{T_1}){\parallel }_2^2& \le \left(\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}+{\delta }_{2k}{k}^{1-2/q}\right)\\ \times {\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{2/q},\end{array}$$

(24)

By the definition of δ _2k  and using H$ö$lder’s equality, we then further have

$$\begin{array}{l}\parallel \Phi ({\mathbf{h}}_{T_0}+{\mathbf{h}}_{T_1}){\parallel }_2^2\\ \ge (1-{\delta }_{2k})\parallel {\mathbf{h}}_{T_0}+{\mathbf{h}}_{T_1}{\parallel }_2^2\\ =(1-{\delta }_{2k})(\parallel {\mathbf{h}}_{T_0}{\parallel }_2^2+\parallel {\mathbf{h}}_{T_1}{\parallel }_2^2)\\ \ge (1-{\delta }_{2k})(k^{1-2/q}\parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}^2+k^{1-2/q}\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^2)\\ =(1-{\delta }_{2k})k^{1-2/q}\\ \times \left(\gamma {(\mathbf{h},q)}^2\parallel {\mathbf{h}}_{T_0^c}{\parallel }_{2,q}^2+t^{2/q}{\left(\sum _{i\ge 1}\parallel {\mathbf{h}}_{T_i}{\parallel }_{2,q}^q\right)}^{2/q}\right)\\ =(1-{\delta }_{2k})k^{1-2/q}(\gamma {(\mathbf{h},q)}^2+t^{2/q}){\left(\sum _{i\ge 1}\parallel {\mathbf{h}}_{T_i}{\parallel }_{2,q}^q\right)}^{2/q}.\end{array}$$

(25)

By Equations (24) and (25),

$$\gamma {(\mathbf{h},q)}^2\le \left(\frac{1-t}{t^{1-2}/q}\right)/(1-{\delta }_{2k})/-t^{2/q}\doteq f\left(t\right).$$

Through a straightforward calculation, it is easy to get that the maximum of f(t) occurs at $t_0=\frac{1-q/2}{2-{\delta }_{2k}}$ and

$$f\left(t_0\right)=\frac{{\delta }_{2k}\frac{2-q}{2-{\delta }_{2k}}+q{\left(\frac{1-q/2}{2-{\delta }_{2k}}\right)}^{2/q}}{\frac{2-q}{2-{\delta }_{2k}}(1-{\delta }_{2k})}.$$

If f(t ₀) < 1, then we have γ(h,q) < 1. However, f(t ₀) < 1 amounts to

$${\delta }_{2k}\frac{2-q}{2-{\delta }_{2k}}+q{\left(\frac{1-q/2}{2-{\delta }_{2k}}\right)}^{2/q}<\frac{2-q}{2-{\delta }_{2k}}(1-{\delta }_{2k}).$$

or, equivalently,

$${\delta }_{2k}+\frac{q}{2^{2/q+1}}{\left(\frac{2-q}{2-{\delta }_{2k}}\right)}^{2/q-1}<\frac{1}{2}$$

(26)

Since the second term on the left-hand side of (26) goes to zero as q → 0₊ whenever q ≤ 1, δ _2k  < 1, and

$$\frac{q}{2^{2/q+1}}{\left(\frac{2-q}{2-{\delta }_{2k}}\right)}^{2/q-1}\le q{\left(\frac{2-q}{2}\right)}^{2/q}\approx \frac{q}{e}$$

We thus obtain that for δ _2k  < 1/2, there exists a value q ₀ = q ₀(δ _2k ) ∈ (0,1] such that for all q ∈ (0,q ₀) the above inequality (26) is true.

From the definition of γ(h,q), we have

$$\parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}^2\le \frac{{\delta }_{2k}\frac{2-q}{2-{\delta }_{2k}}+q{\left(\frac{1-q/2}{2-{\delta }_{2k}}\right)}^{2/q}}{\frac{2-q}{2-{\delta }_{2k}}(1-{\delta }_{2k})}\parallel {\mathbf{h}}_{T_0^c}{\parallel }_{2,q}^2.$$

As ∥x ^∗∥ _q  = ∥x + h∥ _q  is the minimum, using the equation ${\mathbf{x}}^{\ast }=\mathbf{x}+{\mathbf{h}}_{T_0}+{\mathbf{h}}_{T_0^c}$, we get

$$\begin{array}{ll}\parallel \mathbf{x}{\parallel }_{2,q}^q& =\parallel {\mathbf{x}}_{T_0}{\parallel }_{2,q}^q+\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}^q\\ \ge \parallel {\mathbf{x}}^{\ast }{\parallel }_{2,q}^q=\parallel \mathbf{x}+{\mathbf{h}}_{T_0}+{\mathbf{h}}_{T_0^c}{\parallel }_{2,q}^q\\ =\parallel {(\mathbf{x}+{\mathbf{h}}_{T_0}+{\mathbf{h}}_{T_0^c})}_{T_0}{\parallel }_{2,q}^q+\parallel {(\mathbf{x}+{\mathbf{h}}_{T_0}+{\mathbf{h}}_{T_0^c})}_{T_0^c}{\parallel }_{2,q}^q\\ =\parallel \mathbf{x}+{\mathbf{h}}_{T_0}{\parallel }_{2,q}^q+\parallel {\mathbf{h}}_{T_0^c}{\parallel }_{2,q}^q\\ \ge \parallel \mathbf{x}{\parallel }_{2,q}^q-\parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}^q+\parallel {\mathbf{h}}_{T_0^c}{\parallel }_{2,q}^q\\ \ge \parallel {\mathbf{x}}_{T_0}{\parallel }_{2,q}^q-\parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}^q+\parallel {\mathbf{h}}_{T_0^c}{\parallel }_{2,q}^q-\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}^q.\end{array}$$

(27)

Since $\parallel \mathbf{x}{\parallel }_{2,q}^q=\parallel {\mathbf{x}}_{T_0}{\parallel }_{2,q}^q+\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}^q$, this then implies

$$\begin{array}{ll}\parallel {\mathbf{h}}_{T_0^c}{\parallel }_{2,q}^q& \le \parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}^q+2\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}^q\\ \le {\left(\frac{{\delta }_{2k}\frac{2-q}{2-{\delta }_{2k}}+q{\left(\frac{1-q/2}{2-{\delta }_{2k}}\right)}^{2/q}}{\frac{2-q}{2-{\delta }_{2k}}(1-{\delta }_{2k})}\right)}^{q/2}\parallel {\mathbf{h}}_{T_0^c}{\parallel }_{2,q}^q\\ +2\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}^q,\end{array}$$

That is,

$$\parallel {\mathbf{h}}_{T_0^c}{\parallel }_{2,q}^q\le \frac{2}{1-{\left(\frac{{\delta }_{2k}\frac{2-q}{2-{\delta }_{2k}}+q{\left(\frac{1-q/2}{2-{\delta }_{2k}}\right)}^{2/q}}{\frac{2-q}{2-{\delta }_{2k}}(1-{\delta }_{2k})}\right)}^{q/2}}\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}^q.$$

(28)

Thus, we have

$$\begin{array}{ll}\parallel \mathbf{h}{\parallel }_{2,q}^q& =\parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}^q+\parallel {\mathbf{h}}_{T_0^c}{\parallel }_{2,q}^q\\ \le \left({\left(\frac{{\delta }_{2k}\frac{2-q}{2-{\delta }_{2k}}+q{\left(\frac{1-q/2}{2-{\delta }_{2k}}\right)}^{2/q}}{\frac{2-q}{2-{\delta }_{2k}}(1-{\delta }_{2k})}\right)}^{q/2}+1\right)\parallel {\mathbf{h}}_{T_0^c}{\parallel }_{2,q}^q\\ \le \frac{2\left({\left(\frac{{\delta }_{2k}\frac{2-q}{2-{\delta }_{2k}}+q{\left(\frac{1-q/2}{2-{\delta }_{2k}}\right)}^{2/q}}{\frac{2-q}{2-{\delta }_{2k}}(1-{\delta }_{2k})}\right)}^{q/2}+1\right)}{1-{\left(\frac{{\delta }_{2k}\frac{2-q}{2-{\delta }_{2k}}+q{\left(\frac{1-q/2}{2-{\delta }_{2k}}\right)}^{2/q}}{\frac{2-q}{2-{\delta }_{2k}}(1-{\delta }_{2k})}\right)}^{q/2}}\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}^q\\ =C_q^q\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}^q.\end{array}$$

(29)

This justifies that the first equality of (13) is proved. In the following, we further prove the second equality of (13).

In effect, from (24) and (25), we have

$$\parallel {\mathbf{h}}_{T_0\cup T_1}{\parallel }_2^2\le \frac{\left(\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}+{\delta }_{2k}{k}^{1-2/q}\right){\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{2/q}}{1-{\delta }_{2k}}$$

(30)

which, together with (22) and (30), then implies

$$\begin{array}{l}\begin{array}{ll}\parallel \mathbf{h}{\parallel }_2& \le \sqrt{\parallel {\mathbf{h}}_{T_0\bigcup T_1}{\parallel }_2^2+\parallel {\mathbf{h}}_{{(T_0\bigcup T_1)}^c}{\parallel }_2^2}\\ \le \sqrt{\frac{\left(\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}+{\delta }_{2k}{k}^{1-2/q}\right){\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{2/q}}{1-{\delta }_{2k}}+\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{2/q}}\\ =\sqrt{\left(\frac{\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}+{\delta }_{2k}{k}^{1-2/q}}{1-{\delta }_{2k}}+\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}\right){\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{2/q}}\\ =\sqrt{\frac{k^{1-2/q}(\frac{1-t}{t^{1-2/q}}+{\delta }_{2k}+(1-{\delta }_{2k}\left)\frac{1-t}{t^{1-2/q}}\right)}{1-{\delta }_{2k}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}\\ =k^{1/2-1/q}\sqrt{\frac{(2-{\delta }_{2k})(1-t)t^{2/q-1}+{\delta }_{2k}}{1-{\delta }_{2k}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}\\ \le k^{1/2-1/q}\sqrt{\frac{(2-{\delta }_{2k})q{(2-q)}^{2/q-1}+2^{2/q}{\delta }_{2k}}{(1-{\delta }_{2k})2^{2/q}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}\end{array}\end{array}$$

(31)

where we have used the fact that

$$\begin{array}{ll}\underset{t\in [0,1]}{\text{max}}(1-t)t^{2/q-1}& =(1-t)t^{2/q-1}{|}_{t=1-q/2}\\ =\frac{q}{2}{\left(1-\frac{q}{2}\right)}^{2/q-1}\end{array}$$

By (28), we further have

$$\begin{array}{ll}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}& \le \frac{2^{1/q}\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}}{{(1-{\left(f\right(t_0\left)\right)}^{q/2})}^{1/q}}\\ \le \frac{2^{2/q-1}\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}}{{(1-{\left(f\right(t_0\left)\right)}^{q/2})}^{1/q}}\end{array}$$

(32)

Thus, it follows from (31) and (32) that

$$\begin{array}{ll}\parallel \mathbf{h}{\parallel }_2& \le k^{1/2-1/q}\sqrt{\frac{(2-{\delta }_{2k})q{(2-q)}^{2/q-1}+2^{2/q}{\delta }_{2k}}{(1-{\delta }_{2k})2^{2/q}}}\\ \frac{2^{2/q-1}\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}}{{(1-{\left(f\right(t_0\left)\right)}^{q/2})}^{1/q}}\\ =\frac{C_1(q,{\delta }_{2k})\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}}{k^{1/q-1/2}}\end{array}$$

(33)

That is, the second inequality in (13) holds. With this, the proof of Theorem 1 is completed.

We further consider the recovery problem of any high-dimensional signals in the presence of noise. In this situation, the measurement can be expressed as

$$\mathbf{y}=\Phi \mathbf{x}+\mathbf{z},$$

where $\mathbf{\text{z}}\in {\mathbb{R}}^N$ is an unknown noise term. In order to reconstruct x, we adopt the unconstraint mixed l ₂/l _q  minimization scheme with 0 < q < 1:

$$\begin{array}{l}\begin{array}{l}\underset{\mathbf{x}}{\text{min}}\parallel \mathbf{x}{\parallel }_{2,q}^q\\ \text{s.t.}\parallel \mathbf{y}-\Phi \mathbf{x}{\parallel }_2\le ϵ,\end{array}\end{array}$$

(34)

where ϵ > 0 is a bound on the noisy level. We below show that one can also recover x stably and robustly under the same assumption as those in Theorem 1.

Theorem 2.

 (Noisy recovery). Let y = Φ x+z( ∥z∥₂ ≤ ϵ) be noisy measurements of a signal x. If the matrix Φ satisfies the block RIP (5) with

$${\delta }_{2k}<1/2,$$

then there exists a number q ₀(δ _2k ) ∈ (0,1] such that for any q < q ₀, the mixed l ₂/l _q  method (34) can stably and robustly recover the original signal x. More precisely, the solution x ^∗ of (34) obeys to

$$\parallel \mathbf{x}-{\mathbf{x}}^{\ast }{\parallel }_2\le \frac{C_1(q,{\delta }_{2k})\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}}{k^{1/q-1/2}}+C_2(q,{\delta }_{2k})ϵ$$

(35)

where C ₁(q,δ _2k ) and C ₂(q,δ _2k ) are positive constants dependent on q and δ _2k , T ₀ is the block index set over the k blocks with largest l ₂ norm of the original signal x.

Remark 2.

 The inequality (35) in Theorem 2 offers an upper bound estimation on the recovery of the mixed l ₂/l _q minimization ( q ∈ (0,q ₀)). In particular, this estimation shows that the recovery accuracy of the mixed l ₂/l _q  minimization can be controlled by the degree of sparsity of the signal and the exponential number q. It reveals also the close connections among the recovery precision of the mixed l ₂/l _q  minimization method may achieve, the sparsity of the signal and the index q used in the recovery procedure. In addition, the estimation (35) shows that the recovery precision of the method (34) can be bounded by the noise level. In this sense, Theorem 2 shows that under certain conditions, a block k-sparse signal can be robustly recovered by the method (34).

Proof of Theorem 2.

 The proof of Theorem 2 is similar to the procedure of the proof of Theorem 1 with minor differences.

To be more detail, we also set x ^∗ = x + h. Due to the existence of noise, in this case, h is not necessarily in the null space of Φ any more. But we can still prove Theorem 2 under the same assumption.

We still denote by T ₀ the block index set over the k blocks with largest l ₂ norm of x, and ${\mathbf{h}}_{T_0}$ the restriction of h onto these blocks. We also denote ${\mathbf{h}}_{T_j}(j\ge 1)$ similar to the proof of Theorem 1. By ∥z∥₂≤ ϵ and the triangle inequality, we first have

$$\parallel \Phi \mathbf{h}{\parallel }_2=\parallel \Phi (\mathbf{x}-{\mathbf{x}}^{\ast }){\parallel }_2\le \parallel \Phi \mathbf{x}-\mathbf{y}{\parallel }_2+\parallel \Phi {\mathbf{x}}^{\ast }-\mathbf{y}{\parallel }_2\le 2\mathrm{ϵ.}$$

(36)

Since $\Phi ({\mathbf{h}}_{T_0}+{\mathbf{h}}_{T_1})=\Phi \left(\mathbf{h}\right)-\Phi \left(\sum _{j\ge 2}{\mathbf{h}}_{T_j}\right)$, from definition of the block-RIP, we get

$$\begin{array}{l}\begin{array}{ll}(1-{\delta }_{2k})\parallel {\mathbf{h}}_{T_0\bigcup T_1}{\parallel }_2^2& =(1-{\delta }_{2k})(\parallel {\mathbf{h}}_{T_0}{\parallel }_2^2+\parallel {\mathbf{h}}_{T_1}{\parallel }_2^2)\\ \le \parallel \Phi \left({\mathbf{h}}_{T_0\bigcup T_1}\right){\parallel }_2^2\\ =\parallel \Phi \mathbf{h}-\sum _{j\ge 2}\Phi \left({\mathbf{h}}_{T_j}\right){\parallel }_2^2\\ \le {\left(\parallel \Phi \mathbf{h}{\parallel }_2+\parallel \sum _{j\ge 2}\Phi \left({\mathbf{h}}_{T_j}\right){\parallel }_2\right)}^2\\ \le {\left(2ϵ+\parallel \sum _{j\ge 2}\Phi \left({\mathbf{h}}_{T_j}\right){\parallel }_2\right)}^2\end{array}\end{array}$$

(37)

Hence,

$$\parallel {\mathbf{h}}_{T_0}{\parallel }_2^2+\parallel {\mathbf{h}}_{T_1}{\parallel }_2^2\le \frac{{\left(2ϵ+\parallel \sum _{j\ge 2}\Phi \left({\mathbf{h}}_{T_j}\right){\parallel }_2\right)}^2}{1-{\delta }_{2k}}$$

(38)

On the other hand,

$$\parallel {\mathbf{h}}_{T_0}{\parallel }_2^2+\parallel {\mathbf{h}}_{T_1}{\parallel }_2^2\ge k^{1-2/q}\parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}^2+k^{1-2/q}\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^2$$

(39)

From (38) and (39), we thus have

$$\begin{array}{l}\parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}^2\\ \le \frac{k^{2/q-1}\left(\right.2ϵ+\parallel \sum _{j\ge 2}\Phi \left({\mathbf{h}}_{T_j}\right){\parallel }_2{}^2-(1-{\delta }_{2k})\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^2}{1-{\delta }_{2k}}\\ \le \frac{{\left(2ϵ{\left(2k\right)}^{1/q-1/2}\right)}^2+4{\left(2k\right)}^{1/q-1/2}ϵ·{(k/2)}^{1/q-1/2}\parallel \sum _{j\ge 2}\Phi \left({\mathbf{h}}_{T_j}\right){\parallel }_2+k^{2/q-1}\parallel \sum _{j\ge 2}\Phi \left({\mathbf{h}}_{T_j}\right){\parallel }_2^2-(1-{\delta }_{2k})\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^2}{1-{\delta }_{2k}}\end{array}$$

(40)

Let $\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^q=t\sum _{i\ge 1}\parallel {\mathbf{h}}_{T_i}{\parallel }_{2,q}^q,t\in [0,1]$ and we also have (23), thus

$$\begin{array}{l}{(k/2)}^{1/q-1/2}\parallel \sum _{j\ge 2}\Phi \left({\mathbf{h}}_{T_j}\right){\parallel }_2\\ \le \sqrt{\frac{(1-t)t^{2/q-1}+{\delta }_{2k}}{2^{2/q-1}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}\end{array}$$

(41)

If we denote

$$g\left(t\right)\doteq \frac{(1-t)t^{2/q-1}+{\delta }_{2k}}{2^{2/q-1}},$$

then, by a easy calculation, we can easily obtain that the maximum of g(t) occurs at $t_1=\frac{2-q}{2}$ and

$$g\left(t_1\right)=\frac{{\delta }_{2k}+\frac{q}{2}{\left(\frac{2-q}{2}\right)}^{2/q-1}}{2^{2/q-1}}\le {\delta }_{2k}+\frac{q}{2^{2/q}}{\left(\frac{2-q}{2-{\delta }_{2k}}\right)}^{2/q-1}$$

Therefore, we have

$$\begin{array}{l}{(k/2)}^{1/q-1/2}\parallel \sum _{j\ge 2}\Phi \left({\mathbf{h}}_{T_j}\right){\parallel }_2\\ \le \sqrt{{\delta }_{2k}+\frac{q}{2^{2/q}}{\left(\frac{2-q}{2-{\delta }_{2k}}\right)}^{2/q-1}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}\end{array}$$

(42)

By (23) and the condition of f(t) in the proof of Theorem 1, we also have

$$\begin{array}{l}{k}^{2/q-1}\parallel \sum _{j\ge 2}\Phi \left({\mathbf{h}}_{T_j}\right){\parallel }_2^2-(1-{\delta }_{2k})\parallel {\mathbf{h}}_{T_1}{\parallel }_{2,q}^2\\ \le (1-{\delta }_{2k})f\left(t\right)\left(\right.\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q{}^{2/q}\\ \le (1-{\delta }_{2k})f\left(t_0\right)\left(\right.\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q{}^{2/q}\\ =\left(\right.{\delta }_{2k}+\frac{q}{2^{2/q}}\left(\right.\frac{2-q}{2-{\delta }_{2k}}{}^{2/q-1}\left)\right.\left(\right.\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q{}^{2/q}\end{array}$$

(43)

Plugging (42) and (43) into (40), it is easy to see that

$$\parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}\le \frac{2{\left(2k\right)}^{1/q-1/2}ϵ}{\sqrt{1-{\delta }_{2k}}}+\sqrt{f\left(t_0\right)}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}$$

Consequently, we obtain

$$\parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}^q\le \frac{{\left(2{\left(2k\right)}^{1/q-1/2}ϵ\right)}^q}{{(1-{\delta }_{2k})}^{q/2}}+{\left(f\right(t_0\left)\right)}^{q/2}\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)$$

(44)

In the following, we further prove that f(t ₀) < 1 one can obtain the conclusion of Theorem 2. Precisely, under the same condition on δ _2k , we can prove Theorem 2.

Note that from (27), we also have

$$\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\le \parallel {\mathbf{h}}_{T_0}{\parallel }_{2,q}^q+2\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}^q$$

Plugging (44) into the above inequality and by f(t ₀) < 1, one can show that

$$\begin{array}{ll}\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q& \le \frac{{\left(2{\left(2k\right)}^{1/q-1/2}ϵ\right)}^q}{{(1-{\delta }_{2k})}^{q/2}(1-{\left(f\right(t_0\left)\right)}^{q/2})}\\ +\frac{2\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}^q}{1-{\left(f\right(t_0\left)\right)}^{q/2}}\end{array}$$

Since ∥v∥ _q  ≤ 2^1/q-1∥v∥₁ for $\mathbf{v}\in {\mathbb{R}}^2$, we further have

$$\begin{array}{ll}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}& \le \frac{2^{2/q-1/2}{k}^{1/q-1/2}ϵ}{\sqrt{1-{\delta }_{2k}}{(1-{\left(f\right(t_0\left)\right)}^{q/2})}^{1/q}}\\ +\frac{2^{2/q-1}\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}}{{(1-{\left(f\right(t_0\left)\right)}^{q/2})}^{1/q}}\end{array}$$

(45)

and by (37), (22), and (23), we also have

$$\begin{array}{ll}\parallel \mathbf{h}{\parallel }_2^2& =\parallel {\mathbf{h}}_{T_0\bigcup T_1}{\parallel }_2^2+\parallel {\mathbf{h}}_{{(T_0\bigcup T_1)}^c}{\parallel }_2^2\le \frac{{(2ϵ+\parallel \sum _{j\ge 2}\Phi ({\mathbf{h}}_{T_j}\left){\parallel }_2\right)}^2}{1-{\delta }_{2k}}+\sum _{j\ge 2}\parallel {\mathbf{h}}_{T_j}{\parallel }_2^2\\ \le \frac{{\left(2ϵ+{(\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}+{\delta }_{2k}{k}^{1-2/q})}^{1/2}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}\right)}^2}{1-{\delta }_{2k}}+\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{2/q}\end{array}$$

(46)

since $\sqrt{{(a+\sqrt{b})}^2+c}\le a+\sqrt{b+c}(a,b,c>0)$, it gives

$$\begin{array}{ll}\parallel \mathbf{h}{\parallel }_2& \le \sqrt{\frac{{\left(2ϵ+{\left(\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}+{\delta }_{2k}{k}^{1-2/q}\right)}^{1/2}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}\right)}^2}{1-{\delta }_{2k}}+\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{2/q}}\\ \le \frac{2ϵ}{\sqrt{1-{\delta }_{2k}}}+\sqrt{\frac{\left(\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}+{\delta }_{2k}{k}^{1-2/q}\right){\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{2/q}}{1-{\delta }_{2k}}+\frac{1-t}{t^{1-2/q}{k}^{\frac{2-q}{q}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{2/q}}\\ =\frac{2ϵ}{\sqrt{1-{\delta }_{2k}}}+\sqrt{\frac{k^{1-2/q}\left(\frac{1-t}{t^{1-2/q}}+{\delta }_{2k}+(1-{\delta }_{2k})\frac{1-t}{t^{1-2/q}}\right)}{1-{\delta }_{2k}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}\\ =\frac{2ϵ}{\sqrt{1-{\delta }_{2k}}}+k^{1/2-1/q}\sqrt{\frac{(2-{\delta }_{2k})(1-t)t^{2/q-1}+{\delta }_{2k}}{1-{\delta }_{2k}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}\\ \le \frac{2ϵ}{\sqrt{1-{\delta }_{2k}}}+k^{1/2-1/q}\sqrt{\frac{(2-{\delta }_{2k})q{(2-q)}^{2/q-1}+2^{2/q}{\delta }_{2k}}{(1-{\delta }_{2k})2^{2/q}}}{\left(\sum _{j\ge 1}\parallel {\mathbf{h}}_{T_j}{\parallel }_{2,q}^q\right)}^{1/q}\end{array}$$

(47)

where we have used the fact that

$$\underset{t\in [0,1]}{\text{max}}(1-t)t^{2/q-1}\le (1-t)t^{2/q-1}{|}_{t=1-q/2}=\frac{q}{2}{\left(1-\frac{q}{2}\right)}^{2/q-1}$$

Thus, it then follows from (45) and (47) that

$$\begin{array}{l}\begin{array}{l}\parallel \mathbf{h}{\parallel }_2\\ \le \frac{2ϵ}{\sqrt{1-{\delta }_{2k}}}+k^{1/2-1/q}\sqrt{\frac{(2-{\delta }_{2k})q{(2-q)}^{2/q-1}+2^{2/q}{\delta }_{2k}}{(1-{\delta }_{2k})2^{2/q}}}\\ \times \left(\frac{2^{2/q-1/2}{k}^{1/q-1/2}ϵ}{\sqrt{1-{\delta }_{2k}}{(1-{\left(f\right(t_0\left)\right)}^{q/2})}^{1/q}}+\frac{2^{2/q-1}\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}}{{(1-{\left(f\right(t_0\left)\right)}^{q/2})}^{1/q}}\right)\\ =k^{1/2-1/q}\sqrt{\frac{(2-{\delta }_{2k})q{(2-q)}^{2/q-1}+2^{2/q}{\delta }_{2k}}{(1-{\delta }_{2k})}}\frac{2^{1/q-1}\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}}{{(1-{\left(f\right(t_0\left)\right)}^{q/2})}^{1/q}}\\ +\frac{2ϵ}{\sqrt{1-{\delta }_{2k}}}\left(1+k^{1/2-1/q}\sqrt{\frac{(2-{\delta }_{2k})q{(2-q)}^{2/q-1}+2^{2/q}{\delta }_{2k}}{(1-{\delta }_{2k})}}\frac{2^{1/q-3/2}}{{(1-{\left(f\right(t_0\left)\right)}^{q/2})}^{1/q}}\right)\\ =\frac{C_1(q,{\delta }_{2k})\parallel {\mathbf{x}}_{T_0^c}{\parallel }_{2,q}}{k^{1/q-1/2}}+C_2(q,{\delta }_{2k})ϵ\end{array}\end{array}$$

(48)

This arrives to the conclusion of Theorem 2.

6 An IRLS algorithm

Inspired by the ideas of [28, 33], in this section, we present an efficient IRLS algorithm for the solution of the mixed l ₂/l _q  norm minimization problem (34). We first rewrite the problem as the following regularized version of the unconstrained smoothed l ₂/l _q  minimization:

$$\underset{\mathbf{x}}{\text{min}}\parallel \mathbf{x}{\parallel }_{2,q}^{ϵ,q}+\frac{1}{2\tau }\parallel \mathbf{y}-\Phi \mathbf{x}{\parallel }_2^2,$$

(49)

where τ is an regularization parameter and

$$\parallel \mathbf{x}{\parallel }_{2,q}^{ϵ,q}=\sum _{i=1}^m{(\parallel \mathbf{x}[i]{\parallel }_2^2+{ϵ}^2)}^{q/2}.$$

Let J _2,q (x,ϵ,τ) be the objective function associated with (49), that is,

$$J_{2,q}(\mathbf{x},ϵ,\tau )=\sum _{i=1}^m{(\parallel \mathbf{x}[i]{\parallel }_2^2+{ϵ}^2)}^{q/2}+\frac{1}{2\tau }\parallel \mathbf{y}-\Phi \mathbf{x}{\parallel }_2^2.$$

(50)

Then, the first-order necessary optimality condition for the solution of x is

$${\left[\frac{q\mathbf{x}\left[i\right]}{{({ϵ}^2+\parallel \mathbf{x}[i\left]{\parallel }_2^2\right)}^{1-q/2}}\right]}_{1\le i\le m}+\frac{1}{\tau }{\Phi }^T(\Phi \mathbf{x}-\mathbf{y})=0.$$

Hence, we define the diagonal weighting matrix W as W _i  = d i a g(q ^1/2(ϵ ² + ∥ x[i] ∥ 22)^q/4-1/2) for i th block, and after simple calculations, we can obtain the following necessary optimality condition:

$$(\tau W^2+{\Phi }^T\Phi )\mathbf{x}={\Phi }^T\mathbf{y}.$$

(51)

Due to the nonlinearity, there is no straightforward method to solve the above system of equations. But if we fix W = W ^(t) to be that determined already in the t th iteration step, the solution of (51), set as the (t + 1)th iterate, then can be found to be

$$\begin{array}{ll}{\mathbf{x}}^{(t+1)}& ={\left(W^{\left(t\right)}\right)}^{-1}{\left(\Phi {\left(W^{\left(t\right)}\right)}^{-1}\right)}^T\\ \times {(\Phi {\left(W^{\left(t\right)}\right)}^{-1}+\tau \mathbf{I})}^{-1}{\left(\Phi {\left(W^{\left(t\right)}\right)}^{-1}\right)}^T\mathbf{y}.\end{array}$$

(52)

This defines a nature iterative method for solution of (49). We formalize such reweighted algorithm as the following:

Algorithm 1. An IRLS algorithm for the unconstrained smoothed l ₂ /l _q (0 < q < ≤ 1) minimization problem

•  Step 1. Initialize ${\mathbf{x}}^{\left(0\right)}=arg\underset{\mathbf{x}}{\text{min}}\parallel \mathbf{y}-\Phi \mathbf{x}\underset{2}{\overset{2}{\parallel }}$ and $\widehat{k}$ be the estimated block-sparsity. Set ϵ ₀ = 1,t = 0.

•  Step 2. while loop do

  $$\begin{array}{ll}{W}^{\left(t\right)}& \leftarrow \text{diag}\left(q^{1/2}{({ϵ}_t^2+\parallel {\mathbf{x}}^{\left(t\right)}[i\left]{\parallel }_2^2\right)}^{q/4-1/2}\right),\\ i=1,\dots ,m\\ A^{\left(t\right)}& \leftarrow \Phi {\left(W^{\left(t\right)}\right)}^{-1}\\ {\mathbf{x}}^{(t+1)}& \leftarrow {\left(W^{\left(t\right)}\right)}^{-1}{\left({\left(A^{\left(t\right)}\right)}^T\right(A^{\left(t\right)})+\tau \mathbf{I})}^{-1}{\left(A^{\left(t\right)}\right)}^T\mathbf{y}\\ {ϵ}_{t+1}& \leftarrow \text{min}\{{ϵ}_t,\mathrm{\alpha r}{\left({\mathbf{x}}^{(t+1)}\right)}_{\widehat{k}+1}/N\}\\ t& \leftarrow t+1\end{array}$$

•  until ϵ _t + 1 = 0 otherwise repeat the above computation within a reasonable time

•  Step 3. Output x ^(t + 1) to be an approximation solution.

In Algorithm Algorithm 1. An IRLS algorithm for the unconstrained smoothed l ₂ /l _q (0 < q < ≤ 1) minimization problem, $r{\left(\mathbf{x}\right)}_{\widehat{k}+1}$ is the $(\widehat{k}+1)$th largest l ₂ norm value of the block of x in the decreasing order, α ∈ (0,1) is a number such that α r(x ⁽¹⁾)/N < 1, and τ is an appropriately chosen parameter which controls the tolerance of noise term. Although the best τ may change continuously with respect to noise level, we used the fixed value of τ in our numerical implementations.

Obviously, from the update equation (52), we can see that x ^(t) can be understood as the minimizer of

$$\frac{1}{2\tau }\parallel \mathbf{y}-\Phi \mathbf{x}{\parallel }_2^2+\frac{1}{2}\parallel W^{\left(t\right)}\mathbf{x}{\parallel }_2^2.$$

Due to the iteratively fixing feature of W ^(t), this gives the name IRLS. Majumdar and Ward [12] first adopted the IRLS methodology to solve a non-convex l ₂/l _q  block-sparse optimization problem. But their algorithm is unsuitable for noisy block-sparse recovery. Lai et al. [28] proposed an IRLS algorithm for the unstrained l _q (0 < q ≤ 1) minimization problem and they made an detailed analysis which includes convergence, local convergence rate and error bound of the algorithm. To some extent, our proposed algorithms can be considered as a generalization of their algorithm to the setting of block-sparse recovery.

Note that {ϵ _t } in Algorithm Algorithm 1. An IRLS algorithm for the unconstrained smoothed l ₂ /l _q (0 < q < ≤ 1) minimization problem is a bounded non-increasing sequence, which must converge to some ϵ _∗ ≥ 0. Then using a similar argument as that in [28], we can prove that x ^(t) must have a convergent subsequence and the limit of the subsequence is a critical point of (50) whenever ϵ _∗ > 0. In addition, there exists a convergent subsequence whose limit x _∗ is a sparse vector with block-sparsity $\parallel {\mathbf{x}}_{\ast }{\parallel }_{2,0}\le \widehat{k}$ when ϵ _∗ = 0. Furthermore, we can also verify the super-linear local convergence rate for the proposed Algorithm Algorithm 1. An IRLS algorithm for the unconstrained smoothed l ₂ /l _q (0 < q < ≤ 1) minimization problem. Due to space limitations, we leave the detailed analysis to the interested reader.

7 Numerical experiments

In this section, we conduct two numerical experiments to compare the non-convex l ₂/l _q (0 < q < 1) minimization method with the l ₂/l ₁ minimization method and the standard l _q (0 < q ≤ 1) method in the context of block-sparse signal recovery. Note that this is possible because Algorithm Algorithm 1. An IRLS algorithm for the unconstrained smoothed l ₂ /l _q (0 < q < ≤ 1) minimization problem can apply to the standard l _q (0 < q ≤ 1) minimization method too. For all compared methods, we use the same starting point x ⁽⁰⁾ = arg minx $\parallel \mathbf{y}-\Phi \mathbf{x}{\parallel }_2^2$.

In our experiments, the measurement matrix Φ was generated by creating an M × N matrix with i.i.d draws from a standard Gaussian distribution N (0,1). We considered four different values of q = 0.1,0.5,0.7,1 for both the l ₂/l _q  minimization method and the l _q  minimization method. The purpose of experiments was to compare the recovery performance of the mixed l ₂/l _q  method and the l _q  method for block-sparse signals without noise and with noise respectively.

7.1 Noiseless recovery

In this set of experiments, we considered the case that the signals were perfectly measured without noise. We first randomly generated the block-sparse signal x with values chosen from a Gaussian distribution of mean 0 and standard deviation 1 and then randomly drew a measurement matrix Φ from Gaussian ensemble. Then we observed the measurements y from the model y = Φ x. In all the experiment cases, if ϵ _t+1 < 10^-7 or ∥x ^(t+1)-x ^(t) ∥ ₂ < 10^-8, the iteration terminates and outputs x ^(t+1) as an approximation solution of original signal x; otherwise, we let the algorithms run to the maximum number of iterations m a x = 2000. We also set parameters τ and α to 10^-5 and 0.7. We tested Algorithm Algorithm 1. An IRLS algorithm for the unconstrained smoothed l ₂ /l _q  (0 < q < ≤ 1) minimization problem for different initial block-sparsity estimates and did find that any overestimated $\widehat{k}$ of k would yield similar results. A typical simulation result is shown in Figure 1. Therefore, for simplicity, we set $\widehat{k}=k+1$ in our implementation.

Figure 1

A typical simulation result with different estimated block-sparsity. The randomly generated signals have length N = 256, m = 64 uneven size blocks, and k = 9 active blocks with total sparsity k ₀ = 22. We observed m = 64 measurements and the average RMSE over 100 independent random trails in the logarithmic scale is shown.

Figure 2a depicts an instance of the generated block-sparse signal with signal length N = 512. There are 128 blocks with uneven block size and 16 active blocks with the sparsity: k ₀ = ∥x ∥₀ = 101. Figure 2b–d shows that the recovery results by the standard l _q  method with q = 1, the standard l _q  method with q = 0.5, and the mixed l ₂/l ₁ method, respectively, when M = 225. Since the sample size M is only around 2.2 times the signal sparsity k ₀, the standard l ₁ method does not yield good recovery results, whereas the mixed l ₂/l ₁ method and the non-convex l _q (q = 0.5) method achieve near perfect recovery of the original signal. The results illustrate that if one incorporates the block-sparsity structure into the recovery procedure, the block version of convex l ₁ minimization does also reduce the number of measurement as the standard non-convex l _q  minimization with some q < 1.

Figure 2

Recovery results with  M = 225 . (a) Original signal; (b) results with l ₁ minimization (RMSE = 0.1301); (c) results with l _q (q = 0.5) minimization (RMSE = 0.000061); (d) results with l ₂/l ₁ minimization(RMSE = 0.000033).

We further compared the recovery performance of the standard l _q  method and the mixed l ₂/l _q  method for different values of q. Figure 3 shows a similar instance as Figure 2. We generated a block-sparse signal with the same non-zero block locations as in Figure 2a, and we observed M = 144 measurements that is only around 1.4 times the signal sparsity k ₀. From Figure 3, we can see that only the non-convex l ₂/l _q  method with q = 0.5,0.1 achieve near optimal recovery results while other methods fail. The results illustrate that for any q ≤ 0.5, the mixed l ₂/l _q  method can exactly recover the original signal. In addition, the results also demonstrate the outperformance of the non-convex mixed l ₂/l _q (0 < q < 1) method over the standard non-convex l _q (0 < q < 1) method.

Figure 3

Recovery results with  M = 144 . (a) Original signal; (b) results with l _q (q = 0.7) minimization (RMSE = 0.3589); (c) results with l _q (q = 0.5) minimization (RMSE = 0.3857); (d) results with l _q (q = 0.1) minimization (RMSE = 0.4124); (e) results with l ₂/l ₁ minimization (RMSE = 0.1688); (f) results with l ₂/l _q (q = 0.7) minimization (RMSE = 0.1164); (g) results with l ₂/l _q (q = 0.5) minimization (RMSE = 4.21 × 10^-7); (h) results with l ₂/l _q (q = 0.1) minimization (RMSE = 8.23 × 10^-8).

Figure 4a shows the effect of sample size, where we report the average root mean squares error (RMSE) over 100 independent random trails in the logarithmic scale for each sample size. In this case, we set signal length N = 256, and there are 64 blocks with uneven block size and the k = 4 active blocks were randomly extracted from the 64 blocks. The figure indicates the decay in recovery error as a function of sample size for all the algorithms. We can observe that both the l _q and the mixed l ₂/l _q  methods improve the recovery performance as q decreases, and for a fixed q, the mixed l ₂/l _q  method is clearly superior to the standard l _q  method in this block-sparse setting. To further study the effect of the active block number k (with k ₀ fixed), we drew a matrix Φ of size 128×256 from Gaussian ensemble. We also set the signal x with the even block size and the total sparsity k ₀ = 64. The block size was changed while keeping other parameters unchanged. Figure 4b shows the average RMSE over 100 independent random runs in the logarithmic scale. One can easily see that the recovery performance for the standard l _q  method is independent to the active block number, while the recovery errors for the mixed l ₂/l _q  method are significantly better when the active block number k is far smaller than the total signal sparsity k ₀. As expected, the performance of the mixed l ₂/l _q  method becomes identical to the standard l _q  method when k = k ₀. This illustrates that the mixed method favors large sized block when the total sparsity k ₀ is fixed. Moreover, similar to the standard l _q  method, the mixed l ₂/l _q  method performs better and better as q decreases.

Figure 4

Recovery performance: (a) average RMSE (log-scale) versus sample size ratio  M/k ₀ ; (b) average RMSE (log-scale) versus active block number k .

7.2 Noisy recovery

In this experiment, we considered the case of recovering the block-sparse signals in the presence of noise. We observed the measurements y from the model y = Φ x+z, here Φ and x were generated as the last subsection and z was zero-mean Gaussian noise with standard deviation σ. In our implementation of this experiment, we set τ = 10^-1 max|Φ ^T y| and kept other parameters unchanged.

Table 1 lists the comparison results of the relative errors of the true solutions and the approximate solutions yielded, respectively, by the mixed l ₂/l _q  method and the l _q  method with active block number k varying in {4,12}, sample size r = M/k ₀ in {3,4}, and the noise level σ in {0.02,0.05,0.10}. Here, the relative errors are defined as ∥x-x ^∗∥₂/∥x∥₂. It is reported in Table 1 that the average relative errors and the standard deviations over 100 random trails. From the table, it is seen that, for a fixed q, the mixed l ₂/l _q  method always get better results than the standard l _q  method. And in the low-noisy cases (say, σ = 0.02,0.05), as q decreases, the mixed l ₂/l _q  method improves the recovery performance. However, when σ=0.10, the mixed l ₂/l _q  method is not always able to improve the recovery results when q ≤ 0.7. Thus, we may reasonably infer that there exits a q ₀ ≤ 0.1 such that for any q < q ₀, all the l ₂/l _q minimization can obtain similar recovery results when the noise level is low ( σ = 0.02,0.05); while there exits a q ₀ ≤ 0.7 such that as q < q ₀ decreases, the mixed l ₂/l _q  method is unable to improve the recovery results when σ = 0.10.

Table 1 Comparison results ( ∥x-x ^∗ ∥ _2/ ∥x∥ ₂ ) of randomly generated signals with length N = 256 and m = 64 uneven size blocks by mixed  l ₂ / l _q method and  l _q  method with four different values of q and two different sampling sizes r = M/k ₀

8 Conclusion

In this article, we have investigated the block-sparse recovery performance of the mixed l ₂/l _q  minimization approach, especially for the non-convex case of 0 < q < 1. Under the assumption that the measurement matrix Φ has the RIP with δ _2k  < 1/2, we have proved that the non-convex l ₂/l _q (0 < q < 1) method can exactly and stably recover original block-sparse signals in noiseless case and noisy case, respectively. The sufficient recovery condition we obtained is weaker than those of l ₂/l ₁ method ( δ _2k  < 0.414), which implies the better block-sparse recovery ability of the mixed l ₂/l _q (0 < q < 1) method. We have conducted a series of numerical experiments to support the correctness of the theory and the outperformance of the mixed method.

Our study so far is only concerned with the block-sparse signal recovery without overlapping blocks. While in many real applications, such as the gene expression data in bioinformatics, the blocks of elements could potentially be overlapped. Rao et al. [36] derived some tight bounds for the number of measurements required for exact and stable recovery of block-sparse signals with overlapping blocks by the mixed l ₂/l ₁ minimization method. Their analysis can naturally be extended to the non-convex l ₂/l _q (0 < q < 1) method. All these extensions will be part of our future research.

Although our simulation studies in this article demonstrate clearly that the recovery ability of the mixed l ₂/l _q  method would be better and better as q decreases, there is still lack of further theoretical analysis to support for such observation. The works of [37, 38] addressed the block-sparse recovery ability by an accurate analysis of the breakdown behavior of the mixed l ₂/l ₁ method. One could then have an interest in extending these results to the case of the mixed l ₂/l _q (0 < q < 1) method as well. It is noted that since the resultant mixed minimization problem is non-convex, it seems very difficult to make such similar theoretical analysis and perhaps we need more powerful tools of geometric functional analysis for the extension.