Adaptive matching pursuit with constrained total least squares

Abstract

Compressive sensing (CS) can effectively recover a signal when it is sparse in some discrete atoms. However, in some applications, signals are sparse in a continuous parameter space, e.g., frequency space, rather than discrete atoms. Usually, we divide the continuous parameter into finite discrete grid points and build a dictionary from these grid points. However, the actual targets may not exactly lie on the grid points no matter how densely the parameter is grided, which introduces mismatch between the predefined dictionary and the actual one. In this article, a novel method, namely adaptive matching pursuit with constrained total least squares (AMP-CTLS), is proposed to find actual atoms even if they are not included in the initial dictionary. In AMP-CTLS, the grid and the dictionary are adaptively updated to better agree with measurements. The convergence of the algorithm is discussed, and numerical experiments demonstrate the advantages of AMP-CTLS.

1 Introduction

A new class of techniques called compressed sampling or compressive sensing (CS) has been widely used recently, due to the fact that CS techniques have shown good performance in different areas such as signal processing, communication and statistics; see, e.g., [1]. Generally, CS finds the sparsest vector x from measurements y = Φx, where Φ is often referred to as dictionary with more columns than rows, and each column of the dictionary is called an atom or a basis.

Matching pursuit (MP) is a set of popular greedy approaches to compressive sensing. The basic idea is to sequentially find the support set of x and then project on the selected atoms. The atoms selected in the support set are mainly determined by correlations between atoms and the regularized measurements [2]. MP methods include standard MP [3], and several other examples, such as orthogonal matching pursuit (OMP) [4], regularized OMP (ROMP) [5], stage-wise OMP (StOMP) [6], compressive sampling matching pursuit (CoSaMP) [7] and subspace pursuit (SP) [2].

These MP methods [2–7] do not consider the off-grid problem in grid-based CS approaches. In some applications of CS, such as harmonic retrieval and radar signal processing (e.g., range profiling [8, 9], direction of arrival estimate [10–12]), we usually divide a continuous parameter space into discrete grid points to generate the dictionary. For example, in harmonic retrieval, frequency space is divided and dictionary is a discrete Fourier transform (DFT) matrix. The off-grid problem emerges when the actual frequencies are placed off the predefined grid. The mismatch between the predefined and actual atom can lead to performance degradation in sparse recovery (e.g., [13–15]).

The grid misalignment problem in CS has recently received growing interest. The sensitivity of CS to the mismatch between the predefined and actual atoms is studied in [13]; however, the focus of that article is mainly on mismatch analysis rather than development of an algorithm. Cabrera et al. [16] and Zhu et al. [14], respectively, provided an iterative re-weighted (IRW)-based and a Lasso-based method to recover an unknown vector considering the atom misalignment, whereas we focus on MP methods in this article. Compared with IRW or Lasso, MP methods greedily find the support set and greatly reduce the dimension of the CS problem; thus, they have an advantage in computability. Gabriel [17] proposed best basis compressive sensing in a tree-structured dictionary, but some dictionaries (e.g., DFT matrix) do not possess a tree structure.

To alleviate the off-grid problem in matching pursuit, we developed adaptive matching pursuit with constrained total least squares (AMP-CTLS). In AMP-CTLS, we model the grid as an unknown parameter, and adaptively search for the best one. We choose harmonic retrieval to demonstrate the performance of AMP-CTLS. The algorithm can also be applied to jointly estimate range and velocity in randomized step frequency (RSF) radar. Note that in the RSF scenario range-velocity estimation is hard to be directly solved by subspace-based methods, e.g., Capon's method, MUSIC and ESPRIT [18]. Since only one snapshot data is available in RSF radar, to obtain the covariance matrix these subspace-based methods need to apply smoothing method, which requires uniform and linear condition [18]. However, this condition is not satisfied in the case of random frequency model.

This article is structured as follows. Section 2 introduces grid-based CS and outlines the procedures of AMP-CTLS. In Sections 3 and 4, we discuss the implementation of AMP-CTLS in harmonic retrieval and RSF radar, respectively. In Section 5, numerical examples are presented to illustrate merits of AMP-CTLS. Section 6 is dedicated to a brief conclusion. Notations: (·)^H denotes conjugate transpose matrix; (·)^T transpose matrix; (·)* conjugate matrix; (·)^† pseudo-inverse matrix; I _L /0 _L the L × L identity/zero matrix; || · ||₂ the ℓ₂ norm; {·} denotes a set; | · | the absolute value of a complex number or the cardinality of a set; (·)_Λ denotes elements/columns indexed in the set Λ of a vector/matrix; supp(·) is the support set of a vector, that is, the indices of the nonzero elements in the vector; Re(·) the real part of a complex number; ⊗ denotes the right Kronecker product [19]; and E[·] denotes the expectation of a random variable.

2 Grid-based CS and the AMP-CTLS algorithm

The signal model of grid-based CS is introduced in Section 2.1. We combine the greedy idea of MP methods and the constrained total least squares (CTLS) technique [20], and thus produce AMP-CTLS to alleviate the off-grid problem. In AMP-CTLS, the grid is cast as an unknown parameter, and is jointly estimated together with x. In Section 2.2, the framework of AMP-CTLS is given. Section 2.3 is dedicated to the iterative joint estimator (IJE) algorithm, which is implemented in AMP-CTLS. In the IJE algorithm, the CTLS technique is used, which is presented in Section 2.4. Section 2.5 summarizes the entire procedure of AMP-CTLS. In Section 2.6, the convergence of IJE is analyzed.

2.1 Grid-based CS

CS promises efficient recovery of sparse signals. In many applications, signals are sparse in a continuous parameter space rather than finite discrete atoms. Usually, we divide the continuous parameter into discrete grid points and cast the problem as a grid-based CS model:

\mathbf{y}=\mathbf{\Phi }\left(\mathbf{g}\right)\mathbf{x}+\mathbf{w},

(1)

where y ∈ ℂ^{M × 1}and w ∈ ℂ^{M × 1}are measurement vector and white Gaussian noise (WGN) vector, respectively. x ∈ ℂ^{N × 1}is to be learned. g ∈ ℂ^{N × 1}are discrete grid points g = [g₁, g₂, . . . , g _N ]. Φ(g) ∈ ℂ ^{M × N} is built from g, Φ(g) = [ϕ(g₁), ϕ(g₂), . . . , ϕ(g _N )], and the mapping g → Φ is known. For example, to recover a frequency sparse signal, we grid the frequency space into discrete frequency points \mathbf{g}={\left[0,\frac{1}{N},\frac{2}{N},\dots ,\frac{N-1}{N}\right]}^{\mathsf{\text{T}}}. Φ is a DFT matrix, of which the m th-row, n th-column element is exp \left(j2\pi \frac{n}{N}m\right). However, the signal is only sparse in the DFT atoms if all of the sinusoids are exactly at the pre-defined grid points [13]. In some cases, no matter how densely we grid the frequency space, the sinusoids could be off-grid, which saps the performance of CS methods [13].

2.2 Main idea of AMP-CTLS

The off-grid problem usually emerges because we do not often have enough priori knowledge to generate a perfect grid to guarantee that all of the signals exactly lie on grid points. Thus, we cast the grid as an unknown parameter, and search for the best grid g as well as the sparsest x by solving the optimum problem:

\widehat{\mathbf{x}},\widehat{\mathbf{g}}=\underset{\mathbf{x}\mathsf{\text{,}}\mathbf{g}}{\mathsf{\text{arg }}\mathsf{\text{min }}}\left|\right|\mathbf{x}|{|}_0\mathsf{\text{, s}}.\mathsf{\text{t}}.\left|\right|\mathbf{y}-\mathbf{\Phi }\left(\mathbf{g}\right)\mathbf{x}|{|}_2^2\le \eta ,

(2)

where η is the noise power. Equation (2) is similar to that used in traditional MP methods [2–7], except that we recover x and simultaneously estimate the grid. In most cases, solving (2) is a complex non-linear optimum problem. In this article, an iterative method is introduced.

AMP-CTLS inherits the greedy idea from MP methods, which use correlations to iteratively find the support set. In each iteration, one or more atoms are added into the support set. Suppose the support set is obtained as Λ^(k)after the k th iteration, and denote the corresponding grid points as {\widehat{\mathbf{g}}}_{\text{Λ}}^{\left(k\right)}. In traditional MP methods [2–7], x_Λ is estimated by solving a least squares problem. In AMP-CTLS, considering the off-grid problem, we jointly search for x_Λ and the best grid points in the neighboring continuous region of {\widehat{\mathbf{g}}}_{\text{Λ}}^{\left(k\right)} via (3), in which we minimize norm of the residual error, which is defined as r = y - Φ (g_Λ) x_Λ.

{\widehat{\mathbf{x}}}_{\mathbf{\Lambda }}^{\left(k+1\right)}\mathsf{\text{,}}{\widehat{\mathbf{g}}}_{\mathbf{\Lambda }}^{\left(k+1\right)}=\underset{{\mathbf{x}}_{\mathbf{\Lambda }},{\mathbf{g}}_{\mathbf{\Lambda }}}{\mathsf{\text{arg }}\mathsf{\text{min }}}{\left|\right|\mathbf{y}-\mathbf{\Phi }\left({\mathbf{g}}_{\mathbf{\Lambda }}\right){\mathbf{x}}_{\mathbf{\Lambda }}\left|\right|}_2^2

(3)

We develop the iterative joint estimator (IJE) algorithm to solve (3), which is detailed in ensuing section.

2.3 IJE algorithm

It is difficult to find an analytical solution to (3). The IJE algorithm is devised to seek a numerical solution. Given initial grid points {\widehat{\mathbf{g}}}_{\text{Λ}}\left(0\right), IJE searches for the best grid points g_Λ in the neighborhood of {\widehat{\mathbf{g}}}_{\text{Λ}}\left(0\right). The mismatch of the grid is denoted as \text{Δ}{\mathbf{g}}_{\text{Λ}}={\mathbf{g}}_{\text{Λ}}-{\widehat{\mathbf{g}}}_{\text{Λ}}\left(0\right)={\left[\text{Δ}{g}_1,...,\text{Δ}{g}_{\left|\text{Λ}\right|}\right]}^{\mathsf{\text{T}}}. IJE includes three steps: calculate the estimation of the mismatch, {\widehat{\Delta \mathbf{g}}}_{\Lambda }; update the grid with {\widehat{\Delta \mathbf{g}}}_{\Lambda }; and estimate x_Λ with projection onto the new grid points. These three steps are executed iteratively to pursue more accurate results. To distinguish from iterations in search for the support set in (3), we denote l as the counter of loops in IJE; thus, IJE is expressed as follows:

{\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right),{\mathbf{x}}_{\mathsf{\text{CTLS}}}=\underset{\text{Δ}{\mathbf{g}}_{\text{Λ}},{\mathbf{x}}_{\text{Λ}}}{\mathsf{\text{arg }}\mathsf{\text{min }}}{C}_{\mathsf{\text{CTLS}}},

(4)

{\widehat{\mathbf{g}}}_{\text{Λ}}\left(l+1\right)={\widehat{\mathbf{g}}}_{\text{Λ}}\left(l\right)+{\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right),

(5)

{\widehat{\mathbf{x}}}_{\text{Λ}}\left(l+1\right)=\underset{{\mathbf{x}}_{\text{Λ}}}{\mathsf{\text{arg }}\mathsf{\text{min}}}{∥\mathbf{y}-\mathbf{\Phi }\left({\widehat{\mathbf{g}}}_{\text{Λ}}\left(l+1\right)){\mathbf{x}}_{\text{Λ}}\right)∥}_2^2.

(6)

In (4), CTLS technique is applied to simultaneously search for the mismatch Δg_Λ and x _Λ , and {\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right) and x_CTLS are the results. C_CTLS denotes the penalty function of CTLS, which is detailed in Section 2.4. Since (6) is a linear least squares problem, the closed-form solution is

{\widehat{\mathbf{x}}}_{\text{Λ}}\left(l+1\right)={\left(\mathbf{\Phi }\left({\widehat{\mathsf{\text{g}}}}_{\text{Λ}}\left(l+1\right)\right)\right)}^{†}\mathbf{y}={\left({\mathbf{\Phi }}^{\mathsf{\text{H}}}\mathbf{\Phi }\right)}^{-1}{\mathbf{\Phi }}^{\mathsf{\text{H}}}\mathbf{y}.

(7)

The loops are terminated when the norm of residual error is scarcely reduced.

2.4 CTLS technique

Traditional MP methods [2–7] apply least squares to calculate amplitudes of x_Λ after finding the support set. When there are off-grid signals, mismatches occur in the dictionary; thus, we replace the least squares model with total least squares (TLS) criterion, which is appropriate to deal with the fitting problem when perturbations exist in both the measurement vector and in the dictionary [21]. Since the dictionary mismatches are constrained by errors of grid points, we introduce the constrained total least squares (CTLS) technique [20] in AMP-CTLS to jointly estimate the grid point errors and x_Λ, i.e., solving (4). It has been proved that CTLS is a constrained space state maximum likelihood estimator [20].

Suppose that we obtain the estimate of grid points as {\widehat{\mathbf{g}}}_{\text{Λ}}\left(l\right) after l th IJE iteration. Assume that the mismatch Δg_Λ is significantly small; thus we can approximate the perfect dictionary Φ(g _Λ ) as a linear combination of the mismatch Δg with Taylor expansion:

\mathbf{\Phi }\left({\mathbf{g}}_{\text{Λ}}\right)=\mathbf{\Phi }\left({\widehat{\mathbf{g}}}_{\text{Λ}}\left(l\right)\right)+\sum _{i=1}^{\left|\text{Λ}\right|}{\text{R}}_i\left({\widehat{\mathsf{\text{g}}}}_{\text{Λ}}\left(l\right)\right)\text{Δ}{g}_i+\sum _{i=1}^{\left|\text{Λ}\right|}o\left(\text{Δ}{\mathbf{g}}_i^2\right),

(8)

where R _i ∈ ℂ^{M ×|Λ|}is

{\mathbf{R}}_i\left({\widehat{\mathbf{g}}}_{\text{Λ}}\left(l\right)\right)={\left.\frac{\partial \mathbf{\Phi }\left({\mathbf{g}}_{\text{Λ}}\right)}{\partial g_i}\right|}_{{\mathbf{g}}_{\text{Λ}}={\widehat{\mathbf{g}}}_{\text{Λ}}\left(l\right)}

(9)

and o(·) denotes higher order terms. For simplicity, in this section we ignore the iteration counter in the notations, and {\mathbf{R}}_i\left({ĝ}_{\text{Λ}}\left(l\right)\right),\mathbf{\Phi }\left({ĝ}_{\text{Λ}}\left(l\right)\right) are, respectively, simplified as R _i , Φ _Λ . Neglect o\left(\text{Δ}{g}_i^2\right) and the signal model in (1) is replaced by:

\mathbf{y}=\left({\mathbf{\Phi }}_{\text{Λ}}+\sum _{i=1}^{\left|\text{Λ}\right|}{\mathbf{R}}_i\text{Δ}{g}_i\right){\mathbf{x}}_{\text{Λ}}+\mathbf{w}.

(10)

CTLS models Δg _Λ as an unknown random perturbation vector. The grid misalignment and the noise vector are combined into a (M +| Λ|)-dimensional vector \mathbf{v}={\left[{\left(\text{Δ}{\mathbf{g}}_{\text{Λ}}\right)}^{\mathsf{\text{T}}},{\mathbf{w}}^{\mathsf{\text{T}}}\right]}^{\mathsf{\text{T}}}, and CTLS aims at minimizing \left|\right|\mathbf{v}|{|}_2^2. It has been proved that CTLS is a constrained space state maximum likelihood estimator if v is a WGN vector [20]. Thus, we first whiten v. Assume that Δg _Λ is independent of w. The covariance matrix of Δg _Λ is {\mathbf{C}}_{\mathbf{g}}=E\left[\text{Δ}{\mathbf{g}}_{\text{Λ}}{\left(\text{Δ}{\mathbf{g}}_{\text{Λ}}\right)}^{\mathsf{\text{H}}}\right]\in {ℂ}^{\left|\text{Λ}\right|\times \left|\text{Λ}\right|}. D ∈ ℂ^|Λ|×|Λ| obeys {\mathbf{C}}_{\mathbf{g}}^{-1}={\mathbf{D}}^{\mathsf{\text{H}}}\mathbf{D}. The variance of white noise w is {\sigma }_{\mathbf{w}}^2. We denote an unknown normalized vector u ∈ ℂ^{(M+| Λ|) × 1}as (11); thus, u is a WGN vector.

\mathbf{u}=\left[\begin{array}{c}\hfill \mathbf{D}\text{Δ}{\mathbf{g}}_{\text{Λ}}\hfill \\ \hfill \frac{1}{{\sigma }_{\mathbf{w}}}\mathbf{w}\hfill \end{array}\right]

(11)

Minimize the penalty function C_{\mathsf{\text{CTLS}}}=\left|\right|\mathbf{u}{\left|\right|}_2^2 and (4) is detailed as follows:

\widehat{\mathbf{u}},{\mathbf{x}}_{\mathsf{\text{CTLS}}}=\underset{\mathbf{u},{\mathbf{x}}_{\text{Λ}}}{\mathsf{\text{arg }}\mathsf{\text{min}}}{\left|\right|\mathbf{u}\left|\right|}_2^2,

(12)

\mathsf{\text{s}}.\mathsf{\text{t}}.-\mathbf{y}+\left({\mathbf{\Phi }}_{\text{Λ}}+\sum _i^{\left|\text{Λ}\right|}{\mathbf{R}}_i\text{Δ}{g}_i\right){\mathbf{x}}_{\text{Λ}}+\mathbf{w}=0.

(13)

The constraint condition (13) can be rewritten as:

\mathsf{\text{s}}.\mathsf{\text{t}}.-\mathbf{y}+{\mathbf{\Phi }}_{\text{Λ}}{\mathbf{x}}_{\text{Λ}}+{\mathbf{W}}_{\mathbf{x}}\mathbf{u}=0,

(14)

where W _x = [H σ _w I _M ] ∈ ℂ^{M × (| Λ|+M)}. H ∈ ℂ^{M ×| Λ|}is defined as

H=\mathbf{G}\left({\mathbf{D}}^{-1}\otimes {\mathbf{I}}_{\left|\text{Λ}\right|}\right)\left({\mathbf{I}}_{\left|\text{Λ}\right|}\otimes {\mathbf{x}}_{\text{Λ}}\right),

(15)

where \text{G}=[{\text{R}}_1,\dots ,{\text{R}}_{|\Lambda |}]\in {ℂ}^{M\times {|\Lambda |}^2}. The equivalence between (13) and (14) is proved as follows:

\begin{array}{cc}\hfill \left(\sum _{i=1}^{\left|\text{Λ}\right|}{\mathbf{R}}_i\text{Δ}{g}_i\right){\mathbf{x}}_{\text{Λ}}& =\mathbf{G}\left(\text{Δ}{\mathbf{g}}_{\text{Λ}}\otimes {\mathbf{I}}_{\left|\text{Λ}\right|}\right){\mathbf{x}}_{\text{Λ}}\hfill \\ =\mathbf{G}\left({\mathbf{D}}^{-1}\otimes {\mathbf{I}}_{\left|\text{Λ}\right|}\right)\left(\mathbf{D}\text{Δ}{\mathbf{g}}_{\text{Λ}}\otimes {\mathbf{I}}_{\left|\text{Λ}\right|}\right){\mathbf{x}}_{\text{Λ}}\hfill \\ =\mathbf{G}\left({\mathbf{D}}^{-1}\otimes {\mathbf{I}}_{\left|\text{Λ}\right|}\right)\left({\mathbf{I}}_{\left|\text{Λ}\right|}\otimes {\mathbf{x}}_{\text{Λ}}\right)\mathbf{D}\text{Δ}{\mathbf{g}}_{\text{Λ}}\hfill \\ =\mathbf{H}\mathbf{D}\text{Δ}{\mathbf{g}}_{\text{Λ}}.\hfill \end{array}

(16)

When W _x is of full-row rank, the optimum problem (12, 14) are equivalent to (17)-(19), which has been proved in [20].

{\mathbf{x}}_{\mathsf{\text{CTLS}}}=\underset{{\mathbf{x}}_{\mathbf{\Lambda }}}{\mathsf{\text{min}}}{\left|\right|{\mathbf{W}}_{\mathbf{x}}^{†}\left(\mathbf{y}-{\mathbf{\Phi }}_{\mathbf{\Lambda }}{\mathbf{x}}_{\mathbf{\Lambda }}\right)\left|\right|}_2^2

(17)

\widehat{\mathbf{u}}={\mathbf{w}}_{\mathbf{x}}^{†}\left(\mathbf{y}-{\mathbf{\Phi }}_{\text{Λ}}{\mathbf{x}}_{\text{Λ}}\right){|}_{\mathbf{x}={\mathbf{x}}_{\mathsf{\text{CTLS}}}}

(18)

{\mathbf{w}}_{\mathbf{x}}^{†}={\mathbf{w}}_{\mathbf{x}}^{\mathsf{\text{H}}}{\left({\mathbf{w}}_{\mathbf{x}}{\mathbf{w}}_{\mathbf{x}}^{\mathsf{\text{H}}}\right)}^{-1}

(19)

It is quite difficult to obtain analytical solution to (17). A complex version of Newton method is developed in [20], which is presented in Appendix 1. Initial value of x _Λ required in Newton's method for (17) can be given as:

{\mathbf{x}}_{\mathsf{\text{ini}}}={\mathbf{\Phi }}_{\text{Λ}}^{†}\mathbf{y}={\left({\mathbf{\Phi }}_{\text{Λ}}^{\mathsf{\text{H}}}{\mathbf{\Phi }}_{\text{Λ}}\right)}^{-1}{\mathbf{\Phi }}_{\text{Λ}}^{\mathsf{\text{H}}}\mathbf{y}.

(20)

{\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}} is extracted from û via {\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}=\left[{\mathbf{D}}^{-1}{0}_N\right]\widehat{\mathbf{u}}, thus (4) is solved. The sketch of CTLS is given in Algorithm 1. As the authors' best knowledge, the convergence guarantees for this Newton method are still open question.

2.5 Sketch of AMP-CTLS

Similarly to traditional MP methods [2–7], AMP-CTLS first greedily finds the support set. Then AMP-CTLS adaptively optimizes the grid points indexed in the support set. In this article, we imitate the greedy approach of OMP, in which only one atom is added to the support set in each iteration. If the number of atoms is known, terminate the iterations when the cardinality of the support set reaches the pre-specified number. If it is not known, we can apply some other successfully used stopping criterions, e.g., norm of residual being below a threshold [22]. A sketch of AMP-CTLS is presented in Algorithm 2.

2.6 Convergence of the IJE algorithm

Here, we analyze convergence of the IJE algorithm. Assume that the mapping g_Λ→ Φ (g_Λ) is linear, which means

\mathbf{\Phi }\left({\mathbf{g}}_{\text{Λ}}+\text{Δ}{\mathbf{g}}_{\text{Λ}}\right)=\mathbf{\Phi }\left({\mathbf{g}}_{\text{Λ}}\right)+\mathbf{g}\left(\text{Δ}{\mathbf{g}}_{\text{Λ}}\otimes {\mathbf{I}}_{\left|\text{Λ}\right|}\right),

(21)

and G should be a constant matrix.

Proposition. If the measurement y is perturbed by WGN and (21) is obeyed, IJE monotonically reduces values of the penalty function in (3). The estimates of x _Λ and g _Λ satisfy:

{\left|\right|\mathbf{y}-\mathbf{\Phi }\left({\widehat{\mathbf{g}}}_{\text{Λ}}\left(l\right)\right){\widehat{\mathbf{x}}}_{\text{Λ}}\left(l\right)\left|\right|}_2^2\ge {\left|\right|\mathbf{y}-\mathbf{\Phi }\left({\widehat{\mathbf{g}}}_{\text{Λ}}\left(l+1\right)\right){\widehat{\mathbf{x}}}_{\text{Λ}}\left(l+1\right)\left|\right|}_2^2.

(22)

Proof. Define a penalty function as follows:

f_p\left(\text{Δ}{\mathbf{g}}_{\text{Λ}},{\mathbf{x}}_{\text{Λ}}\right)={\sigma }_{\mathbf{w}}^2{\left|\right|\mathbf{u}\left|\right|}_2^2={\sigma }_{\mathbf{w}}^2{\left(\text{Δ}{\mathbf{g}}_{\text{Λ}}\right)}^{\mathsf{\text{H}}}{\mathbf{C}}_{\mathbf{g}}^{-1}\text{Δ}{\mathbf{g}}_{\text{Λ}}+{\left|\right|\mathbf{y}-\mathbf{\Phi }\left({\widehat{\mathbf{g}}}_{\text{Λ}}\left(l\right)\right){\mathbf{x}}_{\text{Λ}}-\mathbf{G}\left(\text{Δ}{\mathbf{g}}_{\text{Λ}}\otimes {\mathbf{I}}_{\left|\text{Λ}\right|}\right){\mathbf{x}}_{\text{Λ}}\left|\right|}_2^2;

(23)

thus, {\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right) and x_CTLS are obtained by solving

{\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right),{\mathbf{x}}_{\mathsf{\text{CTLS}}}=\underset{\text{Δ}{\mathbf{g}}_{\text{Λ}},{\mathbf{x}}_{\text{Λ}}}{\mathsf{\text{arg }}\mathsf{\text{min}}}{f}_p\left(\text{Δ}{\mathbf{g}}_{\text{Λ}},{\mathbf{x}}_{\text{Λ}}\right),

(24)

which is the same as (4), for {\sigma }_{\mathbf{w}}^2 is a constant. Thus, it is satisfied that

f_p\left({\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right),{\mathbf{x}}_{\mathsf{\text{CTLS}}}\right)\le f_p\left(0,{\widehat{\mathbf{x}}}_{\text{Λ}}\left(l\right)\right)={\left|\right|\mathbf{y}-\mathbf{\Phi }\left({\widehat{\mathbf{g}}}_{\text{Λ}}\left(l\right)\right){\widehat{\mathbf{x}}}_{\text{Λ}}\left(l\right)\left|\right|}_2^2.

(25)

Substitute (5), (21) into f_p\left({\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right),{\mathbf{x}}_{\mathsf{\text{CTLS}}}\right), and note that {\mathbf{C}}_{\mathbf{g}}^{-1} is a positive definite matrix; thus,

\begin{array}{cc}\hfill f_p\left({\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right),{\mathbf{x}}_{\mathsf{\text{CTLS}}}\right)& =\left|\right|\mathbf{y}-\mathbf{\Phi }\left({\widehat{\mathbf{g}}}_{\text{Λ}}\left(l+1\right)\right){\mathbf{x}}_{\mathsf{\text{CTLS}}}|{|}_2^2+{\sigma }_{\mathbf{w}}^2{\left({\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right)\right)}^{\mathsf{\text{H}}}{\mathbf{C}}_{\mathbf{g}}^{-1}{\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right)\hfill \\ \ge \left|\right|\mathbf{y}-\mathbf{\Phi }\left({\widehat{\mathbf{g}}}_{\text{Λ}}\left(l+1\right)\right){\mathbf{x}}_{\mathsf{\text{CTLS}}}|{|}_2^2\hfill \\ \ge \left|\right|\mathbf{y}-\mathbf{\Phi }\left({\widehat{\mathbf{g}}}_{\text{Λ}}\left(l+1\right)\right){\widehat{\mathbf{x}}}_{\text{Λ}}\left(l+1\right)|{|}_2^2,\hfill \end{array}

(26)

where the last inequality is taken from (6). The inequalities in (25) and (26) are transformed to equalities if and only if {\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right)=0. □

For simplicity, we assume that the transform Φ(g_Λ) is linear. In some practical applications like harmonic retrieval, linearity is not strictly guaranteed. However, when atom mismatch \Delta \text{g} is significantly small, the higher order errors due to Taylor expansion (8) are ignorable, and (21) is approximately satisfied. Numerical examples are performed in Section 5, which demonstrate the convergence of the proposed algorithm in the case of harmonic retrieval.

3 Application in the harmonic retrieval

In this section, we apply AMP-CTLS in harmonic retrieval. In Section 3.1, the signal model of harmonic retrieval is presented and adverse effects of MP approaches [2–7] in harmonic retrieval is discussed. In Section 3.2, we detail the implementation of AMP-CTLS in harmonic retrieval.

3.1 Signal model of harmonic retrieval

Consider a complex sinusoidal signal

y_m=\sum _{k=1}^K{\alpha }_k\mathsf{\text{ exp }}\left(j2\pi f_{k}m\right)+w_m,

(27)

where y _m is the m th measurement, and w _m is the m th noise, m = 0, 1, . . . , M - 1. There are K sinusoids, and amplitude α _k , frequency f _k of the k th sinusoid are unknown parameters. When the sinusoids are sparse, i.e., K << M, harmonic retrieval problem can be solved by grid-based CS approaches. Divide the digital frequency f ∈ [0 1) into N grid points g = [g₁, g₂, . . . , g _N ]^T. When all frequencies are exactly at grid points, rewrite (27) as

y_m=\sum _{n=1}^N{x}_n\cdot \mathsf{\text{exp }}\left(j2\pi g_{n}m\right)+w_m,

(28)

where g _n is the frequency of the n th grid point and

x_n=\left\{\begin{array}{c}\hfill {\alpha }_k,\mathsf{\text{the}}k\mathsf{\text{th}}\mathsf{\text{sinusoid}}\mathsf{\text{is}}\mathsf{\text{present}}\mathsf{\text{at}}n\mathsf{\text{th}}\mathsf{\text{grid}}\mathsf{\text{point}},\hfill \\ \hfill 0,\mathsf{\text{no}}\mathsf{\text{sinusoid}}\mathsf{\text{is}}\mathsf{\text{present}}\mathsf{\text{at}}n\mathsf{\text{th}}\mathsf{\text{grid}}\mathsf{\text{point}}.\hfill \end{array}\right.

(29)

Rewrite (28) in matrix form as

\mathbf{y}=\mathbf{\Phi }\mathbf{x}+\mathbf{w},

(30)

where the m th-row, n th-column element of Φ is of the form ϕ(m, n) = exp(j 2πg _n m). Apply CS methods to seek the sparsest solution of (30). Then, estimates of the frequencies and amplitudes are obtained with the indices and magnitudes of nonzero coefficients in x, respectively. The sparsest solution can be obtained with computational MP methods, which greedily minimize the ℓ₀ norm. It can also be obtained by minimizing the ℓ₁ norm [23], the quasi-norm [24, 25] or the ℓ_{p ≤ 1}p-norm-like diversity [26].

We focus on MP methods in this article for the high computation efficiency. However, conventional MP methods [2–7] suffer from performance degradation if the frequency space is not perfectly grided. When the frequency is sparsely divided, sinusoids may lie off the grid points, and accuracy of frequency estimates is limited by the gap between neighboring grid points. MP methods iteratively search for the sinusoids. If an off-grid sinusoid emerges, the energy of this sinusoid can not be totally canceled and performs as an interference in the next iterations. The leakage of the energy may mask the weak sinusoids. On the other hand, if the frequency space is densely divided, correlations between atoms are enhanced [27], which also reduces the performance of MP methods. Especially in those MP methods that select multiple atoms into the support set in a single iteration, e.g., CoSaMP, SP, ROMP, and StOMP, highly correlated atoms could be chosen in a same iteration, which impairs the numerical stability of projection onto the adopted atoms.

3.2 Harmonic retrieval with AMP-CTLS

The AMP-CTLS algorithm can be applied for harmonic retrieval. AMP-CTLS adaptively finds the atoms and recovers the sinusoids. In those MP approaches with constant predefined atoms, frequency estimates are discrete values, depending on grid points. In AMP-CTLS frequency estimates are continuous, since estimates of the grid misalignments are continuous. In this section, we adjust two steps of AMP-CTLS presented in Section 2 to better fit the harmonic retrieval problem.

Calculate the R matrix in (9). According to (9), the m th-row, i th-column element of the R _i is expressed as follows:

{\mathbf{R}}_i\left(m,i\right)=\mathsf{\text{exp}}\left(j2\pi g_{i}m\right)\cdot j2\pi m.

(31)

Elements in other columns are all zeros.

Adjust the grid-updating formula in (5). In CTLS as presented in Section 2.4, the grid misalignment Δg _Λ is assumed to be a complex vector; therefore, the estimate {\widehat{\Delta \mathbf{g}}}_{\Lambda } is complex. However, frequency grid points are restrained to be real, so regularization Δg _Λ = (Δg _Λ )* should be added to (12) in the case of harmonic retrieval. Unfortunately, the solver becomes complex, which is derived in Appendix 2. For simplicity, (5) is replaced with (32) to approximatively update the grid points:

{\widehat{\mathbf{g}}}_{\text{Λ}}\left(l+1\right)={\widehat{\mathbf{g}}}_{\text{Λ}}\left(l\right)+\mathsf{\text{Re}}\left({\widehat{\text{Δ}\mathbf{g}}}_{\text{Λ}}\left(l\right)\right).

(32)

4 Application in RSF radar

AMP-CTLS can also be applied in randomized step frequency (RSF) radar. RSF radar can improve the range-velocity resolution and avoid range-velocity coupling problems [28, 29]. However, RSF radar suffers from the sidelobe pedestal problem, which results in small targets being masked by noise-like components due to dominant targets [29]. Our problem of interest is to recover small targets. When the observed scene is sparse, i.e., only few targets exists, we can use sparse recovery to exploit the sparseness [9]. AMP-CTLS relieves the sidelobe pedestal problem in RSF radar and recovers small targets well.

Correlation-matrix-based spectral analysis methods, e.g., MUSIC, ESPRIT [18], are hard to be directly utilized in range-Doppler estimation in RSF scheme. Since only one snapshot of radar data is available and radar echoes from different scatterers are coherent, smoothing technique is invoked to obtain a full rank correlation matrix [30]. Smoothing method requires that the array is uniform and linear [18]. However, in RSF radar, the echoes are determined by a random permutation of integers, see (34); thus, the uniform and linear condition is not satisfied, which restricts application of correlation-matrix-based methods.

We discuss a specific example of RSF radar, in which the waveform is a monotone pulse signal and the frequency of the m th pulse is f₀ + C _m δf, m = 0, 1, . . . , M - 1, where f₀ is carrier frequency and δf is frequency step size. C _m is a random permutation of integers from 0 to M - 1. The m th echo of radar can be expressed as (see [8, 28, 29]):

y_m=\sum _{k=1}^K{\alpha }_k{s}_m\left(p_k,q_k\right)+w_m,

(33)

s_m\left(p,q\right)=\mathsf{\text{exp}}\left(-j2\pi {\mathbf{C}}_{m}p-j2\pi m\left(1+{\mathbf{C}}_m\delta f/f_0\right)q\right),

(34)

where w _m is noise in the m th echo. K denotes the number of targets and k denotes k th target. α _k , p _k , and q _k are to be learned. α _k presents the scattering intensity. p _k ∈ [0 1) and q _k ∈ [0 1) are determined by range and radial velocity of the k th target, respectively. Note that in (34) the echo is simultaneously related to the sequence m and the random integer C _m .

Divide p space into C grid points p _c = c/C, c = 0, 1, . . . ,C - 1. Divide q space into D grid points p _d = d/D, d = 0, 1, . . . , D - 1. Rewrite (33) as:

\mathbf{y}=\mathbf{\Phi }\left(\mathbf{p},\mathbf{q}\right)\mathbf{x}+\mathbf{w},

(35)

where the m th-row, (c + dC)th-column element of Φ (p, q) ∈ ℂ ^{M × CD} is s _m (p _c , q _d ).

AMP-CTLS is implemented to solve (35). First, we find the support set Λ and then use IJE and CTLS to adjust the grid points, though CTLS described in Section 2.4 requires modification. The grid misalignment vector consists of two parts: p mismatch Δp_Λ ∈ ℝ^{|Λ|× 1}and q mismatch Δq_Λ ∈ ℝ^{|Λ|× 1}, \text{Δ}{\mathbf{g}}_{\text{Λ}}={\left[{\left(\text{Δ}{\mathbf{p}}_{\text{Λ}}\right)}^{\mathsf{\text{T}}},{\left(\text{Δ}{\mathbf{q}}_{\text{Λ}}\right)}^{\mathsf{\text{T}}}\right]}^{\mathsf{\text{T}}}\in {ℝ}^{2\left|\text{Λ}\right|\times 1}. The R ∈ ℂ ^{M × CD} matrix

{\mathbf{R}}_{p_i}=\frac{\partial \mathbf{\Phi }\left({\mathbf{p}}_{\text{Λ}},{\mathbf{q}}_{\text{Λ}}\right)}{\partial p_i},{\mathbf{R}}_{q_i}=\frac{\partial \mathbf{\Phi }\left({\mathbf{p}}_{\text{Λ}},{\mathbf{q}}_{\text{Λ}}\right)}{\partial q_i}.

(36)

Assume that Δp_Λ and Δq_Λ are independent of each other and of the noise. The covariance matrix of Δg_Λ is

{\text{C}}_{\text{g}}=\left[\begin{array}{cc}{\text{C}}_{\text{p}}& 0_{|\Lambda |}\\ 0_{|\Lambda |}& {\text{C}}_{\text{q}}\end{array}\right]\in {ℂ}^{2|\Lambda |\times 2|\Lambda |},

(37)

where {\mathbf{C}}_{\mathbf{p}}=E\left[\text{Δ}{\mathbf{p}}_{\text{Λ}}{\left(\text{Δ}{\mathbf{p}}_{\text{Λ}}\right)}^{\mathsf{\text{H}}}\right]\in {ℂ}^{\left|\text{Λ}\right|\times \left|\text{Λ}\right|}, {\mathbf{C}}_{\mathbf{q}}=E\left[\text{Δ}{\mathbf{q}}_{\text{Λ}}{\left(\text{Δ}{\mathbf{q}}_{\text{Λ}}\right)}^{\mathsf{\text{H}}}\right]\in {ℂ}^{\left|\text{Λ}\right|\times \left|\text{Λ}\right|}, {\mathbf{C}}_{\mathbf{p}}^{-1}={\mathbf{D}}_{\mathbf{p}}^{\mathbf{H}}{\mathbf{D}}_{\mathbf{p}} and {\mathbf{C}}_{\mathbf{q}}^{-1}={\mathbf{D}}_{\mathbf{q}}^{\mathbf{H}}{\mathbf{D}}_{\mathbf{q}}. In the case of RSF radar \mathbf{u}={\left[{\left({\mathbf{D}}_{\mathbf{p}}\text{Δ}{\mathbf{p}}_{\text{Λ}}\right)}^{\mathsf{\text{T}}},{\left({\mathbf{D}}_{\mathbf{q}}\text{Δ}{\mathbf{q}}_{\text{Λ}}\right)}^{\mathsf{\text{T}}},{\left(\frac{1}{{\sigma }_{\mathbf{w}}}\mathbf{w}\right)}^{\mathsf{\text{T}}}\right]}^{\mathsf{\text{T}}}\in {ℂ}^{\left(2\left|\text{Λ}\right|+M\right)\times 1}, and W _x = [H _p , H _q , σ _w I _N ] ∈ ℂ^{M × (2|Λ|+M)}, where {\mathbf{G}}_{\mathbf{p}}=\left[{\mathbf{R}}_{p_1},..,{\mathbf{R}}_{p_{\left|\text{Λ}\right|}}\right]\in {ℂ}^{M\times |\text{Λ}{|}^2}, {\mathbf{H}}_{\mathbf{p}}={\mathbf{G}}_{\mathbf{p}}\left({\mathbf{D}}_{\mathbf{p}}^{-1}\otimes {\mathbf{I}}_{\left|\text{Λ}\right|}\right)\left({\mathbf{I}}_{\left|\text{Λ}\right|}\otimes {\mathsf{\text{x}}}_{\text{Λ}}\right)\in {ℂ}^{M\times \left|\text{Λ}\right|}, {\mathbf{G}}_{\mathbf{q}}=\left[{\mathbf{R}}_{q_1},..,{\mathbf{R}}_{q_{\left|\text{Λ}\right|}}\right]\in {ℂ}^{M\times |\text{Λ}{|}^2} and {\mathbf{H}}_{\mathbf{q}}={\mathbf{G}}_{\mathbf{q}}\left({\mathbf{D}}_{\mathbf{q}}^{-1}\otimes {\mathbf{I}}_{\left|\text{Λ}\right|}\right)\left({\mathbf{I}}_{\left|\text{Λ}\right|}\otimes {\mathbf{x}}_{\text{Λ}}\right)\in {ℂ}^{M\times \left|\text{Λ}\right|}. Since p and q are both real, formula (32) s used to update grid points, in which the imaginary parts of Δp and Δq estimates are abandoned.

5 Simulations

Numerical results are provided to illustrate the performance of the new algorithm. In all examples, the noise is additive Gaussian white noise.

5.1 Accuracy of AMP-CTLS

We compare the accuracy of AMP-CTLS with standard OMP. We assume that there is a single sinusoid in the measurements of form (27), where α = 1 and the signal to noise ratio SNR = α²/σ² = 5 dB, where σ² is the variance of noise. The number of measurements M is 32. The frequency of sinusoid is varied between two adjoining frequency grid points. The mean square errors (MSEs) of frequency estimates are calculated. The MSEs are compared with the corresponding Cramer-Rao lower bound (CRB) [31]. The frequency is uniformly divided into m grid points in xMP _m (OMP _M , OMP_10M, CoSaMP _M , etc.) and into M points in AMP-CTLS. AMP-CTLS is configured as follows: IJE loops no more than 14 times; the normalization factors in (11) are D = I /(σ_Δf), σ_Δf= 0.005, and σ _w = 1. As shown in Figure 1, MSEs of AMP-CTLS are close to the CRB and lower than those of OMP, except when the sinusoid is in the vicinity of the grid point.

Figure 1

MSEs of the frequency estimates obtained from 1,000 independent Monte-Carlo trials via OMP and AMP-CTLS. The distance to the nearest grid point is normalized by 1/M. CRB denotes Cramer-Rao bound.

5.2 Convergence of AMP-CTLS

We first discuss the convergence speed of the proposed IJE algorithm in noise-free case. Suppose that the sinusoid is located at f = 9.5/M, M = 32. Other conditions are the same as described in Section 5.1. In the l th iteration of IJE, we can obtain a grid point ĝ\left(l\right) with (5) and residual error \mathbf{r}\left(l\right)=\mathbf{y}-\mathbf{\Phi }\left(\mathit{ĝ}\right)\widehat{\mathit{x}}\left(l\right) after (6). we calculate the norm of residual ||r(l)||₂ and the grid error |\mathit{ĝ}\left(l\right)-f|, and normalize the results with ||r(0)||₂ and |\mathit{ĝ}\left(0\right)-f|, respectively. As shown in Figure 2, both the residual error and the grid error converge fast (about five steps) to 0 in noiseless case.

The purpose of what follows is to discuss the feasible zone of the initial grid points in noisy circumstance. In Section 2.6, the convergence analysis of IJE is based on the assumption that the transform Φ is linear. This is only approximately satisfied in harmonic retrieval when the higher order terms of Taylor expansion (8) are ignorable, which means that the grid points indexed in the support set are required to be close to the actual frequencies. We assign SNR = 5 dB and the initial frequency grid point as g(0) = 9/M. The true frequency of the sinusoid varies from 9/M to 11/M.

Figure 2

The normalized errors at different IJE iterations in noiseless case. ||r(l)||₂ denotes the norm of residual error after l th iteration. f is actual frequency of the sinusoid and \widehat{g}(l) is the estimate of grid point. In the plots, ||r(l)||₂ and |\mathit{ĝ}\left(l\right)-f| are normalize with ||r(0)||₂ and |\mathit{ĝ}\left(0\right)-f|, respectively.

As shown in Figure 3, when the distance (normalized by 1/M) between the true frequency and the initial grid point is less than 0.7, the initial grid is adjusted to be close to the actual value, and MSEs of the frequency estimates converge to CRB. When the distance is greater than 1, the AMP-CTLS curve is close to the initial distance, which means that AMP-CTLS fails to improve the initial grid, because errors of Taylor expansion cannot be ignored and affect convergence of the algorithm.

Figure 3

MSEs of the frequency estimates versus distances between the initial grid point and the actual values. The distance is normalized by 1/M. The estimates are obtained from 1,000 independent Monte-Carlo trials. 'Initial distance (dB)' in the legend denotes the square differences between the frequency grid point and the true ones.

5.3 Input of sparsity

In Sections 5.1 and 5.2, we assume that the sparsity K, i.e., the number of modes, is known and we use K to terminate AMP-CTLS, while a priori sparsity is not obligatory. When K is unknown, we can use norm of residual error r = y - Φ (g _Λ ) x _Λ as termination criterion.

Furthermore, AMP-CTLS does not seriously rely on the given sparsity K', and the performance is slightly affected when K' > K. Suppose there are three sinusoids denoted as Si₁, Si₂, and Si₃, where α₁ = 20, α₂ = 15, α₃ = 1, f₁ = 3.15/M, f₂ = 4.2/M, f₃ = 7.25/M, M = 32. \mathsf{\text{SN}}{\mathsf{\text{R}}}_{\mathsf{\text{3}}}={\alpha }_3^2/{\sigma }^2=10\mathsf{\text{dB}}\mathsf{\text{.}} In AMP-CTLS, frequency is uniformly grided to 2M points, and other configurations are the same as described in Section 5.1.

We calculate means of the final residual norm ||r^(K')||₂ versus K' and present the results in Figure 4. When all of the sinusoids have been chosen into the support set and K' ≥ K, energy of the sinusoids are canceled thoroughly and only noise exist in the residual. The norm of residual error becomes small and is slowly reduced along with K'. The results illustrate that we can use threshold of values or decrease rate of the norm of residual to end AMP-CTLS loops.

Figure 4

Means of residual norm ||r^(K')|| ₂ versus given sparsity K' obtained from 10,000 independent Monte-Carlo trials.

Spurious sinusoids emerge when K' > K. Denote the amplitude estimates by {\widehat{\alpha }}_1,{\widehat{\alpha }}_2,\dots ,{\widehat{\alpha }}_{K^{\prime }} in descend order of magnitudes and their counterparts of frequency estimates by {\widehat{f}}_1,{\widehat{f}}_2,\dots ,{\widehat{f}}_{K^{\prime }}. MSEs of f₃ estimates versus K' are presented in Figure 5, which indicates that accuracy of frequency estimates of Si₃ is slightly affected (< 2 dB) by K'.

Figure 5

MSEs of f ₃ estimates versus given sparsity K' obtained from 10,000 independent Monte-Carlo trials. The results are compared with corresponding CRB.

We also calculate |{\widehat{\alpha }}_{K+1}/{\alpha }_K| as measurement of the level of spurious sinusoids. Figure 6 presents the results and shows that the ratios |{\widehat{\alpha }}_{K+1}/{\alpha }_K| stay at low level (< 0.2) and are not sensitive to K'.

Figure 6

Means of |{\widehat{\alpha }}_{K+1}/{\alpha }_K|versus given sparsity K' obtained from 10,000 independent Monte-Carlo trials, where {\widehat{\alpha }}_{K+1}denotes the biggest magnitude estimate of spurious sinusoid.

Figure 7 presents MSEs of f₃ estimates versus SNR at different K'. Noise variance σ² is altered such that SNR₃ varies. The MSEs converges to CRB at high SNR (SNR₃> 2 dB) when K' = K = 3. The results of K' = 6 are close to those of K' = 3.

Figure 7

MSEs of f ₃ estimates versus SNR at different given sparsity K' obtained from 1,000 independent Monte-Carlo trials. The results are compared with CRB.

5.4 Recovering small sinusoids

We compare the performance on recovering weak sinusoids of AMP-CTLS with CS methods, e.g., OMP and CoSaMP, and conventional spectral analysis methods, e.g., ESPRIT and root MUSIC [18]. Suppose there are three sinusoids denoted as Si₁, Si₂, and Si₃, where α₁ = 20, α₂ = 15, α₃ = 1, f₁ = 3.15/M, f₂ = 5.2/M, f₃ = 3.95/M, M = 32.\mathsf{\text{SN}}{\mathsf{\text{R}}}_{\mathsf{\text{3}}}={\alpha }_3^2/{\sigma }^2=5\mathsf{\text{dB}}. The number of sinusoids K = 3 is assumed to be known. CoSaMP iterates 50 times. In both ESPRIT and root MUSIC, the model orders are set as K, and the covariance matrix orders are M/ 2 according to [32]. ESPRIT and root MUSIC output frequency estimates, and the corresponding magnitudes are obtained by projection on these frequencies. AMP-CTLS is configured the same as mentioned in Section 5.3.

Some intuitive results are presented in Figure 8. The Si₃ is recovered by the AMP-CTLS algorithm and is masked via other tested algorithms. CoSaMP_100Mis also tested, but the results are not displayed because the amplitude estimates are too large (> 1,000), which is caused by projection onto the ill-conditioned matrix consisting of highly correlated atoms. In OMP_2M, the sinusoids are not exactly at the grid points, so the energies of Si₁ and Si₂ cannot be totally canceled in the beginning two iterations, and the leakage of the energies masks the smallest signal Si₃. In OMP_100M, all sinusoids are placed at the grid points, and Si₁ and Si₂ are better recovered than in OMP_2M, but energy leakage of dominant sinusoids still exists. In AMP-CTLS, the grid points are adaptively adjusted to match the sinusoids, so the algorithm is less sensitive to grid mismatch and can achieve better performance than OMP and CoSaMP even if the frequency space is sparsely divided. ESPRIT and root MUSIC do not correctly recover Si₃ as AMP-CTLS does. Since there is only one snapshot data, smoothing method [30] is used in these two methods to estimate the covariance matrix, which results in aperture loss [18].

Figure 8

Amplitude and frequency estimates of the sinusoids obtained with OMP, CoSaMP, ESPRIT, root MUSIC and AMP-CTLS from 15 independent Monte-Carlo trials. The estimates are compared with the true parameters. Subscripts of OMP and CoSaMP denote numbers of grid points. Note that in OMP_100M, all of the sinusoids exactly lie on the grid points. The results of conventional MP methods and spectral analysis methods are, respectively, shown in (a) and (b) for better readability.

5.5 Range-velocity joint estimate in RSF radar

In this section, we discuss merits of AMP-CTLS in recovering small targets with RSF radar. Suppose there are three targets: two large targets T₁ and T₂ and a small target T₃. The number of measurements M is 32. The scattering intensities are α₁ = α₂ = 10, α₃ = 1, and the ranges and the velocities are set such that the p, q parameters are p₁ = 10.1/M, p₂ = 10.7/M, p₃ = 20/M, q₁ = 19.4/M, q₂ = 10.2/M and q₃ = 15.2/M. AMP-CTLS is configured as follows: the p, q spaces are both uniformly divided into M grid points; the normalization factors are D _p = D _q = I /σ_Δ, σ_Δ = 0.025, σ _w = 1; and the IJE algorithm iterates fewer than 14 times. In OMP _m , both the p and q spaces are uniformly divided into m grid points. Note that all of the targets lie on the grid points in OMP_10M. We focus on the results of recovering the weakest target T₃. Change the noise covariance σ²; thus, the signal to noise ratio \mathsf{\text{SN}}{\mathsf{\text{R}}}_{\mathsf{\text{3}}}={\alpha }_3^2/{\sigma }^2 varies. Calculate MSEs of p₃, q₃ parameters. As shown in Figure 9, the MSEs with AMP-CTLS are lower than those with OMP and converge to the CRB when the SNR₃ is no less than 2 dB. The difference between these MSEs of AMP-CTLS at high SNR and CRB is less than 0.5 dB.

Figure 9

MSEs of p ₃ and q ₃ estimates of OMP and AMP-CTLS versus SNR ₃ obtained from 1,000 independent Monto-Carlo trials. The results are compared with CRB.

5.6 DoA estimation

In this section, AMP-CTLS is compared with the Lasso-based TLS method WSS-TLS [14] on direction of arrival (DoA) estimation. The goal is estimating DoA of plane waves from far-field, narrowband sources with uniform linear array of antennas [14]. We focus on the single-snapshot case. Suppose the antenna array contains M = 8 elements and the interval between neighboring elements d = 1/ 2 wavelength. There are two sources (K = 2) from angles θ₁ = -29° and θ₂ = 13°. The amplitudes α₁ = α₂ = 1 and \mathsf{\text{SN}}{\mathsf{\text{R}}}_{\mathsf{\text{1}}}=\mathsf{\text{SN}}{\mathsf{\text{R}}}_{\mathsf{\text{2}}}={\alpha }_1^2/{\sigma }^2, where σ² is the noise variance. The angle space from -90° to 90° are uniformly divided to N = 90 grid points; thus both sources are 1° off the nearest grid points. The WSS-TLS algorithm is set according to [14]. Since WSS-TLS returns multiple nonzero DoA estimates, we choose two peaks with largest magnitudes as the estimates of θ₁ and θ₂. AMP-CTLS models DoA estimation as an harmonic retrieval problem and outputs frequency estimates \widehat{f}\in [01). Denote \tilde{f}=\widehat{f}-0.5\left(\mathsf{\text{sgn}}\left(\widehat{f}-0.5\right)+1\right), where sgn(·) represents signum function; thus the DoA estimate with AMP-CTLS is obtained by \widehat{\theta }={\text{arcsin}}^{-1}(\tilde{f}/d). In AMP-CTLS, IJE loops no more than 50 times; the normalization factors in (11) are D = I /(σ_Δf), σ_Δf= 0.005, σ _w = 1. MSEs of θ₁ and θ₂ estimates versus SNR are shown in Figure 10a, b, respectively. The results indicate that MSEs of AMP-CTLS are closer to CRB than those of WSS-TLS.

Figure 10

MSEs of θ ₁ and θ ₂ estimates of WSS-TLS and AMP-CTLS obtained from 1,000 independent Monto-Carlo trials.

6 Conclusion

To alleviate the off-grid problem in grid-based MP methods, we implement CTLS into the OMP framework and propose a new algorithm, namely AMP-CTLS. Unlike traditional MP methods, AMP-CTLS adaptively adjusts the grid and dictionary. The convergence of AMP-CTLS is analyzed, and the initial conditions of the algorithm are discussed. Numerical examples indicate that the advantages of AMP-CTLS over OMP and CoSaMP are twofold: (1) it is still efficient even when the continuous parameter space is sparsely divided, but OMP or CoSaMP suffers from performance degradation when the space is not divided reasonably; (2) it can achieve higher accuracy, and the MSEs converge to CRB.

Appendix 1

We recall the complex version of Newton's method in [20]. Minor changes are made because of differences in notation. The recursion formula of the complex Newton method for (17) is:

{\mathbf{x}}_{\mathsf{\text{CTLS}}}^{\left(t+1\right)}={\mathsf{\text{x}}}_{\mathsf{\text{CTLS}}}^{\left(t\right)}+{\left({\mathbf{A}}^{*}{\mathbf{B}}^{-1}\mathbf{A}-{\mathbf{B}}^{*}\right)}^{-1}\left({\mathbf{a}}^{*}-{\mathbf{A}}^{*}{\mathbf{B}}^{-1}\mathbf{a}\right),

(38)

where

\mathbf{h}={\left({\mathbf{W}}_{\mathbf{x}}{\mathbf{W}}_{\mathbf{x}}^{\mathsf{\text{H}}}\right)}^{-1}\left({\mathbf{\Phi }}_{\text{Λ}}{\mathbf{x}}_{\mathsf{\text{CTLS}}}^{\left(t\right)}-\mathbf{y}\right),

(39)

\widehat{\mathsf{\text{u}}}=-{\mathbf{W}}_{\mathbf{x}}^{\mathsf{\text{H}}}\mathbf{h},

(40)

\tilde{\mathbf{B}}={\mathbf{\Phi }}_{\text{Λ}}+\mathbf{G}\left({\mathbf{D}}^{-1}\otimes {\mathbf{I}}_{\left|\text{Λ}\right|}\right)\left(\widehat{\mathbf{u}}\otimes {\mathbf{I}}_{\left|\text{Λ}\right|}\right),

(41)

{\mathbf{Q}}_j=\left[\left[{\left({\mathbf{R}}_1\right)}_{\left\{j\right\}},\dots ,{\left({\mathbf{R}}_{\left|\text{Λ}\right|}\right)}_{\left\{j\right\}}\right]{\mathbf{D}}^{-1},0_N\right],j\le \left|\text{Λ}\right|,

(42)

\tilde{\mathbf{Q}}=\left[{\mathbf{Q}}_1^{\mathsf{\text{H}}}\mathbf{h},{\mathbf{Q}}_2^{\mathsf{\text{H}}}\mathbf{h},\dots ,{\mathbf{Q}}_{\left|\text{Λ}\right|}^{\mathsf{\text{H}}}\mathbf{h}\right],

(43)

\mathbf{a}={\left({\mathbf{h}}^{\mathsf{\text{H}}}\widetilde{\mathsf{\text{B}}}\right)}^{\mathsf{\text{T}}},

(44)

\mathbf{A}=-{\tilde{\mathbf{Q}}}^{\mathsf{\text{H}}}{\mathbf{W}}_{\mathbf{x}}^{†}\tilde{\mathbf{B}}-{\left({\tilde{\mathbf{Q}}}^{\mathsf{\text{H}}}{\mathbf{W}}_{\mathbf{x}}^{†}\tilde{\mathbf{B}}\right)}^{\mathsf{\text{T}}},

(45)

\text{B}={\tilde{\text{Q}}}^{\text{H}}\left({\text{W}}_{\text{x}}^{†}{\text{W}}_{\text{x}}-\text{I}\right)\tilde{\text{Q}}+{\left({\tilde{\text{B}}}^{\text{H}}{\left({\text{W}}_{\text{x}}{\text{W}}_{\text{x}}^{\text{H}}\right)}^{-1}\tilde{\text{B}}\right)}^{\text{T}}

(46)

Appendix 2

When the frequencies are constrained to be real, the CTLS solver becomes more complex. Some notations are introduced for simplicity as follows: W₁ = H, W₂ = σ _w I _N , \mathbf{u}={\left[{\mathbf{u}}_1^{\mathsf{\text{T}}},{\mathbf{u}}_2^{\mathsf{\text{T}}}\right]}^{\mathsf{\text{T}}}, z = y - Φ_Λx_Λ. Notice that the matrix H is relative to x_Λ. Replace the optimum problem in (12), (14) with:

\widehat{\mathbf{u}},{\mathbf{x}}_{\mathsf{\text{CTLS}}}=\underset{{\mathbf{u}}_{\mathsf{\text{1}}},{\mathbf{u}}_{\mathsf{\text{2}}},{\mathbf{x}}_{\text{Λ}}}{\mathsf{\text{arg }}\mathsf{\text{min }}}{\mathbf{u}}_1^{\mathsf{\text{T}}}{\mathbf{u}}_1+{\mathbf{u}}_2^{\mathsf{\text{H}}}{\mathbf{u}}_2,

(47)

\mathsf{\text{s}}\mathsf{\text{.t}}\mathsf{\text{.}}\left\{\begin{array}{c}\mathsf{\text{z}}-\left[{\mathbf{W}}_1,{\mathbf{W}}_2\right]\left[\begin{array}{c}{\mathbf{u}}_1\hfill \\ {\mathbf{u}}_2\hfill \end{array}\right]\hfill \\ {\mathbf{u}}_1^{*}={\mathbf{u}}_1\hfill \end{array}\right.=0.

(48)

First, suppose x_Λ is known, and seek the solution of u. If both W₁ and W₂ are of full-row rank, we have

{\mathbf{u}}_1=2\mathsf{\text{Re}}\left({\mathbf{W}}_1^{\mathsf{\text{H}}}\mathbf{v}\right),

(49)

{\mathbf{u}}_2=2{\mathbf{W}}_2^{\mathsf{\text{H}}}\mathbf{v},

(50)

where

\mathbf{v}=-{\left({\mathbf{C}}_2^{-1}{\mathbf{C}}_1-{\left({\mathbf{C}}_1^{-1}{\mathbf{C}}_2\right)}^{*}\right)}^{-1}\left({\mathbf{C}}_2^{-1}z-{\left({\mathbf{C}}_1^{-1}z\right)}^{*}\right),

(51)

{\mathbf{C}}_1={\mathbf{W}}_1{\mathbf{W}}_1^{\mathsf{\text{H}}}+2{\mathbf{W}}_2{\mathbf{W}}_2^{\mathsf{\text{H}}},

(52)

{\mathbf{C}}_2={\mathbf{W}}_1{\mathbf{W}}_1^{\mathsf{\text{T}}}.

(53)

The solution u₁, u₂ depends on x_Λ. Then, calculate x_Λ as

{\mathbf{x}}_{\mathsf{\text{CTLS}}}=\underset{{\mathbf{x}}_{\text{Λ}}}{\mathsf{\text{arg }}\mathsf{\text{min }}}{\mathbf{u}}_1^{\mathsf{\text{T}}}{\mathbf{u}}_1+{\mathbf{u}}_2^{\mathsf{\text{H}}}{\mathbf{u}}_2.

(54)

However, it is difficult to solve (54), because the Jacobi matrix and Hessian matrix of {\mathbf{u}}_1^{\mathsf{\text{T}}}{\mathbf{u}}_1+{\mathbf{u}}_2^{\mathsf{\text{H}}}{\mathbf{u}}_2 versus x_Λ are complex. In this article, we simply consider the frequencies to be complex and ignore the imaginary parts.

Proof: We prove that when x_Λ is known, the solution of u is given as (49) to (53). The Lagrangian

L\left(\mathbf{u},\mathbf{v}\right)=\frac{1}{2}{\mathbf{u}}_1^{\mathsf{\text{T}}}{\mathbf{u}}_1-{\mathbf{v}}^{\mathsf{\text{H}}}\left(z+{\mathbf{W}}_1{\mathbf{u}}_1+{\mathbf{W}}_2{\mathbf{u}}_2\right)+\frac{1}{2}{\mathbf{u}}_2^{\mathsf{\text{H}}}{\mathbf{u}}_2-{\mathbf{v}}^{\mathsf{\text{T}}}\left(z^{*}+{\mathbf{W}}_1^{*}{\mathbf{u}}_1+{\mathbf{W}}_2^{*}{\mathbf{u}}_2^{*}\right)

(55)

can be expressed as follows:

\begin{array}{c}L\left(\mathbf{u},\mathbf{v}\right)=\frac{1}{2}{\left({\mathbf{u}}_1-2\mathsf{\text{Re}}\left({\mathbf{W}}_1^{\mathsf{\text{H}}}\mathbf{v}\right)\right)}^{\mathsf{\text{T}}}\left({\mathbf{u}}_1-2\mathsf{\text{Re}}\left({\mathbf{W}}_1^{\mathsf{\text{H}}}\mathbf{v}\right)\right)-2{\left(\mathsf{\text{Re}}\left({\mathbf{W}}_1^{\mathsf{\text{H}}}\mathbf{v}\right)\right)}^{\mathsf{\text{T}}}\mathsf{\text{Re}}\left({\mathbf{W}}_1^{\mathsf{\text{H}}}\mathbf{v}\right)\\ -2{\mathbf{v}}^{\mathsf{\text{H}}}{\mathbf{W}}_2{\mathbf{W}}_2^{\mathsf{\text{H}}}\mathbf{v}+\frac{1}{2}{\left({\mathbf{u}}_2-2{\mathbf{W}}_2^{\mathsf{\text{H}}}\mathbf{v}\right)}^{\mathsf{\text{H}}}\left({\mathbf{u}}_2-2{\mathbf{W}}_2^{\mathsf{\text{H}}}\mathbf{v}\right)-2\mathsf{\text{Re}}\left[{\mathbf{v}}^{\mathsf{\text{H}}}z\right].\end{array}

(56)

When {\mathbf{u}}_1=2\mathsf{\text{Re}}\left({\mathbf{W}}_1^{\mathsf{\text{H}}}\mathbf{v}\right) and {\mathbf{u}}_2=2{\mathbf{W}}_2^{\mathbf{H}}\mathbf{v}, the Lagrangian reaches the infimum; thus, the Lagrange dual function is obtained as

\gamma \left(\mathbf{v}\right)=\underset{\mathbf{u}}{\mathsf{\text{inf }}}L\left(\mathbf{u},\mathbf{v}\right)=-4{\mathbf{v}}^{\mathsf{\text{T}}}{\mathbf{W}}_2^{*}{\mathbf{W}}_2^{\mathsf{\text{T}}}{\mathbf{v}}^{*}-4\mathsf{\text{Re}}\left({\mathbf{v}}^{\mathsf{\text{H}}}\mathbf{z}\right)-\left({\mathbf{v}}^{\mathsf{\text{H}}}{\mathbf{W}}_1+{\mathbf{v}}^{\mathsf{\text{T}}}{\mathbf{W}}_1^{*}\right)\left({\mathbf{W}}_1^{\mathsf{\text{H}}}\mathbf{v}+{\mathbf{W}}_1^{\mathsf{\text{T}}}{\mathbf{v}}^{*}\right).

(57)

Calculate the Jacobi and Hessian matrix of γ(v):

\frac{\partial \gamma }{\partial {\mathbf{v}}^{*}}=-\frac{1}{2}{\mathbf{W}}_1{\mathbf{W}}_1^{\mathsf{\text{H}}}\mathbf{v}-\frac{1}{2}{\mathbf{W}}_1{\mathbf{W}}_1^{\mathsf{\text{T}}}{\mathbf{v}}^{*}-{\mathbf{W}}_2{\mathbf{W}}_2^{\mathsf{\text{H}}}\mathbf{v}-\frac{1}{2}\mathbf{z},

(58)

\frac{{\partial }^2\gamma }{\partial {\mathbf{v}}^{\mathsf{\text{T}}}\partial {\mathbf{v}}^{*}}=-\frac{1}{2}{\mathbf{W}}_1{\mathbf{W}}_1^{\mathsf{\text{H}}}-{\mathbf{W}}_2{\mathbf{W}}_2^{\mathsf{\text{H}}}.

(59)

Because W₁, W₂ are of full-row rank, \frac{{\partial }^2\gamma }{\partial {\mathbf{v}}^{\mathsf{\text{T}}}\partial {\mathbf{v}}^{*}} is a negative definite matrix. We solve v with the optimum condition \frac{\partial \theta }{\partial {\mathbf{v}}^{*}}=0, and obtain (51) to (53). The proof is complete.

Algorithm 1. The CTLS technique

• 1) Input the dictionary Φ _Λ and all the coefficient matrices R _i .

• 2) Compose W _x and solve (17) with the initial value given in (20).

• 3) Calculate û via (18) and (19).

• 4) Extract {\widehat{\Delta \mathbf{g}}}_{\Lambda } from û .

Algorithm 2. The AMP-CTLS algorithm

• 1) Divide the continuous parameter f into grid point {\widehat{\mathbf{g}}}^{\left(0\right)}; input the sparsity level K or residual threshold δ.

  Set the support set Λ⁽⁰⁾ = ∅, and the residual error r⁽⁰⁾ = y.

• 2) Calculate the correlations p_i=⟨r^{\left(k\right)},\mathbf{\Phi }\left({\mathit{ĝ}}_i^{\left(k\right)}\right)⟩.

• 3) Find the index n = arg max |p _i |.

• 4) Merge the support set Λ^(k+1)= Λ^(k)∪ {n}.

5) Solve (3) with the IJE algorithm. Then we get {\widehat{\mathbf{g}}}_{\Lambda }^{(k+1)} and {\widehat{\mathbf{x}}}_{\Lambda }^{(k+1)}.

• 6) Update the residual error {\mathbf{r}}^{(k+1)}=\mathbf{y}-\Phi \left({\widehat{\mathbf{g}}}_{\Lambda }^{(k+1)}\right){\widehat{\mathbf{x}}}_{\Lambda }^{(k+1)}.

• 7) Increase k. Return to Step 2 until stop criterion, e.g., k < K, ||r ^(k)||₂ < δ or ||r ^(k)||₂ < δ||r ^{(k- 1)}||₂, is satisfied.

• 8) Simultaneously output {\widehat{\mathbf{x}}}_{\text{Λ}}^{\left(k\right)} and {\widehat{\mathbf{g}}}_{\text{Λ}}^{\left(k\right)}, and set the elements of x not indexed in Λ to 0.