The signal model of grid-based CS is introduced in Section 2.1. We combine the greedy idea of MP methods and the constrained total least squares (CTLS) technique , and thus produce AMP-CTLS to alleviate the off-grid problem. In AMP-CTLS, the grid is cast as an unknown parameter, and is jointly estimated together with x. In Section 2.2, the framework of AMP-CTLS is given. Section 2.3 is dedicated to the iterative joint estimator (IJE) algorithm, which is implemented in AMP-CTLS. In the IJE algorithm, the CTLS technique is used, which is presented in Section 2.4. Section 2.5 summarizes the entire procedure of AMP-CTLS. In Section 2.6, the convergence of IJE is analyzed.
2.1 Grid-based CS
CS promises efficient recovery of sparse signals. In many applications, signals are sparse in a continuous parameter space rather than finite discrete atoms. Usually, we divide the continuous parameter into discrete grid points and cast the problem as a grid-based CS model:
where y ∈ ℂM × 1and w ∈ ℂM × 1are measurement vector and white Gaussian noise (WGN) vector, respectively. x ∈ ℂN × 1is to be learned. g ∈ ℂN × 1are discrete grid points g = [g1, g2, . . . , g
]. Φ(g) ∈ ℂ M × N is built from g, Φ(g) = [ϕ(g1), ϕ(g2), . . . , ϕ(g
)], and the mapping g → Φ is known. For example, to recover a frequency sparse signal, we grid the frequency space into discrete frequency points . Φ is a DFT matrix, of which the m th-row, n th-column element is exp . However, the signal is only sparse in the DFT atoms if all of the sinusoids are exactly at the pre-defined grid points . In some cases, no matter how densely we grid the frequency space, the sinusoids could be off-grid, which saps the performance of CS methods .
2.2 Main idea of AMP-CTLS
The off-grid problem usually emerges because we do not often have enough priori knowledge to generate a perfect grid to guarantee that all of the signals exactly lie on grid points. Thus, we cast the grid as an unknown parameter, and search for the best grid g as well as the sparsest x by solving the optimum problem:
where η is the noise power. Equation (2) is similar to that used in traditional MP methods [2–7], except that we recover x and simultaneously estimate the grid. In most cases, solving (2) is a complex non-linear optimum problem. In this article, an iterative method is introduced.
AMP-CTLS inherits the greedy idea from MP methods, which use correlations to iteratively find the support set. In each iteration, one or more atoms are added into the support set. Suppose the support set is obtained as Λ(k)after the k th iteration, and denote the corresponding grid points as . In traditional MP methods [2–7], xΛ is estimated by solving a least squares problem. In AMP-CTLS, considering the off-grid problem, we jointly search for xΛ and the best grid points in the neighboring continuous region of via (3), in which we minimize norm of the residual error, which is defined as r = y - Φ (gΛ) xΛ.
We develop the iterative joint estimator (IJE) algorithm to solve (3), which is detailed in ensuing section.
2.3 IJE algorithm
It is difficult to find an analytical solution to (3). The IJE algorithm is devised to seek a numerical solution. Given initial grid points , IJE searches for the best grid points gΛ in the neighborhood of . The mismatch of the grid is denoted as . IJE includes three steps: calculate the estimation of the mismatch, ; update the grid with ; and estimate xΛ with projection onto the new grid points. These three steps are executed iteratively to pursue more accurate results. To distinguish from iterations in search for the support set in (3), we denote l as the counter of loops in IJE; thus, IJE is expressed as follows:
In (4), CTLS technique is applied to simultaneously search for the mismatch ΔgΛ and x
, and and xCTLS are the results. CCTLS denotes the penalty function of CTLS, which is detailed in Section 2.4. Since (6) is a linear least squares problem, the closed-form solution is
The loops are terminated when the norm of residual error is scarcely reduced.
2.4 CTLS technique
Traditional MP methods [2–7] apply least squares to calculate amplitudes of xΛ after finding the support set. When there are off-grid signals, mismatches occur in the dictionary; thus, we replace the least squares model with total least squares (TLS) criterion, which is appropriate to deal with the fitting problem when perturbations exist in both the measurement vector and in the dictionary . Since the dictionary mismatches are constrained by errors of grid points, we introduce the constrained total least squares (CTLS) technique  in AMP-CTLS to jointly estimate the grid point errors and xΛ, i.e., solving (4). It has been proved that CTLS is a constrained space state maximum likelihood estimator .
Suppose that we obtain the estimate of grid points as after l th IJE iteration. Assume that the mismatch ΔgΛ is significantly small; thus we can approximate the perfect dictionary Φ(g
) as a linear combination of the mismatch Δg with Taylor expansion:
∈ ℂM ×|Λ|is
and o(·) denotes higher order terms. For simplicity, in this section we ignore the iteration counter in the notations, and are, respectively, simplified as R
. Neglect and the signal model in (1) is replaced by:
CTLS models Δg
as an unknown random perturbation vector. The grid misalignment and the noise vector are combined into a (M +| Λ|)-dimensional vector , and CTLS aims at minimizing . It has been proved that CTLS is a constrained space state maximum likelihood estimator if v is a WGN vector . Thus, we first whiten v. Assume that Δg
is independent of w. The covariance matrix of Δg
is . D ∈ ℂ|Λ|×|Λ| obeys . The variance of white noise w is . We denote an unknown normalized vector u ∈ ℂ(M+| Λ|) × 1as (11); thus, u is a WGN vector.
Minimize the penalty function and (4) is detailed as follows:
The constraint condition (13) can be rewritten as:
= [H σ
] ∈ ℂM × (| Λ|+M). H ∈ ℂM ×| Λ|is defined as
where . The equivalence between (13) and (14) is proved as follows:
is of full-row rank, the optimum problem (12, 14) are equivalent to (17)-(19), which has been proved in .
It is quite difficult to obtain analytical solution to (17). A complex version of Newton method is developed in , which is presented in Appendix 1. Initial value of x
required in Newton's method for (17) can be given as:
is extracted from û via , thus (4) is solved. The sketch of CTLS is given in Algorithm 1. As the authors' best knowledge, the convergence guarantees for this Newton method are still open question.
2.5 Sketch of AMP-CTLS
Similarly to traditional MP methods [2–7], AMP-CTLS first greedily finds the support set. Then AMP-CTLS adaptively optimizes the grid points indexed in the support set. In this article, we imitate the greedy approach of OMP, in which only one atom is added to the support set in each iteration. If the number of atoms is known, terminate the iterations when the cardinality of the support set reaches the pre-specified number. If it is not known, we can apply some other successfully used stopping criterions, e.g., norm of residual being below a threshold . A sketch of AMP-CTLS is presented in Algorithm 2.
2.6 Convergence of the IJE algorithm
Here, we analyze convergence of the IJE algorithm. Assume that the mapping gΛ→ Φ (gΛ) is linear, which means
and G should be a constant matrix.
Proposition. If the measurement y is perturbed by WGN and (21) is obeyed, IJE monotonically reduces values of the penalty function in (3). The estimates of x
Proof. Define a penalty function as follows:
thus, and xCTLS are obtained by solving
which is the same as (4), for is a constant. Thus, it is satisfied that
Substitute (5), (21) into , and note that is a positive definite matrix; thus,
where the last inequality is taken from (6). The inequalities in (25) and (26) are transformed to equalities if and only if . □
For simplicity, we assume that the transform Φ(gΛ) is linear. In some practical applications like harmonic retrieval, linearity is not strictly guaranteed. However, when atom mismatch is significantly small, the higher order errors due to Taylor expansion (8) are ignorable, and (21) is approximately satisfied. Numerical examples are performed in Section 5, which demonstrate the convergence of the proposed algorithm in the case of harmonic retrieval.