Our goal is to estimate DOA of weak signals under strong interference. Based on the array signal model proposed in Sect. 2, the proposed method is illustrated in this section. Figure 1 presents the flow chart of the proposed algorithm. The parallel adaptive beamforming algorithm based on PI is applied to the received signal in Formula (3) to suppress strong interference and form new array data. To reduce the spurious peaks in the spectrum under strong interference, the weighted matrix is determined according to the optimized subspace algorithm of the subspace projection. Then, the DOA estimation, which is calculated by weighted l_{1}norm sparse representation, is formed by the weighted matrix and new array data.
3.1 The parallel algorithm of adaptive beamforming based on PI
It can be seen in Sect. 1 that many interference suppression algorithms, such as DOA estimation, require prior knowledge, and the performance is affected by SIR. To suppress strong interferences in the mixed signal, we propose a parallel algorithm of adaptive beamforming based on PI. We assumed that the signal received by the first array element is used as the reference signal, and the weighted output of the remaining M1 array elements is expressed as
$${\mathbf{y}}_{1,M  1} = {\mathbf{w}}_{M  1}^{H} {\mathbf{X}}_{M  1}$$
(6)
In the above formula, w_{M1} = [w_{2}, w_{3}, …, w_{M}]^{T} is the weighted vector, and the signal of the 2nd to Mth array elements is X_{M1} = [x_{2}, x_{3}, …, x_{M}]^{T}. According to the minimum mean square error criterion [20], the 2nd to Mth channel can be used as input to obtain the weighted matrix
$${\mathbf{w}}_{M  1} { = }{\mathbf{R}}_{b}^{  1} {\mathbf{R}}_{a}$$
(7)
where R_{b} = E[X_{M1}XH M1] is the autocorrelation of the second to M array element data, and \(\mathbf{R}_a = E\left[\mathbf{X}_{M1}{x^*_1}\right]\) is the crosscorrelation between the second to M array element data and the first array element data. Therefore, the optimal weighted vector under this model is
$${\mathbf{w}}_{{{\text{opt1}}}} { = }\left[ {\begin{array}{*{20}c} 1 \\ {{\mathbf{w}}_{M  1} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} 1 \\ {{\mathbf{R}}_{b}^{  1} {\mathbf{R}}_{a} } \\ \end{array} } \right]$$
(8)
To obtain the best weight vector and update it in time, we use the least mean square algorithm to achieve adaptive weight calculation [21]; that is, the weight at time n + 1 is determined by the weight and correction number at time n. The adaptive weight is
$${\mathbf{w}}_{M  1} \left( {n + 1} \right){ = }{\mathbf{w}}_{M  1} \left( n \right){ + }\mu {\mathbf{X}}_{M  1} \left( n \right)\left[ {x_{1} \left( n \right)  {\mathbf{w}}_{M  1}^{H} \left( n \right){\mathbf{X}}_{M  1} \left( n \right)} \right]^{*} , \, \mu \left( {0 < \mu < {\text{tr}}\left( {{\mathbf{R}}_{b} } \right)} \right)$$
(9)
where μ is the convergence factor, and the weighted output signal of M array elements can be expressed as
$${\mathbf{y}}_{1} = {\mathbf{w}}_{{{\text{opt1}}}} {\mathbf{X}}$$
(10)
At this time, y_{1} is the signal vector after suppressing strong interference, but the signal vector cannot form a steering vector, which is unable be processed by spatial array signal processing. Therefore, the DOA estimation cannot be performed with y_{1}. To solve this problem, we propose a parallel PI algorithm as shown in Fig. 2. One of the array elements is used as the reference signal and other M−1 array elements to solve the weight vector. Each of the reference array data and other M−1 array data are substituted into Eqs. (6)–(10) to obtain the new Mchannel data Y = [y_{1}, y_{2}, …, y_{M}]^{T} after interference suppression. For the reference array data, w_{optm} is a fixed constant (m = 1, …, M), so the phase of y_{M} is the same as original signal x_{m} according to Eq. (10). That is, the spatial processing cannot change the phase of the original signal, and the phase of the original signal is basically unchanged after the Mline parallel PI algorithm. Therefore, the output signal can be used as the input for subsequent DOA estimation.
We derive the new array signal obtained by the Mline parallel PI algorithm as a matrix form
$${\mathbf{Y}} = {\mathbf{A}}_{k} {\mathbf{S}} + {\mathbf{N}}_{y}$$
(11)
where N_{y} = [N_{y1}, N_{y2}, …, N_{yM}]^{T} is an M × T additive white Gaussian noise matrix with zero mean and σ_{n}^{2} variance, and N_{ym} is the noise received by the mth antenna. The sparse signal S contains the angle of the desired signal. Remarkably, DOA estimation is the process of reconstructing signal S by using the new array signal Y and steering matrix A_{k}. Therefore, we can determine the DOA estimation of the desired signal according to the number of nonzero elements in S. Equation (11) can be expressed as a l_{0}norm model, but the l_{0}norm is an NPhard problem, which can often be expressed as a l_{1}norm model. Therefore, Eq. (11) can be expressed as
$$\min \left\ {{\mathbf{Y}}  {\mathbf{A}}_{k} {\mathbf{S}}} \right\_{F}^{2} + h\left\ {{\mathbf{S}}^{{l_{2} }} } \right\_{{l_{1} }}$$
(12)
where h is the regularization parameter, which is affected by noise. S^{l2} is the l_{2}norm of each row in the matrix S. YA_{k}S2 F =vec(Y − A_{k}S)2 l_{2} is used to straighten matrix Y − A_{k}S by the matrix vec operator and calculate the l_{2}norm. To reduce the calculation, the array signal can be sparsely represented as [22]
$$\min \left\ {{\mathbf{Y}}_{svw}  {\mathbf{A}}_{kw} {\mathbf{S}}_{sv} } \right\_{F}^{2} + h\left\ {{\mathbf{S}}^{{l_{2} }} } \right\_{{l_{1} }}$$
(13)
where Y_{svw} = R_{y}^{−1/2} Y_{sv}, A_{kw} = R_{y}^{− 1/2} A_{k}, S_{sv} = SV_{y}D_{yk}, R_{y} is the covariance matrix of Y, Y_{sv} = YV_{y}D_{yk} is the dimension reduction matrix after the singular value decomposition of Y, V_{y} is a T × T dimensional right singular value matrix after the singular value decomposition of Y, D_{yk} = [I_{yk}, 0], I_{yk} is a K × K dimensional identity matrix, and 0 is a K × (T − K) dimensional zero matrix.
3.2 Weighted l
_{1}norm sparse representation DOA estimation for subspace projections
According to Formula (10), we can estimate the DOA of weak signals under strong interference. However, the l_{1}norm will destroy the sparsity of the solution and create more spurious peaks in the spectrum under low SNR, resulting in poor accuracy of weak signals DOA estimation and even failure. To solve this problem, we propose a weighted l_{1}norm sparse representation DOA estimation for subspace projection. Since the desired signals, interference signals and noise received in the original array are not correlated with each other, the covariance matrix of the received array signals is expressed as
$${\mathbf{R}}_{x} = E\left\{ {{\mathbf{X}}\left( t \right){\mathbf{X}}^{H} \left( t \right)} \right\} = {\mathbf{R}}_{s} + {\mathbf{R}}_{j} { + }{\mathbf{R}}_{n}$$
(14)
where R_{s}, R_{j} and R_{n} are the covariance matrices of the desired signals, interference signals and noise, respectively. The covariance eigenvalue of the received array signal is decomposed as
$$\begin{gathered} {\mathbf{R}}_{x} = \sum\limits_{m = 1}^{M} {\lambda_{m} {\mathbf{u}}_{m} {\mathbf{u}}_{m}^{H} } \hfill \\ { = }\sum\limits_{m = 1}^{Q} {\lambda_{m} {\mathbf{u}}_{m} {\mathbf{u}}_{m}^{H} } + \sum\limits_{m = Q + 1}^{K + Q} {\lambda_{m} {\mathbf{u}}_{m} {\mathbf{u}}_{m}^{H} } + \sum\limits_{m = K + Q + 1}^{M} {\lambda_{m} {\mathbf{u}}_{m} {\mathbf{u}}_{m}^{H} } , \, \lambda_{m} \left( {m = 1,2, \cdots ,M,\lambda_{1} > \lambda_{2} > \cdots > \lambda_{M} } \right) \hfill \\ { = }{\mathbf{U}}_{j} {{\varvec{\Sigma}}}_{j} {\mathbf{U}}_{j}^{H} + {\mathbf{U}}_{s} {{\varvec{\Sigma}}}_{s} {\mathbf{U}}_{s}^{H} + {\mathbf{U}}_{n} {{\varvec{\Sigma}}}_{n} {\mathbf{U}}_{n}^{H} \hfill \\ \end{gathered}$$
(15)
where λ_{m}(m = 1,2, …, M, λ_{1} > λ_{2} > … > λ_{M}) is the M eigenvalue of R_{x}, and u_{m} is the eigenvector. ∑_{j} = diag{λ_{1}, λ_{2}, …, λ_{Q}}, ∑_{s} = diag{λ_{Q+1}, λ_{Q+2}, …, λ_{Q+K}} and ∑_{n} = diag{λ_{Q+K+1}, λ_{Q+K+2}, …, λ_{M}}are eigenmatrices of the interference signals, desired signals and noise, respectively. U_{j} = span{u_{1}, u_{2}, …, u_{Q}}, U_{s} = span{u_{Q+1}, u_{Q+2}, …, u_{Q+K}}, and U_{n} = span{u_{Q+K+1}, u_{Q+K+2}, …, u_{M}} are the interference subspace, desired signal subspace and noise subspace, respectively. Since the eigenvalues of the strong interference are much larger than desired signal, we can select the eigenvector U_{j} after selecting the eigenvalue. Therefore, the orthogonal complement space of the interference subspace is
$${\mathbf{U}}_{j}^{ \bot } = {\mathbf{I}}  {\mathbf{U}}_{j} {\mathbf{U}}_{j}^{H}$$
(16)
We project the array data onto this space to obtain the signal after suppressing interference
$${\mathbf{z}} = {\mathbf{U}}_{j}^{ \bot } {\mathbf{X}}$$
(17)
In the l_{1}norm sparse representation DOA estimation algorithm, the l_{1}norm is used to replace the l_{0}norm. There are only 0 or 1 in the l_{0}norm model, and their contributions to the objective function are equal [23]. The l_{1}norm model represents the minimum modulus, where the large number is the large modulus value and the small number is a small modulus value in S. To make the contribution of S to the objective function equal and the l_{1}norm closer to the l_{0}norm, a small weight punishment is applied to the large modulus, and a large weight punishment is applied to the small modulus in S, thus the reconstructed signal can obtain the same constraint. Therefore, S needs to be weighted so that the large modulus in Formula (13) is multiplied by the small weight, and the small modulus is multiplied by the large weight. The signal obtained by Formula (17) is used as the input to obtain the quotient of the l_{2}norm of the products of the direction vector with the noise subspace and the signal subspace. The weighted value of the l_{1}norm under a strong interference signal is
$$w_{zi} = {\raise0.7ex\hbox{${\left\ {{\mathbf{E}}_{zn}^{H} {\mathbf{b}}\left( {\theta_{i} } \right)} \right\_{2} }$} \!\mathord{\left/ {\vphantom {{\left\ {{\mathbf{E}}_{zn}^{H} {\mathbf{b}}\left( {\theta_{i} } \right)} \right\_{2} } {\left\ {{\mathbf{E}}_{zs}^{H} {\mathbf{b}}\left( {\theta_{i} } \right)} \right\_{2} }}}\right.\kern\nulldelimiterspace} \!\lower0.7ex\hbox{${\left\ {{\mathbf{E}}_{zs}^{H} {\mathbf{b}}\left( {\theta_{i} } \right)} \right\_{2} }$}}$$
(18)
where b(θ_{i}) is the steering vector, E_{zn} is the noise subspace and E_{zs} is the desired signal subspace. The weighted matrix is expressed as
$${\mathbf{W}}_{z} = {\text{diag}}\left\{ {w_{zi} } \right\}$$
(19)
Substituting Formula (19) into Formula (13), the weighted l_{1}norm sparse representation of the array signal can be derived as
$$\min \left\ {{\mathbf{Y}}_{svw}  {\mathbf{A}}_{kw} {\mathbf{S}}_{sv} } \right\_{F}^{2} + h\left\ {{\mathbf{W}}_{z} {\mathbf{S}}^{{l_{2} }} } \right\_{{l_{1} }}$$
(20)
Formula (20) is solved by the following secondorder cone programming problem
$$\begin{gathered} \min p + hq \hfill \\ s.t.\left\ {{\mathbf{Y}}_{svw}  {\mathbf{A}}_{kw} {\mathbf{S}}_{sv} } \right\_{F}^{2} \le p \hfill \\ \left\ {{\mathbf{W}}_{z} {\mathbf{S}}^{{l_{2} }} } \right\_{{l_{1} }} \le q \hfill \\ \end{gathered}$$
(21)
The DOA estimation can be obtained by optimizing S from Formula (21). The detailed steps are as follows:

(1)
According to the PI adaptive beamforming algorithm, y_{1} is calculated from the original received array signal in Formula (10).

(2)
The proposed parallel algorithm is used to calculate y_{2}, y_{3}, …, y_{M}, and construct a new array signal Y after suppressing strong interference.

(3)
The new array signal Y is converted into Formula (13) according to the sparse representation model.

(4)
The original received array signal is projected onto the subspace, and the weighted matrix W_{z} is obtained from Formulas (18), (19).

(5)
The DOA estimation is obtained from Eq. (21) using secondorder cone programming.