The Upper Bound of Multi-source DOA Information in Sensor Array and Its Application in Performance Evaluation

Direction of arrival (DOA) estimation has been discussed extensively in the array signal processing (cid:12)eld. In this paper, we focus on the DOA information which is de(cid:12)ned as the mutual information between the DOA and the received signal with complex additive white Gaussian noise. A theoretical expression of DOA information with multiple sources is presented in the uniform linear array. Specially, the upper bound of DOA information for sparse sources with high SNR is derived and compared with the information of single source. Moreover, the relationship between Cramer-Rao bound and the upper bound of DOA information is given. Finally, the paper investigate the performance evaluation of estimation based on the DOA information. We de(cid:12)ne the entropy error (EE) as a new performance evaluation index and (cid:12)nd that EE is better than mean square error. Moreover, the lower bound of the EE can be regarded as the Generalized Cramer-Rao bound considering the sources’ order in multi-source scenario.


Introduction
It has long been a focus of research using sensor arrays, e.g., radar, sonar, wireless communication to locate the far-field sources and estimate the parameters in various fields [1]. The problem of DOA estimation of multiple sources has been an active research area for decades [2,3,4]. Many high-resolution Direction of arrival (DOA) estimation algorithms (multiple signal classification (MUSIC) [5] [6], maximum-likelihood (ML) [7], estimating signal parameter via rotational invariance techniques (ESPRIT) [8], etc) are designed to determine the DOA of multiple narrowband noncoherent signals. The question followed is how to evaluate these algorithms. Therefore, estimation performance and error analysis of DOA estimation algorithm have also been studied widely [9,10,11,12,13]. Mean square error(MSE) is usually used to evaluate system performance and the Cramer-Rao bound (CRB) provides a fundamental physical limit on system accuracy. Stoica and Nehorai [14] introduced stochastic and deterministic signal models and derived the general expressions for the corresponding CRBs in the multi-source case.
The comparisons of multiple signal classification MU-SIC, ML and CRB are presented [15] [16]. However, CR-B is not a tight bound of MSE in low SNR region [17]. When the received signal is given, the probability distribution of DOA no longer obeys gaussian distribution in low SNR region. Thus, using MSE (a second order statistic) to evaluate the estimation results of the actual algorithm is insufficient when SNR is low. In this paper, we use information theory to define new performance evaluation index of multi-source DOA estimation algorithm.
The information theory [18] was proposed by Shannon in 1948 and plays a fundamental role in the field of information transmission, channel coding, data compression and etc. Similar to the communication system, the radar system and the sensor array system are both information acquisition systems. Woodward and Davies [19] [20] utilized mutual information to investigate the problem of measurement for target's range. Xu [21] have also employed the thoughts and methodologies of Shannon's information theory to systematically establish an information theory for a radar system in the presence of complex Gaussian noise. However, existing investigations based on Shannon information theory for DOA estimation mainly focus on the enumeration of source signals. Wax [22][23] introduced the information theory criterion into the problems of signal detection and proposed the methods to estimate the number of sources. To the best of our knowledge, only a few researchers employ the information theory to address the performance analysis of DOA estimation. Xu and Yan [24] studied the spatial status estimation process with a sensor array in view of information theory and provided the quantity of information obtained from the sensor array. In their study, the upper bound of DOA information in the single source scenario is derived. Furthermore, the entropy error(EE) is defined to measure the estimation performance. The relationship between EE, MSE, and CRB was presented. However, their research is not yet complete in the multi-source scenario. In this paper, the research of DOA information in multi-source scenario will be further promoted.
The remaining of this paper is organized as follows. In Section 2, We review the DOA information theory which includes system model and the definition of DOA information. Then a theoretical expression of DOA information in multi-source scenario is presented. Moreover, the upper bound of DOA information in multi-source scenario for sparse sources with high SNR is derived and compared with the DOA information of single source. We give the simulation comparison and discuss the obtained results in Section 3: The relationship between CRB and the upper bound of DOA information is given; The expression of EE and its low bound(EEB) in multi-source scenario is proposed; The comparison between EE, EEB, MSE and CRB in the case of two signals is presented. The Section 4 concludes the paper.

System Model
Suppose that there are K narrowband far-field sources impinging on a uniform linear antenna array with M elements, as shown in Fig. 1. The received signal at the m-th array element is given by is given by where s k (t) = α k e jφ k denotes the k-th(k = 1, 2, · · · , K) source signal. The source signal's amplitude α k is constant and its phase φ is random. w 0 is the angular frequency of carrier signal. w m (t) stands for the CAWGN at the m-th array element. And the noise added to different arrays is independent of each other. τ m (θ k ) represents the time delay of the k-th source signal with DOA θ k to the m-th array element. Suppose the distance between any two adjacent elements in the uniform linear array is d, then time delay τ m (θ k ) can be expressed by τ m (θ k ) = md sin θ k /v, where v is the propagation velocity of the signal. Constructing a ma- where in which where a (θ k ) is a so-called transfer vector between the k-th source and received signal.
Considering a single snapshot scenario, omitting time t, we can rewrite (2) as X = A (θ) S + W (8) Assuming that the source obeys uniform distribution within the observation interval of angle Q = [−∥Θ∥/2, ∥Θ∥/2], where ∥Θ∥ is the observation interval, then the prior PDF of the Θ is given by When the carrier frequency is very high, a small change in time delay will lead to a large change in phase. Therefore, Φ is regarded as a random variable subject to uniform distribution on the interval [0, 2π], so the prior PDF of Φ is given by Next, note that noise is CAWGN, and obeys where I is an identity matrix and E{·} denotes the expectation. N 0 is the power spectral density of noise, which represents the power of noise when the bandwidth is normalized. Then, we difine the signal to noise ratio as where α k 2 is the power of the useful signal. We will derive the expression of DOA information in the following sections.

DOA Information
In this section, we will provide the theoretical expression of the DOA information. The DOA information is defined as the mutual information between DOA and received source signal, i.e. I (X; Θ).we suppose the actual value of DOA is θ 0 = [θ 10 , θ 20 , · · · , θ K0 ] T . Considering CAWGN, the multi-dimensional PDF of X conditioned on Θ and Φ is given by With (9) and (13) , the joint probability density of X and Θ conditioned on Φ is derived as Then, the joint probability distribution of X and Θ is given by Consequently, the probability distribution of Θ conditioned on X is given by by omitting the terms independent of Θ, this expression can be simplified to where g (x, θ, φ) is given by Since the posteriori probability density of Θ is given, the quantity of DOA information obtained from multiple sources scenario is the difference of the priori entropy and the conditional entropy of Θ, i.e.
where h (Θ) denotes the prior information of Θ and h (Θ|X) denotes the conditional entropy of Θ when X is obtained. Clearly, the DOA information is algorithm-independent. It can provide a bound for the performance of algorithms, which has important theoretical guidance.

Upper Bound of DOA Information
The upper bound of DOA information in single-source scenario is obtained in previous papers. In this section, we will use some reasonable assumptions and approximation methods to derive the upper bound of DOA information in multi-source scenario.

Approximate expression of posterior PDF
Obviously, when the DOA of signal sources are close to each other, part of the DOA information will be lost because of the interference between sources. Therefore, in order to obtain the maximum DOA information, we suppose there are K(K << M ) independent sources with large spacing between any two sources to avoid this interference, i.e. sparse sources assumption.
Similar to single source scenario, p(θ|x) presents Gaussian-like distribution centered on the actual location of the source θ 0 . Thus, we obtain p(θ|x) in the neighborhood of θ 0 .
Clearly, in the case of multi-source, we have and β = 2πd sin θ/λ, λ is the wavelength of the signal.
Notice that when i = j, a(θ i ) H a (θ j ) = M . Furthermore, (21) has a distribution like the sinc function, its side lobe is quite small compared to the main lobe. Base on the sparse sources assumption, we have Therefore, (20) can be approximated to it follows that Substituting (23) in (18) results in In addition, for the actual received signal, we have where θ 0 is the actual value of DOA, and Same as (20), A H (θ) A(θ 0 ) can be approximated when θ is in the neighborhood of θ 0 . At this time, a(θ k ) H a (θ k0 ) is the only element left in its k-th row and the rest is approximated to 0, i.e.
Moreover, suppose that the signal amplitude of each source is equal, i.e. α k = α. It follows that ∮ exp dφ can be further derived as shown by ∮ exp where I 0 {·} is the first kind of zero-order Bessel function [25], and where β k = 2πd sin θ k /λ , β k0 = 2πd sin θ k0 /λ. And G (θ k ) can be regarded as the influence of the signal to the posteriori probability density of Θ.
By the way, is the direction diagram of the array pointing to θ k0 .
Similarly, ξ (θ, w) can be regarded as the influence of the noise to the posteriori probability density of Θ.
Therefore, under the sparse sources condition, Eq(17) can be rewritten as We have known that in the single source scenario, DOA information will approach to an upper bound with the increasing of SNR. The closed expression of the upper bound was derived under the condition of high SNR. Therefore, we follow this condition to derive the upper bound of multi-source DOA information. Considering the posterior PDF is composed of signal and noise components, in the case of high SNR, we can neglect the noise components to approximate p(θ|x) when θ is in the neighborhood of θ 0 . Moreover, p(θ|x) tends to 0 out of the neighborhood. Thus, we have in which where κ is a normalizing constant. In order to obtain the approximation of DOA information, we approximate |G (θ k )| using the first-order Taylor series expansion at θ k = θ k0 , it follows that where L 2 = π 2 L 2 /3 is root mean square aperture width, L = M d/λ denotes the normalized aperture width, cosθ k0 is direction cosine of sensor arrays. Substituting (35) in (34) and using the expansion of the Bessel function It follows that the approximation of (34) is given by is covariance matrix of K-dimensional Gaussian distribution.

Correction expression of posterior PDF
Using the expression of p(θ|x) , we can observe the posterior PDF through numeral calculation. We take the two-source scenario as an examplethe actual value of DOA is set as θ 0 = [θ 10 , θ 20 ] T . As shown in Fig. 2, the posterior PDF presents a two-dimensional probability distribution with two peaks, which are located at θ = [θ 10 , θ 20 ] T and θ = [θ 20 , θ 10 ] T . According to the simulation results, We find that the distribution of the posterior probability is mainly in the neighborhood of P π l θ 0 .
Since the order of sources is not concerned and the K elements of θ 0 = [θ 10 , θ 20 , · · · , θ K0 ] T have K! different permutations, the posterior probability distribution presents a K-dimensional probability distribution with K! peaks when there are K sources. To facilitate further derivation, we introduce the concept of permutation matrix.
where e π l (k) is the unit row vector whose π l (k)-th element is 1. Because identity matrix is given by In other words, P π l is the matrix obtained by permuting the rows of the identity matrix. Then the permutation of θ 0 can be represented as P π l θ 0 . According to the numeral calculation results, We find that the distribution of the posterior probability is mainly located in the neighborhood of P π l θ 0 . At this time, a(θ k ) H a ( θ π l (k)0 ) is the only element left in its k-th row and the rest is approximated to 0. Now, (27) can be rewritten as The subsequent derivation is the same as (28)-(37). Therefore, the correction expression of posterior PDF in the neighborhood of P π l θ 0 is given by where κ ′ ≈ 1/K! because p(θ|x) presents a Kdimensional probability distribution with K! same peaks when α k = α. And is the probability density function of K-dimensional Gaussion distribution. In which

Approximate expression of the upper bound
Then, we divide the domain of integration into K! domains centered on each peak. The PDF in the neighborhood of each peak is given by (42). Next, we can extend the integral domain to the whole domain when calculating the integral of each domain for convenience. The error caused by such approximation is acceptable because the value of the Gaussian distribution outside the neighborhood of each peak is close to zero. The calculation process is given by The differential entropy formula of multi-dimensional Gaussian distribution is used here.
By substituting (44) in (19) we obtain an approximation for the upper bound of DOA information where the first term of (46) is the sum of DOA information of every single source and the second term is the loss of information due to the uncertainty of sources' order. This is our main result.

Results and discussion
In this section, we provide the numerical results to illustrate the theoretical result in multi-source scenario with CAWGN. Taking the dual-source scenario for example, we consider the reflection coefficient α 1 = α 2 = 1 and the phase follows a uniform distribution in the interval [0,2π]. In the derivation process, θ 10 is located at −5 • and θ 20 is located at 5 • for convenience. However, In the actual simulation, θ 0 should be randomly distributed in the observation interval.

Upper bound of DOA information and CRB
CRB provides a fundamental physical limit on system accuracy. And the expression of CRB in the multisource scenario is given by [15], which is shown by recall that d (θ) = ∂a (θ) /∂θ.
Similar to single source scenario, DOA information can be calculated using CRB as the covariance matrix of K-dimensional Gaussian distribution.
Through numerical calculation, it can be found |C θ l | is close to the |CRB (θ)|. Thus I CRB is log K! bit bigger than the upper bound of DOA information. We can obtain few DOA information in the low SNR region as the noise is significant. Then DOA information increases monotonically with the increasing of SNR. And the slope first increases and then decreases to a constant in the large SNR region, i.e. the relation of the DOA information and the SNR via logarithm becomes linear. The upper bound of DOA information we derived is also shown in the figure which can verify the effectiveness of theoretical analysis in this study.

Entropy error and MSE
As we mentioned in the section of introduction, MSE is usually used to evaluate system performance and CRB provides the best accuracy achievable by any unbiased estimator of the signal parameters and provides a fundamental physical limit on system accuracy. We know that h (Θ|X) represents uncertainty of Θ when received signal is given. And the smaller the h (Θ|X), the better the performance of the system. In fact, EE is the entropy power of the posterior probability distribution, which can better measure the theoretical performance of the system. In the case of single source, Xu and Yan [24] have found that EE is consistent with MSE and tends to CRB in the high SNR region. In this section, We will discuss the relationship between the EE, MSE and CRB in the multi-source scenario.
Firstly, EE is defined as the entropy power of p (Θ |X ) to evaluate the DOA estimation [24]. The EE in the multi-source scenario is given by In the previous section, we have derived the conditional entropy of DOA in multi-source scenario for sparse sourses. Since (44) is the approximation of the conditional entropy in high SNR region, we can obtain an approximation of EE by substituting (44) into (49), it follows that EE's low bound(EEB) is given by From the expression (44), we find the term log(K!) is caused by the uncertainty of sources' order. It is also reflected in the expression of EE. However, MSE and CRB don't reflect the uncertainty of sources' order.
MSE is obtained by simulation of the maximum likelihood algorithm of DOA estimation.
For the convenience of comparison, CRB will be further calculated by Next, We compare EE and MSE for various SNR through simulation to show their relationship. According to the result of simulation showen in Fig. 3, we find MSE decrease monotonically with increasing SNR and tends to CRB in high SNR region. Similarly, EE decrease monotonically with increasing SNR and tends to EEB. However, unlike the single source scenario, EEB does not conincide with CRB in multi-source scenario.
The result above may explain why EE is better than MSE.
1) The posterior probability distribution of Θ no longer obeys gaussian distribution in the case of medium and low SNR. MSE is invalid as a second-order statistic when SNR is low.
2) EE reflects the influence of the uncertainty of sources' order in multi-source scenario. Fig. 4 shows the relationship of EE, EEB, MSE and CRB in the dual-source scenario. The EE is calculated by substituting the conditional entropy obtained by simulation into (49). EEB is given by (50). The MSE is obtained by the simulation of maximum likelihood  In addition, the main conclusions of this paper is given under the condition of high SNR and sparse sources. However, the research and conclusions of this paper provide guidance for further study of multisource DOA information and estimation performance evaluation in the general scenarios.

Conclusions
In this paper, a theoretical expression of DOA information in multi-source scenario is proposed. Moreover, we show that DOA information in multi-source scenario increases monotonically with the increasing of SNR and tends to an upper bound. And the upper bound of DOA information for sparse sources with high S-NR is given and compared with the DOA information of single source. Furthermore, we find that the upper bound of DOA information in multi-source scenario is the sum of that in single source scenario with the loss of information due to the uncertainty of sources' order. In addition, I CRB is log K! bit bigger than the upper bound of DOA information.
Then, we do the research of the practical application of DOA information. The expression of EE in multisource scenario is given. The comparison between EE and MSE in the case of two signals is presented by numerical simulation. It is illustrated that EE decrease monotonically with increasing SNR and tends to EEB. However, unlike the single source scenario, EEB does not conincide with CRB in multi-source scenario. On the one hand, the posterior probability distribution of Θ no longer obeys gaussian distribution in the case of medium and low SNR. MSE is invalid as a secondorder statistic when SNR is low. On the other hand, EE reflects the influence of the uncertainty of sources' order in multi-source scenario. Thus EE can be used as a better criterion to measure the performance of the sensor array detection system. In addition, EEB can be regarded as the generalized CRB considering the sources' order in multi-source scenario.
Further investigations will be undertaken in future works in order to complete the research of DOA information theory in other scenarios.