 Research
 Open Access
Matched, mismatched, and robust scatter matrix estimation and hypothesis testing in complex tdistributed data
 Stefano Fortunati^{1, 2}Email authorView ORCID ID profile,
 Fulvio Gini^{1, 2} and
 Maria S. Greco^{1, 2}
https://doi.org/10.1186/s1363401604170
© The Author(s). 2016
 Received: 7 July 2016
 Accepted: 29 October 2016
 Published: 21 November 2016
Abstract
Scatter matrix estimation and hypothesis testing are fundamental inference problems in a wide variety of signal processing applications. In this paper, we investigate and compare the matched, mismatched, and robust approaches to solve these problems in the context of the complex elliptically symmetric (CES) distributions. The matched approach is when the estimation and detection algorithms are tailored on the correct data distribution, whereas the mismatched approach refers to the case when the scatter matrix estimator and the decision rule are derived under a model assumption that is not correct. The robust approach aims at providing good estimation and detection performance, even if suboptimal, over a large set of possible data models, irrespective of the actual data distribution. Specifically, due to its central importance in both the statistical and engineering applications, we assume for the input data a complex tdistribution. We analyze scatter matrix estimators derived under the three different approaches and compare their mean square error (MSE) with the constrained CramérRao bound (CCRB) and the constrained misspecified CramérRao bound (CMCRB). In addition, the detection performance and false alarm rate (FAR) of the various detection algorithms are compared with that of the clairvoyant optimum detector.
Keywords
 Covariance matrix estimation
 Complex elliptically symmetric distribution
 Detection problem
 Constrained CramérRao bound
 Misspecified CramérRao bound
1 Introduction
This paper deals with two common inference problems in radar signal processing, namely the estimation of the disturbance covariance matrix and the adaptive detection of a radar target. In addition to the radar detection, the covariance matrix estimation is a fundamental prerequisite for a lot of applications in many different areas: the direction of arrival (DOA) estimation in array processing [1], the principal component analysis (PCA) [2], and the portfolio optimization in finance [3], just to name a few. We put the covariance estimation and the adaptive detection problems in the more general context of the scatter matrix estimation and hypothesis testing in the complex elliptically symmetric (CES) distribution family. CES distributions constitute a wide class of distributions that includes the complex Gaussian, generalized Gaussian, the Kdistribution, complex tdistribution and all the compound Gaussian distributions as special cases. Due to their flexibility and their capability to model a plethora of different data behavior, they are widely applied in many areas, such as radar, sonar, and communications [4, 5]. A CES distribution is completely characterized by the mean value γ, the scatter (or shape) matrix Σ, and the density generator g. Given a particular CES distribution, its density generator could depends on some extra parameters, (e.g., shape and scale parameters for a complex tdistribution) that are in general unknown and need to be estimated from the data along with γ and Σ.
Specifically, because of its generality, several aspects should be taken into account when making inference on the CES class. The first aspect concerns the existence, convergence, and computational complexity of optimal algorithms tailored (matched) to a particular CES distribution at hand. Think for example of the problem of the joint estimation of the mean value, of the scatter matrix, and of the extra parameters that characterize the density generator. As pointed out in, e.g., [6, 7], a joint maximum likelihood (ML) estimation of all these unknown quantities would encounter computational difficulties and convergence (or even existence) issues. To overcome this problem, one has to rely on suboptimal, computationally inexpensive and easy to implement estimators [8]. A different alternative could be to assume a simpler model, e.g., a Gaussian distribution, for the data behavior that allows one for an easy derivation of optimal (but generally mismatched) estimators or detection rules [9]. This consideration leads directly to another issue, namely the robustness to misspecification. In particular, it would be of interest to know whether the inference methods based on an assumed CES distribution can achieve “good” performance even if the data follow a different and, in general, more involved CES model. Finally, as direct consequence of the previous considerations, this analysis culminates in the possibility to derive and implement robust inference algorithms with good performance over the whole class of CES distributions, even if not optimal under any nominal model.
Following the line of the previous discussion, in this paper, we investigate and compare the matched, mismatched, and robust approaches for inference methods in complex tdistributed data. We focus on the multivariate complex tdistribution, since it has long been recognized by several authors from both the statistical (see, e.g., [6] and the references therein) and the signal processing communities (see, e.g., [10–13]) as a suitable and flexible model able to describe the heavytailed behavior of the measurements in many practical applications (e.g., radar detection).
The paper is organized in two parts. In the first part, we investigate the scatter matrix estimation problem. The second part deals with adaptive detection algorithms. In particular, in the first part, we investigate the performance loss in the scatter matrix estimation when the unknown extra parameters of the tdistribution are replaced with low computational complexity estimates obtained via the method of moments (MoM). This represents the matched case. The mean square error (MSE) of the “matched” estimators is compared with the constrained CramérRao bound (CCRB). Then, we address the mismatched case where, following the approach discussed in our recent work [14], the performance of the mismatched ML (MML) scatter matrix estimator derived under Gaussian assumption is evaluated and its MSE compared with the constrained misspecified CramérRao bound (CMRB) [15]. Finally, the minmax robust (among the whole CES class) constrained Tyler (CTyler) estimator [16] is introduced and its performance compared with the CCRB and the other previously derived estimators.
The second part of the paper focuses on the detection performance of three detection algorithms: the linear threshold detector (LTD) [12], i.e., the matched generalized likelihood ratio test (GLRT) detector for complex tdistributed data; Kelly’s detector [17], i.e., the GLRT detector derived under the misspecified Gaussian distribution; and the adaptive normalized matched filter (ANMF), that represents the robust detector among the CES class. The ANMF has been derived and analyzed by many authors under different names (see, e.g., [4, 18–23]). The three detectors are compared in terms of (i) constant false alarm rate (CFAR) property with respect to (w.r.t.) the scatter matrix and the extra parameters estimation and (ii) receiver operating characteristic (ROC) curves.
The remainder of the paper is organized as follows. In Section 2, a brief review of the main properties of the CES distribution class and of the complex tdistribution is provided. In Section 3, the scatter matrix estimation problem is introduced and the application of the matched, mismatched, and robust approaches extensively analyzed. In Section 4, the hypothesis testing problem in complex tdistributed data is investigated. Section 5 collects the simulation results, while Section 6 summarizes our conclusions.
2 The CES distribution class, the compoundGaussian subclass, and the complex tdistribution
The aim of this section is to provide a brief overview of the CES distribution class and makes no claim to completeness. For more comprehensive and detailed discussions, we refer the readers to the excellent works [4] and [5].
An important subclass of the CES distributions is the compoundGaussian (CG) distributions [24]. In particular, a CGdistributed random vector x _{ m } ~ CG _{ N }(γ, Σ, p _{ τ }) admits the following stochastic representation \( {\mathbf{x}}_m={}_d\;\boldsymbol{\upgamma} +\sqrt{\tau}\mathbf{P}\mathbf{w}={}_d\sqrt{\tau \cdot {Q}_w}\mathbf{P}\mathbf{u} \), where, as before, w ~ CN(0, I), u ~ U(ℂS ^{ N }), and Q _{ w } ~ Gam(N, 1). Usually, the positive real random variable τ is called the texture and the complex Gaussian random vector n = _{ d } Pw is called the speckle. It can be noted that a CESdistributed random vector belongs to the subclass of the CGdistributed random vector if and only if the square of its modular variate R ^{2} can be written as a random scaled gamma distribution, i.e., R ^{2} = τ ⋅ Q _{ w } and then R ^{2}τ ~ Gam(N, τ) [5].
 i)
The dataset \( \mathbf{x}={\left\{{\mathbf{x}}_m\right\}}_{m=1}^M \) is composed of M independent and identically distributed (IID) Ndimensional, zeromean, complex tdistributed random vectors.
 ii)
The scatter matrix Σ is a real and full rank matrix.
It must be underlined that the second assumption is quite strong and not always verified in radar/sonar applications. It is wellknown in fact that, if the power spectral density (PSD) of the disturbance is not symmetric around a central frequency, the autocorrelation function of the complex envelope of the data is complex valued and consequently also the scatter matrix (see, e.g., [25]). However, working with complex matrices would require the use of more sophisticated mathematical tools, i.e., the socalled Wirtinger calculus [26], but this general approach falls outside the scope of the paper. The case of complex scatter matrix will be considered in future works.
3 Scatter matrix estimation
This section deals with the scatter matrix estimation from a set of IID complex tdistributed data. As discussed in the previous section, we investigate three different approaches: the matched, the mismatched, and the robust cases. We also provide the relative performance bounds, i.e., the CCRB and the CMCRB.
3.1 The matched case for complex tdistributed data
In this section, we discuss and derive two matched estimators of the scatter matrix Σ and of the extra parameters λ and η by assuming to know perfectly the correct data model, i.e., the complex tdistribution. Building upon previous results, we investigate the performance of the following two estimators: (1) the constrained maximum likelihood (CML) estimator of Σ which uses the method of moments (MoM) estimates of λ and η and (2) a recursive (suboptimal) estimator of Σ, λ, and η.
3.1.1 The constrained MLMoM estimator (CMLMoM)
As we can see, Eq. (4) involves the unknown shape and scale parameter of the tdistribution. To estimate them, we use the low computational complexity (but suboptimal) MoM estimators. The MoM method consists of equating the experimental moments with the corresponding theoretical ones in order to obtain an estimate of the unknown parameters of interest. In particular, given a random variable r whose pdf depends on some unknown parameters, one needs to firstly evaluate analytically the moments m _{ k } ≜ E{r ^{ k }}, i.e., the expected values of powers of the random variable under consideration, and, secondly, equate the obtained expressions (that will depend on the unknown parameters) with the corresponding sample estimates of the moments, i.e., \( {\widehat{m}}_k\triangleq {\displaystyle {\sum}_{m=1}^M{r}_m^k} \), where \( {\left\{{r}_m\right\}}_{m=1}^M \) are M realizations of the random variable r.
It can be noted that the constraint on the trace of \( {\widehat{\boldsymbol{\Sigma}}}_{\mathrm{CML}}^{(k)} \) has to be imposed at each iteration.
3.1.2 The constrained recursive MLWMoM estimator (CMLWMoM)
 1.Initialization (k = 0)$$ {\widehat{\boldsymbol{\Sigma}}}_W^{(0)}=\mathbf{I}, $$(11)
 2.(k + 1)th iteration (for k = 0,…,K)$$ {\tilde{\mathbf{x}}}_m={\left({\widehat{\boldsymbol{\Sigma}}}_W^{(k)}\right)}^{1/2}{\mathbf{x}}_m, $$(12)$$ \left\{\begin{array}{c}\hfill {\widehat{\mu}}^{(k)}=\frac{1}{MN}{\displaystyle \sum_{m=1}^M{\displaystyle \sum_{n=1}^N{\tilde{x}}_{m,n}\kern3.5em }}\hfill \\ {}\hfill {\widehat{\mu}}_2^{(k)}=\frac{1}{MN}{\displaystyle \sum_{m=1}^M{\displaystyle \sum_{n=1}^N{\left{\tilde{x}}_{m,n}{\widehat{\mu}}^{(k)}\right}^2}}\kern0.5em \hfill \\ {}\hfill {\widehat{\mu}}_4^{(k)}=\frac{1}{MN}{\displaystyle \sum_{m=1}^M{\displaystyle \sum_{n=1}^N{\left{\tilde{x}}_{m,n}{\widehat{\mu}}^{(k)}\right}^4\kern0.5em }}\hfill \end{array}\right.\overset{\mathrm{eq}.\ (9)}{\Rightarrow}\kern1em {\lambda}_{WMoM}^{(k)}>2,{\eta}_{WMoM}^{(k)}, $$(13)$$ \left\{\begin{array}{l}{\mathbf{S}}_W^{\left(k+1\right)}={\displaystyle \sum_{m=1}^M\frac{{\mathbf{x}}_m{\mathbf{x}}_m^H}{{\mathbf{x}}_m^H{\left({\widehat{\boldsymbol{\Sigma}}}_W^{(k)}\right)}^{1}{\mathbf{x}}_m+{\widehat{\lambda}}_{WMoM}^{(k)}/{\widehat{\eta}}_{WMoM}^{(k)}}.}\\ {}{\widehat{\boldsymbol{\Sigma}}}_W^{\left(k+1\right)}=N{\mathbf{S}}_W^{\left(k+1\right)}/\mathrm{t}\mathrm{r}\left({\mathbf{S}}_W^{\left(k+1\right)}\right)\end{array}\right. $$(14)
Even if based on more accurate considerations about the marginal pdf of the entries of x _{ m }, the proposed recursive constrained MLwhitened MoM (CMLWMoM) estimator is itself a suboptimal algorithm. Moreover, the convergence of the recursive procedure is not guaranteed.
3.1.3 The constrained CramérRao bound (CCRB)
3.2 The mismatched case
In the matched case, the true data model and the model assumed to derive a joint estimator of the scatter matrix and of the shape and scale parameters are the same; that is, the model is correctly specified. However, a certain amount of mismatch is often inevitable in practice. Among others, the model mismatch can be due to an imperfect knowledge of the true data model or to the need to fulfill some operative constraints on the estimation algorithm (processing time, simple hardware implementation, and so on). In other words, even if the true but involved model is known, in order to derive a simple (mismatched) estimator for practical exploitation, one could decide to assume a simpler model, e.g., a Gaussian distribution. In our recent work [14], we investigated the behavior of the ML estimator of the scatter matrix in CESdistributed data under mismatched conditions, i.e., the mismatched ML (MML) estimator. Moreover, the existence of a lower bound on the error covariance matrix of a certain class of mismatched estimators has been investigated as well (see also [33]). In particular, it has been shown that the asymptotic distribution of the MML estimator is a Gaussian one, whose mean value is the minimizer (also called pseudotrue parameter vector) of the KullbackLeibler (KL) divergence between the true and the assumed data distributions and the covariance matrix is given by the socalled Huber “sandwich” matrix. For brevity, we refer the reader to the recent papers [14] and [33] and references therein for a more comprehensive and insightful review of these topics.
The covariance matrix is \( \mathbf{M}=E\left\{{\mathbf{x}}_m{\mathbf{x}}_m^H\right\}={\sigma}^2\boldsymbol{\Sigma} \), where tr(Σ) = N and σ ^{2} are the power. Hence, the parameter vector to be estimated can be expressed as \( \boldsymbol{\uptheta} ={\left[\begin{array}{cc}\hfill \mathrm{vecs}{\left(\boldsymbol{\Sigma} \right)}^T\hfill & \hfill {\sigma}^2\hfill \end{array}\right]}^T\in \varTheta \). However, the true data are distributed according to the complex tdistribution \( {p}_X\left({\mathbf{x}}_m;\overline{\boldsymbol{\uptheta}}\right)\triangleq {p}_X\left({\mathbf{x}}_m;\overline{\boldsymbol{\Sigma}},\lambda, \eta \right) \) of Eq. (2), where \( \overline{\boldsymbol{\uptheta}}={\left[\begin{array}{cc}\hfill \mathrm{vecs}{\left(\overline{\boldsymbol{\Sigma}}\right)}^T\hfill & \hfill \lambda \kern1em \eta \hfill \end{array}\right]}^T\in T \) is the true parameter vector and \( \overline{\boldsymbol{\Sigma}} \) is the true scatter matrix that could be different to the scatter matrix Σ of the assumed Gaussian distribution. A point need to be clearly highlighted: in the mismatched case, the parameter space Θ that parameterizes the assumed distribution and the (possibly inaccessible and unknown) parameter space T that parameterizes the true distribution may be different. In the case at hand, for example, T ⊂ ℝ ^{ l } × (0, ∞) × (0, ∞) while Θ ⊂ ℝ ^{ l } × (0, ∞) where × indicates the Cartesian product and l = N(N + 1)/2 as before. Moreover, the constraint on the trace of the scatter matrix limits both the true and assumed parameter vector to belong to two lower dimensional smooth manifolds \( \overset{\sim }{\mathrm{T}}=\left\{\overline{\uptheta}\in \mathrm{T}\Big\mathrm{t}\mathrm{r}\operatorname{}\left(\overline{\boldsymbol{\Sigma}}\right)=N\right\} \) and \( \tilde{\varTheta}=\left\{\boldsymbol{\uptheta} \in \varTheta \left\mathrm{t}\mathrm{r}\right.\left(\boldsymbol{\Sigma} \right)=N\right\} \), respectively.
3.2.1 The constrained MML (CMML) estimator
3.2.2 The constrained misspecified CramérRao bound (CMCRB)
3.3 The robust approach
Unlike previous scenarios, where the estimators of the scatter matrix have been derived by assuming the correct tdistributed data model (matched case) or the simpler, but different from the true one, Gaussian data model (mismatched case), we now focus on robust estimation, i.e., we aim at finding an estimator that does not assume any specific model for the data. A robust estimator is supposed to provide good estimation performance over a large set of different models (in the application discussed here, the set of CES models), even if not optimal under any nominal (matched or mismatched) one. Because of its generality, a robust estimator of the scatter matrix over the CES distributions will not rely on any additional estimates of unknown extra parameters, as it is for the matched ML estimator in Eq. (4) that depends on the estimates of λ and η.
4 Hypothesis testing problem for target detection
After having discussed the three approaches for the scatter matrix estimation in tdistributed data, we can introduce the classical radar detection problem. In particular, we address the problem of detecting a complex signal vector s in the received data x = s + c where c represents the unobserved complex noise/clutter random vector. The target signal s is modelled as s = α p where p (generally called target vector response or Doppler steering vector) is the transmitted known radar pulse vector and α = γe ^{ jϕ } ∈ ℂ is an unknown signal parameter accounting for both channel propagation effects and the target backscattering. α can be modelled as an unknown deterministic parameter or as a random variable depending on the application at hand. When modelled as a random quantity, α is assumed to be a circular Gaussian random variable \( \alpha \sim CN\left(0,{\sigma}_{\alpha}^2\right) \) where the amplitude γ is Rayleigh distributed and the phase ϕ is uniformly distributed in [0, 2π) and independent of γ. More general target models are the socalled Swerling models [42]. Regarding the complex noise vector c, it has been successfully modelled as a zeromean CESdistributed random vector with covariance matrix M = σ ^{2} Σ, where Σ and σ ^{2} represent the unknown scatter matrix and the unknown statistical noise power. In particular, c is modelled as a complex tdistributed random vector [12, 13, 24].
4.1 The matched case and the linear threshold detector
4.2 The mismatched case and the Kelly’s GLRT
We note, in passing, that the Kelly’s GLRT emerges also in detection problems involving CESdistributed data. In particular, in [43], it is shown that the Kelly’s GLRT is a robust detector over a wide subclass of CES data distributions. However, it must be noted that the clutter model assumed in [43] is different from the IID model in Eq. (57). The model adopted here corresponds to what Raghavan and Pulsone in [44] called the “independent model,” whereas the one considered in [43] corresponds to the socalled dependent model.
4.3 The robust approach and the ANMF
As a consequence of the consistency of the Tyler’s estimator, the resulting adaptive test statistic Λ _{ANMF} will have approximately a beta(1,N1) distribution for sufficiently large M, i.e., Λ _{ANMF} is asymptotically CFAR w.r.t. Σ, as desired [5]. Further discussions on the asymptotic properties of the Λ _{ANMF} can be found in [45] and [46].
5 Simulation results
In this section, we integrate through extensive numerical simulations, the theoretical findings on scatter matrix estimation and adaptive detection discussed in previous sections. In all the simulation results reported here, the true scatter matrix is assumed to be [Σ]_{ i,j } = ρ ^{i − j}, for i, j = 1,2,…,N. Note that ρ is the clutter onelag correlation coefficient that is assumed to be real. Under this assumption, Σ is real, as well. To exploit this assumption, in all numerical simulations, we took the real part of the scatter matrix estimators. The extension to the more general case of complex scatter matrix will be investigated in future works.
5.1 Estimation performance
 1.
 2.
 3.

Regarding the scatter matrix estimation, the robust CTyler estimator is an “almost” efficient estimator, even if it is not the most efficient estimator for tdistributed data, in fact when λ increases, the other two estimators achieve better performance. The MSE ε _{C ‐ Tyler} is close to the CCRB especially for small λ (see Figs. 1, 5, and 9). In particular, its performance is robust, i.e., it is not affected by the value of the shape parameter λ (see Fig. 5), even if it is not efficient for large λ.

Regarding the CMML estimator, it always achieves the CMCRB, both for the scatter matrix estimation and for the estimation of the average power (see Figs. 1, 5, 8, 9, and 12). The CMML presents a small bias on the estimation of the scatter matrix and then, \( {\widehat{\boldsymbol{\Sigma}}}_{\mathrm{CMML}} \) is not a MSunbiased estimator [9] (at least in the finite sample regime). For this reason, ε _{CMML} is in general slightly below the CMCRB. The loss in estimation accuracy due to the mismatch is particularly high for extremely heavytailed data, i.e., when λ is close to 0 (see Fig. 5). When λ → 0, the CMCRB rapidly increases while the CCRB is quite independent of λ. On the other hand, when λ → ∞, the CMCRB and the CCRB coincide, as expected, and the performance of the CMML estimator converge to that of the CMLMoM and CMLWMoM estimators.

Quite surprisingly, even if the MoMbased estimators fail to provide an accurate estimate of λ as it increases (see Fig. 6), the MSE of the CMLMoM and CMLWMoM estimators achieve the CCRB, as shown in Fig. 5.

Regarding the estimation of λ and η, the recursive WMoM estimator always outperforms the classical MoM estimator (see Figs. 2, 3, 7, 10, and 11), even though it does not achieve the CCRB. In particular, as shown in Fig. 10, the MSE of the WMoM is independent from the value of ρ, while this is not the case for the MSE of the classical MoM estimator. This desirable behavior of the WMoM estimator is due to the whitening operation that makes each entry of the data vectors mutually uncorrelated, as discussed in Section 3.1.
5.2 Detection performance
 1.The probability of false alarm (P _{FA}) as function of the onelag coefficient ρ (Fig. 13). This allows us to verify the CFAR property of the Λ _{LTD ‐ CML ‐ MoM} (Eq. (59)), Λ _{LTD ‐ CML ‐ WMoM} (Eq. (60)), Λ _{Kelly} (Eq. (63)), and Λ _{ANMF ‐ C ‐ Tyler} (Eq. (66)) w.r.t. the correlation shape. Simulation parameters: N = 16, M = 3 N, λ = 3, η = 1, K = 4. The detection thresholds have been set to achieve a nominal P _{FA} of 10^{−3}.
 2.The probability of false alarm (P _{FA}) as function of the shape parameter λ of the true complex tdistribution, i.e., for different spikiness levels (Fig. 14). This is important, since it highlights the CFARness of the four detectors w.r.t. the nonGaussianity level of the data. Simulation parameters: N = 16, M = 3 N, ρ = 0.8, η = 1, K = 4. The detection thresholds have been set to achieve a nominal P _{FA} of 10^{−3}.
 3.The receiver operating characteristic (ROC) curves (Fig. 15). The simulation parameters are the following: N = 16, M = 3 N, ρ = 0.8, λ = 3, η = 1, and K = 4. Moreover, \( \alpha \sim CN\left(0,{\sigma}_{\alpha}^2\right) \) where \( {\sigma}_{\alpha}^2 \) is set to have signal to noise power ratio (SNR) equal to 3 dB.
As we can see from Fig. 13, all the analyzed detectors are CFAR with respect to ρ. Their P _{FA} curves are constant and close to the nominal value 10^{−3}. A different behavior can be observed in Fig. 14, where the P _{FA} curves have been evaluated as function of λ. It can be noted that only Λ _{ANMF ‐ C ‐ Tyler} is a CFAR detector w.r.t. the data spikiness, while the P _{FA} of the other detectors change with λ. Finally, in Fig. 15, the ROC curves of Λ _{LTD ‐ CML ‐ MoM}, Λ _{LTD ‐ CML ‐ WMoM}, Λ _{Kelly ' s GLRT}, and Λ _{ANMF ‐ C ‐ Tyler} are shown. For the sake of comparison, we evaluated also the ROC of the clairvoyant optimum detector for the tdistributed data, i.e., the Λ _{LTD} in Eq. (58) where Σ, λ, and η are perfectly known. As we can see, the performance of Λ _{LTD ‐ CML ‐ MoM}, Λ _{LTD ‐ CML ‐ WMoM}, and Λ _{ANMF ‐ C ‐ Tyler} are close to that of the clairvoyant detector Λ_{LTD}, while Λ _{Kelly ' s GLRT} undergoes some detection loss for relatively low value of the P _{FA}. In particular, the fact that the performance of the robust Λ _{ANMF ‐ C ‐ Tyler} is close to the one of the matched detectors, Λ _{LTD ‐ CML ‐ MoM} and Λ _{LTD ‐ CML ‐ WMoM}, suggests that the detection loss due to the robustness is small. However, it must be highlighted again that we are considering a particular scenario in which the clutter covariance matrix is assumed to be real and full rank. Moreover, due to the high computational load of the Monte Carlo simulations, the detection performance of the proposed detectors has been evaluated only for a P _{FA} greater that 10^{−5}. It would be very useful to investigate the detection performance at an operative value of P _{FA}, e.g., below 10^{−5}.
6 Conclusions
This paper focused on two inference problems, the scatter matrix estimation and the adaptive detection of radar targets in complex tdistributed data. Three different approaches have been investigated and compared: the matched, the mismatched, and the robust approaches. Regarding the classical matched approach, we analyzed the performance of the CML estimator for the scatter matrix, when the shape and scale parameters are estimated through the lowcomplexity and suboptimal MoM method (CMLMoM) and a recursive improvement of it (CMLWMoM). We found that both the CMLMoM and the CMLWMoM estimators achieve the CCRB, while the CMLWMoM estimator outperforms the CMLMoM for the estimation of the shape and scale parameters. Then, the previous two estimators have been adapted to implement the LTD, which is the GLRT decision rule in tdistributed data. Numerical simulations show that the performance of the adaptive LTD are very close to the clairvoyant LTD detector, but it is not CFAR w.r.t. the variation of data spikyness. Regarding the mismatched approach, we proved that the CMML estimator derived under the assumption of Gaussiandistributed data converges almost surely to the true scatter matrix and to the true (tdistributed) data power, so it can be applied for inference problems that require the knowledge of these two quantities. Moreover, its efficiency with respect to the CMCRB has been shown and its performance loss with respect to the matched case discussed. The CMML estimator of the scatter matrix has been used in the “mismatched” Kelly’s GLRT. Numerical simulations proved that Kelly’s GLRT is not CFAR and presents large performance loss for small values of P _{FA}. Finally, the minmax robust CTyler scatter matrix estimator and the adaptive version of the robust NMF detector, that exploits the CTyler estimator, have been introduced and analyzed. In particular, our numerical results demonstrated that Tyler’s estimator is an “almost” efficient estimator w.r.t. the CCRB and its estimation accuracy is independent on the value of the shape parameter. More importantly, the resulting ANMF is CFAR w.r.t. the shape parameter, i.e., w.r.t. the level of data spikiness, and has only a small detection loss w.r.t. the clairvoyant LTD. To summarize, the results discussed in this paper show that the robust approach, thanks to its generality, robustness to misspecification, and small estimation and detection losses, seems to be the a good choice in practical applications.
Declarations
Competing interests
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
 H Krim, M Viberg, Two decades of array signal processing research: the parametric approach. IEEE Signal Process. Mag. 13(4), 67–94 (1996)View ArticleGoogle Scholar
 J Karhunen, J Joutsensalo, Generalizations of principal component analysis, optimization problems, and neural networks. Neural Netw. 8(4), 549–562 (1995)View ArticleGoogle Scholar
 O Ledoit, M Wolf, Improved estimation of the covariance matrix of stock returns with an application to portfolio selection. J. Empir. Finance 10(5), 603–621 (2003)View ArticleGoogle Scholar
 CD Richmond, Adaptive array signal processing and performance analysis in nonGaussian environments. Ph.D. thesis (MIT, Cambridge, MA, 1996). Available at: https://dspace.mit.edu/handle/1721.1/11005 Google Scholar
 E Ollila, DE Tyler, V Koivunen, VH Poor, Complex elliptically symmetric distributions: survey, new results and applications. IEEE Trans. Signal Process. 60(11), 5597–5625 (2012)MathSciNetView ArticleGoogle Scholar
 KL Lange, RJA Little, JMG Taylor, Robust statistical modeling using the t distribution. J. Am. Stat. Assoc. 84(408), 881–896 (1989)MathSciNetGoogle Scholar
 JP Romano, AF Siegel, Counterexamples in probability and statistics, Wadsworth and Brooks/Cole, Monterey, CAGoogle Scholar
 S Fortunati, MS Greco, F Gini, The impact of unknown extra parameters on scatter matrix estimation and detection performance in complex tdistributed data, IEEE Workshop on Statistical Signal Processing 2016 (SSP), (IEE, Palma de Mallorca, 2016), pp 1–4. doi:10.1109/SSP.2016.7551738
 S Fortunati, F Gini, MS Greco, On scatter matrix estimation in the presence of unknown extra parameters: mismatched scenario, EUSIPCO 2016, (Budapest, Hungary, 2016)Google Scholar
 A Balleri, A Nehorai, J Wang, Maximum likelihood estimation for compoundGaussian clutter with inverse gamma texture. IEEE Trans. Aerosp. Electron. Syst. 43(2), 775–779 (2007)View ArticleGoogle Scholar
 J Wang, A Dogandzic, A Nehorai, Maximum likelihood estimation of compoundGaussian clutter and target parameters. IEEE Trans. Signal Process. 54(10), 3884–3898 (2006)View ArticleGoogle Scholar
 KJ Sangston, F Gini, MS Greco, Coherent radar target detection in heavytailed compoundGaussian clutter. IEEE Trans. Aerosp. Electron. Syst. 48(1), 64–77 (2012)View ArticleGoogle Scholar
 A Younsi, M Greco, F Gini, A Zoubir, Performance of the adaptive generalised matched subspace constant false alarm rate detector in nonGaussian noise: an experimental analysis. IET Radar Sonar Navigation 3(3), 195–202 (2009)View ArticleGoogle Scholar
 S Fortunati, F Gini, MS Greco, The misspecified CramérRao bound and its application to the scatter matrix estimation in complex elliptically symmetric distributions. IEEE Trans. Signal Process. 64(9), 2387–2399 (2016)MathSciNetView ArticleGoogle Scholar
 S Fortunati, F Gini, MS Greco, The constrained misspecified CramérRao bound. IEEE Signal Process. Lett. 23(5), 718–721 (2016)View ArticleGoogle Scholar
 D Tyler, A distributionfree Mestimator of multivariate scatter. Ann. Stat. 15(1), 234–251 (1987)MathSciNetView ArticleMATHGoogle Scholar
 EJ Kelly, An adaptive detection algorithm. IEEE Trans. Aerosp. Electron. Syst. AES22(2), 115–127 (1986)View ArticleGoogle Scholar
 LL Scharf, B Friedlander, Matched subspace detectors. IEEE Trans. Signal Process. 42(8), 2146–2157 (1994)View ArticleGoogle Scholar
 F Gini, A cumulantbased adaptive technique for coherent radar detection in a mixture of Kdistributed clutter and Gaussian disturbance. IEEE Trans. Signal Process. 45(6), 1507–1519 (1997)MathSciNetView ArticleMATHGoogle Scholar
 E Conte, A De Maio, G Ricci, Recursive estimation of the covariance matrix of a compoundGaussian process and its application to adaptive CFAR detection. IEEE Trans. Signal Process. 50(8), 1908–1915 (2002)View ArticleGoogle Scholar
 F Gini, Suboptimum coherent radar detection in a mixture of kdistributed and Gaussian clutter. IEE Proc. Part F 144(1), 39–48 (1997)Google Scholar
 F Gini, MV Greco, A Farina, Clairvoyant and adaptive signal detection in nonGaussian clutter: a datadependent threshold interpretation. IEEE Trans. on Signal Processing 47(6), 1522–1531 (1999)View ArticleGoogle Scholar
 F Gini, MV Greco, A suboptimum approach to adaptive coherent radar detection in compoundGaussian clutter. IEEE Trans. Aerosp. Electron. Syst. 35(3), 1095–1104 (1999)View ArticleGoogle Scholar
 E Ollila, DE Tyler, V Koivunen, HV Poor, CompoundGaussian clutter modelling with an inverse Gaussian texture distribution. IEEE Signal Processing Letters 19(12), 876–879 (2012)View ArticleGoogle Scholar
 A Farina, F Gini, MV Greco, L Verrazzani, High resolution sea clutter data: statistical analysis of recorded live data. IEE Proc. Radar Sonar Navigation 144(3), 121–130 (1997)View ArticleGoogle Scholar
 R Remmer, Theory of complex functions (SpringerVerlag, New York, NY, USA, 1991)View ArticleMATHGoogle Scholar
 MS Greco, F Gini, CramérRao lower bounds on covariance matrix estimation for complex elliptically symmetric distributions. IEEE Trans. Signal Process. 61(24), 6401–6409 (2013)MathSciNetView ArticleGoogle Scholar
 P Stinco, MS Greco, F Gini, Adaptive detection in compoundGaussian clutter with inversegamma texture. IEEE CIE International Conference on Radar 2011, 2011, pp. 434–437Google Scholar
 MS Greco, S Fortunati, F Gini, Naive, robust or fullyadaptive: an estimation problem for CES distributions. 8th IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM 2014), 2014, pp. 457–460Google Scholar
 JR Magnus, H Neudecker, The commutation matrix: some properties and applications. Ann. Stat. 7, 381–394 (1979)MathSciNetView ArticleMATHGoogle Scholar
 JD Gorman, AO Hero, Lower bounds for parametric estimation with constraints. IEEE Trans. Inf. Theory 6(6), 1285–1301 (1990)MathSciNetView ArticleMATHGoogle Scholar
 P Stoica, BC Ng, On the CramerRao bound under parametric constraints. IEEE Signal Processing Letters 5(7), 177–179 (1998)View ArticleGoogle Scholar
 CD Richmond, LL Horowitz, Parameter bounds on estimation accuracy under model misspecification. IEEE Trans. on Signal Processing 63(9), 2263–2278 (2015)MathSciNetView ArticleGoogle Scholar
 PJ Huber, The behavior of maximum likelihood estimates under nonstandard conditions. Proc. of the Fifth Berkeley Symposium in Mathematical Statistics and Probability (University of California Press, Berkley, 1967)MATHGoogle Scholar
 H White, Maximum likelihood estimation of misspecified models. Econometrica 50, 1–25 (1982)MathSciNetView ArticleMATHGoogle Scholar
 CC Heyde, R Morton, On constrained quasilikelihood estimation. Biometrika 80(4), 755–61 (1993)MathSciNetView ArticleMATHGoogle Scholar
 QH Vuong, CramérRao bounds for misspecified models, Working paper 652, Division of the Humanities and Social Sciences (Caltech, 1986). Available at: https://www.hss.caltech.edu/content/cramerraoboundsmisspecifiedmodels
 MS Greco, S Fortunati, F Gini, Maximum likelihood covariance matrix estimation for complex elliptically symmetric distributions under mismatched conditions. Signal Process. 104, 381–386 (2014)View ArticleGoogle Scholar
 RA Maronna, Robust Mestimators of multivariate location and scatter. Ann. Stat. 4(1), 51–67 (1976)MathSciNetView ArticleMATHGoogle Scholar
 F Gini, MS Greco, Covariance matrix estimation for CFAR detection in correlated heavy tailed clutter. Signal Process. 82(12), 1847–1859 (2002)View ArticleMATHGoogle Scholar
 F Pascal, Y Chitour, J Ovarlez, P Forster, P Larzabal, Covariance structure maximumlikelihood estimates in compound Gaussian noise: existence and algorithm analysis. IEEE Trans. Signal Process. 56(1), 34–48 (2008)MathSciNetView ArticleGoogle Scholar
 M Skolnik, Introduction to radar systems: third edition (McGrawHill, New York, 2001)Google Scholar
 CD Richmond, A note on nonGaussian adaptive array detection and signal parameter estimation. IEEE Trans. on Signal Processing 3(8), 251–252 (1996)View ArticleGoogle Scholar
 NB Pulsone, RS Raghavan, Analysis of an adaptive CFAR detector in nonGaussian interference. IEEE Trans. Aerosp. Electron. Syst. 35(3), 903–916 (1999)View ArticleGoogle Scholar
 F Pascal, JP Ovarlez, Asymptotic detection performance of the robust ANMF, 23rd European Signal Processing Conference (EUSIPCO), 2015, Nice, 2015, pp. 524528Google Scholar
 JP Ovarlez, F Pascal, A Breloy, Asymptotic detection performance analysis of the robust adaptive normalized matched filter, IEEE CAMSAP 2015, (Cancun, 2015), pp. 137140Google Scholar
 F Gini, JH Michels, Performance analysis of two covariance matrix estimators in compoundGaussian clutter. IEE Proceedings PartF 146(3), 133–140 (1999)Google Scholar