Skip to main content

Locally optimal detector design in impulsive noise with unknown distribution


This paper designs the locally optimal detector (LOD) in additive white impulsive noise with unknown distribution. Unlike traditional LODs derived from a known or approximated noise probability density function (PDF), the LOD proposed in this paper is achieved by designing the zero-memory non-linearity (ZMNL) function based on real data. After the PDF estimation in a nonparametric way by a kernel method, the ZMNL function is designed as a piecewise differentiable function consisting of a polynomial function and inverse proportional functions. Then, we analyze the detection performance and develop the constant false alarm ratio technique. Simulation results show that the LOD design is near-optimal in α-stable noise and the optimal in real atmospheric data, compared with the maximum likelihood detector of α-stable distribution.

1 Introduction

Generally, signal detection in additive noise can be viewed as a problem of binary or multiple hypothesis testing [1]. Most existing digital systems use linear detectors which are the optimal in additive white Gaussian noise. However, nonlinear processing is required for optimal detection in impulsive noise with heavy PDF tails. This is necessary for many environments, such as vehicular communication, power line communication, low-frequency communication, and underwater communications [24]. In low signal-to-noise ratio (SNR), the LOD can be realized via a simple structure by a nonlinear preprocessor based on ZMNL processing followed by a linear correlator [5, 6]. As is well-known, the ZMNL function can be derived directly if the noise PDF in closed form is known.

However, the ZMNL function needs designing when the analytical PDF is unavailable. For instance, the α-stable noise, which is widely used for modeling the impulse noise, does not provide the PDF in closed form generally [4]. As a result, researchers have developed various ZMNL functions for different α-stable distributions [79]. However, real-data processing results demonstrate that the impulsive noise does not always strictly obey the α-stable distribution, though it generally has a unimodal PDF like the α-stable noise. Therefore, the ZMNLs which are designed for α-stable noise are not surely optimal in real-data processing, as illustrated by the simulations in Section 6.4.

This paper focuses on the LOD design when the noise distribution is unknown, which is not fully discussed before. Actually, this is a reasonable consideration since the noise in real world would probably be varying and disobey the assumed distributions, e.g., the α-stable distribution. To solve this problem, this paper proposes to develop a practical approach for designing the ZMNL based on the real-noise data instead of knowing the PDF or assuming the noise distribution.

In the LOD design, lessons are drawn from the existing ZMNL functions of the symmetric α-stable (S αS) noise which is practically useful as a heavy-tailed distribution. In this paper, the ZMNL is designed as a piecewise function which follows linearity, nonlinearity, and differentiability in different regions. Then, the detection performances are analyzed theoretically. In simulations, the proposed approach is demonstrated in simulated S αS noise and in real atmospheric noise. The detection performances will be compared with several existing detectors.

The remainder of this paper is organized as follows. Section 2 briefly reviews several detectors for known noise PDFs. Section 3 introduces the preparation works for unknown noise PDF. Then, the ZMNL function is designed in Section 4, and detection performances are analyzed in Section 5. Section 6 presents the simulations in α-stable noise and real atmospheric noise. Finally, conclusions are drawn in Section 7.

2 Detectors in white noise with known PDF

Detecting a deterministic signal in additive white noise is a classical problem in statistical signal processing. A general solution is by hypothesis testing. Under hypothesis H i , the received signal is modeled as

$$\begin{array}{*{20}l} H_{i}: \boldsymbol{r} = \xi_{i} \cdot \boldsymbol{s}_{i}+ \boldsymbol{x} \end{array} $$

where s i , ξ i , and x denote the desired signal, the signal amplitude, and the additive white noise, respectively [1]. For the case of multiple sample detection, r, s i , and x can be considered as M-dimensional row vectors.

Maximum likelihood detector (MLD) is a well-known detector and requires the noise PDF. Considering multiple sample detection in white noise with a PDF f(x), the MLD is formulated as

$$\begin{array}{*{20}l} \max_{H_{i}}\; \ln\text{Pr}(\boldsymbol{r}, \boldsymbol{s}_{i} | H_{i}) \doteq \max_{H_{i}} \;\sum_{m=1}^{M} \ln f\left(\boldsymbol{r} [\!m]- \xi_{i} \boldsymbol{s}_{i}[\!m] \right) \end{array} $$

where the symbol “ \(\doteq \) ” denotes “being equivalent to.”

The decision rule (2) can be simplified for Gaussian noise. Under a realistic consideration of knowing the sign of ξ i , without loss of generality, assume ξ i >0. Then, simplifying (2) leads to the matched filter detector (MFD), formulated as

$$\begin{array}{*{20}l} \max_{H_{i}}\; \ln \text{Pr}(\boldsymbol{r}, \boldsymbol{s}_{i} ; H_{i})& \doteq \max_{H_{i}}\; -\sum_{m=1}^{M}\left(\boldsymbol{r}[\!m]- \xi_{i} \boldsymbol{s}_{i}[\!m]\right)^{2} \\ & \doteq \max_{H_{i}}\; \boldsymbol{r}\boldsymbol{s}_{i}^{\dag}, \quad \textrm{for Gaussian noise} \end{array} $$

where (·) denotes transpose. Obviously, the MFD depends on the linear correlation.

Decision rule (3) does not hold true for non-Gaussian noise. However, the MLD can be simplified in low SNR cases and results in the LOD, formulated as

$$\begin{array}{*{20}l} \max_{H_{i}} \;\ln \text{Pr}(\boldsymbol{r}, \boldsymbol{s}_{i} ; H_{i}) \doteq \max_{H_{i}} \; g(\boldsymbol{r})\boldsymbol{s}_{i}^{\dag},\quad \textrm{for low\, SNR} \end{array} $$


$$\begin{array}{*{20}l} g(x)=-{f^{\prime}}(x)/f(x) \end{array} $$

is called as the ZMNL function [1].

As can be seen from (5), g(x) is uniquely determined by f(x). Given analytically known f(x), g(x) is obtained directly in closed form. Otherwise, g(x) must be designed. Researchers usually chose to study g(x) by approximating the PDF. Figure 1 shows three reported ZMNL functions proposed for symmetric α-stable (S αS) noise, including the algebraic-tailed ZMNL (AZMNL, depending on α, for S αS distribution) [7], the Cauchy ZMNL (CZMNL, for Cauchy distribution α=1) [8], and the Gaussian-tailed ZMNL (GZMNL, independent on α, robust for S αS distribution) [9]. The ZMNL curves in Fig. 1 show that g(x) varies greatly for different distributions. This demonstrates the necessity of designing the optimal LOD for a unique distribution.

Fig. 1
figure 1

The LODs for S αS noise and reported ZMNL design [7], for σ=1

3 Preparation of LOD design for unknown PDF

This paper proposes to design the LOD and g(x) based on real data when the PDF is unknown. Preparatory work raised by unknown PDF is introduced in this section. The detailed ZMNL design approach will be presented in the next section.

3.1 Guidelines for the LOD design

This paper focuses on impulsive noise which comes from unimodal heavy-tailed distributions and has a PDF similar to the PDFs of existing heavy-tailed distributions such as the S αS noise and the Middleton class A noise [2]. However, on the one hand, due to the remarkable differences among the PDFs, the ZMNL functions of existing noise models are possibly not the optimal for the impulsive noise in various scenarios, giving rise to the necessity of LOD design. On the other hand, since the PDF shapes are similar, the knowledge of S αS noise is referable for our LOD design.

Analysis on the existing ZMNL functions of impulsive noise shows that most of them can be regarded as piecewise functions which consist of the main body in the near-linear region and the tails in the non-linear regions. At the breakpoints, natural ZMNLs deduced from heavy-tailed distributions are generally differentiable, while designed ZMNLs may be continuous or discontinuous.

In this paper, g(x) is designed as a piecewise function, following three guidelines. (i) The main body of g(x) is estimated by polynomial fitting. (ii) The tails of g(x) are modeled as reciprocal functions. (iii) The breakpoints are localized for continuity and differentiability. The detailed reasons of the guidelines will be presented in Section 4 where we develop the LOD design procedures. Before that, the rest of this section will introduce the preparatory work for the LOD design.

3.2 ZMNL sample calculation via the KDE

When the noise distribution is unknown, the PDF can be estimated by nonparametric estimation methods. Herein, this paper proposes to measure the PDF samples by the kernel density estimation (KDE) method, i.e., the Parzen-Rosenblatt window method [10, 11].

Let {x1,x2,…,x L } denote the amplitude vector with uniform increment Δx. Given the noise data \(\widetilde {x}[n], n=1,2,\dots,N\), the PDF samples are modeled as

$$\begin{array}{*{20}l} \widetilde{f}[\!x_{l}]=\frac{1}{Nh}\sum_{n=1}^{N}\mathcal{K}\left(\frac{x_{l}-\widetilde{x}[\!n]}{h}\right) \end{array} $$

where \(\mathcal {K}(\cdot) \) is the kernel function and h>0 is the smoothing parameter called the bandwidth.

Then, by formula (5), the ZMNL samples are calculated as

$$\begin{array}{*{20}l} \widetilde{g}[\!x_{l}]=-{\widetilde{f}_{\Delta}[\!x_{l}]}/{\widetilde{f}[\!x_{l}]} \end{array} $$


$$\begin{array}{*{20}l} \widetilde{f}_{\Delta}[\!x_{l}]=\left(\widetilde{f}[\!x_{l+1}]-\widetilde{f}[\!x_{l-1}]\right) /{(2\Delta x)} \end{array} $$

simulates f(x) at x=x l . Specially, let \(\widetilde {g}[\!x_{1}]=\widetilde {g}[\!x_{L}]=0\).

It is obvious that the value of bandwidth h is critical for the PDF estimation. In fact, since the PDF and ZMNL samples are simulated numerically in discrete points, the increment Δx is also very important. It must be evaluated reasonably so that \(\widetilde {f}[\!x_{l}]\) can describe the shape of f(x) effectively.

As said before, this paper emphasizes on the impulsive noise from heavy-tailed distributions. Although the noise may disobey the α-stable distribution, the α-stable distribution characteristics are also referable in some aspects. In our practice, the α-stable methods are found to be efficient in the evaluation of h and Δx.

The evaluation approach is developed as follows. Firstly, estimate the α-stable distribution parameters based on the noise samples \(\widetilde {x}[\!n]\). An estimation method can refer to paper [12] which is based on the quantile of samples.

Denote the estimated dispersion as \(\widehat {\gamma }\), which has similar meaning to the variance in Gaussian distribution. Then, the parameters related to the KDE are set as

$$\begin{array}{*{20}l} h&=\Delta x=0.1 \widehat{\gamma}, \end{array} $$
$$\begin{array}{*{20}l} x_{1}&= \frac{1}{N}\sum\limits_{n=1}^{N} \widetilde{x}[\!n]-\frac{N}{2}\Delta{x}, \end{array} $$
$$\begin{array}{*{20}l} x_{L}&= \frac{1}{N}\sum\limits_{n=1}^{N} \widetilde{x}[\!n]+\frac{N}{2}\Delta{x}. \end{array} $$

By this approach, the PDF estimate \(\widetilde {f}[\!x_{l}]\) can be smooth and well represent the probability density of noise samples.

3.3 ZMNL sample extraction

The ZMNL samples must be classified before using them for LOD design. The reason is as follows. As can be seen from (5), when \(\widetilde {f}(x_{l})\) is very small, the error between \(\widetilde {f}_{\Delta }[\!x_{l}]\) and f(x l ) can be greatly increased when entering \(\widetilde {g}[\!x_{l}]\), resulting in huge gap between \(\widetilde {g}[\!x_{l}]\) and g(x) (see simulations in Section 6). Thus, \(\widetilde {g}[\!x_{l}]\) samples for small \(\widetilde {f}[\!x_{l}]\) are unreliable.

Considering the ZMNL curves are nearly straight in center where \(\widetilde {f}[\!x_{l}]\) is large, we can extract the reliable ZMNL samples based on linearity, i.e., the cross-correlation coefficient between \(\widetilde {g}[\!x_{i}]\) and x. The linearity of \(\widetilde {g}[\!x_{i}]\) samples in any region \(\widetilde {\Omega }\) is measured by

$$\begin{array}{*{20}l} \rho_{\widetilde{\Omega}}=\frac{\sum\limits_{x_{i} \in {\widetilde{\Omega}}}(x_{i}-\overline{x_{i}})\left(\widetilde{g}[\!x_{i}]-\overline{\widetilde{g}[\!x_{i}]}\right)}{\sqrt{\sum\limits_{x_{i} \in {\widetilde{\Omega}}}(x_{i}-\overline{x_{i}})^{2} \cdot \sum\limits_{x_{i} \in {\widetilde{\Omega}}}\left(\widetilde{g}[\!x_{i}]-\overline{\widetilde{g}[\!x_{i}]}\right)^{2}}} \end{array} $$

where \(\overline {x_{i}}\) and \(\overline {\widetilde {g}[\!x_{i}]}\) denote the mean of x i and \(\widetilde {g}[\!x_{i}]\), respectively, for \(x_{i} \in {\widetilde {\Omega }}\).

The extraction algorithm is summarized in Table 1, where ρth is a threshold for linearity controlling. Generally, it is set 0.5≤ρth≤0.8.

Table 1 Algorithm of ZMNL sample extraction

The extracted ZMNL samples are denoted by

$$\begin{array}{*{20}l} \breve{{g}}[\!x_{i}]=\widetilde{g}[\!x_{i}],\; x_{i} \in \Omega=[\!x_{L_{\text{low}}}, x_{L_{\text{up}}}] \end{array} $$

which has I=LupLlow+1 samples.

4 Design of the LOD and ZMNL function

In the LOD design, the ZMNL function is modeled as a piecewise function. The main body is estimated by polynomial fitting. The tails and the breakpoints are determined for continuity and differentiability.

4.1 Polynomial fitting for near-linear region

The proposal of polynomial fitting is based on the basic property of ZMNL functions. As can be seen in (4), the ZMNL function transforms the observations r into the domain related to probability, acting like a weighting function. It may be linear (in Gaussian case) or nonlinear (in non-Gaussian cases). In either case, g(x) is supposed to keep close correlation to x for the majority of x, so that the test statistic \( g(\boldsymbol {r}) \boldsymbol {s}_{i}^{\dag }\) could be optimized. Therefore, the near-linear region always exists in g(x) for heavy-tailed distributions.

This paper employs a polynomial function to describe the nonlinearity property and also keep certain linearity in the near-linear region. This is unlike traditional ZMNL designs which define the nonlinear region very differently from the linear region and produce unavoidable error to the real LOD.

Considering the degree P polynomial, the ZMNL function is modeled as

$$\begin{array}{*{20}l} \sum_{p=0}^{P} A_{p}x^{p}, \quad x\in \Omega \end{array} $$

where A p , for p=0,1,…,P is the coefficient to be estimated. By using the ZMNL samples in (13), the linear equations can be written as

$$\begin{array}{*{20}l} \boldsymbol{A}\boldsymbol{X}_{\mathrm{v}}=\boldsymbol{G}_{x} \end{array} $$


$$\begin{array}{*{20}l} \boldsymbol{A}&=\left[A_{0} \quad A_{1} \quad \cdots \quad A_{P} \right], \end{array} $$
$$\begin{array}{*{20}l} \boldsymbol{G}_{x}&=\left[ \breve{g}[\!x_{1}] \quad \breve{g}[\!x_{2}] \quad \cdots \quad \breve{g}[\!x_{I}] \right], \end{array} $$
$$\begin{array}{*{20}l} \boldsymbol{X}_{\mathrm{v}}&=\left[{\begin{array}{cccc} 1 & 1& \cdots & 1\\ x_{1}^{1} &x_{2}^{1} & \cdots & x_{I}^{1} \\ x_{1}^{2} &x_{2}^{2} & \cdots & x_{I}^{2} \\ \vdots &\vdots & \ddots & \vdots \\ x_{1}^{P} &x_{2}^{P} & \cdots & x_{I}^{P} \end{array}} \right]. \end{array} $$

Since Xv is a Vandermonde matrix, a solution in least square (LS) sense is guaranteed to hold, provided that IP is satisfied generally. By the LS estimation, the parameter vector estimate is achieved by

$$\begin{array}{*{20}l} \widehat{\boldsymbol{A}}=\boldsymbol{G}_{x} \boldsymbol{X}_{\mathrm{v}}^{\dag} \left(\boldsymbol{X}_{\mathrm{v}} \boldsymbol{X}_{\mathrm{v}}^{\dag} \right)^{-1}. \end{array} $$

Plugging \(\widehat {\boldsymbol {A}}\) into (14) yields

$$\begin{array}{*{20}l} \widehat{g}(x)= \sum\limits_{p=0}^{P} \widehat{A}_{p} x^{p}, \quad & x \in \Omega. \end{array} $$

Thus, the main body of \(\widehat {g}(x)\) is achieved.

4.2 Tail design and breakpoint localization

Usually, the tail of g(x) functions as a controller which limits the effects of large-magnitude samples from heavy-tailed distributions. The tail function can be developed in various methods. For instance, Nikais and Shao proposed two kinds of nonlinearity for xΩ, including a “hole puncher” to null g(x) and a “clipper” to set g(x) equal to the nearest g(x i ) as constants. The tail of g(x) is also modeled as x1−α or x−1 in α-stable noise [4, 7]. Besides, the breakpoints which connect the tails with the main body are also important for the LOD design.

This paper models the tail as a reciprocal function and localizes the breakpoint for differentiability everywhere. Hereby, g(x) is formulated as a piecewise function

$$\begin{array}{*{20}l} \widehat{g}(x)=\left\{{\begin{array}{lc} B_{ne}/x, & x < x_{ne} \\ \sum\limits_{p=0}^{P} \widehat{A}_{p} x^{p}, & x_{ne} \leq x \leq x_{po} \\ B_{po}/x, & x > x_{po} \end{array}} \right. \end{array} $$

where the breakpoint locations x ne and x po and two coefficients B ne and B po are the parameters to determine.

Under the consideration of continuity and differentiability, four equations of x ne , x po , B ne , and B po can be obtained, listed in a set

$$\begin{array}{*{20}l} \left\{{\begin{array}{c} \frac{B_{ne}}{x_{ne}}= \sum\limits_{p=0}^{P} \widehat{A}_{p} x_{ne}^{p}\\ - \frac{B_{ne}}{x^{2}_{ne}}=\sum\limits_{p=1}^{P} p\widehat{A}_{p} x_{ne}^{p-1}\\ \frac{B_{po}}{x_{po}}= \sum\limits_{p=0}^{P} \widehat{A}_{p} x_{po}^{p} \\ - \frac{B_{po}}{x^{2}_{po}}= \sum\limits_{p=1}^{P} p\widehat{A}_{p} x_{po}^{p-1} \\ \end{array}} \right.. \end{array} $$

The solution of (22) is included in Table 2 which summarizes the algorithm of designing \({\widehat {g}}(x)\). Consequently, the LOD can be achieved by using \(\widehat {g}(x)\) in the decision rule (4).

Table 2 Algorithm of ZMNL function design

As far as we know, this is the first approach to use polynomial fitting for designing the ZMNL function \(\widehat {g}(x)\). Herein, we call the proposed ZMNL design as polynomial ZMNL (PZMNL).

One merit of the PZMNL is that it can approximate the main body of real LOD very well. As shown in Fig. 1, the LOD g(x) changes smoothly without a breakpoint to clearly classify the near-linear region and the nonlinear region. However, traditional ZMNL functions change dramatically at the breakpoints, from proportional functions in the near-linear region into nonlinear functions in the nonlinear region. They are greatly different from g(x) and thus have an upper bound of being sub-optimal. Unlike them, the PZMNL can fit well around the near-linear region and achieve nearly optimal performances.

Notation: About the polynomial order P, practical experience shows that 10≤P≤20 is suitable for polynomial fitting. Otherwise, if P is too small, the designed polynomial cannot represent the nonlinearity beyond the near-linear region and also reduces the solution to Eq. (23). On the contrary, if P is too large, the designed polynomial is not smooth enough and also generates unfavorable solutions to (23), resulting in a narrow near-linear region. Examples can refer to the simulations in Section 6.

5 Detection performance analysis of the LOD

This section analyzes the detection performances of the detectors based on various ZMNL functions. Supposing h(x) as the ZMNL function before a linear detector, based on the signal model (1), the test statistic is

$$\begin{array}{*{20}l} T(\boldsymbol{r})= h(\boldsymbol{r})\boldsymbol{s}^{\dag}_{i}=\sum\limits_{m=1}^{M} h(r[\!m])s_{i}[\!m] \end{array} $$

for hypothesis H i . If the PDF of T(r) is known, the detection performance can be deduced accordingly. However, the PDF of T(r) is uneasy to obtain generally.

Herein, we consider this topic in low SNR as a usual scenario in real world. Obviously, low SNR means that ξ i is rather small. As a result, to detect a transmitted signal bit, communication systems need to accumulate massive samples as M is large. Besides, the noise samples are assumed to be independently and identically distributed (i.i.d.). Therefore, by using the central limit theorem, we suppose that the test statistic T(r) obeys the Gaussian distribution. Its PDF is available as long as we know its expectation and covariance.

Before the deduction, the following assumption is made to simplify analysis.

Assumption 1

The impulsive noise is zero-mean, symmetrically distributed; thus, f(x) is even and f(x) is odd. The function h(x) used for ZMNL processing is considered to be almost odd, leading to

$$\begin{array}{*{20}l} \int\limits_{-\infty}^{\infty}h(x)f(x)dx \approx 0,\, \int\limits_{-\infty}^{\infty}h^{2}(x)f'(x)dx \approx 0. \end{array} $$

Then, the asymptotic distribution of T(r) in low SNR is derived, concluded as Theorem 1.

Theorem 1

For the signal model in (1) under Assumption 1 and low SNR, the test statistic T(r) in (25) asymptotically obeys Gaussian distribution

$$\begin{array}{*{20}l} T({\boldsymbol{r}}) \,\substack{a\\\sim\\ \quad}\, \mathcal{N} \left(\mathcal{E}_{hg} \xi_{i} \boldsymbol{s}_{i} \boldsymbol{s}_{i}^{\dag}, \mathcal{E}_{hh}\boldsymbol{s}_{i} \boldsymbol{s}_{i}^{\dag} \right), \quad H_{i} \end{array} $$

for M, where

$$\begin{array}{*{20}l} \mathcal{E}_{hg}&=\int\limits_{-\infty}^{\infty}h(x)g(x)f(x)dx, \end{array} $$
$$\begin{array}{*{20}l} \mathcal{E}_{hh}&=\int\limits_{-\infty}^{\infty}h^{2}(x)f(x)dx. \end{array} $$

The proof is referred in Appendix. Note that \(\mathcal {E}_{hg}\) and \(\mathcal {E}_{hh}\) are actually the expectations of h(x)g(x) and h2(x), respectively.

By Theorem 1, we can analyze the detection performances of a designed ZMNL function in a convenient way. For instance, in binary hypotheses

$$\begin{array}{*{20}l} \boldsymbol{r}=\left\{{\begin{array}{cl} \boldsymbol{x}, & H_{0}\\ \xi \cdot \boldsymbol{s}+ \boldsymbol{x}, & H_{1} \end{array}} \right. \end{array} $$

the asymptotic distribution of T(r) can be written as

$$\begin{array}{*{20}l} T({\boldsymbol{r}}) \,\overset{a}{\sim} \left\{ {\begin{array}{cl} \mathcal{N} \left(0, \mathcal{E}_{hh}\boldsymbol{s} \boldsymbol{s}^{\dag} \right), & H_{0}\\ \mathcal{N} \left(\mathcal{E}_{hg} \xi \boldsymbol{s} \boldsymbol{s}^{\dag}, \mathcal{E}_{hh}\boldsymbol{s} \boldsymbol{s}^{\dag} \right), & H_{1} \end{array}} \right. \end{array} $$

Then, the constant false alarm ratio (CFAR) technique can be achieved. A desired false alarm ratio (FAR) P fa is available by the threshold

$$\begin{array}{*{20}l} \eta=\sqrt{\mathcal{E}_{hh}\boldsymbol{s} \boldsymbol{s}^{\dag} }Q^{-1}(P_{fa}) \end{array} $$

and the corresponding detection probability is given by

$$\begin{array}{*{20}l} P_{D}&=Q\left[\left(\eta-\mathcal{E}_{hg} \xi \boldsymbol{s} \boldsymbol{s}^{\dag}\right)/\sqrt{\mathcal{E}_{hh}\boldsymbol{s} \boldsymbol{s}^{\dag}}\right]\\ &=Q\left[Q^{-1}(P_{fa})-\mathcal{E}_{hg} \xi \sqrt{\boldsymbol{s} \boldsymbol{s}^{\dag}/\mathcal{E}_{hh}}\right] \end{array} $$

where Q(·) is the tail probability of the standard normal distribution, defined as

$$\begin{array}{*{20}l} Q(x)=\frac{1}{2\pi}\int\limits_{x}^{\infty} \exp\left(-\frac{u^{2}}{2}\right)du. \end{array} $$

As clearly shown in (29) and (32), the calculation of threshold η depends on the information of PDF f(x). Thus, for the LOD in the noise with unknown PDF, the CFAR technique is actually unavailable.

The following develops the CFAR technique for the noise with unknown PDF. Notice that false alarm is related to hypothesis H0 and independent of hypothesis H1. In (31), hypothesis H0 requires f(x) to calculate the covariance coefficient \(\mathcal {E}_{hh}\). Though g(x) is also unknown, it is unnecessary in hypothesis H0. In fact, g(x) is only required in hypothesis H1 to calculate the expectation coefficient \(\mathcal {E}_{hg}\) in (28) and finally affects the detection probability P D in (33).

Therefore, this paper proposes the CFAR technique by using the PDF estimate \(\widetilde {f}[x_{l}]\) instead of the unknown real PDF f(x). By employing \(\widetilde {f}[\!x_{l}]\) in (6), we can compute \(\mathcal {E}_{hh}\) in (29) and then the threshold η by formula (32) for the desired FAR. Simulation results show that this approach can achieve a steady FAR which is very close to the desired FAR.

6 Results and discussion

This section presents the results of LOD design in the S αS noise and real atmospheric noise. Discussions about the proposed design of LOD and PZMNL are provided.

6.1 Detector and simulation settings

The general approach to suppress heavy-tailed interference is to pass the received observation through a ZMNL limiter. Two simple examples include a hole puncher and a clipper [4]. Some limiters are designed carefully according to the noise model.

This paper simulates three ZMNL methods which are proposed for S αS noise. The CZMNL is developed for the Cauchy distribution, i.e., S αS noise for α=1, formulated as

$$\begin{array}{*{20}l} g_{c}(x)=2x/\left(x^{2}+\sigma^{2}\right). \end{array} $$

The AZMNL is proposed for standard S αS noise, formulated as

$$\begin{array}{*{20}l} g_{a}(x)=\left\{ {\begin{array}{ll}{x\cdot\Gamma(3/\alpha)}/{\Gamma(1/\alpha)}, & |x| \leq \tau\\ K(\alpha)/{x}, & |x| > \tau \end{array}} \right. \end{array} $$

where \(\tau = \sqrt {K(\alpha)\Gamma (1/\alpha)/\Gamma (3/\alpha)}\) uses K(α)=α2. The GZMNL formula can refer to [7, 9] and is not introduced here for the sake of simplification.

As the analytical PDF of S αS noise is generally unavailable, the MLD is achieved by a numerical approach as follows. Firstly, the discrete PDF is computed by the inverse Fourier transform of the characteristic function. Then, assuming the signal amplitude is known, decision rule (2) is realized by linear interpolation. The ideal LOD by (5) is also obtained by linear interpolation of the discrete differential. Simulation results show that the MLD and the LOD perform very close to each other. Thus, the LOD and the MLD are depicted by the MLD curve. Obviously, linear interpolation bears expansive computational cost, but it is necessary for the MLD.

As S αS noise has an infinite covariance for α<2, herein, the SNR is measured by the generalized SNR (GSNR) as

$$\begin{array}{*{20}l} GSNR = 10\times \log_{10}\left({\xi^{2}}/{ \gamma }\right) \quad\textrm{in \; decibels} \end{array} $$

where γ denotes the dispersion of S αS noise. In simulations, γ is fixed as 1, so that the AZMNL in (36) can be used. The GSNR is changed by adjusting the signal amplitude ξ. The sinusoidal signal is transmitted, with uniform energy ss=M=1024. The probabilities of detection and false alarm are the results of 105 Monte Carlo simulations.

6.2 ZMNL design in S αS noise

α-Stable distribution is widely used in impulse noise modeling. As its PDF is generally unavailable in closed form, the analytical LOD does not exist for most α-stable distributions. Herein, we simulate the LOD of S αS noise numerically. S αS noise data is simulated for α=1.5, N=104. The KDE method uses the Gaussian kernel for \(\mathcal {K}(\cdot)\).

The sample \(\widetilde {g}[\!x_{l}]\) is depicted in Fig. 2. We can see that the curve \(\widetilde {g}[\!x_{l}]\) maintains linearity around zero and changes dramatically when \(\widetilde {f}[\!x_{l}]\) is rather small for |x|>5. Obviously, \(\widetilde {g}[\!x_{l}]\) samples outside the near-linear region are unacceptable for polynomial fitting. It demonstrates the necessity of extraction process. By ρth=0.8, \(\widetilde {g}[\!x_{l}]\) in Ω=[−5.7,6.5] are extracted for polynomial fitting.

Fig. 2
figure 2

An example of the PZMNLs in S αS noise for α=1.5, N=104

Then, \(\widehat A_{p}\) is calculated for various order P. Figure 2 depicts the designed PZMNL functions, as well as the ZMNL of S αS distribution \(\acute {g}(x)\) by numeric simulation. It can be seen that the PZMNL \(\widehat {g}(x)\) for P=10, with the breakpoints at x ne =− 3.3 and x po =3.6, is almost the same as \(\acute {g}(x)\). The PZMNLs for P=5 and 20 are less similar to \(\acute {g}(x)\). Moreover, the PZMNL for P=40 is with a narrow near-linear region and quite different from \(\acute {g}(x)\). Hence, setting P around 10 is suitable.

By comparison with Fig. 1, it is clear that the AZMNL, the CZMNL, and the GZMNL functions fit \(\acute {g}(x)\) worse than the PZMNLs for P=5, 10, and 20. Note that the PZMNL generally changes in another simulation which contains different noise samples, and the PZMNLs for P=40 is not always as bad as that of Fig. 2.

6.3 Detection performances in S αS noise

Detection performances by the CFAR technique are simulated. The results of detection probability in various GSNRs are depicted in Fig. 3, for α=1.5 and P fa =10−3. Figure 3a shows the curves for P D increases from P fa to 1 while the GSNR grows from low (− 25 dB) to high (− 10 dB). The curve shapes coincide with those of conventional CFAR cases since the theoretical P D in formula (33) has a conventional form. Considering Fig. 3a curves are too close for observation, Fig. 3b draws their details in a limited GSNR range, to provide a clear presentation for P D comparison. Considering the optimality of various ZMNLs keeps the same in different GSNRs, the following simulations will draw the figures of details and omit the figures of whole curves.

Fig. 3
figure 3

Detection probability versus the GSNR, for α=1.5, P fa =0.001: a curves for GSNR between − 25 and − 10 dB, b details for GSNR between − 21 and − 16 dB

As can be seen Fig. 3b, the CZMNL and the GZMNL curves are similar, worse than the other ZMNL detectors. The MLD is the best, and the AZMNL and the PZMNLs for P=5,10,20 are close to the MLD. Besides, the PZMNL for P=40 is worse, as a result of less fitting to \(\acute {g}(x)\). It is worth noting that all the ZMNL functions achieve significant improvement compared to the MFD.

The detection performances for various values of α are also simulated. When α grows, the GZMNL performs better while the CZMNL performs worse. The AZMNL and the PZMNLs for P=5,10,20 are near-optimal. However, when α decreases, the AZMNL and the GZMNL become worse while the CZMNL gets better. The PZMNL design approach still works in a near-optimal way. The results for α=1.0 is shown in Fig. 4, where the CZMNL and the MLD are optimal. Here, we conclude that 10≤P≤20 is suitable in the S αS noise for α(1,2).

Fig. 4
figure 4

Detection probability versus the GSNR, for α=1.0, P fa =0.001

6.4 Experimental results on real data

Atmospheric noise is known to possess a significant impulsive nature. This paper uses it to demonstrate the proposed PZMNL method. The raw data is recorded by a magnetotelluric sounding system at sampling frequency 512 Hz and then whitened to eliminate the power line interference. The output shows typical characteristics of impulsive noise and called as “real data.” The real data used for illustration is recorded in a sunny day at about 10:20 a.m., June 7, 2017, in Qianjiang, Hubei province, China. Its estimate of α is 1.42, near to α=1.5 of the previously simulated S αS noise. In preprocessing, the power of real data is adjusted so that the estimate of dispersion γ is 1.

The PZMNLs are designed in Monte Carlo simulations where a piece of 104-length samples is chosen randomly from the set of real data. One simulation result is depicted in Fig. 5. For comparison, we also simulate “ g(x) of S αS” which assumes S αS distribution, estimates the parameters, and generates the numeric ZMNL. As shown in Fig. 5, the PZMNL design produces smooth curves which fit the ZMNL samples well around the near-linear region, while the P=40 curve appears worse. Note that g(x) of S αS noise also performs well, suggesting its effectiveness in real-data processing.

Fig. 5
figure 5

The PZMNLs in real atmospheric noise for N=104. g(x) of S αS is under the assumption of S αS distribution, and the estimate of α is 1.42

The detection performances of PZMNLs are depicted in Fig. 6, by the CFAR technique proposed in Section 5, at P fa =0.001. For comparison, other detectors are also simulated, under the assumption of S αS noise. As can be seen, the PZMNLs for P=10,20 are better than the MLD. The other ZMNLs are much worse. Besides, the simulated FARs do not equal to the set FAR P fa =0.001 since the FAR and the threshold are calculated under the assumption of S αS noise and so are incorrect theoretically. However, the FARs of the PZMNLs are much close to 0.001 because of using the estimate \(\widetilde {f}[x_{l}]\) by the KDE method instead of the assumption of S αS distribution.

Fig. 6
figure 6

Detection probability versus the GSNR in real atmospheric noise. Except the PZMNLs, other detectors make the assumption of S αS distribution. Though P fa is set as 10−3, the simulated FARs are 10−3×[0.8,1.2,1.3,0.8,0.9,1.0,1.0,1.0] for the detectors from top to bottom, respectively

To achieve a constant FAR, the CFAR technique can be simulated by a numeric method as the following. The threshold η is determined based on the simulated test statistics in H0 and adjusted to fit the desired P fa . Then, η is used in H1 to evaluate the detection probability P D . Finally, the detection results are shown in Fig. 7, for GSNR = − 18 dB. We can see that the PZMNL of P=10 is the optimal, better than the MLD and the PZMNLs of P=5,20. The AZMNL and the PZMNL of P=40 are similar, while the GZMNL and CZMNL are much worse. It shows that the PZMNL of a proper order can outperform other ZMNL functions when the impulsive noise disobeys the assumed distribution.

Fig. 7
figure 7

Detection probability versus P fa in real atmospheric noise, for GSNR =− 18 dB. Except the PZMNLs, other detectors make the assumption of S αS distribution

6.5 Extended discussion

The above results demonstrate our proposed methods of LOD and PZMNL design are effective based on noise samples, no matter whether their distribution model is known. This is a significant advantage over other methods which are developed based on priori known models. In fact, impulsive noise has several different models, but there is no clear agreement or conclusion about their optimality. Once any detectors make false assumptions about the model of real noise, their performances degrade greatly. However, our methods do not have such a risk.

As our methods are proposed for impulsive noise, they may be used in other applications. Some limitations are worthy noting. Firstly, the PZMNL algorithm is developed for unimodal noise with heavy tails. Otherwise, multimodal noise has more than one near-linear regions and complicates the polynomial fitting. Secondly, Assumption 1 is necessary for the CFAR technique. It is easily satisfied in impulsive noise whose PDF is generally symmetric to zero, but may not for others. Finally, the PZMNL’s order P may vary for different distributions. It can be optimized in preprocessing and updated continuously in real time.

7 Conclusions

This paper proposes a novel approach for the LOD design in impulsive noise from unknown distributions. The ZMNL function is designed based on noise samples without assuming distribution models. As it is mainly developed by polynomial fitting, we call it as PZMNL. Simulations on α-stable noise show that the PZMNL achieves the detection performances similarly to the maximum likelihood detector which knows the noise PDF. Experimental results on real atmospheric noise demonstrate that the PZMNL outperforms other detectors which make false assumptions on the distribution models of real noise.

8 \thelikesection Appendix

8.1 \thelikesubsection Proof of Theorem 1

By the central limit theorem, as the elements of r are i.i.d., T(r) asymptotically converges to obey the Gaussian distribution for M. Under hypothesis H i , the expectation of h(r) is calculated

$$\begin{array}{*{20}l} E\left[h(r)\right]&=\int\limits_{-\infty}^{\infty}h(r)f(r-\xi_{i} s_{i})dr\\&=\int\limits_{-\infty}^{\infty}h(r)f(r)dr-\int\limits_{-\infty}^{\infty}h(r)f'(r)\xi_{i} s_{i} dr \end{array} $$

where the first-order Taylor series is used for low SNR

$$\begin{array}{*{20}l} f(r-\xi_{i} s_{i})=f(r)-\xi_{i} s_{i}f'(r). \end{array} $$

By the formulas (5), (26), and (28), we achieve

$$\begin{array}{*{20}l} E\left[h(r);H_{i} \right] \approx \xi_{i} s_{i}\mathcal{E}_{hg}. \end{array} $$

Since r[ m] is i.i.d., the expectation of T(r) is given as

$$\begin{array}{*{20}l} E\left[T(\boldsymbol{r});H_{i} \right]=\mathcal{E}_{hg}\xi_{i} \boldsymbol{s}_{i}\boldsymbol{s}_{i}^{\dag}. \end{array} $$

The covariance can be derived as

$$\begin{array}{*{20}l} \mathcal{D}\left[T(\boldsymbol{r});H_{i} \right]=\sum\limits_{m=1}^{M} \mathcal{D}\left[h({\boldsymbol{r}[m]});H_{i} \right]=\mathcal{D}\left[h({r});H_{i} \right]\boldsymbol{s}_{i}\boldsymbol{s}_{i}^{\dag}. \end{array} $$

Then, in low SNR, the expectation of h2(r) is computed as

$$\begin{array}{*{20}l} E\left[h^{2}({r});H_{i} \right]&=\int\limits_{-\infty}^{\infty}h^{2}(r)f(r-\xi_{i} s_{i})dr \\&=\int\limits_{-\infty}^{\infty}h^{2}(r)f(r)dr-\int\limits_{-\infty}^{\infty}h^{2}(r)f'(r)\xi_{i} s_{i} dr\\&\approx \mathcal{E}_{hh}. \end{array} $$

Therefore, the covariance of h(r) is given as

$$\begin{array}{*{20}l} \mathcal{D}\left[h(r);H_{i} \right]&=E\left[h^{2}(r);H_{i} \right]-\left(E\left[h(r);H_{i} \right]\right)^{2} \\&= \mathcal{E}_{hh}- (\xi_{i} s_{i}\mathcal{E}_{hg})^{2} \\&\approx \mathcal{E}_{hh} \end{array} $$

since ξ i is rather small. Then, the covariance of T(r) is

$$\begin{array}{*{20}l} \mathcal{D}\left[T(\boldsymbol{r});H_{i} \right]= \mathcal{E}_{hh}\boldsymbol{s}_{i}\boldsymbol{s}_{i}^{\dag}. \end{array} $$

Finally, the test statistic asymptotically obeys Gaussian distribution

$$\begin{array}{*{20}l} T(\boldsymbol{r}) \,\substack{a\\\sim}\, \mathcal{N} \left(\mathcal{E}_{hg} \xi_{i} \boldsymbol{s}_{i} \boldsymbol{s}_{i}^{\dag}, \mathcal{E}_{hh}\boldsymbol{s}_{i} \boldsymbol{s}_{i}^{\dag} \right), \quad H_{i}. \end{array} $$



algebraic-tailed ZMNL


constant false alarm ratio


Cauchy ZMNL


false alarm ratio


generalized SNR


Gaussian-tailed ZMNL


kernel density estimation


locally optimal detector


matched filter detector


maximum likelihood detector


probability density function


polynomial ZMNL



-stable; SNR:

signal-to-noise ratio


zero-memory nonlinearity


  1. SM Kay, Fundamentals of statistical signal processing, volume II: detection theory (Prentice Hall PTR, Englewood Cliffs, 1998).

    Google Scholar 

  2. H Oh, H Nam, Design and performance analysis of non-linearity preprocessors in an impulsive noise environment. IEEE Trans. Veh. Technol. 66(1), 364–376 (2017).

    Google Scholar 

  3. Y Yabuuchi, et al, in Proc. IEEE ISPLC. Low rate and high reliable modulation schemes for in-vehicle power line communications (IEEE explore digital library, 2011), pp. 249–254.

  4. CL Nikias, M Shao, Signal processing with alpha stable distribution and applications (John Wiley and Sons, Inc., New York, 1995).

    Google Scholar 

  5. X Li, et al, Normalisation-based receiver using BCGM approximation for α-stable noise channels. Electron. Lett. 49(15), 965–967 (2013).

    Article  Google Scholar 

  6. A Kadri, in European Wireless 2014; 20th European Wireless Conference; Proceedings of VDE. Suboptimal receivers for weak M-ary chirp signals in non-Gaussian noise (IEEE explore digital library, 2014), pp. 1–6.

  7. X Li, et al, Near-optimal detection with constant false alarm ratio in varying impulsive interference. IET Signal Proc. 7(9), 824–832 (2013).

    Article  Google Scholar 

  8. GA Tsihrintzis, CL Nikias, Performance of optimum and suboptimum receivers in the presence of impulsive noise modeled as an alpha-stable process. IEEE Trans. Commun. 43(234), 904–914 (1995).

    Article  Google Scholar 

  9. A Swami, BM Sadler, On some detection and estimation problems in heavy-tailed noise. Elsevier J. Signal Process. 82(12), 1829–1846 (2002).

    Article  MATH  Google Scholar 

  10. V Bhatia, B Mulgrew, Non-parametric likelihood based channel estimator for Gaussian mixture noise. Signal Proc. 87(11), 2569–2586 (2007).

    Article  MATH  Google Scholar 

  11. N Markovich, Nonparametric analysis of univariate heavy-tailed data: research and practice (John Wiley and Sons, New York, 2008).

    MATH  Google Scholar 

  12. JH McCulloch, Simple consistent estimators of stable distribution parameters. Commun. Stat. Simul. Comput. 15(4), 1109–1136 (1986).

    Article  MathSciNet  MATH  Google Scholar 

Download references


The authors would like to thank Yangyong Zhang from 722 Research Institute of China Shipbuilding Industry Corporation (CSIC) for assisting in the experiments of collecting the real data of atmospheric noise.


This work was supported by the National Natural Science Foundation of China (nos. 61701067, 61771085, and 61671095) and the Research Project of Chongqing Educational Commission (nos. KJ1600427 and KJ1600429).

Availability of data and materials

In this paper, the observations of non-Gaussian noise are collected by an environmental noise sounding device. The LOD design approach proposed in this paper can be used as long as the noise samples are collected. For the practical applications, the noise samples are collected by the receivers of communication systems, radar systems, etc. Compared with the conventional detector, it raises no extra requirement on the data or material.

Author information

Authors and Affiliations



ZL is the first and corresponding author. He proposes the guidelines for the LOD design and the idea of estimating the ZMNL function by polynomial fitting and designing the tails as inverse proportional functions. He analyzes the detection performances of the LOD and develops the CFAR technique. He also simulates the LOD design algorithm and the detection performances. Finally, he writes this paper. PL investigates the α-stable noise and writes the programs of α-stable distribution, including generating noise samples, simulating the discrete PDF, and estimating the parameters. GZ collects the real data of atmosphere noise in low-frequency band, at a sampling frequency 512 Hz, by the magnetotelluric sounding system of 722 Research Institute, China. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Zhongtao Luo.

Ethics declarations

Competing interests

The authors declared that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional information

Authors’ information

Zhongtao Luo received the B.S. degree in electronic engineering and the Ph.D. degree in signal and information processing from University of Electronic Science and Technology of China (UESTC), Chengdu, China, in 2007 and 2015, respectively. From 2007 to 2008, he was with Haier Group, China, where he worked as an electronics engineer. From 2012 to 2013, he was a Visiting Scholar in Nanjing Research Institute of Electronics Technology, China. Since July 2015, he has been with Chongqing University of Posts and Telecommunications (CQUPT), Chongqing, China. His research interests include statistical signal processing, array signal processing, and their applications in communication and radar systems.

Peng Lu received the bachelor’s degree in Electronic Information Engineering from Liaocheng University, Liaocheng, China, in 2016. He is currently studying for a master’s degree at Chongqing University of Posts and Telecommunications, China. His research interest involves signal processing techniques in low-frequency band.

Gang Zhang received the Ph.D. degree in College of Communication Engineering from Chongqing University, Chongqing, China, in 2009. He is currently an Associate Professor at Chongqing University of Posts and Telecommunication, China. His research interests involve chaotic synchronization, chaotic secure communication, and weak signal detection.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luo, Z., Lu, P. & Zhang, G. Locally optimal detector design in impulsive noise with unknown distribution. EURASIP J. Adv. Signal Process. 2018, 34 (2018).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: