Skip to main content

Blind estimation of statistical properties of non-stationary random variables


To identify or equalize wireless transmission channels, or alternatively to evaluate the performance of many wireless communication algorithms, coefficients or statistical properties of the used transmission channels are often assumed to be known or can be estimated at the receiver end. For most of the proposed algorithms, the knowledge of transmission channel statistical properties is essential to detect signals and retrieve data. To the best of our knowledge, most proposed approaches assume that transmission channels are static and can be modeled by stationary random variables (uniform, Gaussian, exponential, Weilbul, Rayleigh, etc.). In the majority of sensor networks or cellular systems applications, transmitters and/or receivers are in motion. Therefore, the validity of static transmission channels and the underlying assumptions may not be valid. In this case, coefficients and statistical properties change and therefore the stationary model falls short of making an accurate representation. In order to estimate the statistical properties (represented by the high-order statistics and probability density function, PDF) of dynamic channels, we firstly assume that the dynamic channels can be modeled by short-term stationary but long-term non-stationary random variable (RV), i.e., the RVs are stationary within unknown successive periods but they may suddenly change their statistical properties between two successive periods. Therefore, this manuscript proposes an algorithm to detect the transition phases of non-stationary random variables and introduces an indicator based on high-order statistics for non-stationary transmission which can be used to alter channel properties and initiate the estimation process. Additionally, PDF estimators based on kernel functions are also developed. The first part of the manuscript provides a brief introduction for unbiased estimators of the second and fourth-order cumulants. Then, the non-stationary indicators are formulated. Finally, simulation results are presented and conclusions are derived.

1 Introduction

Unlike discrete transmission channels which have limited practical value, continuous random transmission channels are widely accepted and used to illustrate practical concepts [1]. In numerous proposed algorithms, statistical properties or probability density function (PDF) of transmission channels are assumed to be perfectly known or already estimated [26]. To analyze the performance of various wireless transmission schemes, Simon and Alouini in [7] have used channel statistical models and PDF properties. It is widely believed that wireless transmission channels can be modeled using stationary random variables [1, 8, 9]. We should highlight the fact that various PDFs have been used in the context of wireless communications, such as Gausssian, Rayleigh, log-normal, exponential, one-sided Gaussian distribution, Hoyt, Weibull, Rice, Nakagami-m, α-μ, and η-κ (see [1021] and the references therein). It is worth pointing out that parametrical estimation methods can be used if the channel model (or a PDF for the real random variable (RV)) is selected or identified. Additionally, non-parametrical estimation methods can also be found in the literature [2227]. However, these methods are hard to be implemented and mostly time consuming and they strongly relate to the proposed models and applications. For this main reason, a generic method and few required assumptions about the transmission channel are proposed and discussed in this manuscript.

The recent spread of cellular systems (smart sensors, mobile phones, base stations, satellites, surveillance devices, traffic radars, etc.) has increased the complexity of processing algorithms as well as the model of transmission channels. In fact, in various applications, transceiver units are in motion. For such systems, the transmission channels can no longer be considered as static channels, i.e., they cannot be modeled by stationary random variables. To overcome this difficulty, most researchers assume that the transmission channels are static during the processing timea[2830].

In this manuscript, the proposed models are improved by considering that transmission channels are dynamic channels that can be modeled as non-stationary variables. It is worth mentioning that for antennas mounted on moving vehicles or drones, the transmission channels can be approximated by almost static or slowly evolutionary dynamic channels over certain periods. The transition among these periods can be very fast. In addition, during two adjacent periods, transmission channels can have completely different statistical properties and/or PDFs. This scenario illustrates a typical scenario of a mobile phone used in a moving car which is moving among big buildings in a modern city. It can represent as well the scenario of a drone flying at low altitude in a mountainous area.

In what follows, we consider that transmission channels can be modeled as short-term stationary signals and long-term non-stationary signals (many natural and physical signals belong to this category, such as speech signals [31, 32]). Herein, we develop an algorithm to accurately estimate the transition times. The algorithm is based on high-order statistics (HOS). In several study cases, HOS are shown to be more promising than the second-order stochastic methods, namely, power, variance, covariance, and spectra. In fact, HOS have been used to solve many recent and important telecommunication problems [33, 34], such as blind identification or equalization, blind separation of sources, and time delay estimation [3538]. It is worth mentioning that most of these HOS algorithms are only based on the second- and the fourth-order statistics [39]. Once the transition times are estimated, we should be able to estimate the PDF of the local stationary random variable (i.e., during a short-term period).

It is well known that any random variable can be completely described by its PDF [40]. In many cases, an accurate estimation of the PDF of physical parameters of interest (which are random variables) is essential to achieve our goals. This is while an accurate estimation of PDFs is still challenging to researchers. The estimation of PDFs is a relatively old problem that has been considered since the beginning of the last century with the rise and the development of new and modern complex systems (radars, sonar, and wireless communication systems). However, in late 1950s, systematic mathematical approaches have been proposed. One of the pioneering work in this field is the work presented by Rosenblatt in [41]. Parzen in [42] formulated the estimation of PDFs using kernel approaches. Later on, new approaches emerged to overcome the specification of diverse applications such as the sum of Gamma densities in Risk Theory [43] or the sum of exponential random variables in wireless communication [44]. Other researchers have focused on the choice of the kernel and smoothing functions [45, 46]. The estimation of PDFs using orthogonal series has been introduced by Schwartz in [47]. In more recent work, wavelets have been used as a non-parametric estimation of PDFs [48]. Similar to previous work, Engel used Haar’s series to estimate PDFs [49]. Using autoregressive (AR) models, Kay in [50] proved that PDFs can be estimated by appealing the theory of power spectral density (PSD). More recent work reintroduced the estimation of PDF using a windowed Fourier transform [51] or Hermite’s orthonormal basis set [52]. We should mention that all the abovementioned work assume that the random variables are stationary. In countless applications, the stationarity assumption may be invalid [53]. To the best of our knowledge, there is no such PDF estimator for non-stationary random variables. Hence, it is the aim of this paper to propose a PDF estimator based on a smooth kernel PDF estimator that assumes known transition times.

2 Mathematical model and background

In this paper, we consider the case of real non-stationary random variablesb. We also assume (1.1 in Appendix Appendix 1: high-order statistics estimators) that random variables consisting of transmission channels are stationary by part [54, 55]. While this assumption is less strict than the most widely used assumption of stationary transmission channels, its consideration is justified in the case of moving receivers or transmitters.

Hereinafter, Pr(A) stands for the probability of the event A. Let x1,…,x n stand for n independent realization of a continuous RV X. In addition, let us denote by f X x , F X x , f ̂ X x , and F ̂ X x respectively as the PDF, the cumulative distribution function (CDF) of X, and their estimated functions:

F X x =Pr X x = - x f X x dx.

In the following, k(x) denotes a kernel function of x, and the mathematical expectation (i.e., the mean) of a real RV X is denoted by

m X =E X = x f X x dx,

where E stands for the mathematical expectation. By definition [56], the q th-order moment μ q of a stochastic signal X is

μ q =E X q = x q f X x dx.

We mentioned earlier that a real random variable X is completely described by its PDF f X (x). In addition, a random variable can also be described by its first Φ X (u) or second Ψ X (u) characteristic functions as follows:

Φ X u = E e jXu = e jxu f X x dx .
Ψ X u = ln Φ X u ,

where j2=-1. While the second characteristic function can be directly obtained from the first one, the first one is considered as the inverse Fourier transform of the PDF. Theoretically, PDFs may be obtained by the Fourier transform of Φ X (u). The Taylor series of the two characteristic functions yield other important statistical functions which are the moments and the cumulants [56]:

m k X = E X k = - j k k Φ X u x k u = 0 ,
Cum k X = - j k k Ψ X u x k u = 0 .

According to the theorem of Leonov-Shiryayev [56], the r th-order cumulant of X can be calculated from its moments, using the following formulac[53, 57]:

Cum r X = Cum X , . . . , X = - 1 k - 1 k - 1 ! E X v 1 E X v 2 E X v p .

Using this relationship, we can evaluate the fourth-order cumulant of X as

Cum 4 X = E X 4 - 4 E X E X 3 - 3 E 2 X 2 + 12 E 2 X E X 2 - 6 E 4 X .

For a zero-mean stochastic signal (please see [58] and the cited references therein),d the second-order cumulant is equal to the second-order moment and therefore the fourth-order cumulant becomes

Cum 4 X =E X 4 -3 E 2 X 2 .

The HOS are much easier to be estimated than PDF, as will be shown in the next section. For this reason, our non-stationary transition indicator is based on the second- and fourth-order statistics. In fact, the transition of non-stationary random variable can generate discontinuity in its HOS. As the variance of any signal should be different than zero, then it can be used to identify the existence of signals. On the other hand, normalized signals have the same unit variance. Therefore, the variance is not enough to identify the transition in a general case. Besides that, the fourth-order cumulants of Gaussian signals are zeros. For these reasons, we developed a non-stationary transition indicator based on the variance and the fourth-order cumulant.

3 Unbiased and adaptive HOS estimators

Let X be a zero-mean stochastic signal where x i is an event (or a signal sample) of X (1≤iN). The classic estimator of the r th-order moment of X is given by

μ r ̂ = 1 N i = 1 N x i r .

It is easy to verify that (11) is an unbiased estimator of the r th-order moment of X (i.e., E μ r ̂ = μ r ). To estimate the fourth-order cumulant of X, we can derive an estimator from the Leonov-Shiryayev formula (8):

Cum 4 ̂ X = μ 4 ̂ -3 μ 2 ̂ 2 .

It is established [59, 60] that the estimator in (12) is a biased estimator and the estimation error decreases proportional to 1 N . In fact, using (11) and (12) we can write

Cum 4 ̂ X = μ 4 ̂ - 3 μ 2 ̂ 2 = 1 N i = 1 N x i 4 - 3 N 2 i , j = 1 N x i 2 x j 2 ,

and the estimator expectation becomes

E Cum 4 ̂ X = μ 4 - 3 N μ 4 + N - 1 μ 2 2 = Cum 4 X - 3 N Cum 4 X + 2 μ 2 2 .

Using the Leonov-Shiryayev formula, one can develop an unbiased estimator for the cumulant ase

Cum 4 ̂ X = N + 2 N N - 1 i = 1 N x i 4 - 3 N N - 1 i , j = 1 N x i 2 x j 2 .

It is worth mentioning that the above estimators are only unbiased for stationary signals. In case of non-stationary signals, adaptive estimators should be developed.

Let μ r ̂ k be the estimator of the r th-order moment at the k th iteration, and we can develop the following adaptive estimators for the second- and the fourth-order statistics:

μ 2 ̂ k = k - 1 μ 2 ̂ k - 1 + x k 2 k , μ 4 ̂ k = k - 1 μ 4 ̂ k - 1 + x k 4 k , Cum 4 ̂ X k = k + 2 k - 1 μ 4 ̂ k - 3 k k - 1 μ 2 ̂ k 2 ,

for 1<kN. This algorithm is simple and converges quickly in the case of stationary signals. Another adaptive estimator of a fourth-order cross-cumulant (Cum13(X,Y)) which is more suitable for non-stationary signals is given by

C N = N - 2 N γ C N - 1 + 1 N γ μ 13 X , Y + N + 2 N N - 1 γ x N y N 3 - 3 γ x N y N μ 02 X , Y - 3 γ y N 2 μ 11 X , Y + 1 - γ x N y N 3 - 3 1 - γ x N y N μ 02 2 ,

where 0<γ<1 is another forgetting factor.

4 PDF estimators

To get an accurate estimation of f X (x), one should use available observations or measurements (i.e., x1,…,x n ). Using Fubini’s theorem [61]f, Rosenblatt in [41] proved that any estimator F ̂ X x of f X (x) based on the set of x i , {i=1,n}, is biased. However, Parzen in [42] proved that the F n (x) is an unbiased estimator F ̂ X x of the CDF, i.e., E{F n (x)}=F X (x):

F n x = 1 n Card x i x ,

where Card denotes the cardinal of a σ-algebra set. Using the fact that f X x = dF X x dx , Parzen proposed an asymptotically unbiased estimator of the PDF as the sum of scaled and shifted kernel functions:

f n x = - 1 h k x - y h dF n y 1 nh i = 1 n k x - x i h ,

given that lim n h n =0 and the kernel k(x) and x k(x) are Borel’s functions [62]g in L1 function space, which satisfy few conditions such as [41, 47]

k x = 1 .

In [42], Parzen suggests few kernel functions including histograms which are kernel estimators when

k x = u x + 1 - u x - 1 2 ,

with u(x) being the Heaviside step function.

Using the definition in (4), the PDF f X (x) can be approximated by using the Fourier transform of an estimated characteristic function [53]. In fact, the average over a set of realization is an unbiased estimator of the mathematical expectation of a RV. Therefore, the first characteristic function can be approximated using the following equation:

Φ ̂ X u = 1 n i = 1 n e j 2 πu x i .

In [51], the authors mentioned that the Fourier transform of the last equation cannot converge. However, it is possible to obtain the Fourier transform of the product between Φ ̂ X u and a weighting window w(x). In this case the PDF estimator is given by

f ̂ X x = Φ ̂ X u w u e - j 2 πu x i du 1 n i = 1 n w x - x i .

The above equation can be considered as a kernel estimator. Window length and parameters are the key factors which can affect the performance of the estimator [51]. To avoid the problem of this approach and apply the well-established theory of power spectral density, Kay in [50] proposed an AR model of order p to estimate the PDF. Parameters of his AR model are estimated using Yule-Walker equations and Levinson recursion.

Using Gram-Charlier series to expand the PDF in terms of the normal density and its derivatives, Bowers in [43] approximated a risk theoretic distribution by the sum of Gamma RVs. In his expansion, the polynomials which multiply the PDF of a zero-mean normalized normal PDF are the Hermite polynomials (see Appendix Appendix 2: Hermite’s polynomial functions). This study was generalized by Schwartz in [47].

According to Vannucci [24], there are different types of non-parametric density estimators: delta sequence estimators (such as kernel, histogram, and orthogonal series estimators) and penalized maximum likelihood estimators such as spline estimators. She also proved that scaling functions give a good approximation of smooth functions, while waveletsh can deal with functions which have local fluctuations. In order to practically apply wavelet estimators, a truncation should be made [24]. Therefore, the overall performance depends on the truncation order. An optimal choice of such order can be determined by minimizing the integrated square error or the intergraded mean square error. In real-world applications, another problem can arise related to the evaluation of the scaling and the mother functions at arbitrary points [24]. In [49], Engel used Haar’s functions to estimate the PDF and shows that his estimator is equivalent to a histogram on certain dyadic intervals.

Recently, the authors of [51, 52] have compared their approaches to the abovementioned ones. In fact, Xie and Wang in [51] showed that an estimator based on the Fourier transform of the characteristic function gives similar results to the AR model PDF estimator proposed earlier by Kay in [50]. They also showed that in certain cases, histograms can provide similar overall performance to previously mentioned estimators. However, Howard in [52] showed that the Hermite basis estimator can slightly achieve better results than the estimator based on the Fourier transform of the characteristic function proposed in [51]. For these reasons, we only consider the histogram, the Hermite basis estimator and the smooth kernel density estimator proposed by Bowan in [45, 46]. Our simulations have shown that smooth kernel density has slightly better performance than the previous mentioned ones (see Section 7).

Finally, two useful error measures are proposed to evaluate the overall performance of various estimators [41, 47, 51, 52]:

  • The integrated error

    ε 1 = E f X x - f ̂ X x
  • The root of the mean-squared error

    ε 2 = E f X x - f ̂ X x 2

5 Representation of non-stationary PDF

Real-world signals can be modeled by wide or quasi non-stationary random variables [39, 63]. In this case, the PDF is a time-dependent function f X (x,t) where t can be a vector representing discrete time instants. Let us consider for example a wireless transmission channel h(t) which can be modeled by a quasi stationary RV [1]. Let us assume now that h(t) has a Weibull PDF for the first period, a uniform PDF for the second period, then a Gaussian RV in its last parts. In this case, it becomes clear that the PDF of h(t) cannot be represented by simple curve as function of t or X. Hence, the PDF of non-stationary RV should be plotted as a function of a RV X and t (see Figure 1).

Figure 1

Theoretical and estimated PDF of non-stationary RV. The estimation is done using smooth kernel density estimators. (a) Theoretical PDF. (b) Estimated PDF.

In fact, classic PDF estimators cannot provide an accurate estimation of the non-stationary signal PDF. By applying a smooth kernel density estimator over 10,000 independent realization of h(t), a bimodal PDF was obtained (see Figure 1b). It is clear that the obtained PDF cannot correctly represent the PDF shown in Figure 1a.

6 Non-stationary transition indicator

In order to estimate the PDF parts of quasi non-stationary RV, one can easily use the estimator developed in the previous section to detect the number and the size of the stationary parts. Once these parts are well identified and their sizes are relatively enough to estimate a PDF, we can apply any classic PDF estimator. To make this estimation more robust, cumulants of different order (mainly the second and the fourth order) can be used to identify such parts. In fact, Gaussian signals are characterized by their zero fourth-order cumulant (see Appendix Appendix 1: high-order statistics estimators). In these case, the fourth order cannot separate two adjacent Gaussian parts. However, if these two parts represent two normal RVs with different means or variances, then the second or the first moment can be used to identify these parts. If the two Gaussian parts have the same mean and variance then they can describe the same RV due to the basic assumptions about the whiteness of the samples (i.e., the realization of X).

Hereinafter, we consider that the non-stationary RV is formed by successive segments of stationary RV. In order to estimate the PDF of the non-stationary signal, an estimator of the transition times should be developed. In order to simplify our discussion and gain insight, a generic case is considered. Let us consider a zero-mean non-stationary signal X(t) made by four parts of stationary random variables as shown in Figure 2:

  1. 1.

    The first part contains 8,000 samples of uniform random variable included in [ -1,1].

  2. 2.

    The second part is made of 6,000 samples of zero-mean and unite variance Gaussian signal.

  3. 3.

    The 10,000 samples of a uniform random variable in [ -2,2] formed the third part.

  4. 4.

    The fourth part is a zero-mean Gaussian signal with a standard deviation of 2 .

Figure 2

The considered non-stationary signal.

The fourth-order statistics of the uniform parts are given as follows:

E X 2 = A 2 3 , E X 4 = A 4 5 , Cum 4 X = - 2 A 4 15 ,

where A is the maximum amplitude of X. Figure 3 shows that the proposed adaptive estimators can yield a good estimation of the HOS. However, these estimators are not quite enough to estimate the HOS and the transition times with a wide accuracy range. In fact, the estimators shown in Figure 3 can be considered as noisy signals. To reduce the noise level, we adopted two steps:

  1. 1.

    First, the obtained signals are filtered using the smoothing polynomial regression filter proposed by Savitzky-Golay [6466]. The main idea of the Savitzky-Golay filter is to apply a FIR filter such that its coefficients minimize the mean-squared approximation error. This filter is used to filter biomedical signals such as EEG signals (see Figure 4).

Figure 3

Theoretical and estimated values of variances and fourth-order cumulants of non-stationary signals.

  1. 2.

    Second, the noise in the filtered signals are slightly reduced using a Walsh-Hadamard transform which is a generalization of discrete Fourier transform [67]. This transformation is also used in biomedical signals to filter and compress ECG signals (see Figure 5).

Figure 4

A fifth-order Savitzky-Golay filter applied on real EEG signals.

Figure 5

A Walsh-Hadamard transform applied on simulated ECG signals. Noisy, in blue, and filtered using Walsh-Hadamard transform; in red, ECG signal (top). ECG signal and WHT coefficients (bottom).

In order to smooth the HOS estimator and achieve better detection of the transition times, the signal of the HOS estimator was firstly filtered using a Savitzky-Golay filter then a Walsh-Hadamard transform truncation filtering techniques is applied. Experimental results showed that the filtered signals are much more smooth and useful to reach our goal (see Figure 6).

Figure 6

Estimated variance and fourth-order cumulant filtered by a Savitzky-Golay filter and a truncation of its Walsh-Hadamard transform. Low-pass and Savitzky-Golay filter variance (left). Estimated adaptive cumulant filtered by Walsh-Hadamard and Golay (right).

While the filtered signals are much better smoothed than the raw signals, they are still relatively noisy to obtain an accurate estimate of transition time. In fact, Figure 6 shows that the obtained filtered estimators are still suffering from the previously mentioned drawbacks, i.e., the inaccuracy in the estimation of the HOS and the transition times.

To address the previously mentioned problems, we developed the following algorithm:

  • First, the PDF of the filtered HOS estimators are obtained using the kernel PDF estimator (see Figure 7). Then the maximum values of the obtained PDF are recorded as the coefficients of a vector called maxHOS.

  • The coefficients of the vector maxHOS are introduced as the center of clusters. By minimizing Euclidean distance among the samples of the filtered HOS estimator, all samples of filtered estimated cumulants are clustered in rectangular signals which represent theoretical values of the HOS (see Figure 8).

  • Various simulations have been conducted. Our simulations showed that sometimes the rectangular clustered signals suffer from local narrow spurious error windows (see Figure 8). These spurious error windows are normally very narrow. Hence, by supposing that the channel is not a highly dynamic one, i.e., any channel parameter cannot change more than one time in a short period (for example, the short period can be the symbol duration). In this case, one can easily eliminate these windows. In order to clean out the clustered rectangular signals and achieve an accurate non-stationary indicator, the derivative of these signals is evaluated (see Figure 9).

  • Let us assume 1.2 in Appendix Appendix 1: high-order statistics estimators that the stationary parts of the channels have more than 1,000 samples (in many cases, 500 samples were enough to obtain good results). In this case, each of local narrow spurious error windows generates two Dirac delta functions with equal values and opposite signs. Based on 1.2 in Appendix Appendix 1: high-order statistics estimators, the two delta functions should be close to each other within 1,000 samples. Using this fact, we developed and implemented a recursive filtering procedure called continuity process (CP) to eliminate these spurious windows (see Figure 10). The Matlab code is given in Appendix Appendix 3: continuity process algorithm.

  • Figure 10 shows that the impact of the local narrow spurious error windows is completely eliminated. However, our non-stationary indicator still suffers from two-step transition problem, i.e., the transition between two valid states of the cluster rectangular signals is not immediate (this case can be shown in Figure 8 around the 14,000th and 24,000th samples). Using 1.2 in Appendix Appendix 1: high-order statistics estimators and the fact that the two-step transition problem generates two Dirac delta functions with different values but sharing the same sign, we developed another process called two-step transition process to deal with this problem (see Appendix Appendix 3: continuity process algorithm).

  • Finally, a clear and accurate non-stationary indicator is obtained. Our final indicator is the output of an ‘OR’ gate applied on the non-stationary indicator of various HOS (in our case, we used two HOS, i.e., the second and the fourth cumulants). The final indicator is shown in Figure 11. In Figure 12, the synoptic of our proposed algorithm using smoothing filters and nearest-neighbor clustering algorithm is presented.

Figure 7

Kernel PDF estimator of filtered variance and adaptive cumulant. Kernel PDF of estimated adaptive variance (left). Kernel PDF of estimated adaptive cumulant (right).

Figure 8

Clustering algorithm applied on estimated variances filtered by a Savitzky-Golay filter. Adaptive variance filtered by classification (left). Adaptive cumulant filtered by classification (right).

Figure 9

Derivative of filtered cumulant adaptive estimator.

Figure 10

Indicator of non-stationary transition with continuity process.

Figure 11

Indicator of non-stationary transition with continuity and two-step transition processes.

Figure 12

Synoptic of our algorithm.

7 PDF estimator for non-stationary signals

In the previous section, the transition between two stationary parts has been well identified. Hereinafter, we assumed that the different stationary segments of the signal are already known and just the PDF of the non-stationary signal should be estimated. To evaluate their algorithms, the authors of [5052] generate a RV X with its PDF as a moderated sum of PDF of two uncorrelated normal RVs:

f X x = i = 1 2 p i 2 π σ i exp - x - m i 2 2 σ i 2

where m i and σ i are, respectively, the mean and the standard deviation of the normal RV X i and p1+p2=1 are weighting parameters. It was mentioned in [52] without any proof that the previous PDF shown in equation (20) ‘is consistent with a random sum of independent RVs’:

X= i = 1 M X i

where, for example, M1,2, and Pr[ M=1]=p2. We proved (see Appendix 4) that the sum should be just over two independent RV and we give the statistical properties of the newly obtained RV.

It is well known that the overall performance of a histogram depends on the number of samples as well as the theoretical PDF. Performances can deteriorate if the number of samples is not large enough. On the other hand, if realization can be repeated many times, then an accurate estimation of the PDF can be obtained as the average of all obtained histograms (in Figure 13b, the average is done using ten iterations). For a large number of samples, the histogram performs quite well (see Figure 13c).

Figure 13

PDF estimators using histograms (theoretical PDF in continuous curve). (a) One iteration and 151 samples; (b) 151 samples regenerated using ten iterations; (c) 10,000 samples during one iteration.

Our simulations show that the Hermite basis set estimator suffers from two major inconveniences:

  1. 1.

    It depends on the value of k s ,

  2. 2.

    It can generate a negative function, see Figure 14.

Figure 14

PDF estimators using Hermite’s polynomial functions with various values of K s . Theoretical PDF are in continuous curves. (a) K s =4.5 without correction. (b) K s =4.5 with correction. (c) K s =7.5 without correction.

The smooth kernel estimator proposed in [46] seems to overcome the previous two mentioned drawbacks (see Figure 15).

Figure 15

Smooth kernel density estimator.

8 Conclusions

In this manuscript, a transition indicator for non-stationary signals is presented. The new indicator is based on HOS of quasi non-stationary variables (the random variables are considered stationary by parts). To estimate the HOS, unbiased adaptive exponential estimators are presented.

To reduce the noise level of the HOS estimators, we used a cascade filtering procedure based on Savitzky-Golay filters and a truncation of its Walsh Hadamard transform. Then rectangular signals representing the theoretical values of HOS are obtained by using clustering algorithm using the maxima of kernel PDF estimator and minimizing Euclidean distances.

Simulation studies show that the obtained rectangular signals can suffer from local narrow spurious error windows which can be eliminated using a continuity assumption 1.2 in Appendix Appendix 1: high-order statistics estimators and a continuity cleaning procedure called continuity process. In addition to these local narrow spurious error windows, estimated signals suffer another artifact called the two-step transition problem. After solving this problem using assumption 1.2 in Appendix Appendix 1: high-order statistics estimators and a continuity procedure, an accurate transition indicator of non-stationary signals is achieved. Simulation studies corroborate the performance of our proposed algorithm and the accuracy of our non-stationary transition indicator.

Finally, a survey of major PDF estimators is done. A comparative study is also presented and discussed. The advantages and drawbacks of major methods are highlighted and a theoretical study is provided. Simulation results show a slight advantage of smooth kernel estimator methods. It is worth mentioning that the histogram with a large number of samples is still one of the simplest and efficient estimators. The case of non-stationary process was considered and a PDF estimation approach was discussed.


aHowever, the processing time is not standardized. In fact, different authors claim that the parameters of the channel should remain constant during one frame duration, few hundred symbols, or during the convergence time of their adaptive algorithms.

bThe fact that the channel is considered as a real channel is not limiting our approach, as a complex Gaussian channel could be represented by its modulus as a Rayleigh channel.

cThe original formula shows the relationship among the cumulant of r stochastic signals X i (i=1,…,r) and their moments of order p, pr:

Cum X 1 , , X r = - 1 k - 1 k - 1 ! × E i v 2 X i E j v 2 X j E k v p X k ,

where the addition operation is over all the set of v i (1≤ipr) and v i constitutes a partition of {1,…,r}, [40].

dIn many applications, the stochastic signal X is a zero-mean signal. This assumption is not a major one and it has been used in many studies (please see [58] and the cited references therein). In fact, by considering this assumption, the mathematical notations can be simplified. However, all the proposed steps can be straightforwardly derived in the case of a non-zero mean transmission channel. Besides that, the mean can be estimated and canceled out from the other equations.

eFurther details are given in Appendix Appendix 1: high-order statistics estimators.

fAccording to Fubini’s theorem [61], a double integral defined over two measure spaces D a , D b of a measurable function f(x,y) can be computed using iterated integrals:

D , D f x , y d x , y = D D f x , y dx dy = D D f x , y dy dx.

gA function f(x) is called Borel’s function if yA is an open set, the inverse of f(x), and x=f-1(y) is an element of a Borel’s set, i.e., Lebesgue measurable set [62].

hWavelets are basis functions which have quite interesting properties such as their localization in space and frequency [24].

Appendix 1: high-order statistics estimators

In this section, HOS estimators are developed. Generally, HOS estimators can be divided into main families: the arithmetic and the exponential estimators.

1.1 Arithmetic estimators

Let X to be a zero-mean stochastic ergodic signal where x i is an event (or a signal sample) of X (1<i<N). In this case, the arithmetic estimator of the q th-order moment is given by

μ q ̂ = 1 N i = 1 N x q r .

This estimator assumes that the signal X is stationary over N samples. This estimator is a non-biased estimator (i.e., E μ q ̂ = μ q ) and its variance is given by

Var μ q ̂ = μ 2 q - μ q 2 N .

Clearly, it is a consistent estimator; hence for stationary signals, its variance decreases with an increased number of samples. An arithmetic estimator of the q th-order cumulant can be developed from Equation 8:

Cum q ̂ X = - 1 k - 1 k - 1 ! μ v 1 μ v 2 μ v p .

It is proved [59, 60] that the estimator in (24) is a biased consistent estimator where the estimation error decreases proportional to 1 N :

E Cum q ̂ X = p = 1 q - 1 p N p - 1 p - 1 × μ q + N - 1 μ v 1 μ v 1 + N - 1 p - 1 μ v 1 μ v p .

A non-biased cumulant estimator can be deduced from the last equation:

Cum q ̂ X = p = 1 q c p - 1 p p - 1 μ v 1 μ v 2 μ v p ,

where the parameters c p depend on the partitions of the indices v i . These parameters can be estimated as the solution of q linear equations. Let us consider the fourth-order cumulant:

Cum 4 ̂ X =a μ 4 ̂ -4b μ 1 ̂ μ 3 ̂ -3c μ 2 ̂ 2 +12d μ 1 ̂ 2 μ 2 ̂ -6e μ 1 ̂ 4 .

In order to make the last estimator unbiased, one should solve a linear system of equations obtained by comparing term-to-term the expectation of Equation 26 and the theoretical value given by (9)

a = N 3 + N 2 - 24 N + 24 N - 1 N - 2 N - 3 b = N 2 N 2 - 10 N + 9 2 N - 1 N - 2 N - 3 c = N N 2 - N - 6 N - 1 N - 2 N - 3 d = N 2 2 N - 5 2 N - 1 N - 2 N - 3 e = N 3 N - 1 N - 2 N - 3

For zero-mean signals, we can easily prove that

E Cum 4 ̂ X = μ 4 - 3 N μ 4 + N - 1 μ 2 2 = Cum 4 X - 3 N Cum 4 X + 2 μ 2 2 .

This means that the following estimator is an unbiased estimator for the fourth-order cumulant of a zero-mean stationary signal X:

Cum 4 ̂ X = N + 2 N - 1 μ 4 ̂ - 3 N - 1 μ 2 ̂ 2 .

For real-time applications, the estimators should be adaptive ones. The estimator (23) is not an adaptive one, but it is easy to derive an adaptive version:

μ q ̂ k = 1 k i = 1 k x i q = k - 1 μ q ̂ k - 1 + x k q k ,

where μ r ̂ k is the estimator of the r th order moment at the k th iteration.

1.2 Exponential estimators

Exponential estimators are defined as

μ q ̂ = 1 - λ q i = 1 N λ q N - i x i q ,

where 0<λ q <1 represents a forgotting factor. This estimator can be calculated easily in an adaptive way:

μ q ̂ k = λ q μ q ̂ k - 1 + 1 - λ q x k q .

The latest estimator is biased (E μ q ̂ = 1 - λ q N μ q ), but it is asymptotically non-biased. The main interest in such estimator resides on the fact that it can give better estimation for the moments of non-stationary signals. Thus, the closest λ to 1, the more past samples are taking into account. A non-biased exponential estimator can be written as

μ q ̂ = 1 - λ q 1 - λ q N i = 1 N λ q N - i x i q .

Estimator (32) can be also modified to an adaptive version:

μ q ̂ k = 1 1 - λ q N λ q 1 - λ q k - 1 μ q ̂ k - 1 + 1 - λ q x k q .

An adaptive non-biased estimator of the cumulants could be derived using (22) and (33). To simplify our discussion, the fourth-order cumulant unbiased estimator for zero-mean signals could be developed as

Cum q ̂ X k = Cum q ̂ X k - 1 + 1 - γ H k × Cum q ̂ X k - 1

where γ is a forgetting factor and

H k Cum q ̂ X k - 1 = x k 4 - 4 x k 3 μ 1 ̂ k - 1 - 3 x k 2 μ 2 ̂ k - 1 + 12 x k 2 μ 1 ̂ 2 k - 1 - 6 μ 1 ̂ 4 k - 1 - Cum q ̂ X k - 1

1.3 Adaptive unbiased estimators of the fourth-order cumulants

A non-biased estimator of fourth-order cross-cumulants can be obtained from the definition of the cross-cumulants [68]. In fact let us consider K22 an estimator of Cum22(X,Y) defined as

K 22 = a N i x i 2 y i 2 - b N 2 ij x i 2 y j 2 - 2 c N 2 ij x i x j y i y j ,

where a, b, and c should be set in order to make K22 a non-biased and consistent estimator. When samples x i and y i are independent, one can use similar estimators to these proposed in [69, 70]. In the following, we assume that the samples are independent and identically distributed (iid) over time but spatially correlated. In this case, one can prove that K22 become a non-biased and consistent estimator:

E K 22 = aE X 2 Y 2 - b N 2 NE X 2 Y 2 - 2 c N 2 NE X 2 Y 2 + N N - 1 E X 2 E Y 2 + N N - 1 E XY 2 ,

when a= N + 2 N - 1 and b=c= N N - 1 . Similarly, one can develop other estimators:

K 13 = Cum ̂ 13 X , Y = N + 2 N N - 1 i x i y i 3 - 3 N N - 1 ij x i y i y j 2
K 31 = Cum ̂ 31 X , Y = N + 2 N N - 1 i x i 3 y i - 3 N N - 1 ij x i y i x j 2 .

To obtain these estimators, signals are assumed stationary. The last assumption cannot be satisfied in our application. Therefore, some modifications should be considered. Let C13(N)=K13(X,Y) be the adaptive estimator of the cumulant 3×1 using N samples, A N = i N x i y i 3 and B N = ij N x i y i y j 2 . In this case, Equation 35 can be written as

N N - 1 C 13 N = N + 2 A N -3 B N .

Hence, we can prove that

N N + 1 C N + 1 = N + 3 A N + x N + 1 y N + 1 3 - 3 B N + x N + 1 y N + 1 j = 1 N + 1 y j 2 + y N + 1 2 i = 1 N + 1 x i y i = N N - 1 C N + A N + N + 3 x N + 1 y N + 1 3 - 3 x N + 1 y N + 1 j = 1 N y j 2 - 3 y N + 1 2 i = 1 N x i y i .

Finally, the last equation can be modified to

C N = N - 2 N C N - 1 + 1 N μ 13 X , Y + N + 2 N N - 1 x N y N 3 - 3 x N y N μ 02 X , Y - 3 y N 2 μ 11 X , Y ,

where μ nm X , Y = 1 N - 1 i = 1 N - 1 x i n y i m is the estimator of E(XnYm) using N-1 samples. Using the last two equations, we can derive the final form of our adaptive fourth-order cumulant estimator:

C 31 N = N - 2 N C 31 N - 1 + 1 N μ 31 X , Y + N + 2 N N - 1 x N 3 y N - 3 x N y N μ 20 X , Y - 3 x N 2 μ 11 X , Y .

Appendix 2: Hermite’s polynomial functions

According to [71], Hermite polynomials are real orthogonal polynomials with respect to the weight function w x = e - x 2 . The n th-order Hermite polynomial is defined as follows:

H n x = - 1 n e x 2 d n e - x 2 d x n .

Two hermite polynomials of orders n and m satisfy the following properties:

1 2 p p π H n x H m x e - x 2 dx = δ m n lim N n = 0 N H n x t n n = e - t 2 + 2 tx H n + 1 x - 2 x H n x + 2 n H n - 1 x = 0 ,

where δ m n is Kronecker’s symbol. It is worth mentioning that the above recurrence equation is widely used in practice to estimate the Hermite polynomials. In fact, H0(x)=1, H1(x)=2x so H2(x)=4x2-2 and so on. In order to use Hermite’s basis set, one should define the following two functions:

b i N x = H i x exp - x 2 2 2 i i ! π b i x = 1 k s b i N x k s ,

where k s is the scaling factor, H i (x) is the Hermite’s polynomial of the i th order. Figure 16 shows the first eight normalized Hermite polynomial functions, i.e., b i (x) when k s =1.

Figure 16

The first eight normalized Hermite polynomial functions.

Appendix 3: continuity process algorithm

Appendix 4

The PDF of a moderate sum of two uncorrelated RVs

Let us consider two independent RVs X i , i=1,2. It is easy to prove that the proposed random sum can be written as follows:

X = i = 1 M X i = X 1 + a X 2 ,

where, M1,2 is a modified Bernoulli RV, and a is a Bernouilli RV with Pr(a=1)=1- Pr(a=0)=ε. In this case, one can write:

F X x = Pr X 1 + a X 2 x = Pr a = 0 Pr X 1 + a X 2 x a = 0 + Pr a = 1 × Pr X 1 + a X 2 x a = 1 = 1 - ε Pr X 1 x + ε Pr X 1 + X 2 x = 1 - ε F X 1 x + ε F X 1 + X 2 x .

Using the previous equation, one can deduce [53] that

f X x = dF X x dx = 1 - ε f X 1 x + ε f X 1 x f X 2 x ,

where * represents the convolution product. If X1 and X2 are two uncorrelated mutually Gaussian RVs, then Z=X1+X2 is another normal RV with N m 1 + m 2 , σ 1 2 + σ 2 2 , which proves the proposed statement.

The mean of the new variable X is given by

m = E X = m 1 + m 2 .

Using the definition of the variance and the properties of a Bernouilli RV, the second moment of the new variable X becomes

E X 2 = E X 1 2 + E a 2 E X 2 2 + 2 E a X 1 X 2 = σ 1 2 + m 1 2 + ε σ 2 2 + m 2 2 + 2 ε m 1 m 2 .

In this case the variance of X could be obtained as follows:

σ 2 = σ 1 2 + ε σ 2 2 + 2 ε 1 - ε m 2 2 .

Authors’ information

AM received his M.Sc. and Ph.D. degrees in Signal, Image and Speech Processing from the ‘Institut National Polytechnique de Grenoble - INPG’ (Grenoble, France) on July 1993 and January 1997, respectively, and his HDR degree (Habilitation a Diriger des Recherches; in the French system, this is the highest of the higher degrees) on November 2006 from the Universite de Bretagne Occidentale - UBO (Brest, France). His research interests are in the areas of blind separation of sources, high-order statistics, signal processing, passive acoustics, cognitive radio, robotics, and telecommunication. From January 1997 to July 1997, he held a post-doc position at Lab. de Traitement d’Images et Reconnaissance de Forme, France. From August 1997 to September 2001, he was a research scientist at the Bio-Mimetic Control Research Center (BMC) in Riken, Nagoya, Japan. From October 2001 to January 2008, he was holding a teacher-researcher position at the Ecole Nationale Supérieure des Ingénieurs des Etudes et Techniques d’Armement (ENSIETA) in Brest, France. From February 2008 to August 2010, he was a senior lecturer at the ECE Department of Curtin University, Perth, Australia. During January 2009, he held an invited professor position at the Universite du Littoral Cote d’Opale in Calais, France. From September 2010 till June 2012, he was a professor at University of Tabuk, Kingdom of Saudi Arabia. He also served as the electrical department head at the University of Tabuk. Since September 2012, he has been a professor at Ecole Nationale Supérieure de Techniques Avancées Bretagne (ENSTA Bretagne) in Brest, France. He is the author and the co-author of three books. He is the first author of several papers published in international journals. He is also the first author of many papers published in the proceedings of various international conferences. Finally, he was elected to the grade of IEEE Senior Member in February 2006. He has been a member of Technical Program Committees (TPC) and he was a chair, co-chair, and a scientific committee member in various international conferences. He is an active reviewer for a variety of international journals. He was a lead guest editor of EURASIP Journal on Advances in Signal Processing.

RM (S’00-M’08-SM’13) holds a Ph.D. in Electrical Engineering from Jacobs University in Bremen, Germany and several years of post-doctoral wireless communication and optical wireless communication research experience in Germany. In October 2010, he joined University of Tabuk in Saudi Arabia where he is now an assistant professor and the director of research excellence unit. His main research interests are in spatial modulation, MIMO cooperative wireless communication techniques, and optical wireless communication. His publications received more than 800 citations since 2007. He has published more than 50 publications in top-tier journals and conferences, and he holds seven granted patents. He also serves as on the TPC for academic conferences and is a regular reviewer for most of IEEE/OSA Communication Society’s journals and IEEE/OSA Photonics Society’s journals.

HMA received his M.S. and Ph.D. degrees in Electrical Engineering from the University of Washington (UW), Seattle, WA, USA. He is a professional engineer registered in the State of Washington and a senior member of the Institute of the IEEE. He has taught graduate and undergraduate courses in Electrical Engineering at a number of universities in the USA and abroad. He served at many academic ranks including Endowed Chair Professor and Vice President and Provost. He was the winner of the Boeing Supplier Excellence Award. He was also the winner of the IEEE Professor of the Year Award, UW Branch. He is listed as inventor in a major patent assigned to the Boeing Company. His research work is referred to in many patents including patents assigned to ABB, Switzerland, and EPRI, USA. Currently, he is a professor and director of the Sensor Networks and Cellular Systems (SNCS) Research Center, University of Tabuk, Tabuk, Saudi Arabia. He authored many papers in IEEE and other journals and conferences. His research interests include modeling and simulation of large scale networks, sensors and sensor networks, scientific visualization, and control and energy systems.


  1. 1.

    Proakis J, Salehi M: Digital Communications. New York: McGraw-Hill; 2007.

    Google Scholar 

  2. 2.

    Tarokh V, Jafarkhani H, Calderbank AR: Space-time codes high data rate wireless communication: performance crirterion and construction. IEEE Trans. Inf. Theory 1998, 44(2):744. 10.1109/18.661517

    MATH  Article  Google Scholar 

  3. 3.

    Bradaric I, Petropulu AP, Diamantaras K: On blind identification of FIR-MIMO systems with cyclostationary inputs using second order statistics. IEEE Trans. Signal Process 2003, 51(2):434. 10.1109/TSP.2002.806998

    Article  Google Scholar 

  4. 4.

    Srar JA, Chung KS, Mansour A: Adaptive array beamforming using a combined LMS-LMS algorithm. IEEE Trans. Antennas Propagation 2010, 58: 3545.

    Article  Google Scholar 

  5. 5.

    Mesleh R, Ikki S, Aggoune H, Mansour A: Performance analysis of space shift keying (SSK) modulation with multiple cooperative relays. EURASIP J. Adv. Signal Process 2012., 1: doi:10.1186/1687-6180-2012-201

    Google Scholar 

  6. 6.

    Abaza M, Mesleh R, Mansour A, Alfalou A: MIMO techniques for high data rate free space optical communication system in log-normal channel. In International Conference on Technological Advances in Electrical, Electronics and Computer Engineering. Konya: TAEECE,; 9–11 May 2013:161-166.

    Google Scholar 

  7. 7.

    Simon M, Alouini MS: Digital Communication over Fading Channels: a Unified Approach to Performance Analysis. New York: Wiley; 2000.

    Book  Google Scholar 

  8. 8.

    Haykin S, Moher M: Communication Systems. New York: Wiley; 2009.

    Google Scholar 

  9. 9.

    Abaza M, Mesleh R, Mansour A, Aggoune HA: Diversity techniques for FSO communication system in correlated log-normal channels. Opt. Eng 2014., 53(1): 10.1117/1.OE.53.1.016102

    Google Scholar 

  10. 10.

    Skitovic VP: Linear forms of independent random variables and the normal distribution law. Izvestiya Akademii Nauk SSSR. Seriya Matematiceskaya 1954., 18(185): In Russian

    MATH  Google Scholar 

  11. 11.

    Kendall M, Stuart A: The Advanced Theory of Statistics: Distribution Theory. London: Charles Griffin & Company Limited; 1961.

    Google Scholar 

  12. 12.

    Nikias CL, Shao M: Signal Processing with α-Stable Distribution and Applications. New York: Wiley; 1995.

    Google Scholar 

  13. 13.

    Sijbers J, Den Dekker J, Scheunders P, Van Dyck D: Maximum-likelihood estimation of Rician distribution parameters. IEEE Trans. Med. Imaging 1998, 17(3):357. 10.1109/42.712125

    Article  Google Scholar 

  14. 14.

    Karagiannidis GK, Zogas DA, Kotsopoulos SA: On the multivariate Nakagami-m distribution with exponential correlation. IEEE Trans. Commun 2003, 51(8):1240. 10.1109/TCOMM.2003.815071

    Article  Google Scholar 

  15. 15.

    Yacoub MD, Fraidenraich G, Santos Filho JCS: Nakagami-m phase-envelope joint distribution. Electron. Lett 2005., 41(5): doi:10.1049/el:20057014

    Google Scholar 

  16. 16.

    Yacoub MD, Fraidenraich G, Tercius HB, Martins FC: The symmetrical on the η - κ distribution: a general fading distribution. IEEE Trans. Broadcasting 2005, 51: 504. 10.1109/TBC.2005.859240

    Article  Google Scholar 

  17. 17.

    Fraidernraich G, Daoud Yacoub M: The α - η - μ and α - k - μ fading distributions. In IEEE 9th International Symposium on Spread Spectrum Techniques and Applications. Manaus; 28–31 August 2006:255-263.

    Google Scholar 

  18. 18.

    Nadarajah S, Kotz S: On the η - κ distribution. IEEE Trans. Broadcasting 2006, 52(3):411. 10.1109/TBC.2006.879862

    Article  Google Scholar 

  19. 19.

    Yacoub MD: The α - η distribution: a physical fading model for the Stacy distribution. IEEE Trans. Vehicular Technol 2007, 56(1):27.

    Article  Google Scholar 

  20. 20.

    Cotton SL, Scanlon WG, Guy J: The k - μ distribution: applied to the analysis of fading in body to body communication channels for fire and rescue personnel. IEEE Antennas Wireless Propagation Lett 2008, 7: 66.

    Article  Google Scholar 

  21. 21.

    Yacoub MD: Nakagami-m phase-envelope joint distribution: an improved model. In Microwave and Optoelectronics Conference, IMOC’2009. Belem; 3–9 November 2009:335-339.

    Google Scholar 

  22. 22.

    Cox DD: A penalty method for nonparametric estimation of the logarithmic derivative of a density function. Ann. Instit. Stat. Math 1985, 37: 271. 10.1007/BF02481097

    MATH  Article  MathSciNet  Google Scholar 

  23. 23.

    Vidal J, Bonafonte A, Fonollosa AR, De Losada NF: Parametric modeling of PDF using a convolution of one-sided exponentials: application to HMM. In European Signal Processing Conference, ed. by M Holt, C Cowan, P Grant, W Sandham, European Signal Processing Conference. Edinburgh: Elsevier,; 1994:54-57.

    Google Scholar 

  24. 24.

    Vannucci M: Nonparametric density estimation using wavelets. Discussion Paper, Duke University, 1998

  25. 25.

    Archambeau A, Verleysen M: Fully nonparametric probability density function estimation with finite gausssian mixture models. In 5th International Conference on Advances in Pattern Recognition Calcultta, ICAPR’2003. Allied, New Delhi, Calcultta; 2003:81-84.

    Google Scholar 

  26. 26.

    Boscolo R, Pan H, Roychowdhury VP: IEEE Trans. Neural Netw. 2004, 15(1):55.

    Article  Google Scholar 

  27. 27.

    Roueff F, Rydén T: Nonparametric estimation of mixing densities for discrete distributions. Ann. Stat 2005, 33(5):2066. 10.1214/009053605000000381

    MATH  Article  MathSciNet  Google Scholar 

  28. 28.

    Alamouti S: A simple transmit diversity technique for wireless communication. IEEE J. Selected Areas Commun 1998, 16(8):1451. 10.1109/49.730453

    Article  Google Scholar 

  29. 29.

    Larsson EG, Stoica P: Space-time Block Coding for Wireless Communications. Cambridge: The Press Syndicate of the University of Cambridge; 2003.

    MATH  Book  Google Scholar 

  30. 30.

    Choqueuse V, Mansour A, Burel G, Collin L, Yao K: Blind channel estimation for STBC systems using higher-order statistics. IEEE Trans. Wireless Commun 2011, 10: 495.

    Article  Google Scholar 

  31. 31.

    Mansour A, Jutten C, Ohnishi N: Kurtosis: definition and properties. In International Conference on Multisource-Multisensor Information Fusion. Edited by: Arabnia HR, Zhu DD. Las Vegas; 6–9 July 1998:40-46.

    Google Scholar 

  32. 32.

    Mansour A, Jutten C: What should we say about the kurtosis. IEEE Signal Process. Lett 1999, 6(12):321.

    Article  Google Scholar 

  33. 33.

    Mendel JM: Tutorial on higher-order statistics (Spectra) in signal processing and system theory: theoretical results and some applications. IEEE Proc 1991, 79: 277.

    Article  Google Scholar 

  34. 34.

    Cichocki A, Amari SI: Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. New York: Wiley; 2002.

    Book  Google Scholar 

  35. 35.

    Cardoso JF: Source separation using higher order moments. In Proceedings of International Conference on Speech and Signal Processing 1989, ICASSP 1989. Glasgow; 22–25 May 1989:2109-2212.

    Google Scholar 

  36. 36.

    Comon P: Independent component analysis, a new concept. Signal Process 1994, 36(3):287. 10.1016/0165-1684(94)90029-9

    MATH  Article  Google Scholar 

  37. 37.

    Mansour A, Jutten C: A direct solution for blind separation of sources. IEEE Trans. Signal Process 1996, 44(3):746. 10.1109/78.489054

    Article  Google Scholar 

  38. 38.

    Hyvärinen A, Karhunen J, Oja E: Independent Component Analysis. New York: Wiley; 2001.

    Book  Google Scholar 

  39. 39.

    Mansour A, Kardec Barros A, Ohnishi N: Blind separation of sources: methods, assumptions and applications. IEICE Trans. Fundamentals Electron. Commun. Comput. Sci 2000, E83-A(8):1498.

    Google Scholar 

  40. 40.

    Papoulis A: Probability, Random Variables, and Stochastic Processes. New York: McGraw-Hill; 1991.

    MATH  Google Scholar 

  41. 41.

    Rosenblatt M: Remarks on some nonparametric estimates of a density function. Ann. Math. Stat 1956, 27(3):832. 10.1214/aoms/1177728190

    MATH  MathSciNet  Article  Google Scholar 

  42. 42.

    Parzen E: On the estimation of a probability density function and mode. Ann. Math. Stat 1962, 33(3):1065. 10.1214/aoms/1177704472

    MATH  MathSciNet  Article  Google Scholar 

  43. 43.

    Bowers LN: Expansion of probability density functions as a sum of gamma densities with applications in risk theory. Trans. Soc. Actuaries 1966, 18 PT 1(52):125.

    Google Scholar 

  44. 44.

    Khuong HV, Kong HY: General expression for pdf of a sum of independent exponential random variables. IEEE Commun. Lett 2006, 10(3):159. 10.1109/LCOMM.2006.1603370

    Article  Google Scholar 

  45. 45.

    Reinsch CH, Smoothing by spline functions: Numer. Math. 1967, 10: 177. 10.1007/BF02162161

    MATH  MathSciNet  Article  Google Scholar 

  46. 46.

    Bowman AW, Azzalini A: Applied Smoothing Techniques for Data Analysis. New York: Oxford University; 1997.

    MATH  Google Scholar 

  47. 47.

    Schwartz SC: Estimation of probability density by an orthogonal series. Ann. Math. Stat 1967, 38(3):1261.

    MATH  Article  MathSciNet  Google Scholar 

  48. 48.

    Vannucci M: On the application of wavelets in statistics. Ph.D. thesis, Dipartimento di Statistica, University of Florence, 1996

  49. 49.

    Engel J: Density estimation with, Haar series. Stat. Probability Lett 1990, 9: 111. 10.1016/0167-7152(92)90003-N

    MATH  MathSciNet  Article  Google Scholar 

  50. 50.

    Kay S: Model-based probability density function estimation. IEEE Signal Process. Lett 2001, 5(12):318.

    Article  Google Scholar 

  51. 51.

    Xie J, Wang Z: Probability density function estimation based on windowed Fourier transform of characteristic function. In Proceedings of the 2nd International Congress on Image and Signal Processing. Tianjin; 17–19 October 2009.

    Google Scholar 

  52. 52.

    Howard RM: PDF estimation via characteristic function and an orthonormal basis set. In Proceedings of the 14th WSEAS International Conference on Systems. Corfu; 23–25 July 2010:100-105.

    Google Scholar 

  53. 53.

    Mansour A: Probabilités et statistiques pour les ingénieurs: cours, exercices et programmation. Paris: Hermes Science; 2007.

    Google Scholar 

  54. 54.

    Mesleh R, Haas H, Sinanovic S, Ahn CW, Yun S: Spatial modulation. IEEE Trans. Vehicule Technol 2008, 57(4):2228.

    Article  Google Scholar 

  55. 55.

    Wang Z, Giannakis GB: A simple general parameterization quantifying performance in fading channels. IEEE Trans. Commun 2003, 51(8):1389. 10.1109/TCOMM.2003.815053

    Article  Google Scholar 

  56. 56.

    McCullagh P: Tensor Methods in Statistics. London: Chapman and Hall; 1987.

    MATH  Google Scholar 

  57. 57.

    Shiryayev AN: Probability. London: Springer; 1984.

    MATH  Book  Google Scholar 

  58. 58.

    Vu MH: Exploiting transmit channel side information in MIMO wireless systems. Ph.D. thesis, Stanford University, 2006

  59. 59.

    Kotz S, Johnson NL: Encyclopedia of Statistical Sciences. Amesterdam: University of Amesterdam; 1993.

    Google Scholar 

  60. 60.

    Lacoume JL, Amblard PO, Comon P: Statistiques d’ordre supérieur pour le traitement du signal. Paris: Mason; 1997.

    Google Scholar 

  61. 61.

    Bass J: Cours de Mathématiques: Tome 1, Fasicule 2. Masson, Paris, New York, Barcelone et Milan; 1977.

    Google Scholar 

  62. 62.

    Bass J: Cours de Mathématiques: Tome 2. Masson, Paris, New York, Barcelone et Milan; 1977.

    MATH  Google Scholar 

  63. 63.

    Mansour A, Kawamoto M: ICA papers classified according to their applications & performances. IEICE Trans. Fundamentals Electron. Commun. Comput. Sci 2003, E86-A(3):620.

    Google Scholar 

  64. 64.

    Savitzky A, Golay MJE: Soothing and differentiation of data by simplified least squares procedures. Anal. Chem 1994, 36: 1627-1639.

    Article  Google Scholar 

  65. 65.

    Orfanidis SJ: Introduction to Signal Processing. New Jersey: Prentice-Hall; 1995.

    Google Scholar 

  66. 66.

    Schafer RW: What is a Savitzky-Golay filter? IEEE Signal Process. Mag 2011, 111-117.

    Google Scholar 

  67. 67.

    Beauchamp K: Applications of Walsh and Related Functions. New York: Academic; 1984.

    Google Scholar 

  68. 68.

    Kendall M, Stuart A: The Advanced Theory of Statistics: Design and Analysis, and Time-series. London: Charles Griffin & Company Limited; 1961.

    MATH  Google Scholar 

  69. 69.

    Mansour A, Kardec Barros A, Ohnishi N: Comparison among three estimators for high order statistics. In Fifth International Conference on Neural Information Processing (ICONIP’98). Edited by: Usui S, Omori T. Kitakyushu; 21–23 October 1998:899-902.

    Google Scholar 

  70. 70.

    Martin A, Mansour A: Comparative study of high order statistics estimators. In International Conference on Software, Telecommunications and Computer Networks. Split, Dubrovnik, Venice; 10–13 October 2004:511-515.

    Google Scholar 

  71. 71.

    Bouvier A, George M, Le Lionnais F: Dictionnaire des mathématiques. Paris: Presses Universitaires De France,; 1993.

    MATH  Google Scholar 

Download references


The authors gratefully acknowledge the support for this work by SNCS Research Center at the University of Tabuk under the grant from the Saudi Ministry of Higher Education.

Author information



Corresponding author

Correspondence to Ali Mansour.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2

Authors’ original file for figure 3

Authors’ original file for figure 4

Authors’ original file for figure 5

Authors’ original file for figure 6

Authors’ original file for figure 7

Authors’ original file for figure 8

Authors’ original file for figure 9

Authors’ original file for figure 10

Authors’ original file for figure 11

Authors’ original file for figure 12

Authors’ original file for figure 13

Authors’ original file for figure 14

Authors’ original file for figure 15

Authors’ original file for figure 16

Authors’ original file for figure 17

Authors’ original file for figure 18

Authors’ original file for figure 19

Authors’ original file for figure 20

Authors’ original file for figure 21

Authors’ original file for figure 22

Authors’ original file for figure 23

Authors’ original file for figure 24

Authors’ original file for figure 25

Authors’ original file for figure 26

Authors’ original file for figure 27

Authors’ original file for figure 28

Authors’ original file for figure 29

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Mansour, A., Mesleh, R. & Aggoune, eH.M. Blind estimation of statistical properties of non-stationary random variables. EURASIP J. Adv. Signal Process. 2014, 21 (2014).

Download citation


  • Wireless communication
  • Dynamic transmission channel
  • Higher-order statistics
  • Cumulants
  • Cumulative distribution function
  • Kernel density estimators
  • Hermite basis set
  • Spline functions
  • Characteristic functions
  • Non-stationary signals