 Research
 Open Access
 Published:
Blind estimation of statistical properties of nonstationary random variables
EURASIP Journal on Advances in Signal Processing volume 2014, Article number: 21 (2014)
Abstract
To identify or equalize wireless transmission channels, or alternatively to evaluate the performance of many wireless communication algorithms, coefficients or statistical properties of the used transmission channels are often assumed to be known or can be estimated at the receiver end. For most of the proposed algorithms, the knowledge of transmission channel statistical properties is essential to detect signals and retrieve data. To the best of our knowledge, most proposed approaches assume that transmission channels are static and can be modeled by stationary random variables (uniform, Gaussian, exponential, Weilbul, Rayleigh, etc.). In the majority of sensor networks or cellular systems applications, transmitters and/or receivers are in motion. Therefore, the validity of static transmission channels and the underlying assumptions may not be valid. In this case, coefficients and statistical properties change and therefore the stationary model falls short of making an accurate representation. In order to estimate the statistical properties (represented by the highorder statistics and probability density function, PDF) of dynamic channels, we firstly assume that the dynamic channels can be modeled by shortterm stationary but longterm nonstationary random variable (RV), i.e., the RVs are stationary within unknown successive periods but they may suddenly change their statistical properties between two successive periods. Therefore, this manuscript proposes an algorithm to detect the transition phases of nonstationary random variables and introduces an indicator based on highorder statistics for nonstationary transmission which can be used to alter channel properties and initiate the estimation process. Additionally, PDF estimators based on kernel functions are also developed. The first part of the manuscript provides a brief introduction for unbiased estimators of the second and fourthorder cumulants. Then, the nonstationary indicators are formulated. Finally, simulation results are presented and conclusions are derived.
1 Introduction
Unlike discrete transmission channels which have limited practical value, continuous random transmission channels are widely accepted and used to illustrate practical concepts [1]. In numerous proposed algorithms, statistical properties or probability density function (PDF) of transmission channels are assumed to be perfectly known or already estimated [2–6]. To analyze the performance of various wireless transmission schemes, Simon and Alouini in [7] have used channel statistical models and PDF properties. It is widely believed that wireless transmission channels can be modeled using stationary random variables [1, 8, 9]. We should highlight the fact that various PDFs have been used in the context of wireless communications, such as Gausssian, Rayleigh, lognormal, exponential, onesided Gaussian distribution, Hoyt, Weibull, Rice, Nakagamim, αμ, and ηκ (see [10–21] and the references therein). It is worth pointing out that parametrical estimation methods can be used if the channel model (or a PDF for the real random variable (RV)) is selected or identified. Additionally, nonparametrical estimation methods can also be found in the literature [22–27]. However, these methods are hard to be implemented and mostly time consuming and they strongly relate to the proposed models and applications. For this main reason, a generic method and few required assumptions about the transmission channel are proposed and discussed in this manuscript.
The recent spread of cellular systems (smart sensors, mobile phones, base stations, satellites, surveillance devices, traffic radars, etc.) has increased the complexity of processing algorithms as well as the model of transmission channels. In fact, in various applications, transceiver units are in motion. For such systems, the transmission channels can no longer be considered as static channels, i.e., they cannot be modeled by stationary random variables. To overcome this difficulty, most researchers assume that the transmission channels are static during the processing time^{a}[28–30].
In this manuscript, the proposed models are improved by considering that transmission channels are dynamic channels that can be modeled as nonstationary variables. It is worth mentioning that for antennas mounted on moving vehicles or drones, the transmission channels can be approximated by almost static or slowly evolutionary dynamic channels over certain periods. The transition among these periods can be very fast. In addition, during two adjacent periods, transmission channels can have completely different statistical properties and/or PDFs. This scenario illustrates a typical scenario of a mobile phone used in a moving car which is moving among big buildings in a modern city. It can represent as well the scenario of a drone flying at low altitude in a mountainous area.
In what follows, we consider that transmission channels can be modeled as shortterm stationary signals and longterm nonstationary signals (many natural and physical signals belong to this category, such as speech signals [31, 32]). Herein, we develop an algorithm to accurately estimate the transition times. The algorithm is based on highorder statistics (HOS). In several study cases, HOS are shown to be more promising than the secondorder stochastic methods, namely, power, variance, covariance, and spectra. In fact, HOS have been used to solve many recent and important telecommunication problems [33, 34], such as blind identification or equalization, blind separation of sources, and time delay estimation [35–38]. It is worth mentioning that most of these HOS algorithms are only based on the second and the fourthorder statistics [39]. Once the transition times are estimated, we should be able to estimate the PDF of the local stationary random variable (i.e., during a shortterm period).
It is well known that any random variable can be completely described by its PDF [40]. In many cases, an accurate estimation of the PDF of physical parameters of interest (which are random variables) is essential to achieve our goals. This is while an accurate estimation of PDFs is still challenging to researchers. The estimation of PDFs is a relatively old problem that has been considered since the beginning of the last century with the rise and the development of new and modern complex systems (radars, sonar, and wireless communication systems). However, in late 1950s, systematic mathematical approaches have been proposed. One of the pioneering work in this field is the work presented by Rosenblatt in [41]. Parzen in [42] formulated the estimation of PDFs using kernel approaches. Later on, new approaches emerged to overcome the specification of diverse applications such as the sum of Gamma densities in Risk Theory [43] or the sum of exponential random variables in wireless communication [44]. Other researchers have focused on the choice of the kernel and smoothing functions [45, 46]. The estimation of PDFs using orthogonal series has been introduced by Schwartz in [47]. In more recent work, wavelets have been used as a nonparametric estimation of PDFs [48]. Similar to previous work, Engel used Haar’s series to estimate PDFs [49]. Using autoregressive (AR) models, Kay in [50] proved that PDFs can be estimated by appealing the theory of power spectral density (PSD). More recent work reintroduced the estimation of PDF using a windowed Fourier transform [51] or Hermite’s orthonormal basis set [52]. We should mention that all the abovementioned work assume that the random variables are stationary. In countless applications, the stationarity assumption may be invalid [53]. To the best of our knowledge, there is no such PDF estimator for nonstationary random variables. Hence, it is the aim of this paper to propose a PDF estimator based on a smooth kernel PDF estimator that assumes known transition times.
2 Mathematical model and background
In this paper, we consider the case of real nonstationary random variables^{b}. We also assume (1.1 in Appendix Appendix 1: highorder statistics estimators) that random variables consisting of transmission channels are stationary by part [54, 55]. While this assumption is less strict than the most widely used assumption of stationary transmission channels, its consideration is justified in the case of moving receivers or transmitters.
Hereinafter, Pr(A) stands for the probability of the event A. Let x_{1},…,x_{ n } stand for n independent realization of a continuous RV X. In addition, let us denote by {f}_{X}\left(x\right),{F}_{X}\left(x\right),{\widehat{f}}_{X}\left(x\right), and {\widehat{F}}_{X}\left(x\right) respectively as the PDF, the cumulative distribution function (CDF) of X, and their estimated functions:
In the following, k(x) denotes a kernel function of x, and the mathematical expectation (i.e., the mean) of a real RV X is denoted by
where E stands for the mathematical expectation. By definition [56], the q thorder moment μ_{ q } of a stochastic signal X is
We mentioned earlier that a real random variable X is completely described by its PDF f_{ X }(x). In addition, a random variable can also be described by its first Φ_{ X }(u) or second Ψ_{ X }(u) characteristic functions as follows:
where j^{2}=1. While the second characteristic function can be directly obtained from the first one, the first one is considered as the inverse Fourier transform of the PDF. Theoretically, PDFs may be obtained by the Fourier transform of Φ_{ X }(u). The Taylor series of the two characteristic functions yield other important statistical functions which are the moments and the cumulants [56]:
According to the theorem of LeonovShiryayev [56], the r thorder cumulant of X can be calculated from its moments, using the following formula^{c}[53, 57]:
Using this relationship, we can evaluate the fourthorder cumulant of X as
For a zeromean stochastic signal (please see [58] and the cited references therein),^{d} the secondorder cumulant is equal to the secondorder moment and therefore the fourthorder cumulant becomes
The HOS are much easier to be estimated than PDF, as will be shown in the next section. For this reason, our nonstationary transition indicator is based on the second and fourthorder statistics. In fact, the transition of nonstationary random variable can generate discontinuity in its HOS. As the variance of any signal should be different than zero, then it can be used to identify the existence of signals. On the other hand, normalized signals have the same unit variance. Therefore, the variance is not enough to identify the transition in a general case. Besides that, the fourthorder cumulants of Gaussian signals are zeros. For these reasons, we developed a nonstationary transition indicator based on the variance and the fourthorder cumulant.
3 Unbiased and adaptive HOS estimators
Let X be a zeromean stochastic signal where x_{ i } is an event (or a signal sample) of X (1≤i≤N). The classic estimator of the r thorder moment of X is given by
It is easy to verify that (11) is an unbiased estimator of the r thorder moment of X (i.e., E\left[\hat{{\mu}_{r}}\right]={\mu}_{r}). To estimate the fourthorder cumulant of X, we can derive an estimator from the LeonovShiryayev formula (8):
It is established [59, 60] that the estimator in (12) is a biased estimator and the estimation error decreases proportional to \frac{1}{N}. In fact, using (11) and (12) we can write
and the estimator expectation becomes
Using the LeonovShiryayev formula, one can develop an unbiased estimator for the cumulant as^{e}
It is worth mentioning that the above estimators are only unbiased for stationary signals. In case of nonstationary signals, adaptive estimators should be developed.
Let \hat{{\mu}_{r}}\left\{k\right\} be the estimator of the r thorder moment at the k th iteration, and we can develop the following adaptive estimators for the second and the fourthorder statistics:
for 1<k≤N. This algorithm is simple and converges quickly in the case of stationary signals. Another adaptive estimator of a fourthorder crosscumulant (Cum_{13}(X,Y)) which is more suitable for nonstationary signals is given by
where 0<γ<1 is another forgetting factor.
4 PDF estimators
To get an accurate estimation of f_{ X }(x), one should use available observations or measurements (i.e., x_{1},…,x_{ n }). Using Fubini’s theorem [61]^{f}, Rosenblatt in [41] proved that any estimator {\widehat{F}}_{X}\left(x\right) of f_{ X }(x) based on the set of x_{ i }, {i=1,⋯n}, is biased. However, Parzen in [42] proved that the F_{ n }(x) is an unbiased estimator {\widehat{F}}_{X}\left(x\right) of the CDF, i.e., E{F_{ n }(x)}=F_{ X }(x):
where Card denotes the cardinal of a σalgebra set. Using the fact that {f}_{X}\left(x\right)=\frac{{\mathit{\text{dF}}}_{X}\left(x\right)}{\mathit{\text{dx}}}, Parzen proposed an asymptotically unbiased estimator of the PDF as the sum of scaled and shifted kernel functions:
given that {\text{lim}}_{n\to \infty}h\left(n\right)=0 and the kernel k(x) and x k(x) are Borel’s functions [62]^{g} in L1 function space, which satisfy few conditions such as [41, 47]
In [42], Parzen suggests few kernel functions including histograms which are kernel estimators when
with u(x) being the Heaviside step function.
Using the definition in (4), the PDF f_{ X }(x) can be approximated by using the Fourier transform of an estimated characteristic function [53]. In fact, the average over a set of realization is an unbiased estimator of the mathematical expectation of a RV. Therefore, the first characteristic function can be approximated using the following equation:
In [51], the authors mentioned that the Fourier transform of the last equation cannot converge. However, it is possible to obtain the Fourier transform of the product between {\widehat{\Phi}}_{X}\left(u\right) and a weighting window w(x). In this case the PDF estimator is given by
The above equation can be considered as a kernel estimator. Window length and parameters are the key factors which can affect the performance of the estimator [51]. To avoid the problem of this approach and apply the wellestablished theory of power spectral density, Kay in [50] proposed an AR model of order p to estimate the PDF. Parameters of his AR model are estimated using YuleWalker equations and Levinson recursion.
Using GramCharlier series to expand the PDF in terms of the normal density and its derivatives, Bowers in [43] approximated a risk theoretic distribution by the sum of Gamma RVs. In his expansion, the polynomials which multiply the PDF of a zeromean normalized normal PDF are the Hermite polynomials (see Appendix Appendix 2: Hermite’s polynomial functions). This study was generalized by Schwartz in [47].
According to Vannucci [24], there are different types of nonparametric density estimators: delta sequence estimators (such as kernel, histogram, and orthogonal series estimators) and penalized maximum likelihood estimators such as spline estimators. She also proved that scaling functions give a good approximation of smooth functions, while wavelets^{h} can deal with functions which have local fluctuations. In order to practically apply wavelet estimators, a truncation should be made [24]. Therefore, the overall performance depends on the truncation order. An optimal choice of such order can be determined by minimizing the integrated square error or the intergraded mean square error. In realworld applications, another problem can arise related to the evaluation of the scaling and the mother functions at arbitrary points [24]. In [49], Engel used Haar’s functions to estimate the PDF and shows that his estimator is equivalent to a histogram on certain dyadic intervals.
Recently, the authors of [51, 52] have compared their approaches to the abovementioned ones. In fact, Xie and Wang in [51] showed that an estimator based on the Fourier transform of the characteristic function gives similar results to the AR model PDF estimator proposed earlier by Kay in [50]. They also showed that in certain cases, histograms can provide similar overall performance to previously mentioned estimators. However, Howard in [52] showed that the Hermite basis estimator can slightly achieve better results than the estimator based on the Fourier transform of the characteristic function proposed in [51]. For these reasons, we only consider the histogram, the Hermite basis estimator and the smooth kernel density estimator proposed by Bowan in [45, 46]. Our simulations have shown that smooth kernel density has slightly better performance than the previous mentioned ones (see Section 7).
Finally, two useful error measures are proposed to evaluate the overall performance of various estimators [41, 47, 51, 52]:

The integrated error
\phantom{\rule{2em}{0ex}}{\epsilon}_{1}=E\left\{\mid {f}_{X}\left(x\right){\widehat{f}}_{X}\left(x\right)\mid \right\} 
The root of the meansquared error
\phantom{\rule{2em}{0ex}}{\epsilon}_{2}=\sqrt{E\left\{\mid {f}_{X}\left(x\right){\widehat{f}}_{X}\left(x\right){\mid}^{2}\right\}}
5 Representation of nonstationary PDF
Realworld signals can be modeled by wide or quasi nonstationary random variables [39, 63]. In this case, the PDF is a timedependent function f_{ X }(x,t) where t can be a vector representing discrete time instants. Let us consider for example a wireless transmission channel h(t) which can be modeled by a quasi stationary RV [1]. Let us assume now that h(t) has a Weibull PDF for the first period, a uniform PDF for the second period, then a Gaussian RV in its last parts. In this case, it becomes clear that the PDF of h(t) cannot be represented by simple curve as function of t or X. Hence, the PDF of nonstationary RV should be plotted as a function of a RV X and t (see Figure 1).
In fact, classic PDF estimators cannot provide an accurate estimation of the nonstationary signal PDF. By applying a smooth kernel density estimator over 10,000 independent realization of h(t), a bimodal PDF was obtained (see Figure 1b). It is clear that the obtained PDF cannot correctly represent the PDF shown in Figure 1a.
6 Nonstationary transition indicator
In order to estimate the PDF parts of quasi nonstationary RV, one can easily use the estimator developed in the previous section to detect the number and the size of the stationary parts. Once these parts are well identified and their sizes are relatively enough to estimate a PDF, we can apply any classic PDF estimator. To make this estimation more robust, cumulants of different order (mainly the second and the fourth order) can be used to identify such parts. In fact, Gaussian signals are characterized by their zero fourthorder cumulant (see Appendix Appendix 1: highorder statistics estimators). In these case, the fourth order cannot separate two adjacent Gaussian parts. However, if these two parts represent two normal RVs with different means or variances, then the second or the first moment can be used to identify these parts. If the two Gaussian parts have the same mean and variance then they can describe the same RV due to the basic assumptions about the whiteness of the samples (i.e., the realization of X).
Hereinafter, we consider that the nonstationary RV is formed by successive segments of stationary RV. In order to estimate the PDF of the nonstationary signal, an estimator of the transition times should be developed. In order to simplify our discussion and gain insight, a generic case is considered. Let us consider a zeromean nonstationary signal X(t) made by four parts of stationary random variables as shown in Figure 2:

1.
The first part contains 8,000 samples of uniform random variable included in [ 1,1].

2.
The second part is made of 6,000 samples of zeromean and unite variance Gaussian signal.

3.
The 10,000 samples of a uniform random variable in [ 2,2] formed the third part.

4.
The fourth part is a zeromean Gaussian signal with a standard deviation of \sqrt{2}.
The fourthorder statistics of the uniform parts are given as follows:
where A is the maximum amplitude of X. Figure 3 shows that the proposed adaptive estimators can yield a good estimation of the HOS. However, these estimators are not quite enough to estimate the HOS and the transition times with a wide accuracy range. In fact, the estimators shown in Figure 3 can be considered as noisy signals. To reduce the noise level, we adopted two steps:

1.
First, the obtained signals are filtered using the smoothing polynomial regression filter proposed by SavitzkyGolay [64–66]. The main idea of the SavitzkyGolay filter is to apply a FIR filter such that its coefficients minimize the meansquared approximation error. This filter is used to filter biomedical signals such as EEG signals (see Figure 4).

2.
Second, the noise in the filtered signals are slightly reduced using a WalshHadamard transform which is a generalization of discrete Fourier transform [67]. This transformation is also used in biomedical signals to filter and compress ECG signals (see Figure 5).
In order to smooth the HOS estimator and achieve better detection of the transition times, the signal of the HOS estimator was firstly filtered using a SavitzkyGolay filter then a WalshHadamard transform truncation filtering techniques is applied. Experimental results showed that the filtered signals are much more smooth and useful to reach our goal (see Figure 6).
While the filtered signals are much better smoothed than the raw signals, they are still relatively noisy to obtain an accurate estimate of transition time. In fact, Figure 6 shows that the obtained filtered estimators are still suffering from the previously mentioned drawbacks, i.e., the inaccuracy in the estimation of the HOS and the transition times.
To address the previously mentioned problems, we developed the following algorithm:

First, the PDF of the filtered HOS estimators are obtained using the kernel PDF estimator (see Figure 7). Then the maximum values of the obtained PDF are recorded as the coefficients of a vector called maxHOS.

The coefficients of the vector maxHOS are introduced as the center of clusters. By minimizing Euclidean distance among the samples of the filtered HOS estimator, all samples of filtered estimated cumulants are clustered in rectangular signals which represent theoretical values of the HOS (see Figure 8).

Various simulations have been conducted. Our simulations showed that sometimes the rectangular clustered signals suffer from local narrow spurious error windows (see Figure 8). These spurious error windows are normally very narrow. Hence, by supposing that the channel is not a highly dynamic one, i.e., any channel parameter cannot change more than one time in a short period (for example, the short period can be the symbol duration). In this case, one can easily eliminate these windows. In order to clean out the clustered rectangular signals and achieve an accurate nonstationary indicator, the derivative of these signals is evaluated (see Figure 9).

Let us assume 1.2 in Appendix Appendix 1: highorder statistics estimators that the stationary parts of the channels have more than 1,000 samples (in many cases, 500 samples were enough to obtain good results). In this case, each of local narrow spurious error windows generates two Dirac delta functions with equal values and opposite signs. Based on 1.2 in Appendix Appendix 1: highorder statistics estimators, the two delta functions should be close to each other within 1,000 samples. Using this fact, we developed and implemented a recursive filtering procedure called continuity process (CP) to eliminate these spurious windows (see Figure 10). The Matlab code is given in Appendix Appendix 3: continuity process algorithm.

Figure 10 shows that the impact of the local narrow spurious error windows is completely eliminated. However, our nonstationary indicator still suffers from twostep transition problem, i.e., the transition between two valid states of the cluster rectangular signals is not immediate (this case can be shown in Figure 8 around the 14,000th and 24,000th samples). Using 1.2 in Appendix Appendix 1: highorder statistics estimators and the fact that the twostep transition problem generates two Dirac delta functions with different values but sharing the same sign, we developed another process called twostep transition process to deal with this problem (see Appendix Appendix 3: continuity process algorithm).

Finally, a clear and accurate nonstationary indicator is obtained. Our final indicator is the output of an ‘OR’ gate applied on the nonstationary indicator of various HOS (in our case, we used two HOS, i.e., the second and the fourth cumulants). The final indicator is shown in Figure 11. In Figure 12, the synoptic of our proposed algorithm using smoothing filters and nearestneighbor clustering algorithm is presented.
7 PDF estimator for nonstationary signals
In the previous section, the transition between two stationary parts has been well identified. Hereinafter, we assumed that the different stationary segments of the signal are already known and just the PDF of the nonstationary signal should be estimated. To evaluate their algorithms, the authors of [50–52] generate a RV X with its PDF as a moderated sum of PDF of two uncorrelated normal RVs:
where m_{ i } and σ_{ i } are, respectively, the mean and the standard deviation of the normal RV X_{ i } and p_{1}+p_{2}=1 are weighting parameters. It was mentioned in [52] without any proof that the previous PDF shown in equation (20) ‘is consistent with a random sum of independent RVs’:
where, for example, M∈1,2, and Pr[ M=1]=p_{2}. We proved (see Appendix 4) that the sum should be just over two independent RV and we give the statistical properties of the newly obtained RV.
It is well known that the overall performance of a histogram depends on the number of samples as well as the theoretical PDF. Performances can deteriorate if the number of samples is not large enough. On the other hand, if realization can be repeated many times, then an accurate estimation of the PDF can be obtained as the average of all obtained histograms (in Figure 13b, the average is done using ten iterations). For a large number of samples, the histogram performs quite well (see Figure 13c).
Our simulations show that the Hermite basis set estimator suffers from two major inconveniences:

1.
It depends on the value of k _{ s },

2.
It can generate a negative function, see Figure 14.
The smooth kernel estimator proposed in [46] seems to overcome the previous two mentioned drawbacks (see Figure 15).
8 Conclusions
In this manuscript, a transition indicator for nonstationary signals is presented. The new indicator is based on HOS of quasi nonstationary variables (the random variables are considered stationary by parts). To estimate the HOS, unbiased adaptive exponential estimators are presented.
To reduce the noise level of the HOS estimators, we used a cascade filtering procedure based on SavitzkyGolay filters and a truncation of its Walsh Hadamard transform. Then rectangular signals representing the theoretical values of HOS are obtained by using clustering algorithm using the maxima of kernel PDF estimator and minimizing Euclidean distances.
Simulation studies show that the obtained rectangular signals can suffer from local narrow spurious error windows which can be eliminated using a continuity assumption 1.2 in Appendix Appendix 1: highorder statistics estimators and a continuity cleaning procedure called continuity process. In addition to these local narrow spurious error windows, estimated signals suffer another artifact called the twostep transition problem. After solving this problem using assumption 1.2 in Appendix Appendix 1: highorder statistics estimators and a continuity procedure, an accurate transition indicator of nonstationary signals is achieved. Simulation studies corroborate the performance of our proposed algorithm and the accuracy of our nonstationary transition indicator.
Finally, a survey of major PDF estimators is done. A comparative study is also presented and discussed. The advantages and drawbacks of major methods are highlighted and a theoretical study is provided. Simulation results show a slight advantage of smooth kernel estimator methods. It is worth mentioning that the histogram with a large number of samples is still one of the simplest and efficient estimators. The case of nonstationary process was considered and a PDF estimation approach was discussed.
Endnotes
^{a}However, the processing time is not standardized. In fact, different authors claim that the parameters of the channel should remain constant during one frame duration, few hundred symbols, or during the convergence time of their adaptive algorithms.
^{b}The fact that the channel is considered as a real channel is not limiting our approach, as a complex Gaussian channel could be represented by its modulus as a Rayleigh channel.
^{c}The original formula shows the relationship among the cumulant of r stochastic signals X_{ i } (i=1,…,r) and their moments of order p, p≤r:
where the addition operation is over all the set of v_{ i } (1≤i≤p≤r) and v_{ i } constitutes a partition of {1,…,r}, [40].
^{d}In many applications, the stochastic signal X is a zeromean signal. This assumption is not a major one and it has been used in many studies (please see [58] and the cited references therein). In fact, by considering this assumption, the mathematical notations can be simplified. However, all the proposed steps can be straightforwardly derived in the case of a nonzero mean transmission channel. Besides that, the mean can be estimated and canceled out from the other equations.
^{e}Further details are given in Appendix Appendix 1: highorder statistics estimators.
^{f}According to Fubini’s theorem [61], a double integral defined over two measure spaces {\mathcal{D}}_{a},{D}_{b} of a measurable function f(x,y) can be computed using iterated integrals:
^{g}A function f(x) is called Borel’s function if \forall y\in \mathcal{A} is an open set, the inverse of f(x), and x=f^{1}(y) is an element of a Borel’s set, i.e., Lebesgue measurable set [62].
^{h}Wavelets are basis functions which have quite interesting properties such as their localization in space and frequency [24].
Appendix 1: highorder statistics estimators
In this section, HOS estimators are developed. Generally, HOS estimators can be divided into main families: the arithmetic and the exponential estimators.
1.1 Arithmetic estimators
Let X to be a zeromean stochastic ergodic signal where x_{ i } is an event (or a signal sample) of X (1<i<N). In this case, the arithmetic estimator of the q thorder moment is given by
This estimator assumes that the signal X is stationary over N samples. This estimator is a nonbiased estimator (i.e., E\left(\hat{{\mu}_{q}}\right)={\mu}_{q}) and its variance is given by
Clearly, it is a consistent estimator; hence for stationary signals, its variance decreases with an increased number of samples. An arithmetic estimator of the q thorder cumulant can be developed from Equation 8:
It is proved [59, 60] that the estimator in (24) is a biased consistent estimator where the estimation error decreases proportional to \frac{1}{N}:
A nonbiased cumulant estimator can be deduced from the last equation:
where the parameters c_{ p } depend on the partitions of the indices v_{ i }. These parameters can be estimated as the solution of q linear equations. Let us consider the fourthorder cumulant:
In order to make the last estimator unbiased, one should solve a linear system of equations obtained by comparing termtoterm the expectation of Equation 26 and the theoretical value given by (9)
For zeromean signals, we can easily prove that
This means that the following estimator is an unbiased estimator for the fourthorder cumulant of a zeromean stationary signal X:
For realtime applications, the estimators should be adaptive ones. The estimator (23) is not an adaptive one, but it is easy to derive an adaptive version:
where \hat{{\mu}_{r}}\left\{k\right\} is the estimator of the r th order moment at the k th iteration.
1.2 Exponential estimators
Exponential estimators are defined as
where 0<λ_{ q }<1 represents a forgotting factor. This estimator can be calculated easily in an adaptive way:
The latest estimator is biased (E\left(\hat{{\mu}_{q}}\right)=\left(1{\lambda}_{q}^{N}\right){\mu}_{q}), but it is asymptotically nonbiased. The main interest in such estimator resides on the fact that it can give better estimation for the moments of nonstationary signals. Thus, the closest λ to 1, the more past samples are taking into account. A nonbiased exponential estimator can be written as
Estimator (32) can be also modified to an adaptive version:
An adaptive nonbiased estimator of the cumulants could be derived using (22) and (33). To simplify our discussion, the fourthorder cumulant unbiased estimator for zeromean signals could be developed as
where γ is a forgetting factor and
1.3 Adaptive unbiased estimators of the fourthorder cumulants
A nonbiased estimator of fourthorder crosscumulants can be obtained from the definition of the crosscumulants [68]. In fact let us consider K_{22} an estimator of Cum_{22}(X,Y) defined as
where a, b, and c should be set in order to make K_{22} a nonbiased and consistent estimator. When samples x_{ i } and y_{ i } are independent, one can use similar estimators to these proposed in [69, 70]. In the following, we assume that the samples are independent and identically distributed (iid) over time but spatially correlated. In this case, one can prove that K_{22} become a nonbiased and consistent estimator:
when a=\frac{N+2}{N1} and b=c=\frac{N}{N1}. Similarly, one can develop other estimators:
To obtain these estimators, signals are assumed stationary. The last assumption cannot be satisfied in our application. Therefore, some modifications should be considered. Let C_{13}(N)=K_{13}(X,Y) be the adaptive estimator of the cumulant 3×1 using N samples, {A}_{N}=\sum _{i}^{N}{x}_{i}{y}_{i}^{3} and {B}_{N}=\sum _{\mathit{\text{ij}}}^{N}{x}_{i}{y}_{i}{y}_{j}^{2}. In this case, Equation 35 can be written as
Hence, we can prove that
Finally, the last equation can be modified to
where {\mu}_{\mathit{\text{nm}}}\left(X,Y\right)=\frac{1}{N1}\sum _{i=1}^{N1}{x}_{i}^{n}{y}_{i}^{m} is the estimator of E(X^{n}Y^{m}) using N1 samples. Using the last two equations, we can derive the final form of our adaptive fourthorder cumulant estimator:
Appendix 2: Hermite’s polynomial functions
According to [71], Hermite polynomials are real orthogonal polynomials with respect to the weight function w\left(x\right)={e}^{{x}^{2}}. The n thorder Hermite polynomial is defined as follows:
Two hermite polynomials of orders n and m satisfy the following properties:
where {\delta}_{m}^{n} is Kronecker’s symbol. It is worth mentioning that the above recurrence equation is widely used in practice to estimate the Hermite polynomials. In fact, H_{0}(x)=1, H_{1}(x)=2x so H_{2}(x)=4x^{2}2 and so on. In order to use Hermite’s basis set, one should define the following two functions:
where k_{ s } is the scaling factor, H_{ i }(x) is the Hermite’s polynomial of the i th order. Figure 16 shows the first eight normalized Hermite polynomial functions, i.e., b_{ i }(x) when k_{ s }=1.
Appendix 3: continuity process algorithm
Appendix 4
The PDF of a moderate sum of two uncorrelated RVs
Let us consider two independent RVs X_{ i }, i=1,2. It is easy to prove that the proposed random sum can be written as follows:
where, M∈1,2 is a modified Bernoulli RV, and a is a Bernouilli RV with Pr(a=1)=1 Pr(a=0)=ε. In this case, one can write:
Using the previous equation, one can deduce [53] that
where * represents the convolution product. If X_{1} and X_{2} are two uncorrelated mutually Gaussian RVs, then Z=X_{1}+X_{2} is another normal RV with \mathcal{N}\left({m}_{1}+{m}_{2},\sqrt{{\sigma}_{1}^{2}+{\sigma}_{2}^{2}}\right), which proves the proposed statement.
The mean of the new variable X is given by
Using the definition of the variance and the properties of a Bernouilli RV, the second moment of the new variable X becomes
In this case the variance of X could be obtained as follows:
Authors’ information
AM received his M.Sc. and Ph.D. degrees in Signal, Image and Speech Processing from the ‘Institut National Polytechnique de Grenoble  INPG’ (Grenoble, France) on July 1993 and January 1997, respectively, and his HDR degree (Habilitation a Diriger des Recherches; in the French system, this is the highest of the higher degrees) on November 2006 from the Universite de Bretagne Occidentale  UBO (Brest, France). His research interests are in the areas of blind separation of sources, highorder statistics, signal processing, passive acoustics, cognitive radio, robotics, and telecommunication. From January 1997 to July 1997, he held a postdoc position at Lab. de Traitement d’Images et Reconnaissance de Forme, France. From August 1997 to September 2001, he was a research scientist at the BioMimetic Control Research Center (BMC) in Riken, Nagoya, Japan. From October 2001 to January 2008, he was holding a teacherresearcher position at the Ecole Nationale Supérieure des Ingénieurs des Etudes et Techniques d’Armement (ENSIETA) in Brest, France. From February 2008 to August 2010, he was a senior lecturer at the ECE Department of Curtin University, Perth, Australia. During January 2009, he held an invited professor position at the Universite du Littoral Cote d’Opale in Calais, France. From September 2010 till June 2012, he was a professor at University of Tabuk, Kingdom of Saudi Arabia. He also served as the electrical department head at the University of Tabuk. Since September 2012, he has been a professor at Ecole Nationale Supérieure de Techniques Avancées Bretagne (ENSTA Bretagne) in Brest, France. He is the author and the coauthor of three books. He is the first author of several papers published in international journals. He is also the first author of many papers published in the proceedings of various international conferences. Finally, he was elected to the grade of IEEE Senior Member in February 2006. He has been a member of Technical Program Committees (TPC) and he was a chair, cochair, and a scientific committee member in various international conferences. He is an active reviewer for a variety of international journals. He was a lead guest editor of EURASIP Journal on Advances in Signal Processing.
RM (S’00M’08SM’13) holds a Ph.D. in Electrical Engineering from Jacobs University in Bremen, Germany and several years of postdoctoral wireless communication and optical wireless communication research experience in Germany. In October 2010, he joined University of Tabuk in Saudi Arabia where he is now an assistant professor and the director of research excellence unit. His main research interests are in spatial modulation, MIMO cooperative wireless communication techniques, and optical wireless communication. His publications received more than 800 citations since 2007. He has published more than 50 publications in toptier journals and conferences, and he holds seven granted patents. He also serves as on the TPC for academic conferences and is a regular reviewer for most of IEEE/OSA Communication Society’s journals and IEEE/OSA Photonics Society’s journals.
HMA received his M.S. and Ph.D. degrees in Electrical Engineering from the University of Washington (UW), Seattle, WA, USA. He is a professional engineer registered in the State of Washington and a senior member of the Institute of the IEEE. He has taught graduate and undergraduate courses in Electrical Engineering at a number of universities in the USA and abroad. He served at many academic ranks including Endowed Chair Professor and Vice President and Provost. He was the winner of the Boeing Supplier Excellence Award. He was also the winner of the IEEE Professor of the Year Award, UW Branch. He is listed as inventor in a major patent assigned to the Boeing Company. His research work is referred to in many patents including patents assigned to ABB, Switzerland, and EPRI, USA. Currently, he is a professor and director of the Sensor Networks and Cellular Systems (SNCS) Research Center, University of Tabuk, Tabuk, Saudi Arabia. He authored many papers in IEEE and other journals and conferences. His research interests include modeling and simulation of large scale networks, sensors and sensor networks, scientific visualization, and control and energy systems.
References
Proakis J, Salehi M: Digital Communications. New York: McGrawHill; 2007.
Tarokh V, Jafarkhani H, Calderbank AR: Spacetime codes high data rate wireless communication: performance crirterion and construction. IEEE Trans. Inf. Theory 1998, 44(2):744. 10.1109/18.661517
Bradaric I, Petropulu AP, Diamantaras K: On blind identification of FIRMIMO systems with cyclostationary inputs using second order statistics. IEEE Trans. Signal Process 2003, 51(2):434. 10.1109/TSP.2002.806998
Srar JA, Chung KS, Mansour A: Adaptive array beamforming using a combined LMSLMS algorithm. IEEE Trans. Antennas Propagation 2010, 58: 3545.
Mesleh R, Ikki S, Aggoune H, Mansour A: Performance analysis of space shift keying (SSK) modulation with multiple cooperative relays. EURASIP J. Adv. Signal Process 2012., 1: doi:10.1186/168761802012201
Abaza M, Mesleh R, Mansour A, Alfalou A: MIMO techniques for high data rate free space optical communication system in lognormal channel. In International Conference on Technological Advances in Electrical, Electronics and Computer Engineering. Konya: TAEECE,; 9–11 May 2013:161166.
Simon M, Alouini MS: Digital Communication over Fading Channels: a Unified Approach to Performance Analysis. New York: Wiley; 2000.
Haykin S, Moher M: Communication Systems. New York: Wiley; 2009.
Abaza M, Mesleh R, Mansour A, Aggoune HA: Diversity techniques for FSO communication system in correlated lognormal channels. Opt. Eng 2014., 53(1): 10.1117/1.OE.53.1.016102
Skitovic VP: Linear forms of independent random variables and the normal distribution law. Izvestiya Akademii Nauk SSSR. Seriya Matematiceskaya 1954., 18(185): In Russian
Kendall M, Stuart A: The Advanced Theory of Statistics: Distribution Theory. London: Charles Griffin & Company Limited; 1961.
Nikias CL, Shao M: Signal Processing with αStable Distribution and Applications. New York: Wiley; 1995.
Sijbers J, Den Dekker J, Scheunders P, Van Dyck D: Maximumlikelihood estimation of Rician distribution parameters. IEEE Trans. Med. Imaging 1998, 17(3):357. 10.1109/42.712125
Karagiannidis GK, Zogas DA, Kotsopoulos SA: On the multivariate Nakagamim distribution with exponential correlation. IEEE Trans. Commun 2003, 51(8):1240. 10.1109/TCOMM.2003.815071
Yacoub MD, Fraidenraich G, Santos Filho JCS: Nakagamim phaseenvelope joint distribution. Electron. Lett 2005., 41(5): doi:10.1049/el:20057014
Yacoub MD, Fraidenraich G, Tercius HB, Martins FC: The symmetrical on the η  κ distribution: a general fading distribution. IEEE Trans. Broadcasting 2005, 51: 504. 10.1109/TBC.2005.859240
Fraidernraich G, Daoud Yacoub M: The α  η  μ and α  k  μ fading distributions. In IEEE 9th International Symposium on Spread Spectrum Techniques and Applications. Manaus; 28–31 August 2006:255263.
Nadarajah S, Kotz S: On the η  κ distribution. IEEE Trans. Broadcasting 2006, 52(3):411. 10.1109/TBC.2006.879862
Yacoub MD: The α  η distribution: a physical fading model for the Stacy distribution. IEEE Trans. Vehicular Technol 2007, 56(1):27.
Cotton SL, Scanlon WG, Guy J: The k  μ distribution: applied to the analysis of fading in body to body communication channels for fire and rescue personnel. IEEE Antennas Wireless Propagation Lett 2008, 7: 66.
Yacoub MD: Nakagamim phaseenvelope joint distribution: an improved model. In Microwave and Optoelectronics Conference, IMOC’2009. Belem; 3–9 November 2009:335339.
Cox DD: A penalty method for nonparametric estimation of the logarithmic derivative of a density function. Ann. Instit. Stat. Math 1985, 37: 271. 10.1007/BF02481097
Vidal J, Bonafonte A, Fonollosa AR, De Losada NF: Parametric modeling of PDF using a convolution of onesided exponentials: application to HMM. In European Signal Processing Conference, ed. by M Holt, C Cowan, P Grant, W Sandham, European Signal Processing Conference. Edinburgh: Elsevier,; 1994:5457.
Vannucci M: Nonparametric density estimation using wavelets. Discussion Paper, Duke University, 1998
Archambeau A, Verleysen M: Fully nonparametric probability density function estimation with finite gausssian mixture models. In 5th International Conference on Advances in Pattern Recognition Calcultta, ICAPR’2003. Allied, New Delhi, Calcultta; 2003:8184.
Boscolo R, Pan H, Roychowdhury VP: IEEE Trans. Neural Netw. 2004, 15(1):55.
Roueff F, Rydén T: Nonparametric estimation of mixing densities for discrete distributions. Ann. Stat 2005, 33(5):2066. 10.1214/009053605000000381
Alamouti S: A simple transmit diversity technique for wireless communication. IEEE J. Selected Areas Commun 1998, 16(8):1451. 10.1109/49.730453
Larsson EG, Stoica P: Spacetime Block Coding for Wireless Communications. Cambridge: The Press Syndicate of the University of Cambridge; 2003.
Choqueuse V, Mansour A, Burel G, Collin L, Yao K: Blind channel estimation for STBC systems using higherorder statistics. IEEE Trans. Wireless Commun 2011, 10: 495.
Mansour A, Jutten C, Ohnishi N: Kurtosis: definition and properties. In International Conference on MultisourceMultisensor Information Fusion. Edited by: Arabnia HR, Zhu DD. Las Vegas; 6–9 July 1998:4046.
Mansour A, Jutten C: What should we say about the kurtosis. IEEE Signal Process. Lett 1999, 6(12):321.
Mendel JM: Tutorial on higherorder statistics (Spectra) in signal processing and system theory: theoretical results and some applications. IEEE Proc 1991, 79: 277.
Cichocki A, Amari SI: Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. New York: Wiley; 2002.
Cardoso JF: Source separation using higher order moments. In Proceedings of International Conference on Speech and Signal Processing 1989, ICASSP 1989. Glasgow; 22–25 May 1989:21092212.
Comon P: Independent component analysis, a new concept. Signal Process 1994, 36(3):287. 10.1016/01651684(94)900299
Mansour A, Jutten C: A direct solution for blind separation of sources. IEEE Trans. Signal Process 1996, 44(3):746. 10.1109/78.489054
Hyvärinen A, Karhunen J, Oja E: Independent Component Analysis. New York: Wiley; 2001.
Mansour A, Kardec Barros A, Ohnishi N: Blind separation of sources: methods, assumptions and applications. IEICE Trans. Fundamentals Electron. Commun. Comput. Sci 2000, E83A(8):1498.
Papoulis A: Probability, Random Variables, and Stochastic Processes. New York: McGrawHill; 1991.
Rosenblatt M: Remarks on some nonparametric estimates of a density function. Ann. Math. Stat 1956, 27(3):832. 10.1214/aoms/1177728190
Parzen E: On the estimation of a probability density function and mode. Ann. Math. Stat 1962, 33(3):1065. 10.1214/aoms/1177704472
Bowers LN: Expansion of probability density functions as a sum of gamma densities with applications in risk theory. Trans. Soc. Actuaries 1966, 18 PT 1(52):125.
Khuong HV, Kong HY: General expression for pdf of a sum of independent exponential random variables. IEEE Commun. Lett 2006, 10(3):159. 10.1109/LCOMM.2006.1603370
Reinsch CH, Smoothing by spline functions: Numer. Math. 1967, 10: 177. 10.1007/BF02162161
Bowman AW, Azzalini A: Applied Smoothing Techniques for Data Analysis. New York: Oxford University; 1997.
Schwartz SC: Estimation of probability density by an orthogonal series. Ann. Math. Stat 1967, 38(3):1261.
Vannucci M: On the application of wavelets in statistics. Ph.D. thesis, Dipartimento di Statistica, University of Florence, 1996
Engel J: Density estimation with, Haar series. Stat. Probability Lett 1990, 9: 111. 10.1016/01677152(92)90003N
Kay S: Modelbased probability density function estimation. IEEE Signal Process. Lett 2001, 5(12):318.
Xie J, Wang Z: Probability density function estimation based on windowed Fourier transform of characteristic function. In Proceedings of the 2nd International Congress on Image and Signal Processing. Tianjin; 17–19 October 2009.
Howard RM: PDF estimation via characteristic function and an orthonormal basis set. In Proceedings of the 14th WSEAS International Conference on Systems. Corfu; 23–25 July 2010:100105.
Mansour A: Probabilités et statistiques pour les ingénieurs: cours, exercices et programmation. Paris: Hermes Science; 2007.
Mesleh R, Haas H, Sinanovic S, Ahn CW, Yun S: Spatial modulation. IEEE Trans. Vehicule Technol 2008, 57(4):2228.
Wang Z, Giannakis GB: A simple general parameterization quantifying performance in fading channels. IEEE Trans. Commun 2003, 51(8):1389. 10.1109/TCOMM.2003.815053
McCullagh P: Tensor Methods in Statistics. London: Chapman and Hall; 1987.
Shiryayev AN: Probability. London: Springer; 1984.
Vu MH: Exploiting transmit channel side information in MIMO wireless systems. Ph.D. thesis, Stanford University, 2006
Kotz S, Johnson NL: Encyclopedia of Statistical Sciences. Amesterdam: University of Amesterdam; 1993.
Lacoume JL, Amblard PO, Comon P: Statistiques d’ordre supérieur pour le traitement du signal. Paris: Mason; 1997.
Bass J: Cours de Mathématiques: Tome 1, Fasicule 2. Masson, Paris, New York, Barcelone et Milan; 1977.
Bass J: Cours de Mathématiques: Tome 2. Masson, Paris, New York, Barcelone et Milan; 1977.
Mansour A, Kawamoto M: ICA papers classified according to their applications & performances. IEICE Trans. Fundamentals Electron. Commun. Comput. Sci 2003, E86A(3):620.
Savitzky A, Golay MJE: Soothing and differentiation of data by simplified least squares procedures. Anal. Chem 1994, 36: 16271639.
Orfanidis SJ: Introduction to Signal Processing. New Jersey: PrenticeHall; 1995.
Schafer RW: What is a SavitzkyGolay filter? IEEE Signal Process. Mag 2011, 111117.
Beauchamp K: Applications of Walsh and Related Functions. New York: Academic; 1984.
Kendall M, Stuart A: The Advanced Theory of Statistics: Design and Analysis, and Timeseries. London: Charles Griffin & Company Limited; 1961.
Mansour A, Kardec Barros A, Ohnishi N: Comparison among three estimators for high order statistics. In Fifth International Conference on Neural Information Processing (ICONIP’98). Edited by: Usui S, Omori T. Kitakyushu; 21–23 October 1998:899902.
Martin A, Mansour A: Comparative study of high order statistics estimators. In International Conference on Software, Telecommunications and Computer Networks. Split, Dubrovnik, Venice; 10–13 October 2004:511515.
Bouvier A, George M, Le Lionnais F: Dictionnaire des mathématiques. Paris: Presses Universitaires De France,; 1993.
Acknowledgements
The authors gratefully acknowledge the support for this work by SNCS Research Center at the University of Tabuk under the grant from the Saudi Ministry of Higher Education.
Author information
Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Mansour, A., Mesleh, R. & Aggoune, eH.M. Blind estimation of statistical properties of nonstationary random variables. EURASIP J. Adv. Signal Process. 2014, 21 (2014). https://doi.org/10.1186/16876180201421
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/16876180201421
Keywords
 Wireless communication
 Dynamic transmission channel
 Higherorder statistics
 Cumulants
 Cumulative distribution function
 Kernel density estimators
 Hermite basis set
 Spline functions
 Characteristic functions
 Nonstationary signals