A variable-parameter normalized mixed-norm (VPNMN) adaptive algorithm

Abstract

Since both the least mean-square (LMS) and least mean-fourth (LMF) algorithms suffer individually from the problem of eigenvalue spread, so will the mixed-norm LMS-LMF algorithm. Therefore, to overcome this problem for the mixed-norm LMS-LMF, we are adopting here the same technique of normalization (normalizing with the power of the input) that was successfully used with the LMS and LMF separately. Consequently a new normalized variable-parameter mixed-norm (VPNMN) adaptive algorithm is proposed in this study. This algorithm is derived by exploiting a time-varying mixing parameter in the traditional mixed-norm LMS-LMF weight update equation. The time-varying mixing parameter is adjusted according to a well-known technique used in the adaptation of the step-size parameter of the LMS algorithm. In order to study the theoretical aspects of the proposed VPNMN adaptive algorithm, our study also addresses its convergence analysis, and assesses its performance using the concept of energy conservation. Extensive simulation results corroborate our theoretical findings and show that a substantial improvement, in both convergence time and steady-state error, can be obtained with the proposed algorithm. Finally, the VPNMN algorithm proved its usefulness in a noise cancellation application where it showed its superiority over the normalized least-mean square algorithm.

1 Introduction

Due to its simplicity, the least mean-square (LMS) [1, 2] algorithm is the most widely used algorithm for adaptive filters in many applications. The least mean-fourth (LMF) [3] algorithm was also proposed later as a special case of the more general family of steepest descent algorithms [4] with 2k error norms, k being a positive integer.

But for both of these algorithms, the convergence behavior depends on the condition number, i.e., on the ratio of the maximum to the minimum eigenvalues of the input signal autocorrelation matrix, $\mathbf{R}=E\left[{\mathbf{x}}_n{\mathbf{x}}_n^T\right]$ where x _n is the input signal. This is clearly seen from their respective time constants [1, 3]

$${\tau }_{i\mathsf{\text{LMS}}}=\frac{1}{\mu {\lambda }_i},i=0,1,\dots ,N-1,$$

(1)

and

$${\tau }_{i\mathsf{\text{LMF}}}=\frac{1}{6\mu {\sigma }_{\eta }^2{\lambda }_i},i=0,1,\dots ,N-1,$$

(2)

where ${\sigma }_{\eta }^2$ is the noise power, λ _i is the ith eigenvalue of the autocorrelation matrix of the input signal, µ is the step size used in the adaptation scheme and N is the number of coefficients in the adaptive filter. As seen from (1) and (2), the ratio of $\left(\frac{{\tau }_{\text{max}}}{{\tau }_{\text{min}}}\right)$ is constant for both algorithms and is given by the eigenvalue spread (i.e., condition number), $\frac{{\lambda }_{\text{max}}}{{\lambda }_{\text{min}}}$ i.e.,

$$\frac{{\tau }_{\text{max}}}{{\tau }_{\text{min}}}=\frac{{\lambda }_{\text{max}}}{{\lambda }_{\text{min}}}.$$

(3)

To remove the dependency of the convergence of the LMS algorithm on the condition number, the normalized least-mean square (NLMS) [5] was introduced. As reported in [5], a great improvement in convergence is obtained through the use of the NLMS algorithm over that of the LMS algorithm at the expense of a larger steady-state error. Similar results were obtained for the case of the normalized LMF (NLMF) algorithm [6–8].

A mixed-norm algorithm [9–11], combining both the LMS and the LMF algorithms, will suffer as well from the problem of the eigenvalue spread dependency. Since both of these algorithms suffer individually from this problem, to circumvent this problem for the mixed-norm LMS-LMF, we are adopting here the same technique of normalization that was successfully used with the LMS and LMF separately.

It is well known that fast convergence and lower steady-state error are two conflicting parameters in general adaptive filtering. When compared to the LMS algorithm, the NLMS algorithm results in a faster convergence but only at the expense of a higher steady-state error [12, 13]. A promising solution to this conflict is a time-varying normalized mixed-norm LMS-LMF algorithm. In this mixed-norm algorithm and during the transient state, the NLMS algorithm is used to speed up the algorithm's convergence. However when steady-state is reached, the algorithm automatically switches from the NLMS to the NLMF [7], thanks to a built-in "gear shifting" property, to secure a lower steady-state error.

In this work, the performance of a variable-parameter normalized mixed-norm (VPNMN) LMS-LMF algorithm is evaluated. It will be shown that a better performance in both convergence and steady-state error will be achieved by the VPNMN algorithm than either the NLMS or the NLMF algorithm.

The rest of the article is organized as follows. Section 2 deals with a more explicit development of the proposed algorithm, and Section 3 treats its convergence analysis. The steady-state analysis of the proposed algorithm is detailed in Section 4, while its tracking analysis is given in Section 5. Performance evaluation of the resulting algorithm is carried out in Section 6. Finally, the conclusion section summarizes this work.

2 Algorithm development

The mixed-norm LMS-LMF algorithm is based on the minimization of the following cost function [9, 10]:

$$J_n=\alpha E\left[e_n^2\right]+\left(1-\alpha \right)E\left[e_n^4\right],$$

(4)

where α is a positive mixing parameter in the interval [0, 1] and the error e _n is defined as

$$e_n=d_n+{\eta }_n-{\mathbf{x}}_n^T{\mathbf{w}}_n,$$

(5)

where d _n is the desired value, w _n is the filter coefficient of the adaptive filter, x _n is the input signal and η _n is the additive noise.

A major drawback of this algorithm is, however, the choice of the mixing parameter that is hard to fix a priori for an unknown system. In [14], a self-adapting LMS-LMF algorithm with a time-varying weighting factor was proposed. This time-variation of the weighting factor was achieved by allowing for a variable mixing factor that is updated every iteration using the modified variable step-size (MVSS) algorithm proposed in [15]. The variable weight mixed-norm LMS-LMF algorithm was defined to minimize the following performance measure [14]:

$$J_n={\alpha }_{n}E\left[e_n^2\right]+\left(1-{\alpha }_n\right)E\left[e_n^4\right],$$

(6)

where α _n , chosen in [0, 1] such that the unimodal character of the above cost is preserved, is a time-varying parameter updated according to [15]

$${\alpha }_{n+1}=\delta {\alpha }_n+\gamma p_n^2,$$

(7)

and

$$p_n=\beta p_{n-1}+\left(1-\beta \right)e_n{e}_{n-1}.$$

(8)

The parameters δ and β, both confined to the interval [0,1], are exponential weighting parameters that govern the averaging time constant, i.e., the quality of estimation of the algorithm, and γ > 0. Note that the algorithm defined by (4) is restored when δ = 1 and γ = 0, which forces α _n to have a fixed value.

Based on this motivation, the weight mixed-norm LMS-LMF algorithm for recursively adjusting the coefficients of the system is expressed in the following form:

$${\mathbf{w}}_{n+1}={\mathbf{w}}_n+\mu \left[{\alpha }_n{e}_n+2\left(1-{\alpha }_n\right)e_n^3\right]{\mathbf{x}}_n,$$

(9)

where μ is the step size.

As mentioned earlier and because of its reliance on the LMS and the LMF, the algorithm defined by (9) will be affected by the eigenvalue spread of the autocorrelation matrix of the input signal. To overcome this dependency, a VPNMN adaptive algorithm is introduced and its weight update recursion is given by the following expression:

$${\mathbf{w}}_{n+1}={\mathbf{w}}_n+\mu \left[{\alpha }_n{e}_n+2\left(1-{\alpha }_n\right)e_n^3\right]\frac{{\mathbf{x}}_n}{\parallel {\mathbf{x}}_n{\parallel }^2},$$

(10)

where ║x _n ║² is the Euclidean norm of the input signal x _n . In the case of zero input, the ε-VPNMN algorithm defined as follows:

$${\mathbf{w}}_{n+1}={\mathbf{w}}_n+\mu \left[{\alpha }_n{e}_n+2\left(1-{\alpha }_n\right)e_n^3\right]\frac{{\mathbf{x}}_n}{\epsilon +\parallel {\mathbf{x}}_n{\parallel }^2},$$

(11)

must be used for regularization purposes.

3 Convergence analysis of the VPNMN algorithm

In this section, the convergence analysis of the proposed VPNMN algorithm is carried out. Both the mean and the mean-square behaviors of the weight error vector are presented in the ensuing analysis.

3.1 Mean behavior

In the ensuing analysis, the following assumptions are used in the derivations of the convergence in the mean for the normalized mixed-norm LMS-LMF algorithm. These are quite similar to what is usually assumed in literature [2–4, 16] and which can also be justified in several practical instances

A.1 The noise sequence {η _n } is statistically independent of the input signal sequence {x _n } and both sequences have zero mean.

A.2 The weight error vector (v _n ), to be defined later, is independent of the input x _n .

A.3 The mixing parameter is independent of both the input signal and the error.

Examining the mean behavior of (10) under the above assumptions, sufficient conditions for convergence of the proposed algorithm in-the-mean can be derived and are stated as follows.

Proposition 1 For the algorithm defined by (10) to converge in-the-mean, a sufficient condition is that μ be chosen in the following range:

$$0<\mu <\frac{2}{{\bar{\alpha }}_n+3\left(1-{\bar{\alpha }}_n\right)\left({\sigma }_{\eta }^2+\mathcal{C}\right)},$$

(12)

where${\sigma }_{\eta }^2$is the noise power, ${\bar{\alpha }}_n=E\left[{\alpha }_n\right]$is the mean of the mixing parameter, and$\mathcal{C}$is is the Cramer-Rao bound associated with the problem of estimating the random quantity${\mathbf{x}}_n^T{\mathbf{w}}_{\mathsf{\text{opt}}}$by using${\mathbf{x}}_n^T{\mathbf{w}}_n$.

Proof: The mean convergence of the proposed algorithm is now studied by taking the expectation of the weight error vector, v _n = w _n - w_opt. In this regard, the error e _n can be set up in the following way:

$$e_n={\eta }_n-{\mathbf{x}}_n^T{\mathbf{v}}_n,$$

(13)

and hence (10) becomes

$${\mathbf{v}}_{n+1}={\mathbf{v}}_n+\mu \left[{\alpha }_n{e}_n+2\left(1-{\alpha }_n\right)e_n^3\right]\frac{{\mathbf{x}}_n}{\parallel {\mathbf{x}}_n{\parallel }^2}.$$

(14)

Consequently, taking the expectation on both sides of (14), under A.1-A.3, the mean weight-error vector of the proposed algorithm evolves as

$$E\left[{\mathbf{v}}_{n+1}\right]=E\left[{\mathbf{v}}_n\right]+\mu \left\{{\bar{\alpha }}_{n}E\left[e_n\frac{{\mathbf{x}}_n}{\parallel {\mathbf{x}}_n{\parallel }^2}\right]+\left(1-{\bar{\alpha }}_n\right)E\left[e_n^3\frac{{\mathbf{x}}_n}{\parallel {\mathbf{x}}_n{\parallel }^2}\right]\right\}.$$

(15)

Now, considering the second expectation in the above equation, This will be especially true when the filter is long enough. Consequently, the independence assumption can be invoked to obtain the following:

$$E\left[e_n\frac{{\mathbf{x}}_n}{\parallel {\mathbf{x}}_n{\parallel }^2}\right]\approx \frac{E\left[e_n{\mathbf{x}}_n\right]}{\mathsf{\text{tr}}\left(\mathbf{R}\right)}.$$

(16)

To solve the expectation E[e _n x _n ] we use the technique of [17], and thus it results in

$$E\left[e_n{\mathbf{x}}_n\right]=-\mathsf{\text{tr}}\left(\mathbf{R}\right)E\left[{\mathbf{v}}_n\right].$$

(17)

Now, considering the second expectation in the above equation, This will be especially true when the filter is long enough. Consequently, the independence assumption can be invoked to obtain the following:

$$E\left[e_n^3\frac{{\mathbf{x}}_n}{\parallel {\mathbf{x}}_n{\parallel }^2}\right]\approx \frac{E\left[e_n^3{\mathbf{x}}_n\right]}{\mathsf{\text{tr}}\left(\mathbf{R}\right)}.$$

(18)

To solve the expectation $E\left[e_n^3{\mathbf{x}}_n\right]$ we use the technique of [17, 18], which does not employ any linearization of $e_n^3$ As a result, $E\left[e_n^3{\mathbf{x}}_n\right]$ is found to be

$$E\left[e_n^3{\mathbf{x}}_n\right]=-3\left({\sigma }_{\eta }^2+{\zeta }_n\right)\mathbf{R}E\left[{\mathbf{v}}_n\right].$$

(19)

Ultimately, (15) can be set up in the following form:

$$E\left[{\mathbf{v}}_{n+1}\right]\approx \left\{\mathbf{I}-\mu \left[{\bar{\alpha }}_n+3\left(1-{\bar{\alpha }}_n\right)\frac{\left({\sigma }_{\eta }^2+{\zeta }_n\right)}{\mathsf{\text{tr}}\left(\mathbf{R}\right)}\mathbf{R}\right]\right\}E\left[{\mathbf{v}}_n\right].$$

(20)

If $\mathcal{C}\le {\zeta }_n$ is the Cramer-Rao bound associated with the problem of estimating the random quantity ${\mathbf{x}}_n^T{\mathbf{w}}_{\mathsf{\text{opt}}}$ by using ${\mathbf{x}}_n^T{\mathbf{w}}_n$, then after taking into account the fact that the eigenvalues of R are all real and positive, λ_max being the largest eigenvalue of R and in general λ_max< tr(R) [19], it follows that a sufficient condition for convergence of the proposed algorithm is that the step-size parameter μ satisfies (12). ▪

Two extreme scenarios can be considered here for the value of the mixing parameter α _n

• (1) Scenario 1: When α _n = 0, the VPNMN algorithm reduces to the NLMF algorithm [6], and it can be shown that (12) becomes

  $$0<\mu <\frac{2}{3\left({\sigma }_{\eta }^2+\mathcal{C}\right)}.$$

  (21)

• (2) Scenario 2: When α _n = 1, both the NLMS algorithm and its step size range, that is 0 < μ < 2, are recovered.

Remarks:

• (1) It can be seen from (10) that the VPNMN algorithm can be viewed as a variable step-size LMS-LMF algorithm with time varying step size.

• (2) The error is usually large during the initial adaptation and gradually decreases toward a minimum. Therefore, the signal power, ║x _n║², will act as a threshold to avoid taking large step sizes when the error converges to a minimum in the recursive updating equation.

• (3) The bound for the step-size (μ) of the proposed algorithm that guarantees convergence of the mean weight-vector, given by (12), shows that the mean-weight-vector stability depends on the Cramer-Rao bound. Therefore, the convergence of the mean-weight-vector of the proposed algorithm depends on its mean-square stability. A similar fact was observed in [18] for the LMF algorithm.

3.2 Mean square behavior

In this section the performance of the VPNMN algorithm in the mean-square sense is analyzed. Here, we have used a unified approach to the transient analysis of adaptive filters with error nonlinearities. This approach does not restrict the regression data to be Gaussian and avoids the need for explicit recursions for the covariance matrix of the weight-error vector. This approach assumes that the adaptive filter is long enough to justify the following assumptions which are realistic for longer adaptive filters:

A.4 The residual or a priori error e_an, to be defined later, can be assumed to be Gaussian.

A.5 The norm of the input regressor (║ x _n ║²) can be assumed to be uncorrelated with f²(e _n ) (f(e _n ) is defined in (23)).

The framework is based on the concept of energy conservation relation which was first noted in [20] and in general the adaptation scheme defined in (14) can be written in the following form:

$${\mathbf{v}}_{n+1}={\mathbf{v}}_n+\mu {\mathbf{x}}_{n}f\left(e_n\right),$$

(22)

where f(e _n ) denotes a general scalar function of the output estimation error e _n and in our case it is given by

$$f\left(e_n\right)=\frac{{\alpha }_n{e}_n+2\left(1-{\alpha }_n\right)e_n^3}{\parallel {\mathbf{x}}_n{\parallel }^2}.$$

(23)

We are interested in studying the time-evolution and the steady-state values of $E\left[\left|e_{\mathsf{\text{a}}n}^2\right|\right]$ and E[║v _n ║²] which represent the mean-square-error and the mean-square-deviation performances of the filter, respectively, whereas their time-evolution relate to the learning or the transient behavior of the filter.

Then, for some symmetric positive definite weighting matrix A to be specified later, the weighted a priori and a posteriori estimation errors are, respectively, defined as [21]

$$e_{an}^{\mathbf{A}}={\mathbf{x}}_n^T\mathbf{A}{\mathbf{v}}_n,\mathsf{\text{and}}{e}_{pn}^{\mathbf{A}}={\mathbf{x}}_n^T\mathbf{A}{\mathbf{v}}_{n+1}.$$

(24)

For the special case when A = I, the weighted a priori and a posteriori estimation errors defined above are reduced to standard a priori and a posteriori estimation errors, respectively, that is,

$$e_{\mathsf{\text{a}}n}=e_{\mathsf{\text{a}}n}^{\mathbf{I}}={\mathbf{x}}_n^T{\mathbf{v}}_n,\mathsf{\text{and}}{e}_{\mathsf{\text{p}}n}=e_{\mathsf{\text{p}}n}^{\mathbf{I}}={\mathbf{x}}_n^T{\mathbf{v}}_{n+1}.$$

(25)

It can be shown that the estimation error, e _n , and the a priori error, e_an, are related via e _n = e_an+ η _n . Also, using (10) and (24), it can be shown that

$$e_{\mathsf{\text{p}}n}^{\mathbf{A}}=e_{\mathsf{\text{a}}n}^{\mathbf{A}}-\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2\mu f\left(e_n\right),$$

(26)

where the notation $\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2$ denotes the weighted squared Euclidean norm $\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2={\mathbf{x}}_n^T\mathbf{A}{\mathbf{x}}_n$.

The performance measure in the analysis is the excess mean-square-error (EMSE), denoted by ζ _n , and is defined as follows:

$${\zeta }_n=E\left[{\left|e_n\right|}^2\right]-{\sigma }_{\eta }^2.$$

(27)

Since $e_{\mathsf{\text{a}}n}={\mathbf{x}}_n^T{\mathbf{v}}_n$, the EMSE can also be written as follows:

$$\begin{array}{c}{\zeta }_n=E[{\left|e_{\text{a}n}\right|}^2]\\ =E[{\Vert {\mathbf{v}}_n\Vert }_{\mathbf{R}}^2|.\end{array}$$

(28)

Next, the fundamental weighted-energy conservation relation given in [21] is presented to develop the framework for the transient analysis of the proposed algorithm. Thus, by substituting (26) in (22), the following relation can be obtained:

$${\mathbf{v}}_{n+1}={\mathbf{v}}_n-\frac{{\mathbf{x}}_n}{\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2}\left[e_{\mathsf{\text{a}}n}^{\mathbf{A}}-e_{\mathsf{\text{p}}n}^{\mathbf{A}}\right].$$

(29)

Ultimately, the fundamental weighted-energy conservation relation can be shown to be

$$\parallel {\mathbf{v}}_{n+1}{\parallel }_{\mathbf{A}}^2+\frac{1}{\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2}|e_{\mathsf{\text{a}}n}^{\mathbf{A}}{|}^2=\parallel {\mathbf{v}}_n{\parallel }_{\mathbf{A}}^2+\frac{1}{\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2}|e_{\mathsf{\text{p}}n}^{\mathbf{A}}{|}^2.$$

(30)

This relation shows how the weighted energies of the error quantities evolve in time. It has been shown that different choices of A allow us to evaluate different performance measures of an adaptive filter.

3.2.1 Time evolution of the weighted variance $E\left[\parallel {\mathbf{v}}_n{\parallel }_{\mathbf{A}}^2\right]$

In this section, the time evolution of the weighted variance $E\left[\parallel {\mathbf{v}}_n{\parallel }_{\mathbf{A}}^2\right]$ is derived for the proposed algorithm using the fundamental weighted-energy conservation relation (30). Substituting the expression for a posteriori error from (26) in (30) and taking expectation on both sides to obtain the following relation:

$$E[{\Vert {\mathbf{v}}_{n+1}\Vert }_{\mathbf{A}}^2]=E[{\Vert {\mathbf{v}}_n\Vert }_{\mathbf{A}}^2]-2\mu E[e_{\text{a}n}^{\mathbf{A}}f(e_n)]+{\mu }^{2}E[{\Vert {\mathbf{x}}_n\Vert }_{\mathbf{A}}^2{f}^2(e_n)].$$

(31)

Now, evaluating the two expectations in second and third terms on the right hand side of the above equation, that is, $E\left[e_{\mathsf{\text{a}}n}^{\mathbf{A}}f\left(e_n\right)\right]$ and $E\left[\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2{f}^2\left(e_n\right)\right]$ The details for these two quantities are given next. First, we will use the following assumption which was adopted in [21], that is,

A.6 For any constant matrix A and for all n, e_anand $e_{\mathsf{\text{a}}n}^{\mathbf{A}}$ are jointly Gaussian.

This assumption is reasonable for longer filters using the concept of central limit arguments [21]. Moreover, a similar assumption was used in [22]. Hence, we can simplify the expectation $E\left[e_{\mathsf{\text{a}}n}^{\mathbf{A}}{e}_n\right]$ using Price's Theorem [23, 24] and assumptions A.4 and A.6 as follows:

$$\begin{array}{ll}\hfill E\left[e_{\mathsf{\text{a}}n}^{\mathbf{A}}f\left(e_n\right)\right]& =E\left[e_{\mathsf{\text{a}}n}^{\mathbf{A}}f\left(e_n\right)\right]\\ =E\left[e_{\mathsf{\text{a}}n}^{\mathbf{A}}{e}_{\mathsf{\text{a}}n}\right]\frac{E\left[e_{\mathsf{\text{a}}n}f\left(e_n\right)\right]}{E\left[e_{\mathsf{\text{a}}n}^2\right]}.\end{array}$$

(32)

Since $e_{\mathsf{\text{a}}n}^{\mathbf{A}}={\mathbf{x}}_n^T\mathbf{A}{\mathbf{v}}_n$ and $e_{an}={\mathbf{x}}_n^T\mathbf{I}{\mathbf{v}}_n$ we can simplify the expectation $E\left[e_{\mathsf{\text{a}}n}^{\mathbf{A}}{e}_{\mathsf{\text{a}}n}\right]$ as follows:

$$\begin{array}{ll}\hfill E\left[e_{an}^{\mathbf{A}}{e}_{an}\right]& =E\left[{\mathbf{x}}_n^T\mathbf{A}{\mathbf{v}}_n{\mathbf{x}}_n^T\mathbf{I}{\mathbf{v}}_n\right]\\ =E\left[\parallel {\mathbf{v}}_n{\parallel }_{{\mathbf{A}x}_n^T{\mathbf{x}}_n\mathbf{I}}^2\right]\\ =E\left[\parallel {\mathbf{v}}_n{\parallel }_{\mathbf{A}\mathbf{R}}^2\right].\end{array}$$

(33)

Ultimately, (32) can be written as

$$E\left[e_{\mathsf{\text{a}}n}^{\mathbf{A}}f\left(e_n\right)\right]=E\left[\parallel {\mathbf{v}}_n{\parallel }_{\mathbf{A}\mathbf{R}}^2\right]\frac{E\left[e_{\mathsf{\text{a}}n}f\left(e_n\right)\right]}{E\left[e_{\mathsf{\text{a}}n}^A\right]}.$$

(34)

The term $\frac{E\left[e_{\mathsf{\text{a}}n}f\left(e_n\right)\right]}{E\left[e_{\mathsf{\text{a}}n}^2\right]}$ for the case of proposed algorithm, can be shown to be

$$\begin{array}{ll}\hfill \frac{E\left[e_{\mathsf{\text{a}}n}f\left(e_n\right)\right]}{E\left[e_{\mathsf{\text{a}}n}^2\right]}& =\frac{1}{N}\left[{\bar{\alpha }}_n+6\left(1-{\bar{\alpha }}_n\right)\left({\zeta }_n+{\sigma }_{\eta }^2\right)\right],\\ \triangleq {–Z}_n.\end{array}$$

(35)

Second, to solve the expectation $E\left[\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2{f}^2\left(e_n\right)\right]$, we will resort to the following assumption [21]:

A.7 The adaptive filter is long enough such that $\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2$ and f²(e _n ) are uncorrelated.

This assumption is found to be more realistic as the filter gets longer [21] and unweighted version of this assumption was used in [22, 25]. The assumption enable us to split the expectation $E\left[\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2{f}^2\left(e_n\right)\right]$ as follows:

$$E\left[\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2{f}^2\left(e_n\right)\right]=E\left[\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{A}}^2\right]E\left[f^2\left(e_n\right)\right],$$

(36)

where E[f²(e _n )] can be shown to be (with $\overline{{\alpha }_n^2}=E\left[{\alpha }_n^2\right]$)

$$\begin{array}{ll}\hfill E\left[f^2\left(e_n\right)\right]& =\frac{1}{N^2}\left[\overline{{\alpha }_n^2}\left({\zeta }_n+{\sigma }_{\eta }^2\right)+60\left(1-2{\bar{\alpha }}_n+\overline{{\alpha }_n^2}\right){\left({\zeta }_n+{\sigma }_{\eta }^2\right)}^3+4\left({\bar{\alpha }}_n-\overline{{\alpha }_n^2}\right)\left(3{\zeta }_n^2+6{\zeta }_n{\sigma }_{\eta }^2+3{\sigma }_{\eta }^2\right)\right],\\ \triangleq {ℱ}_n.\end{array}$$

(37)

Ultimately, we can rewrite (31) as follows:

$$E[{\Vert {\mathbf{v}}_{n+1}\Vert }_{\mathbf{A}}^2]=E[{\Vert {\mathbf{v}}_n\Vert }_{\mathbf{A}}^2]-2\mu E[{\Vert {\mathbf{v}}_n\Vert }_{\mathbf{A}\mathbf{>R}}^2]{\sout{Z}}_n+{\mu }^{2}E[{\Vert {\mathbf{x}}_n\Vert }_{\mathbf{A}}^2]{ℱ}_n.$$

(38)

The above equation shows the time evaluation or the transient behavior of the weighted variance $E\left[\parallel {\mathbf{v}}_n{\parallel }_{\mathbf{A}}^2\right]$ for any constant weight matrix A. Different performance measures can be obtained by the proper choice of the weight matrix A.

3.2.2 The EMSE and the MSD learning curves

The learning curves for the EMSE and MSD can be obtained using the fact that $E\left[e_{\mathsf{\text{a}}n}^2\right]=E\left[\parallel {\mathbf{v}}_n{\parallel }_{\mathbf{R}}^2\right]$ while $\mathsf{\text{MSD}}=E\left[\parallel {\mathbf{v}}_n{\parallel }_{\mathbf{I}}^2\right]$. If we choose A = IR...R^N-1, a set of relations can be obtained from (38) which is given by

$$\left\{\begin{array}{c}E\left[{∥{\mathbf{v}}_{n+1}∥}_{\mathbf{I}}^2\right]=E\left[{∥{\mathbf{v}}_n∥}_{\mathbf{I}}^2\right]-2\mu {–Z}_{n}E\left[{∥{\mathbf{v}}_n∥}_{\mathbf{R}}^2\right]+{\mu }^{2}E\left[{∥{\mathbf{x}}_n∥}_{\mathbf{I}}^2\right]{ℱ}_n,\\ E\left[{∥{\mathbf{v}}_{n+1}∥}_{\mathbf{R}}^2\right]=E\left[{∥{\mathbf{v}}_n∥}_{\mathbf{R}}^2\right]-2\mu {–Z}_{n}E\left[{∥{\mathbf{v}}_n∥}_{{\mathbf{R}}^2}^2\right]+{\mu }^{2}E\left[{∥{\mathbf{x}}_n∥}_{\mathbf{R}}^2\right]{ℱ}_n,\\ ⋮\\ E\left[{∥{\mathbf{v}}_{n+1}∥}_{{\mathbf{R}}^{N-1}}^2\right]=E\left[{∥{\mathbf{v}}_n∥}_{{\mathbf{R}}^{N-1}}^2\right]-2\mu {–Z}_{n}E\left[{∥{\mathbf{v}}_n∥}_{{\mathbf{R}}^N}^2\right]+{\mu }^{2}E\left[{∥{\mathbf{x}}_n∥}_{{\mathbf{R}}^{N-1}}^2\right]{ℱ}_n.\end{array}\right.$$

(39)

Now, using Cayley-Hamilton theorem, we can write

$${\mathbf{R}}^N=-p_0\mathbf{I}-p_1\mathbf{R}-\cdots -p_{N-1}{\mathbf{R}}^{N-1},$$

(40)

where

$$p\left(x\right)\triangleq \text{det}\left(x\mathbf{I}-\mathbf{R}\right)=p_0+p_{1}x+\cdots +p_{N-1}{x}^{N-1}+x^N,$$

(41)

is the characteristic polynomial of R. Consequently, the following relation is obtained:

$$\begin{array}{l}E[{\Vert {\mathbf{v}}_{n+1}\Vert }_{{\mathbf{R}}^{N-1}}^2]=E[{\Vert {\mathbf{v}}_n\Vert }_{{\mathbf{R}}^{N-1}}^2]-2\mu (p_{0}E[{\Vert {\mathbf{v}}_n\Vert }_I^2]+p_{1}E[{\Vert {\mathbf{v}}_n\Vert }_{\mathbf{R}}^2]+\cdots \\ +p_{N-1}E[{\Vert {\mathbf{v}}_{n+1}\Vert }_{{\mathbf{R}}^{N-1}}^2]){\sout{Z}}_n+{\mu }^{2}E[{\Vert {\mathbf{x}}_n\Vert }_{{\mathbf{R}}^{N-1}}^2]{ℱ}_n.\end{array}$$

(42)

Ultimately, using (39) and (42), the transient behavior of the proposed algorithm can be shown to be governed by the following recursion:

$${\mathcal{W}}_{n+1}={\mathcal{A}}_n{\mathcal{W}}_n+{\mu }^2\mathcal{Y},$$

(43)

where

$${\mathcal{W}}_n={\left[E\left[\parallel {\mathbf{v}}_n{\parallel }^2\right]E\left[\parallel {\mathbf{v}}_n{\parallel }_{\mathbf{R}}^2\right]\dots E\left[\parallel {\mathbf{v}}_n{\parallel }_{{\mathbf{R}}^{N-1}}^2\right]\right]}^T,$$

(44)

$$\mathcal{Y}={\left[E\left[\parallel {\mathbf{x}}_n{\parallel }^2\right]E\left[\parallel {\mathbf{x}}_n{\parallel }_{\mathbf{R}}^2\right]\dots E\left[\parallel {\mathbf{x}}_n{\parallel }_{{\mathbf{R}}^{N-1}}^2\right]\right]}^T{ℱ}_n,$$

(45)

and

$${\mathcal{A}}_n=\left[\begin{array}{cccccc}\hfill 1\hfill & \hfill -2\mu {–Z}_n\hfill & \hfill 0\hfill & \hfill \dots \hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill & \hfill -2\mu {–Z}_n\hfill & \hfill \dots \hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill \dots \hfill & \hfill 1\hfill & \hfill -2\mu {–Z}_n\hfill \\ \hfill 2\mu p_0{–Z}_n\hfill & \hfill 2\mu p_1{–Z}_n\hfill & \hfill 2\mu {\rho }_2{–Z}_n\hfill & \hfill \dots \hfill & \hfill 2\mu p_{N-2}{–Z}_n\hfill & \hfill 1+2\mu p_{N-1}{–Z}_n\hfill \end{array}\right]$$

(46)

It can be noticed that the learning curves for the MSD and the EMSE can be obtained from the first and second elements of vector ${\mathcal{W}}_n$, respectively.

3.2.3 Mean-square stability

Finally, in this section, the mean-square stability of the proposed algorithm is investigated. Consequently, we provide a nontrivial upper bound on µ for which E[║ v _n ║² remains uniformly bounded for all n.

Starting from (31) with A = I and using the Gaussian behavior of e_an, it can be shown that the proposed algorithm will be mean-square stable provided that

$$\begin{array}{ll}\hfill E\left[\parallel {\mathbf{v}}_{n+1}{\parallel }^2\right]& \le E\left[\parallel {\mathbf{v}}_{n+1}{\parallel }^2\right]\\ \hfill \mu E\left[\parallel {\mathbf{x}}_n{\parallel }^2{f}^2\left(e_n\right)\right]& \le 2E\left[e_{an}f\left(e_n\right)\right].\end{array}$$

(47)

The above inequality, upon substituting the values of the two expectations (E[e_anf(e _n )] and E[║ x _n ║²f²(e _n )]), will lead us to get the following bound:

$$\mu \le \frac{\left[{\bar{\alpha }}_n+6\left(1-{\bar{\alpha }}_n\right)\left(\mathcal{C}+{\sigma }_{\eta }^2\right)\right]N\mathcal{C}}{\left[\overline{{\alpha }_n^2}\left(\mathcal{C}+{\sigma }_{\eta }^2\right)+60\left(1-2{\bar{\alpha }}_n+\overline{{\alpha }_n^2}\right){\left(\mathcal{C}+{\sigma }_{\eta }^2\right)}^3+4\left({\bar{\alpha }}_n-\overline{{\alpha }_n^2}\right)\left(3{\mathcal{C}}^2+6\mathcal{C}{\sigma }_{\eta }^2+3{\sigma }_{\eta }^2\right)\right]\mathsf{\text{tr}}\left(\mathbf{R}\right)}.$$

(48)

4 Steady-state analysis of the VPNMN algorithm

The purpose of the steady state analysis of an adaptive filter is to study the behavior of steady state EMSE. Now, analyzing (31) for the limiting case when n → ∞ . Assuming that the weight error vector reaches a steady-state mean square error value, i.e.,

$$\underset{n\to \infty }{\text{lim}}E\left[\parallel {\mathbf{v}}_{n\mathsf{\text{ + 1}}}{\parallel }_{\mathbf{A}}^2\right]=\underset{n\to \infty }{\text{lim}}E\left[\parallel {\mathbf{v}}_n{\parallel }_{\mathbf{A}}^2\right].$$

(49)

Consequently, for a unity weight matrix (A = I), (31) reduces to the following:

$$\underset{n\to \infty }{\text{lim}}E\left[e_{\mathsf{\text{a}}n}^2\right]=\frac{\mu }{2}\underset{n\to \infty }{\text{lim}}E\left[\parallel {\mathbf{x}}_n{\parallel }^2\right]\frac{{\text{lim}}_{n\to \infty }{ℱ}_n}{{\text{lim}}_{n\to \infty }{–z}_n}.$$

(50)

Now, using the definition of the EMSE given by (28), its steady-state value denoted by ζ_∞ is found to be

$${\zeta }_{\infty }=\frac{\mu }{2}\frac{{\text{lim}}_{n\to \infty }{ℱ}_n}{{\text{lim}}_{n\to \infty }{–Z}_n}\mathsf{\text{tr}}\left(\mathbf{R}\right).$$

(51)

The terms lim_n→∞Z _n and lim_n→∞ℱ _n can be obtained from (35) and (37), respectively.

Since, the EMSE is very close to zero at steady state, therefore, the higher powers of ζ_∞ can be ignored. Ultimately, the steady-state EMSE of the proposed algorithm can be shown to be

$${\zeta }_{\infty }=\frac{\mu \left[\overline{{\alpha }_{\infty }^2}{\sigma }_{\eta }^2+60\left(1-2{\bar{\alpha }}_{\infty }+\overline{{\sigma }_{\infty }^2}\right){\sigma }_{\eta }^6+12\left({\bar{\alpha }}_{\infty }-\overline{{\alpha }_{\infty }^2}\right){\sigma }_{\eta }^2\right]\mathsf{\text{tr}}\left(\mathbf{R}\right)}{2N{\bar{\alpha }}_{\infty }+12N\left(1-{\bar{\alpha }}_{\infty }\right){\sigma }_{\eta }^2-\mu \left[\overline{{\alpha }_{\infty }^2}+180\left(1-2{\bar{\alpha }}_{\infty }+\overline{{\alpha }_{\infty }^2}\right){\sigma }_{\eta }^4+24\left({\bar{\alpha }}_{\infty }-\overline{{\sigma }_{\infty }^2}z\right){\sigma }_{\eta }^2\right]\mathsf{\text{tr}}\left(\mathbf{R}\right)}.$$

(52)

5 Tracking analysis of the VPNMN algorithm

Cyclic and random system nonstationarities are a common impairment in communication systems and especially in applications that involve channel estimation, channel equalization, and inter-symbol-interference cancellation. Random nonstationarity is present due to variations in channel characteristics which is true in most of cases, particularly in the case of a mobile communication environment [26]. Cyclic system nonstationarities arise in communication systems due to mismatches between the transmitter and receiver carrier generator.

The ability of adaptive filtering algorithms to track such system variations is not yet fully understood. In this regard, Rupp [27] presented a first-order analysis of the performance of the LMS algorithm in the presence of the carrier frequency offset. In [21, 25, 28, 29] a general framework for the tracking analysis of adaptive algorithms was developed. It can handle both cyclic as well as random system nonstationarities simultaneously. This framework, based on an energy conservation principle [20], holds for all adaptive algorithms whose recursions are of the form

$${\mathbf{w}}_{n+1}={\mathbf{w}}_n+\mu {\mathbf{x}}_n^{*}f\left(e_n\right).$$

(53)

In the ensuing analysis, the tracking analysis of the proposed algorithm is carried out in the presence of both random and acyclic nonstationarities. It should be noted here that in this case, unlike the convergence analysis which is a linear process, the tracking analysis is a nonlinear one due to the presence of the term (e^{j Ωn}) in (54). This therefore justifies our use of complex signals, instead of real ones, in the (tracking) analysis.

A general system model is presented here which includes both types of nonstationarities, that is random and cyclic ones. To start, consider the noisy measurement d _n that arises in a model of the form:

$$d_n={\mathbf{x}}_n^H{\mathbf{w}}_n^o{e}^{j\mathrm{\Omega }n}+{\eta }_n,$$

(54)

where η _n is the measurement noise and ${\mathbf{w}}_n^o$ is the unknown system to be tracked. The multiplicative term e^{j Ωn}accounts for a possible frequency offset between the transmitter and the receiver carriers in a digital communication scenario. Furthermore it is assumed that the unknown system vector ${\mathbf{w}}_n^o$ is randomly changing according to:

$${\mathbf{w}}_n^o={\mathbf{w}}^o+{\mathbf{q}}_n,$$

(55)

where w^ois a fixed vector, and q _n is assumed to be a zero-mean stationary random vector process with a positive definite autocorrelation matrix ${\mathbf{Q}}_n=E\left[{\mathbf{q}}_n{\mathbf{q}}_n^H\right]$ Moreover, it is also assumed that the sequence {q _n } is mutually independent of the sequences {x _n } and {η _n }. Thus, from the generalized system model given by (54) and (55), it can be seen that the effects of both cyclic and random system nonstationarities are included in this system model.

In the tracking analysis of adaptive algorithms, an important measure of performance is their steady-state tracking EMSE and is given by

$${\zeta }^{\mathsf{\text{tracking}}}=\underset{n\to \infty }{\text{lim}}E\left\{|{\mathbf{x}}_n^{\mathsf{\text{H}}}{\tilde{\mathbf{v}}}_n{|}^2\right\},$$

(56)

where ${\tilde{\mathbf{v}}}_n$ is the weight-error vector for tracking scenario and is defined as follows:

$${\tilde{\mathbf{v}}}_n={\mathbf{w}}_n^o{e}^{j\mathrm{\Omega }n}-{\mathbf{w}}_n.$$

(57)

Using (53), (55) and (57) the following recursion is obtained:

$${\tilde{\mathbf{v}}}_{n+1}={\tilde{\mathbf{v}}}_n-\mu {\mathbf{x}}_n^{*}f\left(e_n\right)+{\mathbf{c}}_n{e}^{j\mathrm{\Omega }n},$$

(58)

where c _n is defined as

$${\mathbf{c}}_n={\mathbf{w}}^o\left(e^{j\mathrm{\Omega }}-1\right)+{\mathbf{q}}_{n+1}{e}^{j\mathrm{\Omega }}-{\mathbf{q}}_n.$$

(59)

Now, let us define the following a priori estimation error, $e_{\mathsf{\text{a}}n}={\mathbf{x}}_n^H{\tilde{\mathbf{v}}}_n$ and a posteriori estimation error, $e_{\mathsf{\text{p}}n}={\mathbf{x}}_n^H\left({\tilde{\mathbf{v}}}_{n+1}-{\mathbf{c}}_n{e}^{j\mathrm{\Omega }n}\right)$ Then, it is very easy to show that the estimation error and the a priori error are related via e _n = e_an+ η _n . Also, from (26) when A = I, the a posteriori error is defined in terms of the a priori error as follows:

$$e_{\mathsf{\text{p}}n}=e_{an}-\frac{\mu }{{\widehat{\mu }}_n}f\left(e_n\right),$$

(60)

where ${\widehat{\mu }}_n=1/\parallel {\mathbf{x}}_n{\parallel }^2$ Substituting (60) into (58) results into the following update relation:

$${\tilde{\mathbf{v}}}_{n+1}={\tilde{\mathbf{v}}}_n-{\widehat{\mu }}_n{\mathbf{x}}_n^{*}\left[e_{an}-e_{\mathsf{\text{p}}n}\right]+{\mathbf{c}}_n{e}^{j\mathrm{\Omega }n}.$$

(61)

By evaluating the energies of both sides of the above equation (taking into account that ${\widehat{\mu }}_n\parallel {\mathbf{x}}_n{\parallel }^2=1$) the following relation is obtained:

$$\parallel {\tilde{\mathbf{v}}}_{n+1}-{\mathbf{c}}_n{e}^{j\mathrm{\Omega }n}{\parallel }^2+{\widehat{\mu }}_n|e_{an}{|}^2=\parallel {\tilde{\mathbf{v}}}_n{\parallel }^2+{\widehat{\mu }}_n|e_{\mathsf{\text{p}}n}{|}^2.$$

(62)

It can be seen that if Ω = 0 (i.e., no frequency offset between the transmitter and the receiver), the above equation reduces to the basic fundamental energy conservation relation.

The energy relation (62) will be used to evaluate the excess-mean-square error at steady state. But before starting the analysis, first the following assumptions are stated:

A.8 In steady-state, the weight error vector ${\tilde{\mathbf{v}}}_n$ takes the generic form z _n e^{j Ωn}with the stationary random process z _n independent of the frequency offset Ω.

Using (60), assumption A.8, and taking expectation of both sides of (62) and the fact that at steady state $E\left[{\tilde{\mathbf{v}}}_{n+1}\right]=E\left[{\tilde{\mathbf{v}}}_n\right]$ the following relation can be obtained:

$$\begin{array}{ll}\hfill E\left[{\widehat{\mu }}_n\parallel e_{an}{\parallel }^2\right]& =2\mathsf{\text{tr}}\left\{{\mathbf{Q}}_n\right\}+\parallel {\mathbf{w}}^o{\parallel }^2|1-e^{j\mathrm{\Omega }}{|}^2-2\mathsf{\text{Re}}\left\{E\left[{\mathbf{q}}_n^{*}\left({\mathbf{z}}_n-\mu {\mathbf{x}}_n^{*}f\left(e_n\right)e^{-j\mathrm{\Omega }n}\right)\right]\right\}\\ -2\mathsf{\text{Re}}\left\{{\left(1-e^{j\mathrm{\Omega }}\right)}^{*}{\mathbf{w}}^{o*}E\left[{\mathbf{z}}_n-\mu {\mathbf{x}}_n^{*}f\left(e_n\right)e^{-j\mathrm{\Omega }n}\right]\right\}+E\left[{\widehat{\mu }}_n{\left|e_{an}-\frac{\mu }{{\widehat{\mu }}_n}f\left(e_n\right)\right|}^2\right].\end{array}$$

(63)

The above equation can be used to solve for the steady-state EMSE. To find the value of z = E[z _n ], (58) is used where it is multiplied by the term e^{-j Ωn}and then expectation is taken on both sides to get

$$\left(1-e^{j\mathrm{\Omega }}\right)\mathbf{z}=\mu E\left({\mathbf{x}}_n^{*}f\left(e_n\right)e^{-j\mathrm{\Omega }n}\right)+{\mathbf{w}}^o\left(1-e^{j\mathrm{\Omega }}\right),$$

(64)

which yields the following value of z at steady-state:

$$\mathbf{z}={\left[\mathbf{I}-\frac{\mu {\gamma }_o}{\left(1-e^{j\mathrm{\Omega }}\right)}\mathbf{R}\right]}^{-1}{\mathbf{w}}^o,$$

(65)

where γ _o is defined as

$${\gamma }_o=\left[{\bar{\alpha }}_n+6\left(1-{\bar{\alpha }}_n\right){\sigma }_{\eta }^2\right]\frac{1}{\mathsf{\text{tr{}}\mathbf{R}\mathsf{\text{}}}}.$$

(66)

Ultimately, the steady-state excess-mean-square error of the proposed algorithm, ζ^tracking, is obtained from (63):

$${\zeta }^{\text{tracking}}=\frac{a}{a-\mu b}\left[\text{tr{}{\mathbf{Q}}_n\mathbf{R}\text{}}+\frac{{\beta }_O}{2\mu {\gamma }_O}+\frac{c\mu }{a}\right],$$

(67)

where

$${\beta }_o={\left|1-e^{j\mathrm{\Omega }}\right|}^{2}Re\left\{\text{tr(}{\Vert {\mathbf{w}}^o\Vert }^{\text{2}}\text{(}\mathbf{I}-2\mathbf{X}))\right\},$$

(68)

$$\mathbf{X}=\left(\mathbf{I}-\mu {\gamma }_o\mathbf{R}\right){\left[\mathbf{I}-\mu {\gamma }_o\mathbf{R}-e^{j\mathrm{\Omega }}\mathbf{I}\right]}^{-1},$$

(69)

and

$$\begin{array}{ll}\hfill a& =2\left\{{\bar{\alpha }}_n+6\left(1-{\bar{\alpha }}_n\right){\sigma }_{\eta }^2\right\},\\ \hfill b& =\left\{E\left[{\alpha }_n^2\right]+12E\left[{\alpha }_n\left(1-{\alpha }_n\right)\right]{\sigma }_{\eta }^2+36E\left[{\left(1-{\alpha }_n\right)}^2\right]{\chi }_{\eta }^4\right\},\\ \hfill c& =\left\{E\left[{\alpha }_n^2\right]{\sigma }_{\eta }^2+4E\left[{\alpha }_n\left(1-{\alpha }_n\right)\right]{\chi }_{\eta }^4+4E\left[{\left(1-{\alpha }_n\right)}^2\right]{\chi }_{\eta }^6\right\},\\ \hfill {\chi }_{\eta }^{\mathsf{\text{6}}}& =E\left[{\eta }_n^6\right],\\ \hfill {\chi }_{\eta }^{\mathsf{\text{4}}}& =E\left[{\eta }_n^4\right].\end{array}$$

It can be seen from the above result that the steady-state tracking EMSE of the NLMS algorithm [28] and the NLMF algorithm [29] can be recovered by substituting α _n = 1 and α _n = 0, respectively, in (67).

For a white Gaussian input signal, the autocorrelation of the input signal $\mathbf{R}={\sigma }_x^2\mathbf{I}$, and therefore (67) will look like the following:

$${\zeta }^{\mathsf{\text{tracking}}}=\frac{a}{a-\mu b}\left[{\sigma }_x^2\mathsf{\text{tr{}}{\mathbf{Q}}_n\mathsf{\text{}}}+\frac{N^2{\sigma }_x^2\left(4N-\mu a\right){\mathrm{\Omega }}^2}{{\mu }^2{a}^2}\parallel {\mathbf{w}}^o{\parallel }^2+\frac{c\mu }{a}\right].$$

(70)

6 Simulation results

The performance of the proposed algorithm, the VPNMN LMS-LMF, is assessed in different scenarios. Experiments are carried out where an unknown system is to be identified under noisy conditions. The unknown system is a non-minimum phase channel. The input signal to both the unknown system and the adaptive filter is obtained by passing a zero-mean white Gaussian sequence through a channel that is used to vary the eigenvalue spread of the autocorrelation matrix of the input signal. The example considered for the sequence {x _n } has an eigenvalue spread of 68.9. The additive noise, η _n , is a zero-mean. The signal to noise ratio is set to be equal to 20 dB and the performance measure considered is the normalized weight error norm 10log₁₀║ w _n - w_opt║²/║ w_opt║². Results are obtained by averaging over 500 independent runs. The proposed algorithm is implemented with the parameters 8 = 0.97, β = 0.98, γ = 10^-2α₀= 0.8 and p₀ = 0. In the ensuing, different aspects of the performance are considered during the course of this study.

6.1 Convergence behavior

Figure 1 compares the fastest convergence characteristics of both the proposed algorithm and the NLMS algorithm. It can be seen from this figure that the proposed algorithm converges as fast as the NLMS algorithm but results in a lower weight mismatch. An improvement of 25 dB is obtained through the use of the proposed algorithm. Also, as shown in Figure 2, the proposed algorithm outperforms the NLMS algorithm, for the lowest steady-state error reached by the later, thanks to its built-in gear-shifting mechanism which gives it an extra degree of freedom in this region.

Figure 1

Fastest convergence characteristics of the VPNMN and NLMS algorithms.

Figure 2

Convergence characteristics of the VPNMN and NLMS algorithms for the lowest steady-state error reached by the NLMS algorithm.

The fast convergence obtained by the proposed algorithm can be justified by the fact that when far from the optimum solution, this algorithm exhibits faster convergence than the NLMS algorithm by automatically increasing the step size (gear-shifting property).

Figure 3 summarizes the performance of the proposed VPNMN algorithm in the three different noise environments with an SNR of 20 dB when the input signal is white. As can be depicted from this figure that the best performance is obtained when the noise statistics are uniform while the worst performance is obtained when the noise statistics are laplacian.

Figure 3

Convergence behaviour of the VPNMN algorithm when the input signal is white.

Similarly, Figure 4 depicts the results for the proposed VPNMN algorithm when the input signal is highly correlated and as can seen from this figure that almost equal performance is obtained by the VPNMN algorithm for the different noise statistics.

Figure 4

Convergence behaviour of the VPNMN algorithm when the input signal is highly correlated.

In order to verify the stability bound on step-size given in (48), we investigate it in a Gaussian environment and an SNR of 20 dB. Here, we choose a misadjustment of five which results in the Cramer-Rao bound to be C ≤ 0.05. Thus, choosing a tr(R) = 5, the upper bound given in (48) is found to be 0.95. It is observed from the various performed simulations that the NCLMF algorithm is stable while µ is less than 1.0 and thus, eventually validating the derived stability bound.

Finally, from the viewpoint of computational load the proposed algorithm requires an additional seven multiplications and three additions when compared to the fixed mixed-norm algorithm defined by (4), and only eleven multiplications and six additions when compared to the NLMS algorithm. The small computational over head of the proposed algorithm is therefore well worth the gain in the steady-state error reduction it brings about.

6.2 Results for the MSE learning curve

Figure 5 depicts the time evolution of the MSE obtained for both the theoretical analysis, the second entry of (44), and the simulations. Excellent agreement between theory and simulation results is obtained; hence, a consistency in performance is obtained by the proposed VPNMN algorithm.

Figure 5

Comparison of analytical and experimental MSE.

6.3 Results for tracking

For tracking, the simulations are carried out for a system identification problem, where the unknown system, having an FIR model, is given by [1.0119 - j 0.7589, - 0.3796 + j 0.5059]^T, while the system characteristics are time-varying according to the system model (54) and (55). Results for the proposed algorithm are presented to validate the theoretical findings for different values of Ω and different values of µ. The input signal x _n to both the unknown system and the adaptive filter is a zero-mean white Gaussian sequence. The signal to noise ratio is set to be equal to 30 dB two values are considered for tr{Q _n }: a very small value of tr{Q _n } = 10^-7, and a very large one of tr{Q _n } = 10^-2.

Figure 6 depicts the comparison of the theory to the simulation results for three different values of Ω, i.e., Ω = 0.001, 0.002, and 0.003. As can be seen from this figure, close agreement between theory and simulation results are obtained. Furthermore, it is observed from this figure that degradation in performance is obtained by increasing the frequency offset Ω and unlike the stationary case, the steady-state EMSE is not a monotonically increasing function of the step-size µ, that is the steady-state EMSE is smaller at larger values of the step-size µ.

Figure 6

Analytical (-) and experimental (Δ) steady-state EMSE for Ω = 0.001, Ω = 0.002, and Ω = 0.003, and tr{ Q _k } = 10 ^-7 .

Figure 6 is obtained for the case when tr{Q _n } = 10^-7 which is represents a small value. Increasing this value to 10^-2, the results depicted in Figure 7 for three larger values of Ω, i.e., 0.01, 0.02, and 0.03, still show that the previously stated observations are similar to those obtained for a smaller value of tr{Q _n }.

Figure 7

Analytical (-) and experimental (Δ) steady-state EMSE for Ω = 0.01, Ω = 0.02, and Ω = 0.03, and tr{ Q _k } = 10 ^-2 .

Finally, the consistency in the performance of the steady-state EMSE of the proposed algorithm is observed in both cases (two different values of tr{Q _n }) and different values of Ω.

6.4 Noise cancelation using VPNMN algorithm

In this example, we study the performance of the VPNMN algorithm for the application of noise cancelation. A pure sinusoidal noise generated by the process (u _n = 0.8 sin (ωn + 0.5π)) with ω = 0.1 π is to be removed from a square wave generated by (s _n = 2 × ((mod(n, 1000) < 1000/ 2) - 0.5)) where mod (n, 1000) computes the modulus of n over 1,000. Summing u _n and s _n gives us the reference signal to the adaptive filter. The input to the adaptive filter is a sinusoidal signal generated by $\left(x_n=\sqrt{2}\text{sin}\left(\omega n\right)\right)$ with ω = 0.1 π. The resulting output error signal e _n will, in time, converge to the desired signal which will be noiseless.

Figure 8 depicts the reference response and the processed results by the VPNMN algorithm and NLMS algorithm. It is clear that both algorithms are able to remove the noise component but VPNMN algorithm exhibits better noise cancelation capabilities as compared to the NLMS algorithm.

Figure 8

Comparison between VPNMN and NLMS algorithms in a noise cancelation application.

7 Conclusion

In this study, a normalized VPNMN algorithm is proposed where a combination of the LMS and the LMF algorithms is incorporated using the concept of variable step-size LMS adaptation. It is found that the proposed algorithm has the fast convergence property of the NLMS algorithm while resulting in a lower steady-state error, therefore eliminating the conflict between these two parameters, i.e., fast convergence and low steady-state error. Moreover, the consistency of the performance of the proposed algorithm has been confirmed by many simulation results which are reported here.

The analytical results of the tracking steady-state EMSE are derived for the proposed algorithm in the presence of both random and cyclic nonstationarities. The results, show that unlike in the stationary case, the steady-state EMSE is not a monotonically increasing function of the step-size µ, while the ability of the algorithm to track the variations in the environment degrades by increasing the frequency offset Ω.

Finally, the VPNMN algorithm proved its usefulness in a noise cancelation scenario where it showed its superiority over the NLMS algorithm.