- Open Access
A family of variable step-size affine projection adaptive filter algorithms using statistics of channel impulse response
© Esfand Abadi and AbbasZadeh Arani; licensee Springer. 2011
- Received: 2 December 2010
- Accepted: 8 November 2011
- Published: 8 November 2011
This paper extends the recently introduced variable step-size (VSS) approach to the family of adaptive filter algorithms. This method uses prior knowledge of the channel impulse response statistic. Accordingly, optimal step-size vector is obtained by minimizing the mean-square deviation (MSD). The presented algorithms are the VSS affine projection algorithm (VSS-APA), the VSS selective partial update NLMS (VSS-SPU-NLMS), the VSS-SPU-APA, and the VSS selective regressor APA (VSS-SR-APA). In VSS-SPU adaptive algorithms the filter coefficients are partially updated which reduce the computational complexity. In VSS-SR-APA, the optimal selection of input regressors is performed during the adaptation. The presented algorithms have good convergence speed, low steady state mean square error (MSE), and low computational complexity features. We demonstrate the good performance of the proposed algorithms through several simulations in system identification scenario.
- Adaptive filter
- Normalized Least Mean Square
- Affine projection
- Selective partial update
- Selective regressor
- Variable step-size
Adaptive filtering has been, and still is, an area of active research that plays an active role in an ever increasing number of applications, such as noise cancellation, channel estimation, channel equalization and acoustic echo cancellation [1, 2]. The least mean squares (LMS) and its normalized version (NLMS) are the workhorses of adaptive filtering. In the presence of colored input signals, the LMS and NLMS algorithms have extremely slow convergence rates. To solve this problem, a number of adaptive filtering structures, based on affine subspace projections [3, 4], data reusing adaptive algorithms [5, 6], block adaptive filters  and multi rate techniques [7, 8] have been proposed in the literatures. In all these algorithms, the selected fixed step-size can change the convergence and the steady-state mean square error (MSE). It is well known that the steady-state MSE decreases when the step-size decreases, while the convergence speed increases when the step-size increases. By optimally selecting the step-size during the adaptation, we can obtain both fast convergence rates and low steady-state MSE. These selections are based on various criteria. In , squared instantaneous errors were used. To improve noise immunity under Gaussian noise, the squared autocorrelation of errors at adjacent times was used in , and in , the fourth order cumulant of instantaneous error was adopted.
In , two adaptive step-size gradient adaptive filters were presented. In these algorithms, the step sizes were changed using a gradient descent algorithm designed to minimize the squared estimation error. This algorithm had fast convergence, low steady-state MSE, and good performance in nonstationary environment. The blind adaptive gradient (BAG) algorithm for code-aided suppression of multiple-access interference (MAI) and narrow-band interference (NBI) in direct-sequence/code-division multiple-access (DS/CDMA) systems was presented in . The BAG algorithm was based on the concept of accelerating the convergence of a stochastic gradient algorithm by averaging. The authors shown that the BAG had identical convergence and tracking properties to recursive least squares (RLS) but had a computational cost similar to the LMS algorithm. In , two low complexity variable step size mechanisms were proposed for constrained minimum variance (CMV) receivers that operate with stochastic gradient algorithms and are also incorporated in the channel estimation algorithms. Also, the low low-complexity variable step size mechanism for blind code-constrained constant modulus algorithm (CCM) receivers was proposed in . This approach was very useful for nonstationary wireless environment.
In , a generalized normalized gradient descent (GNGD) algorithm for linear finite-impulse response (FIR) adaptive filters was introduced that represents an extension of the NLMS algorithm by means of an additional gradient adaptive term in the denominator of the learning rate of NLMS. The simulation results show that the GNGD algorithm is robust to significant variations of initial values of its parameters.
Important examples of two new variable step-size (VSS) versions of the NLMS and the affine projection algorithm (APA) can be found in . In , the step-size is obtained by minimizing the mean-square deviation (MSD). This introduced algorithms show good performance in convergence rate and steady-state MSE. This approach was successfully extended to selective partial update (SPU) adaptive filter algorithm in .
To improve the performance of adaptive filter algorithms, the adaptive filter algorithm was proposed based on channel impulse response statistics [19, 20]. In  a new variable-step-size control was proposed for the NLMS algorithm. In this algorithm, the step-size vector with different values for each filter coefficient was used. In this approach, based on prior knowledge of the channel impulse response statistics, the optimal step-size vector is obtained by minimizing the mean-square deviation (MSD) between the optimal and estimated filter coefficients.
Another feature which is important in adaptive filter algorithms is computational complexity. Several adaptive filters with fixed step-size, such as the adaptive filter algorithms with selective partial updates have been proposed to reduce the computational complexity. These algorithms update only a subset of the filter coefficients in each time iteration. The Max-NLMS , the variants of the SPU-NLMS , and SPU-APA [24, 25] are important examples of this family of adaptive filter algorithms. Recently an affine projection adaptive filtering algorithm with selective regressors (SR) was also proposed to reduce the computational complexity of APA [26–28]. In this algorithm, the recent regressors are optimally selected during the adaptation.
In this paper, we extend the approach in  to the APA, SPU-NLMS, SPU-APA, and SR-APA and four novel VSS adaptive filter algorithms are established. These algorithms are computationally efficient. We demonstrate the good performance of the presented algorithms through several simulation results in system identification scenario. The comparison of the proposed algorithms with other algorithms is also presented.
What we propose in this paper can be summarized as follows:
The establishment of the VSS-APA.
The establishment of the VSS-SPU-NLMS.
The establishment of the VSS-SPU-APA.
The establishment of the VSS-SR-APA.
Demonstrating of the proposed algorithms in system identification scenario.
Demonstrating the tracking ability of the proposed algorithms.
We have organized our paper as follows. In section 2, the NLMS and SPU-NLMS algorithms will be briefly reviewed. Then, the family of APA, SR-APA and SPU-APA are presented and the family of variable step-size adaptive filters is established. In the following, the computational complexity of the VSS adaptive filters is discussed. Finally, before concluding the paper, we demonstrate the usefulness of these algorithms by presenting several experimental results.
Throughout the paper, the following notations are adopted:
|.| norm of a scalar
||.||2 squared Euclidean norm of a vector
(.) T transpose of a vector or a matrix
tr(.) trace of a matrix
E[.] expectation operator
2-1 Background on NLMS
2-2 Selective Partial Update NLMS
which is the max-NLMS algorithm . For M = B and L = B, the SPU-NLMS algorithm becomes identical to NLMS algorithm in (3).
3-1. Affine projection algorithm (APA)
3-2 Selective Regressor APA (SR-APA)
- 1.Compute the following values for 0 ≤ i ≤ K - 1:(17)
The indices of G correspond to P largest values of (17).
3-3. Selective Partial Update APA (SPU-APA)
- 1.Compute the following values for 1 ≤ i ≤ B:(22)
The indices of F correspond to S largest values of relation (22).
4-1. Variable Step-Size NLMS algorithm using statistics of channel response
where h = [h0, h1, ..., hM-1] T is the true unknown system with memory length M, X(n) = [x(n), ..., x(n-M+1)] T is the system input vector, and v(n) is the additive noise.
where the step-size matrix is defined as U(n) = diag[μ0(n), ..., μM-1(n)].
where 0 < λ < 1 is the forgetting factor. Also, the initial value for is given by the second-order statistics of the channel impulse response, i.e. .
4-2. Variable Step-Size Selective Partial Update NLMS algorithm using statistics of channel response
and I SL is the SL × SL identity matrix.
Also, E(e2(n)) is obtained according to (30).
4-3. Variable Step-Size APA using statistics of channel response
and I M is the M × M identity matrix.
4-4. Variable Step-Size Selective Regressor AP algorithm using statistics of channel response
4-5. Variable Step-Size Selective Partial Update AP algorithm using statistics of channel response
Also E[||e(n)||2] is obtained according to (59).
The computational complexity of NLMS, SPU-NLMS, VSS-NLMS, and VSS-SPU-NLMS algorithms.
The computational complexity of APA, SPU-APA, SR-APA, VSS-APA, VSS-SPU-APA, and VSS-SR-APA.
(K2 + 2K)M+K3+K2
(K2 + 2K)SL+K3+K2
(P2 + 2P)M+P3+P2
(K-P)M + K+1
(K2 + 2K)M+K3+K2+KM2
(K2 + 2K)SL+K3+K2+KS2L2
3SL + K + 2
(P2 + 2P)M+P3+P2+PM2
M + K
The SPU-APA needs (K2+2K)SL+K3+K2 multiplications. This algorithm needs 1 additional multiplication and B log2S+O(B) comparisons. The SR-APA needs (P2+2P)M+P3+P2 multiplications and K divisions. This algorithm needs (K-P)M+K+1 additional multiplications and K log2P+O(K) comparisons. Comparing the updated equation for APA and VSS-APA shows that the VSS-APA needs KM2+3M+K+1 additional multiplications and M divisions due to variable step-size. In VSS-SPU-APA, the additional multiplication is KS2+L2+3SL+K+1 and additional division is SL. Also, this algorithm needs B log2S+O(B) comparisons. It is obvious that the computational complexity of VSS-SPU-APA is lower than VSS-APA. The number of reductions in multiplication and division for VSS-SPU-APA is respectively (K2+2K+4)(M-SL)+K(M2-S2L2) and M-SL. In VSS-SR-APA, the additional multiplication is 3M+P+1 and additional division is M compared with SR-APA. Also this algorithm needs K log2P+O(K) comparisons. It is obvious that the computational complexity of VSS-SR-APA is lower than VSS-APA.
The white zero-mean Gaussian noise was added to the filter output such that the SNR = 15dB.
In all simulations, the MSD curves are obtained by ensemble averaging over 200 independent trials.
In this paper, we presented the novel VSS adaptive filter algorithms such as VSS-SPU-NLMS, VSS-APA, VSS-SR-APA and VSS-SPU-APA based on prior knowledge of the channel impulse response statistic. These algorithms exhibit fast convergence while reducing the steady-state MSD as compared to the ordinary SPU-NLMS, APA, SR-APA and SPU-APA algorithms. The presented algorithms were also computationally efficient. We demonstrated the good performance of the presented VSS adaptive algorithms in system identification scenario.
- Widrow B, Stearns SD: Adaptive Signal Processing. Englewood Cliffs, NJ: Prentice-Hall; 1985.Google Scholar
- Haykin S: Adaptive Filter Theory. 4th edition. NJ: Prentice-Hall; 2002.Google Scholar
- Ozeki K, Umeda T: An adaptive filtering algorithm using an orthogonal projection to an affine subspace and its properties. Electron Commun Jpn 1984, 67-A: 19-27.MathSciNetView ArticleGoogle Scholar
- Sayed AH: Fundamentals of Adaptive Filtering. Wiley; 2003.Google Scholar
- Roy S, Shynk JJ: Analysis of data-reusing LMS algorithm. Proc Midwest Symp Circuits Syst 1989, 1127-1130.Google Scholar
- Shin HC, Song WJ, Sayed AH: Mean-square performance of data-reusing adaptive algorithms. IEEE Signal Processing Letters 2005,12(12):851-854.View ArticleGoogle Scholar
- Pradhan SS, Reddy VE: A new approach to sub band adaptive filtering. IEEE Trans Signal Processing 1999, 47: 655-664. 10.1109/78.747773View ArticleGoogle Scholar
- Lee KA, Gan WS: Improving convergence of the NLMS algorithm using constrained sub band updates. IEEE Signal Processing Letters 2004,11(9):736-739. 10.1109/LSP.2004.833445View ArticleGoogle Scholar
- Kwong OW, Johnston ED: A variable step-size LMS algorithm. IEEE Trans Signal Processing 1992,40(7):1633-1642. 10.1109/78.143435View ArticleGoogle Scholar
- Aboulnasr T, Mayyas K: A robust variable step-size lms-type algorithm: Analysis and simulations. IEEE Trans Signal Processing 1997,45(3):631-639. 10.1109/78.558478View ArticleGoogle Scholar
- Pazaitis DI, Constantinides AG: A novel kurtosis driven va riable step-size adaptive algorithm. IEEE Trans Signal Processing 1999,47(3):864-872. 10.1109/78.747793View ArticleGoogle Scholar
- Mathews VJ, Xie Z: A stochastic gradient adaptive filter with gradient adaptive step size. IEEE Trans Signal Processing 1993,41(6):2075-2087. 10.1109/78.218137View ArticleGoogle Scholar
- Krishnamurthy V: Averaged stochastic gradient algorithms for adaptive blind multiuser detection in DS/CDMA systems. IEEE Trans Communications 2000,48(1):125-134. 10.1109/26.818880View ArticleGoogle Scholar
- de Lamare RC, Sampaio-Neto R: Low-complexity blind variable step size mechanisms for minimum variance CDMA receivers. IEEE Transactions on Signal Processing 2006,54(6):2302-2317.MathSciNetView ArticleGoogle Scholar
- Cai Y, de Lamare RC: Low-Complexity Variable Step Size Mechanism for Code-Constrained Constant Modulus Stochastic Gradient Algorithms applied to CDMA Interference Suppression. IEEE Transactions on Signal Processing 2009,57(1):313-323.MathSciNetView ArticleGoogle Scholar
- Mandic DP: A Generalized Normalized Gradient Descent Algorithm. IEEE Signal Processing Letters 2004,11(2):115-118. 10.1109/LSP.2003.821649View ArticleGoogle Scholar
- Shin HC, Sayed AH, Song WJ: Variable step-size NLMS and affine projection algorithms. IEEE Signal Processing Letters 2004,11(2):132-135. 10.1109/LSP.2003.821722View ArticleGoogle Scholar
- Abadi MSE, Mehrdad V, Gholipour A: Family of Variable Step-Size Affine Projection Adaptive Filtering Algorithms with Selective Regressors and Selective Partial Update. International Journal of Science and Technology, Scientia, Iranica 2010,17(1):81-98.Google Scholar
- Makino S, Kaneda Y, Koizumi N: Exponentially weighted step-size NLMS adaptive filter based on the statistics of a room impulse response. IEEE Transactions on Speech Audio Processing 1993,1(1):101-108. 10.1109/89.221372View ArticleGoogle Scholar
- Li N, Zhang Y, Hao Y, Chambers JA: A new variable step-size NLMS algorithm designed for applications with exponential decay impulse responses. Signal Processing 2008,88(9):2346-2349. 10.1016/j.sigpro.2008.03.002View ArticleGoogle Scholar
- Shi Kun, Ma Xiaoli: A variable-step-size NLMS algorithm using statistics of channel response. Signal Processing 2010,90(6):2107-2111. 10.1016/j.sigpro.2010.01.015View ArticleGoogle Scholar
- Douglas SC: Analysis and implementation of the max-NLMS adaptive filter. In Proc 29th Asilomar Conf on Signals, Systems, and Computers. Pacific Grove, CA; 1995:659-663.Google Scholar
- Schertler T: Selective block update NLMS type algorithms. Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing, Seattle, WA 1998, 1717-1720.Google Scholar
- Dogancay K, Tanrikulu O: Adaptive filtering algorithms with selective partial updates. IEEE Trans Circuits, Syst. II: Analog and Digital Signal Processing 2001,48(8):762-769. 10.1109/82.959866View ArticleGoogle Scholar
- Abadi MSE, Husøy JH: Mean-square performance of adaptive filters with selective partial update. Signal Processing 2008,88(8):2008-2018. 10.1016/j.sigpro.2008.02.005View ArticleGoogle Scholar
- Dogancay K: Partial-Update Adaptive Signal Processing: Design, Analysis and Implementation. Academic Press; 2008.Google Scholar
- Hwang KY, Song WJ: An affine projection adaptive filtering algorithm with selective regressors. IEEE Trans Circuits, Syst. II: EXPRESS BRIEFS 2007, 54: 43-46.View ArticleGoogle Scholar
- Abadi MSE, Palangi H: Mean-square performance analysis of the family of selective partial update and selective regressor affine projection algorithms. Signal Processing 2010,90(1):197-206. 10.1016/j.sigpro.2009.06.013View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.