- Research
- Open Access
Filtered-X Least Mean Fourth (FXLMF) and Leaky FXLMF adaptive algorithms
- Ali M. Al Omour^{1},
- Abdelmalek Zidouri^{1},
- Naveed Iqbal^{1} and
- Azzedine Zerguine^{1}Email author
https://doi.org/10.1186/s13634-016-0337-z
© Al Omour et al. 2016
- Received: 30 June 2015
- Accepted: 16 March 2016
- Published: 1 April 2016
Abstract
Adaptive filtering algorithms promise an improvement of the active noise control (ANC) problem encountered in many scenarios. Just to name a few, the Filtered-X Least Mean Square (FXLMS) algorithm, the Leaky FXLMS (LFXLMS) algorithm, and other modified LMS-based algorithms have been developed and utilized to combat the ANC problem. All of these algorithms enjoy great performance when the signal-to-noise ratio (SNR) is high. On the other hand, when the SNR is low, which is a known trend in ANC scenarios, the performance of these algorithms is not attractive. The performance of the Least Mean Fourth (LMF) algorithm has never been tested on any ANC scenario under low or high SNR. Therefore, in this work, reflecting the development in the LMS family on the LMF, we are proposing two new adaptive filtering algorithms, which are the Filtered-X Least Mean Fourth (FXLMF) algorithm and the Leakage-based variant (LFXLMF) of the FXLMF algorithm. The main target of this work is to derive the FXLMF and LFXLMF adaptive algorithms, study their convergence behaviors, examine their tracking and transient conduct, and analyze their performance for different noise environments. Moreover, a convex combination filter utilizing the proposed algorithm and algorithm robustness test is carried out. Finally, several simulation results are obtained to validate the theoretical findings and show the effectiveness of the proposed algorithms over other adaptive algorithms.
Keywords
- Minimum Mean Square Error
- Little Mean Square
- Adaptive Filter
- Little Mean Square Algorithm
- Combine Filter
1 Introduction
Adaptive filtering algorithms are by now omnipresent in a variety of applications, such as plant modeling, adaptive equalization, and system identification, to name a few [1–8]. Add to that, noise control and noise cancelation are important issues whose effects adaptive filtering algorithms strive to mitigate. Active noise control (ANC) techniques use adaptive filtering algorithms to cancel the effect of acoustic noise, by playing anti-noise signal estimated from the noise source itself.
The Least Mean Square (LMS) algorithm suffers from problems, such as a degradation in the algorithm efficiency, due to the presence of a filter in the auxiliary or error path, as in the case of the ANC technique, as well as slow convergence, instability of the algorithm, increased residual noise power, and lower convergence rate. These constraints urged researchers to enhance the performance of the conventional LMS algorithm [9–11].
In the last decade, intensive research was carried out for the purpose of enhancing the performance of the FXLMS algorithm. In [14], a new stochastic analysis for the FXLMS algorithm was introduced, using an analytical model not based on the independence theory [16], to derive the first moment of the adaptive weight filter. The main assumption of this work was to ignore the correlation between the data vector and the weights and compare the correlation between data vectors, preserving past and present data vector correlations. This model was validated for both white and colored primary input signals and shows stability even when using large step sizes.
The FXLMS algorithm is preferred because of its inherent stability and simplicity, but sometimes, the adaptive filter suffers from high noise levels caused by low-frequency resonances, which may cause nonlinear distortion due to overloading of the secondary source. This problem was solved by adding output power constraints to the cost function, as was proposed in the Leaky FXLMS (LFXLMS) algorithm [17]. Moreover, the LFXLMS reduces the numerical error in the finite precision implementation and limits the output power of the secondary source to avoid nonlinear distortion. The LFXLMS increases the algorithm’s stability, especially when a large source strength is used.
Another modification of the FXLMS algorithm is the Modified FXLMS (MFXLMS) algorithm [15]. Since the FXLMS exhibits poor convergence performance, the MFXLMS proposes a better convergence and reduces the computational load.
The LMS may suffer from divergence due to the insufficient spectral excitation, like a sinusoid signal without noise, which consequently may cause overflow for the weight vector during the updating process. This divergence problem can be resolved by proposing a leak term during the update process of the weight vector. Of course, this will result in lesser performance; however, the leakage factor is controlled, which is necessary to balance for the lost performance. In addition, this will add complexity, but more robustness of the adaptive filter is achieved, as was done in the case of the Leaky LMS (LLMS) algorithm.
In [17], a stochastic analysis for the LFXLMS algorithms was proposed without resorting to the independence theory. Furthermore, to strengthen their work, the authors assumed an inexact estimation for the secondary path, which is the case in most practical implementations for the adaptive filter.
Due to very low input signal, the Leaky LMS algorithm proposed in [18] aims to reduce the stalling effect, where the gradient estimate is too small to adjust the coefficients of the algorithm. Moreover, the leakage term stabilized the LMS algorithm successfully. Also, the LLMS solved the problem of bursting in short-distance telephones when they added the adaptive echo canceller [19].
The LMF weights converge proportionally to the LMS weights. The performance of the LMF algorithm has never been tested on any ANC scenario under low or high signal-to-noise ratio (SNR). In this work, we propose two new algorithms, the Filtered-X LMF (FXLMF) and Leaky FXLMF (LFXLMF) algorithms. We analyze the convergence behaviors and examine the performance of both of them. This is carried out under different statistical input signals and noise for the mean and mean square error of the adaptive filter weights, depending on secondary path modeling error using an energy conservation relation framework. These two algorithms are expected to have a high effectiveness on the ANC issue at an extra computational complexity. Monte Carlo simulations used to assess the analytical assumptions, as well as the accuracy of the proposed model, are verified and assessed.
This paper is organized as follows: Section 1 provides an introduction and a literature review. In Section 2, analytical derivations for the FXLMF and LFXLMF algorithms are presented. Section 3 proposes the convex combination of the FXLMF algorithm with the FXLMS algorithm. Simulation results are presented in Section 4, and finally, the conclusions and future work are presented in Section 5.
2 Analysis
Figure 1 illustrates the block diagram of an ANC and illustrates the location of the secondary path S and its estimated secondary path Ŝ. The secondary path is a transfer function which can be represented by a group of a digital-to-analog (D/A) converter, a power amplifier, a canceling loudspeaker, an error microphone, and an (A/D) converter.
Parameters and their descriptions used in Fig. 1
Adaptive filter weights | w(n) = [w _{0}(n) w _{1}(n) … w _{ N − 1}(n)]^{ T } |
Stationary input signal | x(n) = [x(n) x(n − 1) … x(n − N + 1)]^{ T } |
Secondary path | S = [s _{0} s _{1} … s _{ M − 1}]^{ T } |
Estimate of the secondary path | Ŝ = [ŝ _{0} ŝ _{1} … ŝ _{ M − 1}]^{ T } |
Primary (desired) signal | d(n) |
Stationary noise process | z(n) |
Number of tap weight coefficients | N |
Number of the secondary path coefficients | M |
For the case of an exact approximation for the secondary path, that is S = Ŝ, the input signal, x(n), will be filtered by S.
2.1 Development of FXLMF algorithm
During the course of derivations, we will resort to the same assumptions, used in the literature [17–23], to simplify our algorithms. These assumptions are as follows:
2.1.1 Assumption A1
x(n) is the input signal, a zero mean wide-stationary Gaussian process with variance σ _{ x } ^{ 2}, and R _{ i,j } = E[x(n − j)x ^{ T }(n − i)] > 0 is a positive definite autocorrelation matrix of the input signal.
2.1.2 Assumption A2
z(n) is the measurement noise, an independent and identically distributed (i.i.d) random variable with zero mean and variance σ _{ z } ^{2} = E[z ^{2}(n)], and there is no correlation between the input signal and the measurement noise. In other words, the sequence z(n) is independent of x(n) and w(n). The measurement is assumed to have an even probability density function.
2.2 Mean behavior for the FXLMF algorithm
To find the expectations for the terms on the right-hand side of Eq. (13), we resort to the following assumptions [1, 21]:
2.2.1 Assumption A3
Independence theory (IT) states that the taps of the input vector x(n–i), i = 0,1,2,… are statistically dependent so that E[x(n − i)x ^{ T }(n − j)] = E[x(n − i)x ^{ T }(n − j)x(n − k)] = E[x(n − l)x ^{ T }(n − k)x(n − j)x ^{ T }(n − i)] = 0, for any i ≠ j, i ≠ j ≠ k, and i ≠ j ≠ k ≠ l, respectively.
2.2.2 Assumption A4
Take into consideration the correlation between x(n–i), x(n–j), x(n–k), and x(n–l)∀ i, j, k, l and ignore the correlation between w(n–v) and x(n–i) or x(n–j) or x(n–k)∀ i, j, k.
2.3 Second moment analysis for FXLMF algorithm
2.4 FXLMF algorithm stability
where \( {\lambda}_{\max}\left({\tilde{\boldsymbol{R}}}_{s^2}\right) \) represents the maximum eigenvalue of \( {\tilde{\boldsymbol{R}}}_{s^2} \).
2.5 Development of Leaky FXLMF (LFXLMF) algorithm
2.6 Mean behavior of the adaptive weight vector for LFXLMF algorithm
2.7 Second moment analysis for LFXLMF
2.8 LFXLMF algorithm stability
where \( {\lambda}_{\max}\left({\tilde{\boldsymbol{R}}}_{s^2}\right) \) represents the maximal eigenvalue of \( {\tilde{\boldsymbol{R}}}_{s^2} \). As can be seen from Eq. (48), the leakage factor has an effect on the step size.
3 Algorithms’ convex combination
In this section, we examine the behavior of our algorithm through the convex combination approach, namely the convex combination with the FXLMF algorithm. The method of combining two algorithms is an interesting proposal. It aims to mix the output of each filter and highlights the best features of each individual algorithm. Then, it utilizes the features in the overall equivalent filter to improve the performance of the adaptive filter [24–29]. In this section, we will examine our proposed algorithms with members from the LMS and LMF families.
Each filter is updated individually, depending on its own error e _{1}(n) or e _{2}(n), and the overall weight vector is updated according to the total error e(n) = [d(n) − y(n) + z(n)] which adapts the mixing parameter λ(n). Using the gradient descent method, we can minimize the fourth-order e ^{4}(n) and the second-order e ^{2}(n) errors for the overall filter. Based on that, we can use the convex combined filter over two scenarios.
Simulations in Section 4 will investigate four cases, where the comparison will be done by using the FXLMF and FXLMS algorithms, as the two transversal filters are used in the convex combination, according to the second error order minimization.
4 Simulation results
In addition, for simplicity, we assume the secondary path and the estimated secondary path are equal \( \boldsymbol{S} = \widehat{\boldsymbol{S}}=\left[\begin{array}{ccc}\hfill 0.7756\hfill & \hfill 0.5171\hfill & \hfill -0.3620\hfill \end{array}\right] \).
Figure 3 shows a comparison of the mean square error MSE behavior for different algorithms from the LMS family (i.e., NLMS, FXLMS, FeLMS, MFXLMS), the NLMF, and our proposed ones. It can be shown that the FXLMF algorithm converges, and it will reach the white noise level after a large number of iterations. For the LFXLMF algorithm, it reaches the steady state level faster than the others and after almost 5000 iterations, but it converges to a higher white noise level at almost 12 dB. Using a larger step size μ may lead the algorithm to diverge.
Figure 4 shows a comparison of the mean square error MSE behavior for different algorithms with fixed step size but this time for low SNR with a value of 5 dB. We can clearly notice that the FXLMF and LFXLMF algorithms outperform other LMS family algorithms in speed of convergence, an advantage to our proposed algorithms with almost 500 iterations. FXLMF and LFXLMF almost have identical curves because we are using a small leakage factor γ.
Figure 5 shows the effect of changing the step size on the mean weight vector of the FXLMF algorithm; when we increase the values of the step size, the algorithm converges faster to the larger mean of the weight. Moreover, using assumption A4 makes the algorithm converge to a higher mean weight level.
Figure 6 shows the effect of changing the leakage factor on the mean weight of the LFXLMF algorithm. We can see that increasing the value of the leakage factor will increase the mean weight of the LFXLMF algorithm and it does not affect the speed of convergence.
Figure 7 shows the robustness of the proposed algorithms FXLMF and LFXLMF at low SNR and using Gaussian noise, when a sudden change occurred in the weight vector.
Figure 8 reports the performance of the algorithms when the uniform noise ids used instead of Gaussian, using the same conditions as we used before in Fig. 3. As we can see, we have almost the same result, since both the FXLMF and LFXLMF algorithms converge, where the first one keeps converging while the second one reaches the steady state faster.
Figure 9 is the same as Fig. 4 but using a fixed step size and uniform noise. In addition, the FXLMF and LFXLMF algorithms outperform the LMS family in convergence.
Figure 10 shows the effect of changing the step size on the mean weight vector of the FXLMF algorithm, and as depicted in Fig. 5, the algorithm converges faster as we increase the step size using uniform noise.
Figure 11 shows the effect of changing the leakage factor on the mean weight of the LFXLMF algorithm as shown in Fig. 6; increasing the value of the leakage factor will increase the mean weight of the LFXLMF algorithm.
Figure 12 shows the robustness of the proposed algorithms FXLMF and LFXLMF at low SNR and using uniform noise.
Figure 13 illustrates the behavior of the convex-combined filter of FXLMS and FXLMF algorithms; we can see at the beginning that the combined filter followed the FXLMF algorithm since it has a faster speed of convergence. After that, the combined filter moved to the FXLMS algorithm, which showed better convergence at high SNR. Also, we can see from Fig. 14 the behavior of the mixing parameter λ(n). We assume a 50 % mixing percentage as the initial case, then λ(n) followed the FXLMF algorithm at the beginning where the FXLMF shows faster convergence, and after that, the mixing parameter switched to the other algorithm FXLMS where it has a better convergence. In Fig. 15, with the same environment as in Fig. 13 but with low SNR, the FXLMF algorithm outperforms the FXLMS algorithm and the combined filter followed the FXLMF algorithm at the beginning; then, when both algorithms have the same convergence, the mixing parameter is λ(n) = 50 %. This is shown in Fig. 16.
Figure 17 shows the robustness of the convex-combined filter of FXLMF for two different step sizes at low SNR and using Gaussian noise. We can clearly see that the combined filter followed the one with a larger step size, which already shows better performance. Similarly, Fig. 18 shows the robustness of the convex-combined filter of the FXLMF and FXLMS algorithms at high SNR and using Gaussian noise. We can clearly see that the combined filter followed the FXLMF algorithm at the beginning and then switched to the FXLMS algorithm, which shows better performance at high SNR. Finally, Fig. 19 shows the robustness of the convex-combined filter of the FXLMF and FXLMS algorithms at low SNR and using Gaussian noise. We can clearly see that the combined filter followed the FXLMF algorithm all the time since it shows better performance than the FXLMS algorithm at low SNR.
5 Conclusions
Two algorithms FXLMF and LFXLMF were proposed in this work; an analytical study and mathematical derivations for the mean weight adaptive vector and the mean square error for both algorithms have been obtained. Moreover, the step size and the leakage factor bound ranges were investigated.
From the literature, we received a good sense about proposing new algorithms to the LMF family, as was proposed before in the LMS. The FXLMF and LFXLMF algorithms successfully converge under a large range of SNR. Furthermore, we see the ability of both algorithms to converge under different environments of noise: Gaussian and uniform. However, the LMF family requires more computational complexity; our proposed algorithms were faster in convergence than members of the LMS family under some circumstances.
From the simulations, we saw that both algorithms converge well under relatively high SNR but they converge faster under low SNR. In addition, using a step size near the upper boundary will guarantee less time to converge; however, working close to the upper boundary of the step size ensures faster convergence, but we have to take the risk of algorithm divergence. Also, we see that a larger step size will increase the mean of the weight vector. Step size under the upper boundary is given in Eq. (28).
The leakage factor in the LFXLMF algorithm adds more stability to the algorithm, at the expense of reduced performance as was expected from the literature. The leakage factor boundaries were derived in Eq. (47).
The convex combination is an interesting proposal to get the best feature of two or more adaptive algorithms. We were able to successfully apply it using the FXLMF and FXLMS algorithms with different step sizes. In the other scenario, we applied the combination over the FXLMS and FXLMF algorithms and we noticed that the convex-combined filter, at every iteration, followed the best algorithm.
A robustness test was done for all the scenarios used, to ensure that the proposed algorithms are able to adapt in case of a sudden change in the tap weights of the filter, either in the transient or steady state stage.
Declarations
Acknowledgements
The authors would like to thank the anonymous reviewers for their feedback that had improved the quality of the paper. The authors acknowledge the support provided by the Deanship of Scientific Research at KFUPM.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
- S Haykin, Adaptive filter theory, 4th edn. (Prentice-Hall, Englewood Cliffs, NJ, 2002)MATHGoogle Scholar
- AH Sayed, Adaptive filters (Wiley, NJ, USA, 2008)View ArticleGoogle Scholar
- E Hänsler, G Schmidt, Acoustic echo and noise control: a practical approach (Wiley & Sons, New Jersey, 2004)View ArticleGoogle Scholar
- SM Kuo, DR Morgan, Active noise control systems, algorithms and DSP implementation functions (Wiley, New York, 1996)Google Scholar
- B Widrow, D Shur, S Shaffer, On adaptive inverse control, in Proc. 15th Asilomar Conf, 1981, pp. 185–189Google Scholar
- B. Widrow, S.D. Stearns, Adaptive signal processing, (Prentice-Hall, Upper-Saddle River, NJ, 1985)Google Scholar
- WA Gardner, Learning characteristics of stochastic-gradient-descent algorithms: a general study, analysis and critique. Signal Processing 6, 113–133 (1984)MathSciNetView ArticleGoogle Scholar
- B Widrow, ME Hoff, Adaptive switching circuits, in Proc. Of WESCON Conv. Rec., part 4, 1960, pp. 96–140Google Scholar
- P Dreiseitel, E Hänsler, H Puder, Acoustic echo and noise control—a long lasting challenge, in Proc EUSIPCO, 1998, pp. 945–952Google Scholar
- E Hänsler, GU Schmidt, Hands-free telephones—joint control of echo cancellation and postfiltering. Signal Processing 80, 2295–2305 (2000)View ArticleMATHGoogle Scholar
- C Breining, P Dreiscitel, E Hänsler, A Mader, B Nitsch, H Puder, T Schertler, G Schmidt, J Tilp, Acoustic echo control. An application of very-high-order adaptive filters. IEEE Signal Proc. Mag. 16(4), 42–69 (1999)View ArticleGoogle Scholar
- E. Bjarnason, Analysis of the Filtered-X LMS algorithm. IEEE Trans. Speech Audio Process. 3, 504–514 (1995)Google Scholar
- OJ Tobias, JCM Bermudez, NJ Bershad, R Seara, Mean weight behavior of the Filtered-X LMS algorithm, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process, 1998, pp. 3545–3548Google Scholar
- OJ Tobias, Stochastic analysis of the Filtered-X LMS algorithm, in Ph.D dissertation (Federal Univ, Santa Catarina, Brazil, 1999)Google Scholar
- OJ Tobias, JCM Bermudez, NJ Bershad, Mean weight behavior of the Filtered-X LMS algorithm. IEEE Trans. Signal Process. 48(4), 1061–1075 (2000)View ArticleGoogle Scholar
- JE Mazo, On the independence theory of equalizer convergence. Bell Syst. Tech. J. 58, 963–993 (1979)MathSciNetView ArticleMATHGoogle Scholar
- GJ Rey, RR Bitmead, CR Johnson, The dynamics of bursting in simple adaptive feedback systems with leakage. IEEE Trans. Circuits Syst. 38, 475–488 (1991)View ArticleGoogle Scholar
- L Vicente, E Masgrau, Novel FxLMS convergence condition with deterministic reference. IEEE Trans. Signal Process. 54, 3768–3774 (2006)View ArticleGoogle Scholar
- K Mayyas, T Aboulnasr, Leaky LMS algorithm: MSE analysis for Gaussian data. IEEE Trans. Signal Process. 45(4), 927–934 (1997)View ArticleGoogle Scholar
- E. Walach, B. Widrow, The Least Mean Fourth (LMF) adaptive algorithm and its family. IEEE Trans. Inf. Theory. IT-30(2), (1984), pp. 275–283Google Scholar
- O.J. Tobias, R. Seara, Leaky-FXLMS algorithm: stochastic analysis for Gaussian data and secondary path modeling Error. IEEE T. Speech Audi. P. 13(6), 1217–1230 (2005)Google Scholar
- RD Gitlin, HC Meadors Jr, SB Weinstein, The tap-leakage algorithm: an algorithm for the stable operation of a digitally implemented fractionally spaced equalizer. Bell Syst. Tech. J. 61(8), 1817–1839 (1982)View ArticleGoogle Scholar
- O Khattak, A Zerguine, Leaky Least Mean Fourth adaptive algorithm. IET Signal Process. 7(2), 134–145 (2013)MathSciNetView ArticleGoogle Scholar
- LA Azpicueta-Ruiz, M Zeller, AR Figueiras-Vidal, J Arenas-García, Least squares adaptation of affine combinations of multiple adaptive filters, in Proc. of IEEE Intl. Symp. on Circuits and Systems, Paris, France, 2010, pp. 2976–2979Google Scholar
- JC Burgess, Active adaptive sound control in a duct: a computer simulation. J. Acoust. Soc. Am. 70, 715–726 (1981)View ArticleGoogle Scholar
- SS Kozat, AC Singer, Multi-stage adaptive signal processing algorithms, in Proc. 2000 IEEE Sensor Array Multichannel SignalWork- shop, Cambridge, MA, 2000, pp. 380–384Google Scholar
- AC Singer, M Feder, Universal linear prediction by model order weighting. IEEE Trans. Signal Process. 47, 2685–2700 (1999)View ArticleMATHGoogle Scholar
- M Niedz`wiecki, Multiple-model approach to finite memory adaptive filtering. IEEE Trans. Signal Process 40, 470–473 (1992)View ArticleGoogle Scholar
- J. Arenas-Garcia, L.A. Azpicuet-Ruiz, M.T.M. Silva, V.H. Nascimento, A.H. Sayed, Combinations of adaptive filters: performance and convergence properties. IEEE Signal Proc. Mag. 33, 120-140 (2016)Google Scholar