Family of state space least mean power of twobased algorithms
 Muhammad Moinuddin^{1, 2}Email author,
 Ubaid M AlSaggaf^{1, 2} and
 Arif Ahmed^{1, 2}
https://doi.org/10.1186/s1363401502199
© Moinuddin et al.; licensee Springer. 2015
Received: 2 November 2014
Accepted: 24 February 2015
Published: 30 April 2015
Abstract
In this work, a novel family of state space adaptive algorithms is introduced. The proposed family of algorithms is derived based on stochastic gradient approach with a generalized least mean cost function J[k]=E[∥ε[k]∥^{2L }] for any integer L. Since this generalized cost function is having power `2L’, it includes the whole family of the power of twobased algorithms by having different values of L. The novelty of the work resides in the fact that such a cost function has never been used in the framework of state space model. It is a wellknown fact that the knowledge of state space model improves the estimation of state parameters of that system. Hence, by employing the state space model with a generalized cost function, we provide an efficient way to estimate the state parameters. The proposed family of algorithms inherit simplicity in its structure due to the use of stochastic gradient approach in contrast to the other modelbased algorithms such as Kalman filter and its variants. This fact is supported by providing a comparison of the computational complexities of these algorithms. More specifically, the proposed family of algorithms has computational complexity far lesser than that of the Kalman filter. The stability of the proposed family of algorithms is analysed by providing the convergence analysis. Extensive simulations are presented to provide concrete justification and to compare the performances of the proposed family of algorithms with that of the Kalman filter.
Keywords
Adaptive filters State space least mean algorithms State space estimation algorithms Convergence and stability analysis1 Introduction
Over the past few decades, adaptive filters have gained huge recognition in innumerable applications extending over a wide range of fields. Adaptive filtering is an important part of statistical signal processing, and adaptive filters have been successfully applied in diverse fields such as equalization, noise cancellation, linear prediction, and in system identification [1,2]. Adaptive filters are preferred over conventional filters because of their accuracies due to adaptive capability in the domain of the problem in which it is being used. Adaptive filters automatically adjust their weights according to some adaptive algorithm which is usually based on minimization of a function of the difference between the desired signal and the observed signal [1,2]. The most widely used algorithms for adaptive filters are the least mean squares (LMS) algorithm [1,2] and the recursive least squares algorithm [1,2].
It is observed that the adaptive filters designed by incorporating the knowledge of state space (SS) model of the system performs better than the ones without it. Amongst the plethora of literature found on adaptive filtering, there are many algorithms that deal with the SS model. For example, the very wellknown Kalman filter (KF) [3], which gives the linear optimal solution by calculating the minimum mean square error (MMSE) while utilizing the system model. It optimally estimates on the basis of observations which are subjected to noise and other disturbances. In the nonlinear filtering domain, we have the extended Kalman filter (EKF) [3], unscented Kalman filter (UKF) [4], cubature Kalman filter (CKF) [5], quadrature Kalman filter (QKF) [6] and many other variants of KF [3]. These provide suboptimal solution to the filtering problem. Although numerous techniques exist in literature for estimation of state parameters utilizing state space model but most of them are either highly computationally complex or have less accuracy in estimating the state parameters. Recently, the SS version of LMS and RLS are developed in [710] with the aim to provide an alternative to the highlycomplex KF techniques. This is the very reason for carrying out our investigation.
Equation 1 forms the basis of most of the statespacebased estimation algorithms [36]. The usual practice to derive the gain K[k] is to employ least squares solution [1]. Usually the square of the error is minimized; however, nonmean square errors have also been studied [11]. In this paper, we propose to minimize a more general cost function given by J=E[∥ε∥^{2L }], where the notation ‘ ∥.∥’ is used to represent the Euclidean norm and L is a positive integer value where L=1,2,3,... for the basic state space least mean square (SSLMS) algorithms. Our main contribution in this work is to develop a family of adaptive algorithms which has much lesser computational cost as compared to the existing state space modelbased adaptive algorithms [36]. We provide a detailed comparison of computational cost to support this argument in Section 6.
The paper is organized as such: Section 2 of the paper introduces the state space model. Then in Section 3, the proposed general SSLM algorithm is derived. Section 4 presents the convergence analysis followed by simulation results and comparison of the different algorithms in Section 5. Section 6 presents the computational complexity of the algorithms, and finally, X we conclude the paper in Section 7.
2 State space model
where f and h are nonlinear functions and the parameters are as defined before.
3 Derivation of the proposed general SSLM algorithm
It should be noted that for L=1, the algorithm results in the basic SSLMS [7].
4 Convergence in the mean analysis
 1.
The noise vectors w[k] and v[k] are zeromean white processes with covariance matrices \(\textbf {Q}_{w}={\sigma ^{2}_{w}}\textbf {I}\) and \(\textbf {Q}_{v} = {\sigma ^{2}_{v}}\textbf {I}\), respectively. Moreover, they are independent of the input and state variables of the system.
 2.
The system matrices A[k] and C[k] are independent of the state variables to be estimated. Hence, they can be treated as deterministic variables.
 3.
The filter’s length is long enough to apply the law of long adaptive filters [2].
The first assumption is a wellknown assumption and is also true in real practice. The second assumption is true for linear seperable systems but not true for nonlinear and inseparable systems. However, we can employ this assumption to make the analysis tractable. Moreover, this assumption is valid in most of the practical scenarios. The third assumption is also wellknown in literature and often used in the analysis of adaptive filters [2].
 1.The matrices A[k],A[k−1],... should have bounded entries, i.e.:$$ \textbf{A}[k]\{i,j\}<1,~~~\forall~k $$(25)
for i=1,2,3,...,n and j=1,2,3,...,n.
 2.I−μ G C ^{ T }[k]C[k]<1 which implies that the step size μ of the algorithm is bounded by:$$ 0<\mu<\frac{2}{\lambda_{\text{max}}(\textbf{G}\textbf{C}^{T}[k]\textbf{C}[k])}, ~~~\forall~k $$(26)
where λ _{max}(G C ^{ T }[k]C[k]) represents the largest eigenvalue of G C ^{ T }[k]C[k].
 3.
System or the states should be observable, i.e. matrix C[k] is full rank.
The analysis for the higher values of L can be similarly carried out.
5 Simulation results and discussion

Estimation of the state parameters of a noisy sinusoid tracking problem.

Estimation of the state parameters of a Van der pol oscillator.

Estimation of the state parameters of a symmetrical threephase induction machine.
For all of the above experiments, we investigate three different noise environments, namely Gaussian, Uniform, and Laplacian. We also investigate the effect of exponent L for the aforementioned noise environments. More specifically, we set L=1,2,3, and 4 which corresponds to SSLMS, state space least mean fourth (SSLMF), state space least mean sixth (SSLMSi), and state space least mean eighth (SSLME) algorithm, respectively. The results are reported after averaging of 100 independent simulation experiments.
5.1 Example 1. Tracking sinusoids
Root mean square error of example 1
Root mean square error (dB)  

Example 1  
KF  SSLMS  SSLMF  SSLMSi  SSLME  
State x _{1}  Gaussian  −17.3726  −16.4997  −16.4695  −16.7676  −16.9518 
Uniform  −17.1419  −16.4963  −16.4671  −16.7656  −16.9406  
Laplace  −17.6115  −16.4901  −16.5112  −16.8499  −17.0366  
State x _{2}  Gaussian  −17.0558  −16.4167  −16.3911  −16.6666  −16.8323 
Uniform  −16.8535  −16.4157  −16.3910  −16.6672  −16.8250  
Laplace  −17.2894  −16.4092  −16.4314  −16.7432  −16.9055  
State x _{3}  Gaussian  −17.3597  −16.3199  −16.2747  −16.5047  −16.6701 
Uniform  −17.1301  −16.3164  −16.2717  −16.5023  −16.6601  
Laplace  −17.5963  −16.3101  −16.2960  −16.5595  −16.7142  
State x _{4}  Gaussian  −14.0092  −15.5096  −15.4791  −15.6395  −15.7546 
Uniform  −13.8130  −15.5070  −15.4768  −15.6376  −15.7471  
Laplace  −14.2523  −15.5024  −15.4936  −15.6762  −15.7891  
Observation y  Gaussian  −30.4409  −24.3831  −25.3871  −22.6547  −20.8293 
Uniform  −30.4612  −24.3844  −25.3869  −22.6497  −20.9188  
Laplace  −30.4614  −24.3817  −24.8789  −21.9582  −19.3682 
5.2 Example 2. Van der pol oscillator
Root mean square error of Example 2
Root mean square error (dB)  

Example 2  
KF  SSLMS  SSLMF  SSLMSi  SSLME  
State x _{1}  Gaussian  −22.1985  −12.4088  −12.5740  −12.5760  −12.5760 
Uniform  −22.1974  −13.2778  −13.4407  −13.4426  −13.4427  
Laplace  −22.1995  −12.2850  −13.4015  −13.4027  −13.4053  
State x _{2}  Gaussian  −24.0418  −12.5487  −12.7486  −12.7511  −12.7511 
Uniform  −24.0494  −13.4124  −13.6097  −13.6121  −13.6121  
Laplace  −24.0450  −12.2753  −13.5716  −13.5730  −13.5763  
Observation y _{1}  Gaussian  −20.2142  −12.3685  −12.5290  −12.5309  −12.5310 
Uniform  −20.2185  −13.2086  −13.3642  −13.3660  −13.3660  
Laplace  −20.2284  −12.2460  −13.3274  −13.3285  −13.3311  
Observation y _{2}  Gaussian  −20.2526  −12.4860  −12.6800  −12.6824  −12.6824 
Uniform  −20.2403  −13.3160  −13.5048  −13.5071  −13.5071  
Laplace  −20.2426  −12.2205  −13.4735  −13.4749  −13.4780 
5.3 Example 3. Symmetrical three phase induction machine
Root mean square error of Example 3
Root mean square error (dB)  

Example 3  
KF  SSLMS  SSLMF  SSLMSi  SSLME  
State x _{1}  Gaussian  −14.9484  −13.9622  −14.1395  −14.6637  −14.6965 
Uniform  −14.9438  −13.9619  −14.1506  −14.6729  −14.7018  
Laplace  −14.9330  −13.9600  −14.1397  −14.6654  −14.6985  
State x _{2}  Gaussian  −14.2212  −13.1412  −13.7200  −14.3676  −14.3167 
Uniform  −14.2242  −13.1367  −13.7295  −14.3708  −14.4115  
Laplace  −14.2198  −13.1355  −13.7170  −14.3652  −14.3153  
State x _{3}  Gaussian  −8.4244  −7.2323  −7.9750  −8.7182  −8.8679 
Uniform  −8.4201  −7.2197  −7.9772  −8.7156  −8.7881  
Laplace  −8.4095  −7.2197  −7.9665  −8.7122  −8.8612  
State x _{4}  Gaussian  −7.8682  −6.8915  −7.1522  −7.8059  −8.1534 
Uniform  −7.8713  −6.8930  −7.1616  −7.8149  −7.9130  
Laplace  −7.8669  −6.8911  −7.1511  −7.8057  −8.1535  
State x _{5}  Gaussian  −11.1405  −9.9399  −10.2366  −10.7061  −8.1534 
Uniform  −11.1391  −9.9325  −10.2373  −10.7057  −7.9130  
Laplace  −11.1303  −9.9349  −10.2329  −10.7033  −8.1535  
Observation y _{1}  Gaussian  −26.9886  −4.7978  −5.0163  −5.4818  −5.6242 
Uniform  −26.9860  −4.7960  −5.0221  −5.4861  −5.5343  
Laplace  −26.9872  −4.7963  −5.0150  −5.4816  −5.6243  
Observation y _{2}  Gaussian  −23.66444  −7.9033  −8.5351  −9.0782  −8.9918 
Uniform  −23.6616  −7.8941  −8.54861  −9.0833  −9.0834  
Laplace  −23.6558  −7.8925  −8.5351  −9.0773  −8.9919 
An overview of the RMSE of the states and observations can be referred to in Tables 1, 2, and 3.
It is clear from our investigation that as the value of L increases, better performance is observed from the SSLM algorithms. Although the performance of the algorithms is not better in comparison to the KF, nevertheless, the algorithms are extremely low in terms of computational power requirement as will be discussed in Section 6.
6 Computational complexity
Computational complexity of SSLM algorithm
Equation number  Operation  Multiplication  Additions 

6  \(\bar {\mathbf {x}}[k]_{n\times 1} = \mathbf {A}[k1]_{n\times n}\hat {\mathbf {x}}[k1]_{n\times 1}\)  n ^{2}  n ^{2}−n 
8  \({\mathbf {\varepsilon }}[k]_{m\times 1} = \mathbf {y}[k]_{m\times 1}\bar {\mathbf {y}}[k]_{m\times 1}\)  0  m 
9  \(\bar {\mathbf {y}}[k]_{m\times 1} = \mathbf {C}[k]_{m\times n}\bar {\mathbf {x}}[k]_{n\times 1}\)  mn  n m−m 
7  \(\hat {\mathbf {x}}[k]_{n\times 1} = \bar {\mathbf {x}}[k]_{n\times 1} + \textbf {K}[k]_{n\times m}{\mathbf {\varepsilon }}[k]_{m\times 1}\)  mn  nm 
15  K[k]_{ n×m }=μ _{1×1}ε[k]_{ m×1}^{2L−2} G _{ n×n } C ^{ T }[k]_{ n×m }  m n ^{2}+m n+m+L−1  m n ^{2}−m n+m−1 
Total for the SSLM algorithm  3m n+n ^{2}+m n ^{2}+m+L−1  m+m n ^{2}+n ^{2}+m n−n−1 
Computational complexity of KF algorithm
Step  Operation  Multiplication  Additions 

Predict  \(\bar {\mathbf {x}}[k]_{n\times 1} = \mathbf {A}[k1]_{n\times n}\hat {\mathbf {x}}[k1]_{n\times 1}\)  n ^{2}  n ^{2}−n 
\(\bar {\mathbf {P}}[k]_{n\times n} = \mathbf {A}[k1]_{n\times n}\mathbf {P}[k1]_{n\times n}\mathbf {A}^{T}[k1]_{n\times n} + \mathbf {Q}[k]_{n\times n}\)  2n ^{3}  2n ^{3}−n ^{2}  
Update  \({\mathbf {\varepsilon }}[k]_{m\times 1} = \mathbf {y}[k]_{m\times 1} \mathbf {C}[k]_{m\times n}\bar {\mathbf {x}}[k]_{n\times 1}\)  mn  nm 
\(\mathbf {S}[k]_{m\times m} =\mathbf {C}[k]_{m\times n}\bar {\mathbf {P}}[k]_{n\times n}\mathbf {C}^{T}[k]_{n\times m} + \mathbf {R}[k]_{m\times m}\)  m n ^{2}+m ^{2} n  m ^{2} n+m n ^{2}−m n  
\(\mathbf {K}[k]_{n\times m} = \bar {\mathbf {P}}[k]_{n\times n}\mathbf {C}^{T}[k]_{n\times m}\mathbf {S}^{1}[k]_{m\times m}\)  m ^{3}+m ^{2} n+m n ^{2}  m ^{3}+m ^{2} n  
\(\hat {\mathbf {x}}[k]_{n\times 1} =\bar {\mathbf {x}}[k]_{n\times 1} + \mathbf {K}[k]_{n\times m}{\mathbf {\varepsilon }}[k]_{m\times 1}\)  mn  nm  
\(\mathbf {P}[k]_{n\times n} = (\mathbf {I}_{n\times n}\mathbf {K}[k]_{n\times m}\mathbf {C}[k]_{m\times n})\bar {\mathbf {P}}[k]_{n\times n}\)  m n ^{2}+n ^{3}  m n ^{2}+n ^{3}−n ^{2}  
Total for the KF algorithm  m ^{3}+2m ^{2} n+3m n ^{2}+2m n+3n ^{3}+n ^{2}  m ^{3}+2m ^{2} n+3m n ^{2}−m n+3n ^{3}+n ^{2}−n 
Computational complexity summary of Example 1
Example 1 ( m=1 , n=4 )  

Operation  KF  SSLMS  SSLMF  SSLMSi  SSLME 
Multiplications  273  45  46  47  48 
Additions  257  32  32  32  32 
Total operations  530  77  78  79  80 
Computational complexity summary of Example 2
Example 2 ( m=2 , n=2 )  

Operation  KF  SSLMS  SSLMF  SSLMSi  SSLME 
Multiplications  84  26  27  28  29 
Additions  70  15  15  15  15 
Total operations  154  41  42  43  44 
Computational complexity summary of Example 3
Example 3 ( m=2 , n=5 )  

Operation  KF  SSLMS  SSLMF  SSLMSi  SSLME 
Multiplications  618  107  108  109  110 
Additions  583  81  81  81  81 
Total operations  1201  188  189  190  191 
The results clearly suggest that the proposed algorithm has very low complexity as compared to the standard KF algorithm. According to our presented investigation, the KF algorithm perform approximately 6 times as many operations in example 1, 3.5 times as many in example 2, and 6 times as many in example 3 when compared to our algorithms.
7 Conclusions
In this work, the general family of SSLM algorithms is proposed. The proposed family is based on minimizing the general least mean cost function via stochastic gradient optimization. In order to assess the performance of the proposed family, simulation results are carried out for three different examples with different types of noise environments. In these simulations, effect of noise and exponent L on state estimation are investigated. The simulation results show that the performance of the proposed family is efficient and comparable to that of the Kalman filter. However, the computational complexity of the proposed family of algorithms is far lesser than that of the Kalman filter. More specifically, the computational complexity of the proposed family is 3.5 to 6 times lesser than the Kalman filter as presented in the reported examples. This gives a motivation to use our proposed family of algorithm in realtime application where computational complexity is of major concern. For future research, an adaptive μ could be proposed and investigated keeping in focus the effect of varying the sampling time and the noise. Moreover, different variants of KF algorithms and other linear estimation algorithms should be investigated along with the proposed SSLM algorithms family and compared to get more insightful outcome.
Declarations
Acknowledgments
The authors acknowledge the support provided by the Centre of Excellence in Intelligent Engineering Systems http://ceies.kau.edu.sa/ CEIES, King Abdulaziz University, Jeddah, Saudi Arabia and King Abdulaziz University, Jeddah, Saudi Arabia to carry out this work.
Authors’ Affiliations
References
 S Haykin, Adaptive Filter Theory, 3rd edn. (PrenticeHall, UpperSaddle River, NJ, 1996).Google Scholar
 AH Sayed, Fundamentals of Adaptive Filtering (WileyInterscience, New York, 2003).Google Scholar
 CK Chui, G Chen, Kalman Filtering With RealTime Applications, 4th edn. (Springer, Berlin, Heidelberg, 2009).Google Scholar
 EA Wan, R Van der Merwe, in IEEE Proceedings of Symposium on Adaptive Systems for Signal Processing, Communication and Control. The unscented Kalman filter for nonlinear estimation (Lake Louise,Alberta, Canada, 2000).Google Scholar
 I Arasaratnam, S Haykin, Cubature Kalman filters. Automatic Control IEEE Trans 54(6), 1254–1269 (2009).View ArticleMathSciNetGoogle Scholar
 I Arasaratnam, S Haykin, RJ Elliott, Discretetime nonlinear filtering algorithms using GaussHermite quadrature. Proc. IEEE 95(5), 953–977 (2007).View ArticleGoogle Scholar
 MB Malik, Salman M, Statespace least mean square. Digital Signal Process 18(3), 334–345 (2008).View ArticleGoogle Scholar
 MB Malik, RA Bhatti, Tracking of linear timevarying systems using statespace least mean square. IEEE Int. Symp. Commun. Inform. Technol 1, 582–585 (2004).Google Scholar
 MB Malik, Statespace recursive least squares: part I. Signal Process84/9, 1709–1718 (2004).View ArticleGoogle Scholar
 MB Malik, Statespace recursive least squares: part II. Signal Process84/9, 1719–1728 (2004).View ArticleGoogle Scholar
 E Walach, B Widrow, The leastmean fourth (LMF) adaptive algorithm and its family. IEEE Trans. Inform. TheoryIT30, 275–283 (1984).View ArticleGoogle Scholar
 HK Khalil, Nonlinear Systems, 3rd edn. (Prentice Hall, UpperSaddle River, NJ, 2000).Google Scholar
 K Reif, F Sonnemann, R Unbehauen, An EKFbased nonlinear observer with a prescribed degree of stability. Automatica 34(9), 1119–1123 (1998).View ArticleMATHGoogle Scholar
 R Kandepu, B Foss, L Imsland, Applying the unscented Kalman filter for nonlinear state estimation. J. Process Control 18, 753–768 (2008).View ArticleGoogle Scholar
 L Salvatore, S Stasi, L Tarchioni, A new EKF based algorithm for flux estimation in induction machines. IEEE Trans. Ind. Electron 40(5), 496–504 (1993).View ArticleGoogle Scholar
Copyright
This is an Open Access article distributed under the terms of the Creative Commons Attribution License(http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.