An Adaptive Nonlinear Filter for System Identiﬁcation

The primary di ﬃ culty in the identiﬁcation of Hammerstein nonlinear systems (a static memoryless nonlinear system in series with a dynamic linear system) is that the output of the nonlinear system (input to the linear system) is unknown. By employing the theory of a ﬃ ne projection, we propose a gradient-based adaptive Hammerstein algorithm with variable step-size which estimates the Hammerstein nonlinear system parameters. The adaptive Hammerstein nonlinear system parameter estimation algorithm proposed is accomplished without linearizing the systems nonlinearity. To reduce the e ﬀ ects of eigenvalue spread as a result of the Hammerstein system nonlinearity, a new criterion that provides a measure of how close the Hammerstein ﬁlter is to optimum performance was used to update the step-size. Experimental results are presented to validate our proposed variable step-size adaptive Hammerstein algorithm given a real life system and a hypothetical case. Copyright


Introduction
Nonlinear system identification has been an area of active research for decades.Nonlinear systems research has led to the discovery of numerous types of nonlinear systems such as Volterra, Hammerstein, and Weiner nonlinear systems [1][2][3][4].This work will focus on the Hammerstein nonlinear system depicted in Figure 1.Hammerstein nonlinear models have been applied to modeling distortion in nonlinearly amplified digital communication signals (satellite and microwave links) followed by a linear channel [5,6].In the area of biomedical engineering, the Hammerstein model finds application in modeling the involuntary contraction of human muscles [7,8] and human heart rate regulation during treadmill exercise [9].Hammerstein systems are also applied in the area of Neural Network since it provides a convenient way to deal with nonlinearity [10].Existing Hammerstein nonlinear system identification techniques can be divided into three groups: (i) deterministic techniques such as orthogonal leastsquares expansion method [11][12][13], (ii) stochastic techniques based on recursive algorithms [14,15] or nonadaptive methods [16], and (iii) adaptive techniques [17][18][19][20].
Adaptive Hammerstein algorithms have been achieved using block based adaptive algorithms [11,20].In block based adaptive Hammerstein algorithms, the Hammerstein system is overparameterized in such a way that the Hammerstein system is linear in the unknown parameters.This allows the use of any linear estimation algorithm in solving the Hammerstein nonlinear system identification problem.The limitation of this approach is that the dimension of the resulting linear block system can be very large, and therefore, convergence or robustness of the algorithm becomes an issue [18].Recently, Bai reported a blind approach to Hammerstein system identification using least mean square (LMS) algorithm [18].The method reported applied a two-stage identification process (Linear Infinite Impulse Response (IIR) stage and the nonlinear stage) without any knowledge of the internal signals connecting both cascades in the Hammerstein system.This method requires a white input signal to guarantee the stability and convergence of the algorithm.Jeraj and Mathews derived an adaptive Hammerstein system identification algorithm by linearizing the system nonlinearity using a Gram-Schmidt orthogonalizer at the input to the linear subsystem (forming an MISO system) [17].This method also suffers the same limitations as the block-based adaptive Hammerstein algorithms.Thus, to improve the speed of

Hammerstein nonlinear filter
Infinite impulse response filter Figure 1: Adaptive system identification of a Hammerstein system model.
convergence while maintaining a small misadjustment and computational complexity, the Affine Projection theory is used as opposed to LMS [18] or Recursive Least squares (RLSs).
In nonlinear system identification, input signals with high eigen value spread, ill-conditioned tap input autocorrelation matrix can lead to divergence or poor performance of a fixed step-size adaptive algorithm.To mitigate this problem, a number of variable step-size update algorithms have been proposed.These variable step-size update algorithms can be roughly divided into gradient adaptive step-size [21,22] and normalized generalized gradient descent [23].The major limitation of gradient adaptive step-size algorithms is their sensitivity to the time correlation between input signal samples and the value of the additional step-size parameter that governs the gradient adaptation of the step-size.As a result of these limitations, a criteria for the choice of the step-size based on Lyapunov stability theory is proposed to track the optimal step-size required to maintain a fast convergence rate and low misadjustment.
In this paper, we focus on the adaptive system identification problem of a class of Hammerstein output error type nonlinear systems with polynomial nonlinearity.Our unique contributions in the paper are as follows.
(1) Using the theory of affine projections [24], we derive an adaptive Hammerstein algorithm that identifies the linear subsystem of the Hammerstein system without prior knowledge of the input signal z(n).
(2) Employing the Lyapunov stability theory, we develop criteria for the choice of the algorithms step-size which ensures the minimization of the Lyapunov function.This is particularly important for the stability of the linear algorithm regardless of the location of the poles of the IIR filter.
Briefly, the paper is organized as follows.Section 2 describes the nonlinear Hammerstein system identifica-tion problem addressed in this paper.Section 3 contains a detailed derivation of the proposed variable step-size adaptive Hammerstein algorithm.Section 4 provides both a hypothetical and real life data simulation validating the effectiveness of the variable step-size adaptive algorithm proposed.Finally, we conclude with a brief summary in Section 5.

Problem Statement
Consider the Hammerstein model shown in Figure 1, where x(n), v(n), and d(n) are the systems input, noise, and output, respectively.z(n) represents the unavailable internal signal output of the memoryless polynomial nonlinear system.The output of the memoryless polynomial nonlinear system, which is the input to the linear system, is given by Let the discrete linear time-invariant system be an infinite impulse response (IIR) filter satisfying a linear difference equation of the form where p l (n), a i (n), and b j (n) represent the coefficients of the nonlinear Hammerstein system at any given time n.To ensure uniqueness, we set b 0 (n) = 1 (any other coefficient other than b 0 (n) can be set to 1).Thus, (2) can be written as Equation ( 3) can be rewritten in compact form The goal of the Adaptive nonlinear Hammerstein system identification is to estimate the coefficient vector ( θ(n)) in ( 5) of the nonlinear Hammerstein filter based only on the input signal x(n) and output signal d(n) such that d(n) is close to the desired response signal d(n).

Adaptive Hammerstein Algorithm
In this section, we develop an algorithm based on the theory of Affine projection [24] for estimation of the coefficients of the nonlinear Hammerstein system using the plant input and output signals.The main idea of our approach to nonlinear Hammerstein system identification is to formulate a criterion for designing a variable step-size affine projection Hammerstein filter algorithm and then use the criterion in minimizing the cost function.

Stochastic Gradient Minimization
Approach.We formulate the criterion for designing the adaptive Hammerstein filter as the minimization of the square Euclidean norm of the change in the weight vector subject to the set of Q constraints Applying the method of Lagrange multipliers with multiple constraints to ( 6) and ( 7), the cost function for the affine projection filter is written as (assuming real data) (8) where Minimizing the cost function (8) (squared prediction error) with respect to the nonlinear Hammerstein filter weight vector θ(n) gives where Since a portion of the vectors s(n) in S(n) include past d(n) which are dependent on past θ(n) which are used to form the new θ(n), the partial derivative of each element in (10) gives From ( 12), (13), and ( 14) it is necessary to evaluate the derivative of past d(n) with respect to current weight estimates.In evaluating the derivative of d(n) with respect to the current weight vector, we assume that the step-size of the adaptive algorithm is chosen such that [24] Therefore thus, where Let Substituting ( 16), (17), and ( 20) into (11), we get Thus, rewriting ( 10) Setting the partial derivative of the cost function in (24) to zero, we get From ( 7), we can write where Evaluating (27) for λ results in where Substituting ( 28) into ( 25) yields the optimum change in the weight vector

e(n − 1). (30)
Assuming that the input to the linear part of the nonlinear Hammerstein filter is a memoryless polynomial nonlinearity, we normalize (30) as in [25] and exercise control over the change in the weight vector from one iteration to the next keeping the same direction by introducing the stepsize μ.Regularization of the S(n − 1) H Φ(n − 1) matrix is also used to guard against numerical difficulties during inversion, thus yielding

e(n − 1). (31)
To improve the update process Newton's method is applied by scaling the update vector by R −1 (n).The matrix R(n) is recursively computed as where λ n is typically chosen between 0.95 and 0.99.Applying the matrix inversion lemma on (32) and using the result in (31), the new update equation is given by

Variable
Step-Size.In this subsection, we derive an update for the step-size using a Lyapunov function of summed squared nonlinear Hammerstein filter weight estimate error.The variable step-size derived guarantees the stable operation of the linear IIR filter by satisfying the stability condition for the choice of μ in [26].Let where θ represents the optimum Hammerstein system coefficient vector.We propose the Lyapunov function V (n) as which is the general form of the quadratic Lyapunov function [27].The Lyapunov function is positive definite in a range of values close to the optimum θ = θ(n).In order for the multidimensional error surface to be concave, the time derivative of the Lyapunov function must be semidefinite.This implies that From the Hammerstein filter update equation we subtract θ from both sides to yeild From ( 35), (36), and (38) we have Minimizing the Lyapunov function with respect to the step-size μ, and equating the result to zero, we obtain the optimum value for μ as μ opt where Adding the system noise v(n) to the desired output and assuming that the noise is independently and identically distributed and statistically independent of S(n), we have Algorithm 1: Summary of the proposed Variable Step-size Hammerstein adaptive algorithm.
From (40) we write The computation of μ opt requires the knowledge of θ(n − 1) which is not available during adaptation.Thus, we propose the following suboptimal estimate for μ(n): We estimate E Υ(n−1)e(n−1) by time averaging as follows: where μ opt is an rough estimate of μ opt , α is a smoothing factor (0 < α < 1), and C is a constant representing σ 2 v Tr{ Υ(n − 1) 2 } ≈ Q/SNR.We guarantee the stability of the Hammerstein filter by choosing μ opt to satisfy the stability bound in [26].Choosing μ opt to satisfy the stability bound [26] will bound the step-size update μ(n) with an upper limit of μ opt thereby ensuring the slow variation and stability of the linear IIR filter.
A summary of the proposed algorithm is shown in Algorithm 1.In the algorithm, N represents the number and static nonlinearity modeled as a polynomial with coefficients The desired response signal d(n) of the adaptive Hammerstein filter was obtained by corrupting the output of the unknown system with additive white noise of zero mean and variance such that the output signal to noise ratio was 30 dB.The proposed algorithm was initialized as follows: λ n = 0.997, μ opt = 1.5e − 6, δ = 5e − 4, and C = 0.0001.
Figure 2 shows the mean square deviation between the estimated and optimum Hammerstein filter weights for a white input case.The results were obtained, by ensemble averaging over 100 independent trials.The figure shows that the convergence speed of the proposed algorithm is directly correlated to the number of constraints Q used in the algorithm.
Figure 3 shows a comparison of the mean square deviation learning curve obtained, from the proposed algorithm and the algorithm in [17].The proposed algorithm outperformed [17] even though the authors had used the Gram Schmidt process to better enhance the algorithms performance in a colored environment.This result is expected since [17] is sensitive to the additional step-size and regularizing parameters that govern the adaptation of the Gram-Schmidt processor.Results also show that, above a constraint value of 1 (Q > 1), the proposed algorithms performance in the colored environment is similar.

Conclusion
We have proposed a new adaptive filtering algorithm for the Hammerstein model filter based on the theory of Affine Projections.The new algorithm minimizes the norm of the projected weight error vector as a criterion to track the adaptive Hammerstein algorithm's optimum performance.Simulation results confirm the convergence of the parameter estimates from our proposed algorithm to its corresponding parameter in the plants parameter vector.

Figure 3 :
Figure 3: Mean square deviation learning curves of the proposed algorithm for varying constraint and a colored input signal.