Iterative unbiased FIR state estimation: a review of algorithms
- Yuriy S Shmaliy^{1}Email author and
- Dan Simon^{2}
https://doi.org/10.1186/1687-6180-2013-113
© Shmaliy and Simon; licensee Springer. 2013
Received: 6 November 2012
Accepted: 28 April 2013
Published: 30 May 2013
Abstract
In this paper, we develop in part and review various iterative unbiased finite impulse response (UFIR) algorithms (both direct and two‐stage) for the filtering, smoothing, and prediction of time‐varying and time‐invariant discrete state‐space models in white Gaussian noise environments. The distinctive property of UFIR algorithms is that noise statistics are completely ignored. Instead, an optimal window size is required for optimal performance. We show that the optimal window size can be determined via measurements with no reference. UFIR algorithms are computationally more demanding than Kalman filters, but this extra computational effort can be alleviated with parallel computing, and the extra memory that is required is not a problem for modern computers. Under real‐world operating conditions with uncertainties, non‐Gaussian noise, and unknown noise statistics, the UFIR estimator generally demonstrates better robustness than the Kalman filter, even with suboptimal window size. In applications requiring large window size, the UFIR estimator is also superior to the best previously known optimal FIR estimators.
Keywords
1 Review
1.1 Introduction
In optimal estimation theory, unbiasedness is a key condition that is used to derive linear and nonlinear estimators. A classical example is the ordinary least squares (OLS) estimator proposed by Gauss in 1795 [1]. The Gauss‐Markov theorem states that if the noise is white and has the same variance at each time step, the OLS estimator is also the best linear unbiased estimator (BLUE) [2]. In convolution‐based optimal filtering, the unbiasedness constraint [2] leads to the unbiased finite impulse response (UFIR) estimator [3, 4]. An extremely useful property of the BLUE and UFIR is that noise statistics are not required. Another example is the maximum likelihood estimator (MLE), which obtains the estimate at an extremum of the density function of the state conditioned on the measurements [5]. Like the BLUE and UFIR, the MLE is suboptimal for finite data. However, if the sample size (memory) increases to infinity, each of them are optimal.
where ‘Var’ is the error variance. Since the minimum MSE is required by many applications, a minimization of (1) is often desired at the expense of a small increase in bias. That leads to different kinds of optimal solutions such as the minimum variance unbiased estimator (MVUE), the recursive Kalman filter[6], and the optimal FIR (OFIR) filter [7, 8]. The common disadvantage of these filters is that noise statistics and initial errors are required. In view of the fact that noise statistics and initial errors are commonly not well known, especially for time‐variant models, theoretically optimal estimators end up being suboptimal in practical applications. In this regard, engineering experience says the following [9]:
Practical implementation of the Kalman filter is often difficult due to the inability in getting a good estimate of the noise covariance matrices.
where x _{ n } indicates a state variable at discrete time step n, ${\widehat{\mathbf{x}}}_{n}$ its estimate, and E{x} is the expected value of x. Note that the cost of equipment that is required for the characterization of noise statistics cannot commonly be afforded by users, and methods for the estimation of noise covariance matrices via measurements are not well developed. On the other hand, noise statistics are not always necessary to get a good estimate that we illustrate below based on an example.
Example 1
There is no need to use optimal estimators in many applications. UFIR structures that ignore noise statistics and initial estimation error statistics are able to produce acceptable suboptimal estimates.
UFIR estimators have attracted researcher’s attention for decades, beginning with the work of Johnson [12] and others, in which they extended the Wiener filter theory to discrete finite time. Further, the ability of UFIR estimators to produce nearly optimal estimates while ignoring noise statistics was greatly regarded in the development of estimators for polynomial signals [13]‐[15]. Most recently UFIR methods were extended to state space in batch form [3, 4, 16]‐[18] and in an iterative Kalman‐like form [19, 20]. The latter has made the UFIR estimator a significant rival of the Kalman filter and its applications can be found in [10, 21]‐[24]. Even so, UFIR estimators still remain somewhat beyond the typical range of traditional signal processing techniques.
In this paper, we develop in part the results achieved in the field of UFIR filtering and review a family of iterative UFIR algorithms for filtering, smoothing, and prediction of time‐varying (TV) and time‐invariant (TI) discrete state‐space models in white Gaussian noise environments. The following definitions will be used: UFIR estimator satisfies the unbiasedness condition (2), OFIR estimator minimizes the MSE (1), and Optimal UFIR (OUFIR) estimator minimizes the MSE in the UFIR estimator by using a window size N _{opt}.
Section 1.2 presents the linear state‐space model, formulates the problem, and considers the batch p‐shift UFIR estimator along with the generalized NPG. Section 1.3 presents two forms of the p‐shift iterative UFIR algorithm. Section 1.4 discusses the estimation errors of the UFIR estimators. Sections 1.5, 1.6, and 1.7 give the reader a number of practical algorithms for filtering, smoothing, and prediction. Section 1.8 considers an extension to nonlinear systems. Section 1.9 discusses methods for the determination of the optimal memory size N _{opt}. Finally, section 1.10 concludes with some useful generalizations.
1.2 Linear model and batch UFIR estimator
where Q _{ n } and R _{ n } may be unknown to the engineer.
Now suppose that the p‐shift estimate^{a} ${\widehat{\mathbf{x}}}_{n+p|n}$ of x _{ n } is provided at time n+p with the UFIR estimator proposed in [19, 20]. We would like to modify this estimator and review engineering algorithms for different kinds of filtering, q‐lag smoothing, and p‐step prediction. We also wish to estimate the estimation errors and generalize the properties to facilitate a comparison with the OFIR and Kalman algorithms.
1.2.1 Time‐variant models
In convolution‐based filtering (3), we suppose that measurements z _{ n } are available on a time horizon of N points (memory^{b}), from time m=n−N+1 to time n, that the estimator is causal, and that m⩾0. In order to find ${\widehat{\mathbf{x}}}_{n+p}$ in state space, the batch p‐shift UFIR estimator [8, 20] can be applied. For TV models, the p‐shift UFIR estimator was derived in [8], assuming that the negative shift p is no smaller than −N+1. Below, we modify this estimator for arbitrary p, which is needed for one of the smoother forms.
One can notice that (10) is reminiscent of the familiar OLS or BLUE, although the matrices are different.
where A _{ n,m }(p) is the UFIR estimator gain; and p can be arbitrary, − ∞ ⩽ p ⩽ ∞. In the case of −N+1<p<0, one may also use a particular form of (17b) shown in [19], (21) with ${\mathcal{B}}_{n,m}\left(p\right)={\mathcal{F}}_{n+p,0}^{m+1}$.
This suggests that prediction and smoothing can be organized based on the filtering estimate (18) if we use an auxiliary p‐shift gain matrix. We will show below that (19) plays an important part in the design of UFIR algorithms.
1.2.2 Time‐invariant models
A distinctive feature of both TV and TI batch UFIR estimators is that they can be applied to models with noise having arbitrary distributions and covariances. They can also be represented in fast iterative Kalman‐like forms using an auxiliary matrix called the generalized NPG (GNPG), which will be discussed next.
1.2.3 Generalized noise power gain
where the product A(N)A ^{ T }(N) is known as the NPG [11].
to characterize the noise strength at the estimator output. In particular, if the GNPG is an identity matrix, then no noise reduction is provided by the estimator. If the GNPG has components that are equal to zero, then the noise is fully suppressed by the estimator.
Summarizing the generalizations provided for the batch UFIR estimator, we notice again that this estimator ignores noise statistics and initial errors in solving the problems of smoothing, filtering, and prediction in a unified scheme. Its important applied property is that the estimate becomes virtually optimal when N≫1 [20]. On the other hand, large N leads to computational problems owing to the large dimensions of the augmented matrices and vectors. For fast computation, iterative Kalman‐like UFIR forms can be used, which will be discussed next.
1.3 Iterative Kalman‐like UFIR estimation
Similar to the recursive OLS [28], the UFIR estimator can also be represented in a fast iterative form similar to the Kalman filter as shown in [19, 20]. The iterative UFIR estimator requires that we start with initial values that are available from the batch algorithm, which typically requires matrix computations on the order of K×K dimensions, and then we iteratively update the estimator output. The state estimate is taken when an iterative variable reaches the current time n.
1.3.1 Time‐varying models
For TV models, the estimates (17b) and (19) suggest two forms of iterative UFIR computation.
1.3.1.0 The direct form
where s=m+K−1; and the iterative variable l ranges from m+K to n. The estimator output is taken when l=n. Since the UFIR estimate does not require initial conditions, one may approximately set (40) to zero and let (41) be the identity when N≫1. However, this simplification may not always lead to good estimates for smaller values of N.
The estimate at time n+p appears in (36) from an iterative update beginning with time step m+K+p−1. Its flaw is that |p| past points before the N‐point estimator window are formally required for smoothing. This disadvantage is overcome in the two‐stage form, which will be discussed next.
1.3.1.0 The two‐stage form
Here, s=m+K−1 and l ranges from m+K to n, as before.
As can be seen, this second form available does not require extra data points before the filtering window. However, it requires two computational steps, unlike the direct form (36).
1.3.2 Time‐invariant models
Employing (20b) and (24), the p‐shift estimate for TI models can also be found in two equivalent iterative forms.
1.3.2.0 The direct form
where s=m+K−1 and l ranges from m+K to n. The desired state estimate is taken at l=n.
1.3.2.0 The two‐stage form
One may conclude that the algorithm of (53) and (58) is very simple from a programming perspective. As was shown in [8, 19, 20] and in many other studies, the UFIR estimator is a strong rival to the Kalman filter if the noise covariances are not known exactly.
1.4 Estimation errors
In spite of the fact that the UFIR estimator has two equivalent forms (batch and iterative), the MSE can rigorously be determined only via the batch form. Finding closed analytical solutions for the MSE via (19) and (25) implies a large mathematical burden and is still a challenging problem. On the other hand, a rigorous error computation may be unnecessary since estimation error covariances are not used in the UFIR algorithms, and so reasonable approximations can serve us well in practical applications. Such an approximation provided following [23] is given in the Appendix.
The MSE upper bound (UB) ${\mathbf{P}}_{n+p}^{\text{UB}}$ can be obtained from an iterative computation of (114) for the general TV model. Equation (114) implies that process noise covariances are accumulated at each iteration. Therefore, the predicted value from (114) is a bit larger than the actual estimation error covariance for small N and significantly larger for N≫1. For the same reason, the estimate of (114) also diverges as p increases. The UB can thus be very useful for filtering (p=0) when N is not large and for smoothing with small lags. In the case of prediction, the future noise is neglected in (114) so it can serve as a tight upper bound even for very large p.
The MSE lower bound (LB) can be found if we take into consideration the fact that if N⩽N _{opt} the UFIR estimator order fits the system order. Therefore the system noise can be neglected in (114) and the LB ${\mathbf{P}}_{n+p}^{\text{LB}}$ can be found by iterating
Equations (61) and (62) correspond to the direct estimator forms of (36) and (48) respectively.
where ${\mathbf{P}}_{n}^{\text{LB}}$ is provided from (63) with l=n. Note that the LB is associated with the NPG and serves well in the three‐sigma sense [27].
1.5 Filtering
where s=m+K−1, ${\mathcal{F}}_{s,0}^{m+1}=\prod _{i=0}^{K-2}{\mathbf{F}}_{s-i}$, the iteration index l ranges from m+K to n, and the estimate of the current state is taken when l=n.
with the initial value ${\mathbf{P}}_{l-1}^{\text{LB}}$ specified as for the Kalman filter.
where the initial value ${\mathbf{P}}_{l-1}^{\text{LB}}$ can also be specified as in the Kalman filter.
It can easily be shown that (71) is the Kalman a posteriori estimate covariance, if we substitute K _{ l } with the Kalman gain. However, unlike the Kalman filter, (66) can be applied to deterministic models. If that is the case (R _{ l }=0 and Q _{ l }=0), then the estimation error is zero.
Several particular filtering solutions can now be discussed, which will be done in the following sections.
1.5.1 Fixed‐horizon filtering
Fixed-horizon TV UFIR filtering algorithm
Stage | |
---|---|
Given: | K, N, m=n−N+1⩾0, s=m+K−1, |
m+K⩽l⩽n. | |
Set: | ${\widehat{\mathbf{x}}}_{s}$ by (69) and G _{ s } by (70). |
Update: | ${\mathbf{G}}_{l}={[{\mathbf{H}}_{l}^{T}{\mathbf{H}}_{l}+{\left({\mathbf{F}}_{l}{\mathbf{G}}_{l-1}{\mathbf{F}}_{l}^{T}\right)}^{-1}]}^{-1}$, |
${\widehat{\mathbf{x}}}_{l}={\mathbf{F}}_{l}{\widehat{\mathbf{x}}}_{l-1}+{\mathbf{G}}_{l}{\mathbf{H}}_{l}^{T}({\mathbf{z}}_{l}-{\mathbf{H}}_{l}{\mathbf{F}}_{l}{\widehat{\mathbf{x}}}_{l-1})$. | |
Instruction: | Use the estimate when l=n. |
It is implied that measurements are available beginning at time index 0, and the time index n starts at n−1. The initial values ${\widehat{\mathbf{x}}}_{s}$ and G _{ s } are computed using (69) and (70), respectively. For each n, the iterative variable l increments from m+K to n, and the desired estimate is taken when l=n. Note that the estimation error computed by (71) is minimal if one sets N=N _{opt}. A simplification for the TI model is straightforward. One must just let all of the matrices be TI in Table 1.
1.5.2 Full‐horizon filtering
Full-horizon TV UFIR filtering algorithm
Stage | |
---|---|
Given: | K, n⩾K. |
Set: | ${\widehat{\mathbf{x}}}_{K-1}$ by (69) and G _{ K−1} by (70) for m=0. |
Update: | ${\mathbf{G}}_{n}={[{\mathbf{H}}_{n}^{T}{\mathbf{H}}_{n}+{\left({\mathbf{F}}_{n}{\mathbf{G}}_{n-1}{\mathbf{F}}_{n}^{T}\right)}^{-1}]}^{-1}$, |
${\widehat{\mathbf{x}}}_{n}={\mathbf{F}}_{n}{\widehat{\mathbf{x}}}_{n-1}+{\mathbf{G}}_{n}{\mathbf{H}}_{n}^{T}({\mathbf{z}}_{n}-{\mathbf{H}}_{n}{\mathbf{F}}_{n}{\widehat{\mathbf{x}}}_{n-1})$. |
This algorithm is the most simple. It requires only the number of the states K since the filter memory window size changes with time; so, N=n+1. No additional information is needed, and the algorithm thus has extremely desirable engineering features. A natural extension of the TV algorithm (Table 2) to the TI case is provided by removing the time dependencies from the matrices.
The MSE UB and LB can be computed by (71) and (73) if we substitute l with n. Note that the full‐horizon UFIR filter may demonstrate substantial decrease in the output noise as n becomes large.
1.5.3 Tricky‐horizon filtering
The tricky‐horizon (time‐variant memory size N) algorithm can be used in adaptive filtering [29, 30] and whenever some reference information about the process behavior is available. It implies an individual N _{opt} at each time index n. Such flexibility allows better system tracking with minimum residuals [19]. To implement tricky‐horizon filtering, the algorithm (Table 1) can be used if we allow N to be variable.
1.6 Smoothing
Smoothing is commonly associated with a lag q>0 relating the estimate at a given time index to measurements up to and including some past index. By combining ‘future’ and past estimates, it becomes possible to obtain better noise reduction for many practical applications. Note that an infinity of smoother solutions exists [31]. We will discuss two basic schemes for UFIR smoothers in this section.
The direct form
where ${\mathcal{Y}}_{l}={\mathcal{F}}_{l,0}^{l-q}=\prod _{i=0}^{q}{\mathbf{F}}_{l-i}$ and l ranges from m+K to n. The estimate ${\widehat{\mathbf{x}}}_{n-q}$ is traditionally taken at l=n in each iterative cycle.
The two‐stage form
where ${\stackrel{\u0304}{\mathcal{B}}}_{n,m}\left(q\right)\triangleq {\stackrel{\u0304}{\mathcal{B}}}_{n,m}(N,q)$ is given by (77).
where ${\mathbf{P}}_{n}^{\text{LB}}$ is provided by (63) at l=n. As in filtering, here, the LB can serve well in the three‐sigma sense [27].
1.6.1 Fixed‐interval smoothing
Among various smoothing problems, the fixed‐interval one is basic and often associated with smoothing [25, 32]‐[34]. The fixed‐interval UFIR smoother is intended to provide an estimate ${\widehat{\mathbf{x}}}_{n-q|n}$ with any lag 0<q<M utilizing measurement on a fixed interval of M points, from time index n−M+1 to n. Although M may not be equal to N _{opt}, UFIR smoothing is most efficient when M=N _{opt}. In fact, If M>N _{opt}, smoothing is inefficient when N _{opt}<q<M, because q exceeds the length of the averaging interval and smoothing virtually provides the backward prediction, which has an estimation error larger than in filtering. On the other hand, N _{opt} should not be larger than M, because M is commonly associated with an available database.
Provided M=N _{opt}, two traditional forms can be suggested for fixed‐interval UFIR smoothing.
The direct form
Direct fixed-interval TV OUFIR smoothing algorithm
Stage | |
---|---|
Given: | K, N=constant, q, m=n−N+1⩾0, |
s=m+K−1, m+K⩽l⩽n. | |
Set: | ${\widehat{\mathbf{x}}}_{s}$ by (75) and G _{ s } by (76). |
Update: | ${\mathbf{G}}_{l}={[{\mathbf{H}}_{l}^{T}{\mathbf{H}}_{l}+{\left({\mathbf{F}}_{l}{\mathbf{G}}_{l-1}{\mathbf{F}}_{l}^{T}\right)}^{-1}]}^{-1}$, |
${\mathbf{K}}_{l}={\left(\prod _{i=0}^{q-1}{\mathbf{F}}_{l-i}\right)}^{-1}{\mathbf{G}}_{l}{\mathbf{H}}_{l}^{T}$, | |
${\widehat{\mathbf{x}}}_{l-q}={\mathbf{F}}_{l-q}{\widehat{\mathbf{x}}}_{l-q-1}+{\mathbf{K}}_{l}\left({\mathbf{z}}_{l}-{\mathbf{H}}_{l}\prod _{i=0}^{q}{\mathbf{F}}_{l-i}{\widehat{\mathbf{x}}}_{l-q-1}\right)$. | |
Instruction: | The algorithm is valid for any n⩾N−1+q. Use the |
smoothed estimate when l=n. The fixed interval | |
of M=N=N _{opt} points is from time index m to n. |
The two‐stage form
Two-stage fixed-interval TV OUFIR smoothing algorithm
Stage | |
---|---|
Given: | K, N=constant, q, m=n−N+1⩾0, |
s=m+K−1, m+K⩽l⩽n. | |
Set: | ${\widehat{\mathbf{x}}}_{s}$ by (75) and G _{ s } by (76). |
Update: | ${\mathbf{G}}_{l}={\mathbf{H}}_{l}^{T}{\mathbf{H}}_{l}+{\left({\mathbf{F}}_{l}{\mathbf{G}}_{l-1}{\mathbf{F}}_{l}^{T}\right)}^{-1}{]}^{-1}$, |
${\widehat{\mathbf{x}}}_{l}={\mathbf{F}}_{l}{\widehat{\mathbf{x}}}_{l-1}+{\mathbf{G}}_{l}{\mathbf{H}}_{l}^{T}({\mathbf{z}}_{l}-{\mathbf{H}}_{l}{\mathbf{F}}_{l}{\widehat{\mathbf{x}}}_{l-1})$. | |
Use ${\widehat{\mathbf{x}}}_{n}$ when l=n and compute | |
${\widehat{\mathbf{x}}}_{n-q}={\stackrel{\u0304}{\mathcal{B}}}_{n,m}\left(q\right){\left({\mathcal{F}}_{n,0}^{m+1}\right)}^{-1}{\widehat{\mathbf{x}}}_{n}$. | |
Instruction: | This algorithm is valid for any n⩾N−1. The fixed |
interval of M=N=N _{opt} points is from time | |
index m to n. |
1.6.2 Fixed‐lag smoothing
Fixed‐lag smoothing is commonly used for denoising if a time delay of q points is allowed [31, 32, 35, 36]. Two basic fixed‐lag algorithms can be designed based on the UFIR technique.
Fixed‐lag OUFIR smoothing
Provided N _{opt}, the fixed lag q can be specified based on the process properties to obtain the best denoising. Intuition indicates that smoothing is best if the estimation time is the center of the observation interval. This holds true if the polynomial describing the process behavior on the observation interval is of odd degree. Otherwise, if the degree is even, denoising may be better with shorter lags as shown in Figure eight in [17]. The fixed‐lag OUFIR smoothing algorithm is listed in Table 4 if one sets N=N _{opt} and q=constant. Its extension to the TI case can be provided by replacing the ${\widehat{\mathbf{x}}}_{n-q}$ equation in Table 4 with ${\widehat{\mathbf{x}}}_{n-q}={\mathbf{F}}^{-q}{\widehat{\mathbf{x}}}_{n}$.
Fixed‐lag full‐horizon UFIR smoothing
Fixed-lag full-horizon TV UFIR smoothing algorithm
Stage | |
---|---|
Given: | K, q=constant, n⩾K. |
Set: | ${\widehat{\mathbf{x}}}_{K-1}$ by (75) and G _{ K−1} by (76) for m=0. |
Update: | ${\mathbf{G}}_{n}={[{\mathbf{H}}_{n}^{T}{\mathbf{H}}_{n}+{\left({\mathbf{F}}_{n}{\mathbf{G}}_{n-1}{\mathbf{F}}_{n}^{T}\right)}^{-1}]}^{-1}$, |
${\widehat{\mathbf{x}}}_{n}={\mathbf{F}}_{n}{\widehat{\mathbf{x}}}_{n-1}+{\mathbf{G}}_{n}{\mathbf{H}}_{n}^{T}({\mathbf{z}}_{n}-{\mathbf{H}}_{n}{\mathbf{F}}_{n}{\widehat{\mathbf{x}}}_{n-1})$. | |
Compute ${\widehat{\mathbf{x}}}_{n-q}$ for n⩾q as | |
${\widehat{\mathbf{x}}}_{n-q}={\stackrel{\u0304}{\mathcal{B}}}_{n,m}\left(q\right){\left({\mathcal{F}}_{n,0}^{m+1}\right)}^{-1}{\widehat{\mathbf{x}}}_{n}$. |
1.6.3 Fixed‐point smoothing
Fixed-point TV UFIR smoothing algorithm
Stage | |
---|---|
Given: | K, v=constant⩾0, q=n−v, n⩾K. |
Set: | ${\widehat{\mathbf{x}}}_{K-1}$ by (75) and G _{ K−1} by (76) for m=0. |
Update: | ${\mathbf{G}}_{n}={[{\mathbf{H}}_{n}^{T}{\mathbf{H}}_{n}+{\left({\mathbf{F}}_{n}{\mathbf{G}}_{n-1}{\mathbf{F}}_{n}^{T}\right)}^{-1}]}^{-1}$, |
${\widehat{\mathbf{x}}}_{n}={\mathbf{F}}_{n}{\widehat{\mathbf{x}}}_{n-1}+{\mathbf{G}}_{n}{\mathbf{H}}_{n}^{T}({\mathbf{z}}_{n}-{\mathbf{H}}_{n}{\mathbf{F}}_{n}{\widehat{\mathbf{x}}}_{n-1})$. | |
Compute ${\widehat{\mathbf{x}}}_{n-q}$ for n>v as follows: | |
${\widehat{\mathbf{x}}}_{n-q}={\stackrel{\u0304}{\mathcal{B}}}_{n,m}\left(q\right){\left({\mathcal{F}}_{n,0}^{m+1}\right)}^{-1}{\widehat{\mathbf{x}}}_{n}$. |
1.7 Prediction
State prediction plays a key role in many applications. The one‐step predictor is fundamental for digital feedback control systems [38]. It is also commonly provided when measurements are unavailable at some points [39] and when estimates of long‐term future behavior are required [40]. Predictive estimation is necessary for global positioning system (GPS)‐based applications when the GPS receiver temporarily fails or when a signal is temporarily unavailable [27]. Predictive estimation is required for holdover in digital communication networks [41], for maintaining normal functioning of certain systems during down time [42, 43], and for astronomy and climate forecasting. The predictor goal is thus to compensate for unavailable information. In many cases, linear predictors do perform better than nonlinear ones [44].
1.7.1 Fixed‐step prediction
In the fixed‐step case shown in Figure 3a, p is often unity, but in general may be arbitrary (p>0). With p=1, prediction can permanently substitute for unavailable measurements with predicted values.
The direct form
where ${\mathcal{F}}_{s+1,0}^{m+1}=\prod _{i=0}^{K-1}{\mathbf{F}}_{s+1-i}$, s=m+K−1, and l ranges from m+K to n. The desired estimate is obtained when l=n.
Both predictors can be implemented in the algorithm (Figure 3a) to satisfy the following condition: if z _{ n } is unavailable at time n, then set ${\mathbf{z}}_{n}={\mathbf{H}}_{n}{\widehat{\mathbf{x}}}_{n}$ for a TV model and ${\mathbf{z}}_{n}=\mathbf{H}{\widehat{\mathbf{x}}}_{n}$ for a TI one.
The two‐stage form
where ${\widehat{\mathbf{x}}}_{n}$ is the filter estimate. This is the most widely used prediction scheme.
1.7.2 Variable‐step prediction
In the variable‐step case illustrated in Figure 3b, the predicted estimates still compensate for unavailable measurements (points 4, 5, 6), but they are not involved to produce predictions, which is unlike the case of Figure 3a. Instead, p continues to increment until the measurement becomes available. At point 7, all measured and predicted values on a horizon of N _{opt} past points are used to produce a prediction at point 8a. There are no other differences between fixed‐step and variable‐step prediction, and the estimates (36), (47), (48), and (58) can be used in a straightforward manner, along with the relevant error bounds.
1.8 Nonlinear models and extended filtering
where f _{ n }(x _{ n−1}) and h _{ n }(x _{ n }) are time‐varying nonlinear vector functions and all other notations are given in (6) and (7). If f _{ n }(x _{ n−1}) and h _{ n }(x _{ n }) are smooth enough, then the first‐order Taylor series expansion is often applied to make the model approximately linear between two neighboring points.
With (96) and (97) linearized in this way, UFIR filtering can be applied as shown below.
1.8.1 Iterative EFIR filtering
EUFIR filtering algorithm for TV models
Stage | |
---|---|
Given: | K, N, m=n−N+1, |
s=m+K−1, m+K⩽l⩽n. | |
Set: | ${\widehat{\mathbf{x}}}_{s}$ by (69) and G _{ s } by (70) |
Update: | F _{ l } by (100), H _{ l } by (101), |
${\mathbf{G}}_{l}={[{\mathbf{H}}_{l}^{T}{\mathbf{H}}_{l}+{\left({\mathbf{F}}_{l}{\mathbf{G}}_{l-1}{\mathbf{F}}_{l}^{T}\right)}^{-1}]}^{-1}$, | |
${\mathbf{K}}_{l}={\mathbf{G}}_{l}{\mathbf{H}}_{l}^{T}$, | |
${\widehat{\mathbf{x}}}_{l}={\mathbf{f}}_{l}\left({\widehat{\mathbf{x}}}_{l-1}\right)+{\mathbf{K}}_{l}[{\mathbf{z}}_{l}-{\mathbf{h}}_{l}({\widehat{\mathbf{x}}}_{l}^{-}\left)\right]$. | |
Remark: | Use the estimate when l=n. |
As can be seen, it has the same structure as Table 1, except for the nonlinear functions in the estimate. Although the EUFIR algorithm traditionally does not use noise statistics or initial error statistics, the estimation error covariance may be required to characterize the performance. An analysis of error covariances is given in [46]. Note that, in contrast to the first‐order expansion, (98) and (99), the second‐order expansion involves noise statistics. However, as in the extended Kalman filter [28], the higher order expansion typically does not lead to a definitive advantage [46].
1.9 Memory for OUFIR estimators
The window size N plays an important role in UFIR estimators. If N<N _{opt}, denoising appears to be inefficient: the random error dominates, although bias is negligible. If N>N _{opt}, the random error is small, but bias affects the estimate.
Estimation of N _{opt} is still a challenging mathematical problem that requires finding the derivative of the estimate with respect to the convolution length N. Even so, there are several available approaches. For low‐degree polynomial models, N _{opt} was found analytically in [47]. A more general approach employing the bandlimited property of signals was developed in [20]. Finally, an efficient algorithm exploiting measurements was recently proposed in [48]. In any case, it is much simpler to estimate a scalar N _{opt}, rather than accurately estimating matrices Q _{ n } and R _{ n } as is required in the Kalman filter.
1.9.1 Bandlimited signals
In real applications, a measured signal is causal and bandlimited with some maximum frequency W. By the Shannon theorem, the maximum sampling interval that prevents aliasing is T=1/2W. If measurements are obtained with sampling interval T, then only N=K points are available for averaging by the K‐state FIR estimator. If we use larger N, then the estimate will be biased. In order to avoid bias, we would need the model to be represented with a larger number of states, and such a model may not be acceptable or available.
where ⌊x⌋ means the maximum integer that is less than or equal to x. A simple analysis shows that if N>N _{opt}, aliasing causes a bias. If N<N _{opt}, noise reduction is inefficient.
1.9.2 Known reference model
It has been shown in [48] that by increasing n, the first minimum in both (106) and (107) corresponds to N _{opt}. The problem, however, arises when the reference model x _{ n } is unknown, as it usually is.
1.9.3 Unknown reference model
where V _{(k k)n } is the (k k)th component of V _{ n }. The minimization is performed by increasing n, starting with K−1, until the first minimum. To avoid ambiguities when minimizing these functions, the number of points used in the expected value must be sufficiently large, and smoothing of the objective function may be desirable.
1.10 Some generalizations and conclusions
Critical evaluation of the Kalman, OFIR, and OUFIR filters
Kalman | Batch OFIR | Iterative OFIR | Batch OUFIR | Iterative OUFIR | |
---|---|---|---|---|---|
[6] | |||||
Optimality: | Optimal | Optimal | Optimal | Unbiased | Unbiased |
Initial conditions: | A priori | A posteriori | A posteriori | Ignored | A posteriori |
Noise statistics: | Required | Required | Required | Ignored | Ignored |
Noise characteristics: | White | Arbitrary | White | Arbitrary | Arbitrary |
System model: | Stochastic | Arbitrary | Arbitrary | Arbitrary | Arbitrary |
Filter memory (points): | 2 | N _{opt} | N _{opt} | N _{op} | N _{opt} |
Stability: | May diverge | BIBO | BIBO | BIBO | BIBO |
Operation: | Fast | Slow | Medium | Medium | Approximately N _{opt} times slower than |
Kalman; Fast with parallel computing | |||||
Memory consumption: | Small | Large | Medium | Large | Approximately N _{opt} times more than |
Kalman | |||||
Computational complexity: | Low | High | Medium | Medium | Low |
1.10.1 OUFIR vs. OFIR
Beginning with the early limited memory filter of Jazwinski [5], OFIR filtering has been under development for several decades. In control theory, fundamental progress was achieved by Kwon et al. and his followers[7, 35, 49]‐[53]. In signal processing, solutions were found by Shmaliy et al.[8, 20, 27]. It was shown in[52] that different kinds of limited memory filters[5, 54] are equivalent to the OFIR one. The most serious flaws of this technique are high computational complexity and high memory consumption. With such poor engineering features, OFIR estimators still have not gained currency and their development remains mostly at a theoretical level.
Fast‐ and low‐complexity iterative OUFIR algorithms that ignore noise statistics and initial error statistics are practically superior to the best‐known OFIR ones.
Note that this deduction often holds even if N is small. But in some applications, OFIR filters can be more appropriate because of their better accuracy.
1.10.2 OUFIR vs. Kalman filter
The well‐known features of the Kalman filter are optimality, fast computation, and low memory consumption. However, the Kalman filter requires a priori initial condition and noise statistics, and this is recognized as the most annoying flaw of the Kalman filter. Because of this requirement, the Kalman filter is suboptimal for all practical purposes. Moreover, its optimality is guaranteed only if the noise sources are white, which is not the case for many applications. Finally, the Kalman filter applies only to stochastic models.
In turn, the iterative OUFIR filter ignores noise statistics (except for the zero‐mean assumption), allows the noise to have any distribution and covariance, exhibits BIBO stability, and serves for both stochastic and deterministic models. However, it does not guarantee optimality, especially when N _{op} is small. It requires (N _{opt}−1)‐times more computational time and needs about N _{opt} times more memory than the Kalman filter.
Note that the error difference Δ between the two filters decreases with increasing N _{opt}. These observations by diverse authors who have investigated uncertainties, different kinds of noise sources, and other irregular perturbations result in the following important inference:
Under the real‐world operating conditions, and when noise statistics and initial error statistics are not known exactly, the OUFIR estimator is able to outperform the Kalman filter even if N _{opt} is not large.
Simulation results confirming these observations can be found in [19, 23, 46].
2 Conclusions
The UFIR algorithms discussed in this paper cover many applied problems associated with filtering, smoothing, and prediction of discrete‐time state‐space models. The most general conclusions one may arrive at by analyzing these estimators are the following: 1) UFIR algorithms are able to provide nice suboptimal estimates that are acceptable for many applications; 2) The optimal window size N _{opt} can easily be estimated experimentally; 3) The extra time required by the UFIR iterations can be alleviated with parallel computing; and 4) The extra memory required by the UFIR estimators is not a problem for modern computers. So, we conclude that UFIR algorithms are strong rivals to the Kalman filter for real‐world applications. The iterative UFIR estimator commonly outperforms the OFIR one even if N _{opt} is not large, and it is able to outperform the Kalman filter under real‐world operating conditions and when the noise statistics are not known exactly. That makes UFIR algorithms highly attractive for applications. We see the following main trends in further developments of FIR estimators. Optimal FIR filtering and smoothing strongly require fast Kalman‐like algorithms which are similar to those developed for UFIR estimators and considered in this paper. Such algorithms are required for small N _{opt}. In turn, iterative UFIR algorithms need further optimization and robustification in non‐Gaussian environments and under the uncertainties. Special attention should also be paid to fast algorithms for the determination of N _{opt}. Provided such modifications, one may expect new efficient FIR solutions.
Endnotes
^{a} ${\widehat{\mathbf{x}}}_{n+p|n}$ means the estimate at time n+p given measurements up to and including time n. Here, p=0 corresponds to filtering, p>0 corresponds to p‐step prediction, and p<0 corresponds to q‐lag smoothing, where q=−p. We simplify notation by using ${\widehat{\mathbf{x}}}_{n+p}\triangleq {\widehat{\mathbf{x}}}_{n+p|n}$.
^{b} In different applications, the FIR estimator memory is also called the receding horizon[53], sliding window[55], averaging interval[56], etc.
Appendix
The covariance upper bound for TV models
Declarations
Authors’ Affiliations
References
- Gauss CF: Theory of the Combination of Observations Least Subject to Errors. Philadelphia: SIAM Publ; 1995. Transl. by Stewart, GWView ArticleGoogle Scholar
- Kay SM: Fundamentals of Statistical Signal Processing. New York: Prentice Hall; 2001.Google Scholar
- Kim PS, Lee ME: A new FIR filter for state estimation and its applications. J. Comput. Sc. Techn 2007, 22: 779-784. 10.1007/s11390-007-9085-8View ArticleGoogle Scholar
- Shmaliy YS: An unbiased FIR filter for TIE model of a local clock in applications to GPS‐based timekeeping. IEEE Trans. Ultrason., Ferroel. Freq. Contr 2006, 53: 862-870.View ArticleGoogle Scholar
- Jazwinski AH: Stochastic Processes and Filtering Theory. New York: Academic Press; 1970.Google Scholar
- Kalman RE: A new approach to linear filtering and prediction problems. J. Basic Eng 1960, 82: 35-45. 10.1115/1.3662552View ArticleGoogle Scholar
- Kwon OK, Kwon WH, Lee KS: FIR filters and recursive forms for discrete‐time state‐space models. Automatica 1989, 25: 715-728. 10.1016/0005-1098(89)90027-7MathSciNetView ArticleGoogle Scholar
- Shmaliy YS, Ibarra‐Manzano O: Time‐variant linear optimal finite impulse response estimator for discrete‐time state‐space models. Int. J. Adapt. Contr. Signal Process 2012, 26: 95-104. 10.1002/acs.1274MathSciNetView ArticleGoogle Scholar
- Gibbs B: Advanced Kalman Filtering, Least‐Squares and Modeling. New York: Wiley; 2011.View ArticleGoogle Scholar
- Ferrari P, Flammini A, Rinaldi S, Bondavalli A, Brancati F: Experimental characterization of uncertainty sources in a software‐only synchronization system. IEEE Trans. Instrum. Meas 2012, 61: 1512-1521.View ArticleGoogle Scholar
- Blum M: On the mean square noise power of an optimum linear discrete filter operating on polynomial plus white noise input. IRE Trans. Inform. Theory 1957, 3: 225-231. 10.1109/TIT.1957.1057423View ArticleGoogle Scholar
- Johnson KR: Optimum, linear, discrete filtering of signals containing a nonrandom component. IRE Trans. Inform. Theory 1956, 2: 49-55. 10.1109/TIT.1956.1056784View ArticleGoogle Scholar
- Heinonen P, Neuvo Y: FIR‐median hybrid filters with predictive FIR structures. IEEE Trans. Acoust. Speech Signal, Process 1988, 36: 892-899. 10.1109/29.1600View ArticleGoogle Scholar
- Ovaska SJ, Vainio O, Laakso TI: Design of predictive IIR filters via feedback extension of FIR forward predictors. IEEE Trans. Instrum. Meas 1997, 46: 1196-1201. 10.1109/19.676741View ArticleGoogle Scholar
- Samadi S, Nishihara A: Explicit formula for predictive FIR filters and differentiators using Hahn orthogonal polynomials. IEICE Trans. Fundamentals 2007, E90‐A: 1511-1518.View ArticleGoogle Scholar
- Kim PS: An alternative FIR filter for state estimation in discrete‐time systems. Digital Signal Process 2010, 20: 935-943. 10.1016/j.dsp.2009.10.033View ArticleGoogle Scholar
- Shmaliy YS, Morales‐Mendoza LJ: FIR smoothing of discrete‐time polynomial signals in state space. IEEE Trans. Signal Process 2010, 58: 2544-2555.MathSciNetView ArticleGoogle Scholar
- Shmaliy YS: Unbiased FIR filtering of discrete‐time polynomial state‐space models. IEEE Trans. Signal Process 2009, 57: 1241-1249.MathSciNetView ArticleGoogle Scholar
- Shmaliy YS: An iterative Kalman‐like algorithm ignoring noise and initial conditions. IEEE Trans. Signal Process 2011, 59: 2465-2473.MathSciNetView ArticleGoogle Scholar
- Shmaliy YS: Linear optimal FIR estimation of discrete time‐invariant state‐space models. IEEE Trans. Signal Process 2010, 58: 3086-3096.MathSciNetView ArticleGoogle Scholar
- Balbuena DH, Sergiyenko O, Tyrsa V, Burtseva L, Lopez MR: Signal frequency measurement by rational approximations. Measurement 2009, 42: 136-144. 10.1016/j.measurement.2008.04.009View ArticleGoogle Scholar
- Levine J: Invited Review Article: The statistical modeling of atomic clocks and the design of time scales. Rev. Sc. Instr 2012, 83: 021101-1–021101‐28. 10.1063/1.3681448View ArticleGoogle Scholar
- Simon D, Shmaliy YS: Unified forms for Kalman and finite impulse response filtering and smoothing. Automatica 2013, 49: 1892-1899. 10.1016/j.automatica.2013.02.026MathSciNetView ArticleGoogle Scholar
- Wang WQ: Phase noise suppression in GPS‐disciplined frequency synchronization systems. Fluctuation Noise L 2011, 10: 303-313. 10.1142/S0219477511000582View ArticleGoogle Scholar
- Rauch HE, Tung F, Striebel CT: Maximum likelihood estimates of linear dynamic systems. AIAA J 1965, 3: 1445-1450. 10.2514/3.3166MathSciNetView ArticleGoogle Scholar
- Stark H, Woods JW: Probability, Random Processes, and Estimation Theory for Engineers. Upper Saddle River: Prentice Hall; 1994.Google Scholar
- Shmaliy YS: GPS‐based Optimal FIR Filtering of Clock Models. New York: Nova Science Publ.; 2009.Google Scholar
- Simon D: Optimal State Estimation: Kalman, H∞ and Nonlinear Approaches. New York: Wiley; 2006.View ArticleGoogle Scholar
- Gustafsson F: Adaptive Filtering and Change Detection. New York: Wiley; 2000.Google Scholar
- Haykin S: Lessons on adaptive systems for signal processing, communications, and control. IEEE Signal Process. Mag 1999, 16: 39-48.View ArticleGoogle Scholar
- Moore JB: Discrete‐time fixed lag smoothing algorithms. Automatica 1973, 9: 163-173. 10.1016/0005-1098(73)90071-XView ArticleGoogle Scholar
- Bar‐Shalom Y, Li XR, Kirubarajan T: Estimation with Applications to Tracking and Navigation. New York: Wiley; 2001.View ArticleGoogle Scholar
- Helmick RE, Blair WD, Hoffman SA: Fixed‐interval smoothing for Markovian switching systems. IEEE Trans. Inform. Theory 1995, 41: 1845-1855. 10.1109/18.476310View ArticleGoogle Scholar
- Kim JH, Lyou J: Target state estimator design using FIR filter and smoother. Trans. Control, Automation, and Syst. Eng 2002, 4: 305-310.Google Scholar
- Ahn CK, Kim PS: Fixed‐lag maximum likelihood FIR smoother for state‐space models. IEICE Electonics Express 2008, 5: 11-16. 10.1587/elex.5.11View ArticleGoogle Scholar
- Biswas KK, Mahalanabis AK: An approach to fixed‐lag smoothing problems. IEEE Trans. Aerospace Electron. Syst 1972, AES‐8: 676-682.View ArticleGoogle Scholar
- Theodor Y, Shaked U: Game theory approach to H∞‐optimal discrete‐time fixed‐point and fixed‐lag smoothing. IEEE Trans. Autom. Contr 1994, 39: 1944-1948. 10.1109/9.317131MathSciNetView ArticleGoogle Scholar
- Vaccaro RJ: Digital Control: A State‐Space Approach. New York: McGraw‐Hill; 1995.Google Scholar
- Carlin BP, Polson NG, Stoffer DS: A Monte‐Carlo approach to nonnormal and nonlinear state‐space modeling. J. Am. Statist. Assoc 1992, 87: 493-500. 10.1080/01621459.1992.10475231View ArticleGoogle Scholar
- Lamothe M, Auclair M, Hamzaoui C, Huot S: Towards a prediction of long‐term anomalous fading of feldspar IRSL. Radiat. Meas 2003, 37: 493-498. 10.1016/S1350-4487(03)00016-7View ArticleGoogle Scholar
- ITU: Timing Characteristics of Primary Reference Clocks. : ITU‐T Recommendation G.811; 1997.Google Scholar
- Levine J: Time synchronization over the Internet using an adaptive frequency‐locked loop. IEEE Trans. Ultrason., Ferroelect., Freq. Contr 1999, 46: 888-896.View ArticleGoogle Scholar
- Iwata T, Imae M, Suzuyama T, Murakami H, Kawasaki Y: Simulation and ground experiments of remote synchronization system for on‐board crystal oscillator of Quazi‐Zenith Satelite System. J. Inst. Navig 2006, 53: 231-235.View ArticleGoogle Scholar
- Makhoul J: Linear prediction: A tutorial review. Proc. IEEE 1975, 63: 561-580.View ArticleGoogle Scholar
- Cox H: On the estimation of state variables and parameters for noisy dynamic systems. IEEE Trans. Autom. Contr 1964, 9: 5-12. 10.1109/TAC.1964.1105635View ArticleGoogle Scholar
- Shmaliy Y S: Suboptimal FIR filtering of nonlinear models in additive white Gaussian noise. IEEE Trans. Signal Process 2012, 60: 5519-5527.MathSciNetView ArticleGoogle Scholar
- Shmaliy YS, Muñoz‐Diaz J, Arceo‐Miquel L: Optimal horizons for a one‐parameter family of unbiased FIR filters. Digital Signal Process 2008, 18: 739-750. 10.1016/j.dsp.2007.10.002View ArticleGoogle Scholar
- Ramirez‐Echeverria F, Sarr A, Shmaliy YS: Optimal memory for discrete‐time FIR filters in state‐space. IEEE Trans. Signal Process 2013. in pressGoogle Scholar
- Han SH, Kwon WH, Kim PS: Quasi‐deadbeat minimax filters for deterministic state‐space models. IEEE Trans. Automat. Contr 2002, 47: 1904-1908.MathSciNetView ArticleGoogle Scholar
- Ahn CK: Strictly passive FIR filtering for state‐space models with external disturbance. Int. J. Electron. Commun 2012, 66: 944-948. 10.1016/j.aeue.2012.04.002View ArticleGoogle Scholar
- Ahn CK, Han SH: New H∞ FIR smoother for linear discrete‐time state‐space models. IEICE Trans. Commun 2010, E91.B: 896-899.View ArticleGoogle Scholar
- Kwon WH, Suh YS, Lee YI, Kwon OK: Equivalence of finite memory filters. IEEE Trans. Aerosp. Electron. Syst 1994, 30: 968-972. 10.1109/7.303774View ArticleGoogle Scholar
- Han S, Kwon W H: Receding Horizon, Control: Model Predictive Control for State Models. London: Springer; 2005.Google Scholar
- Schweppe FC: Uncertain Dynamic Systems. New York: Prentice‐Hall; 1973.Google Scholar
- Papadias CB, Slock DTM: Normalized sliding window constant modulus and decision‐directed algorithm: a link between blind equalization and classical adaptive filtering. IEEE Trans. Signal Process 1997, 45: 231-235. 10.1109/78.552221View ArticleGoogle Scholar
- Treicher J, Larimore M: New processing technique based on the constant modulus adaptive algorithm. IEEE Trans. Acoust., Speech, Signal Process 1985, 33: 420-431. 10.1109/TASSP.1985.1164567View ArticleGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.