Effect of embedded unbiasedness on discrete-time optimal FIR filtering estimates

Unbiased estimation is an efficient alternative to optimal estimation when the noise statistics are not fully known and/or the model undergoes temporary uncertainties. In this paper, we investigate the effect of embedded unbiasedness (EU) on optimal finite impulse response (OFIR) filtering estimates of linear discrete time-invariant state-space models. A new OFIR-EU filter is derived by minimizing the mean square error (MSE) subject to the unbiasedness constraint. We show that the OFIR-UE filter is equivalent to the minimum variance unbiased FIR (UFIR) filter. Unlike the OFIR filter, the OFIR-EU filter does not require the initial conditions. In terms of accuracy, the OFIR-EU filter occupies an intermediate place between the UFIR and OFIR filters. Contrary to the UFIR filter which MSE is minimized by the optimal horizon of Nopt points, the MSEs in the OFIR-EU and OFIR filters diminish with N and these filters are thus full-horizon. Based upon several examples, we show that the OFIR-UE filter has higher immunity against errors in the noise statistics and better robustness against temporary model uncertainties than the OFIR and Kalman filters.


Introduction
Beginning with the works by Gauss [1], unbiasedness plays a role of the necessary condition that is used to derive linear and nonlinear estimators [2]. In statistics and signal processing, the ordinary least squares (OLS) estimator proposed by Gauss in 1795 is an unbiased estimator. By the Gauss-Markov theorem [3], this estimator is also the best linear unbiased estimator (BLUE) [4] if noise is white and if it has the same variance at each time step [5]. The unbiasedness is obeyed by a condition E{x k } = E{x k } which means that the average of estimatex k is equal to that of the model x k . It leads to the unbiased finite impulse response (UFIR) estimator [6]. Of practical importance is that neither OLS nor UFIR require the noise statistics which are not always known to the engineers [7]. The unbiasedness condition, however, does not guarantee "good estimate" [8]. Therefore, the sufficient conditionminimized noise variance-is often applied along to produce different kinds of estimators which are optimal in the minimum mean square error (MSE) sense or suboptimal: Bayesian, maximum likelihood (MLE), minimum *Correspondence: shmaliy@ugto.mx 2 Department of Electronics Engineering, Universidad de Guanajuato, Salamanca 36885, Mexico Full list of author information is available at the end of the article variance unbiased (MVU), etc. In recent decades, a new class of estimators having FIR (filters, smoothers, and predictors) was developed to have optimal or suboptimal properties.
The FIR filter utilizes finite measurements over the most recent time interval (horizon) of N discrete points. Compared to the filters with infinite impulse response (IIR), such as the Kalman filter (KF) [9], the FIR filter exhibits some useful engineering features such as the bounded input/bounded output (BIBO) stability [10], robustness against temporary model uncertainties and round-off errors [11], and lower sensitivity to noise [12]. The most noticeable early works on optimal FIR (OFIR) filtering are [13][14][15]. At that time, FIR filters were not the ones commonly used for state estimation due to the analytical complexity and large computational burden. Nowadays, the interest to FIR estimators has grown owing to the tremendous progress in the computational resources. Accordingly, we find a number of new solutions on FIR filtering [16][17][18][19][20][21], smoothing [22][23][24], and prediction [25][26][27] as well as efficient applications [28][29][30].
Basically, the unbiasedness can be satisfied in two different strategies: (1) one may test an estimator by the unbiasedness condition or (2) one may embed the unbiasedness constraint into the design. We therefore recognize below the checked (tested) unbiasedness (CU) and the embedded unbiasedness (EU). Accordingly, we denote the FIR filter with CU as FIR-CU and the FIR filter with EU as FIR-EU.
In state estimation, signal processing, tracking, and control, two different state-space models are commonly used. The prediction model which is basic in control is x k+1 = Ax k + Bw k and y k = Cx k + Dv k , in which w k and v k are noise vectors, and A, B, C and D are relevant matrices. Employing this model, the receding horizon FIR estimators were proposed for different types of unbiasedness. In [16], the receding horizon FIR-CU filter was derived from KF with no requirements for the initial state. Soon after, a receding horizon FIR-EU filter was proposed by Kwon, Kim, and Han in [17], where the unbiasedness condition was considered as a constraint to the optimization problem. Later, the receding horizon FIR smoothers were found in [22] for CU by employing the maximum likelihood and in [24] for EU by minimizing the error variance.
The real-time state model x k = Ax k−1 + Bw k is used in signal processing when the prediction is not required (different time index) [31,32]. Employing this model, the FIR-CU filter and smoother were proposed by Shmaliy in [23,33] for polynomial systems. In [12], a p-shift unbiased FIR filter (UFIR) was derived as a special case of the OFIR filter. Here, the unbiasedness was checked a posteriori, and the solution thus belongs to CU. Soon after, the UFIR filter [12] was extended to time-variant systems [18,34]. For nonlinear models, an extended UFIR filter was proposed in [35] and unified forms for FIR filtering and smoothing were discussed in [36]. An important advantage of the UFIR filter against OFIR filter is that the noise statistics are not required. Because noise reduction in FIR structures is provided by averaging, N 1 makes the UFIR filter as successful in accuracy as the OFIR filter.
It has to be remarked now that all of the aforementioned FIR estimators related to real-time state-space model belong to the CU solutions. Still no optimal FIR estimator was addresses of the EU type. It is thus unclear which kind of FIR estimators serves better in particular applications [37][38][39]. So, there is still room for discussion of the best FIR filter.
In this paper, we systematically investigate effect of the embedded unbiasedness on OFIR estimates. To this end, we derive a new FIR filter, called OFIR-EU filter, by minimizing the MSE subject to the unbiasedness constraint. We also learn properties of the OFIR-EU filter in a comparison with the OFIR and UFIR filters and KF. The remaining part of the paper is organized as follows. In Section 2, we describe the model and formulate the problem. The OFIR-EU filter is derived in Section 3. Here, we also consider a unified form for different kinds of OFIR filters. In Section 4, we generalize several FIR filters and discuss special cases of the OFIR-EU filter. The MSEs are compared analytically in Section 5. Extensive simulations are provided in Section 6, and concluding remarks are drawn in Section 7.
The following notations are used: R n denotes the ndimensional Euclidean space; E{·} denotes the expected value; diag (e 1 · · · e m ) represents a diagonal matrix with diagonal elements e 1 , · · · , e m ; tr M is the trace of M; and I is the identity matrix of proper dimensions.

Preliminaries and problem formulation
Consider a linear discrete-time model given with the state-space equations in which k is the discrete time index, x k ∈ R n is the state vector, and y k ∈ R p is the measurement vector. Matrices A ∈ R n×n , B ∈ R n×u , C ∈ R p×n and D ∈ R p×v are timeinvariant and known. We suppose that the process noise w k ∈ R u and the measurement noise v k ∈ R v are zero mean, E{w k } = 0 and E{v k } = 0, mutually uncorrelated, and have arbitrary distributions and known covariances for all i and j, to mean that w k and v k are not obligatorily white Gaussian.
Following [12], the state-space model (1) and (2) can be represented in a batch form on a discrete time interval [l, k] with recursively computed forward-in-time solutions as where l = k − N + 1 is a start point of the averaging horizon. The time-variant state vector X k,l ∈ R Nn×1 , observation vector Y k,l ∈ R Np×1 , process noise vector W k,l ∈ R Nu×1 , and observation noise vector V k,l ∈ R Nv×1 are specified as, respectively, The extended model matrix A k−l ∈ R Nn×n , process noise matrix B k−l ∈ R Nn×Nu , observation matrix C k−l ∈ R Np×n , auxiliary matrix H k−l ∈ R Np×Nu , and measurement noise matrix D k−l ∈ R Np×Nv are all timeinvariant and dependent on the horizon length of N points. Model (1) and (2) suggests that these matrices can be written as, respectively, ) .
Note that at the start horizon point we have an equation x l = x l + Bw l which is satisfied uniquely with zero-valued w l , provided that B is not zeroth. The initial state x l must thus be known in advance or estimated optimally.
The FIR filter applied to N past neighboring measurement points on a horizon [l, k] can be specified witĥ wherex k|k is the estimate 1 , and K k is the FIR filter gain determined using a given cost criterion. Note that a distinctive difference between the FIR with IIR filters is that only one nearest past measurement is used in the recursive IIR (Kalman) filter to provide the estimate, while the convolution-based batch FIR filter requires N most recent measurements.
The estimate (15) will be unbiased if to obey the following unbiasedness condition, in which x k can be specified as if to combine (3) and (4). HereB k−l is the first vector row in B k−l . By substituting (15) and (17) into (16), replacing the term Y k,l with (4), and providing the averaging, one arrives at the unbiasedness constraint which is also known as the deadbeat constraint [19]. Providedx k|k , the instantaneous estimation error e k can be defined as The problem now formulates as follows. Given the models, (1) and (2), we would like to derive an OFIR-EU filter by minimizing the variance of the estimation error (19) as We also wish to investigate effect of the unbiasedness constraint (18) on the OFIR-EU estimate, compare errors in different kinds of FIR filters, and analyze the trade-off between the OFIR-EU filter derived in this paper, UFIR filter [33], OFIR filter [34], and KF under the diverse operation conditions.

OFIR-EU filter
In the derivation of the OFIR-EU filter, the following lemma will be used.  (21) is

Lemma 1. The trace optimization problem is given by
Proof. The proof is provided in Appendix A.

The gain for OFIR-EU filter
Using the trace operation, the optimization problem (20) can be rewritten as subject to (18), where (· · · ) denotes the term that is equal to the relevant preceding term. By substituting x k with (17) andx k|k with (15), the cost function becomes If to take into account constraint (18), provide the averaging, and rearrange the terms, (25) can be transformed to where the fact is invoked that the system noise vector W k,l and the measurement noise vector V k,l are pairwise independent. The auxiliary matrices are Referring to Lemma 1 with θ = 1, the solution to the optimization problem (26) can be obtained by neglecting L, M, and P and using the replacements: where in which The OFIR-EU filter structure can now be summarized in the following theorem. Theorem 1. Given the discrete time-invariant state space model (1) and (2) with zero mean mutually independent and uncorrelated noise vectors w k and v k , the OFIR-EU filter utilizing measurements from l to k is stated byx is the measurement vector given by (6), and K OEUa k and K OEUb k are given by (30) and (31) with C k−l andB k−l specified by (11) and the first row vector of (10), respectively.
Note that the horizon length N for (35) should be chosen such that the first inverse in (30) exists. In general, N can be set as N n, where n is the number of the model states. Table 1 summarizes the steps in the OFIR-EU estimation algorithm, in which the noise statistics are assumed to be known for measurements available from l to k.
Given N, compute K OEUa k and K OEUb k according to (30) and (31), respectively, then the OFIR-EU estimate can be obtained at time index k by (35).

Unified form for OFIR and OFIR-EU filters
In order to ascertain a correspondence between the OFIR filter and its modifications associated with the unbiasedness constraint (18), we rewrite the optimization problem (24) regarding the unified gain K UO k as is the mean square of initial state x l . Using Lemma 1 and substituting we find a solution to (36) as  (31) Compute: In a special case of θ = 1, (37) reduces to (42) where k−l is given by (38), in which¯ x+w+v is specified by (39) with θ = 1. Referring to (30) and (31) and taking into consideration that the second term on the right-hand side of (42) equals to zero, we come up with a deduction that In the unconstrained case of θ = 0, (37) transforms to By multiplying x with identity C T k−l C k−l −1 C T k−l C k−l from the left-hand side, (44) turns up as where the unbiased gain K U k is defined by [6] We thus infer that this case corresponds to the OFIR filter which gain was found in [34]. At this point, we notice that (37) is a unified generalized form for the OFIR filter gain which minimize the MSE in the estimate of discrete time-invariant state-space model. In this regard, the OFIR filter gain derived in [34] and OFIR-EU filter gain specified by Theorem 1 can be considered as special cases of (37).

MVU FIR filter
Owing to its unique properties, the unbiasedness constraint (18) has been employed extensively to derive different kinds of FIR filters [6,[15][16][17]23]. The UFIR filter was shown in [12] to be a special case of the OFIR filter with the unbiased gain specified by (46), where N is chosen as N n to guarantee the invertibility of C T k−l C k−l . The gain (46) can also be obtained by multiplying A N−1 in the constrain (18) from the right-hand side with the identity matrix C T k−l C k−l −1 C T k−l C k−l and neglecting C k−l in both sides. In this sense, the UFIR filter is akin to Gauss's OLS. On the other hand, (46) does not guarantee optimality in the MSE sense. An optimized solution can be provided by minimizing the error variance that leads to the minimum variance unbiased (MVU) FIR filter [40]. Since the properties of the MVU FIR filter are in-between the UFIR and OFIR filters, a unified form for the UFIR filter can also be assumed. Below, we specify the MVU FIR filter and show a unified relationship between the UFIR, MVU FIR, and OFIR-EU filter gains.

Identity of MVU FIR and OFIR-EU filters
It has been shown in [40] that the variance can be minimized in the UFIR filter if to represent the gain of the MVU FIR filter K MVU k as a linear combination of K U k given by (46) and an auxiliary term K a k of the same class, where On the other hand, Lemma 1 suggests that K MVU k does not depend on the initial state matrix x . Any x can thus be supposed in (50), provided that the inverse in (50) exists. This fundamental property was postulated in many papers [11,17,23,33] and, based upon, K MVU k can be rewritten equivalently as where and k−l is given by (32). Referring to (31) and making some rearrangements, we arrive at an aquality which is formalized below with a theorem.

Theorem 2. The MVU FIR filter specified by (47) is identical to the OFIR-EU filter specified by Theorem 1,
Proof. The proof is given in Section 4.1.
It follows from Theorem 2 that the gain K MVU k is not unique. One may suppose any initial state matrix x , compute it by solving the discrete algebraic Riccati equation (DARE) as in [12], or even neglect x as we have done above. Although each of these cases require particular algorithms, Lemma 1 suggests that the estimation accuracy will not be affected by x . We notice that this property of MVU FIR filter was unknown so far. We use it below while comparing different kinds of unbiased FIR filters.

Unified form for UFIR and MVU FIR filters
The basic UFIR filter gain found in [12] is given by (46). There can be found other forms of this gain if to multiply A N−1 in the constraint (18) from the right-hand side with an appropriate identity matrix and remove C k−l from the both sides. The unbiased gain K UU k produced in such a way depends on an auxiliary matrix Z k−l , provided that its inverse exists. However, a class of UFIR filters associated with Z k−l must have some reasonable formulation which can be the following. Let us combine K UU k with two additive components of the same class as where κ can be either 0 or 1, Depending on values of κ and k−l , the following special cases can be recognized: Several other generalizations can also be made regarding the types of systems:

Deterministic state model
If the state model (1) is noiseless, then the term containing w should be omitted in (30) and (31), and (29) reduces to the gain which becomes equals to K UU k with κ = 0 and k−l = −1 v . This gain corresponds to the traditional BLUE and MLE for Gaussian models [5]. The batch form (59) was also shown in [11] for the receding horizon FIR filter with embedded unbiasedness and minimized variance.

Deterministic measurement model
If the observation model (2) is noise-free, one has which is a special case of (55) by κ = 1 and k−l = −1 w .

Deterministic state-space model
Having no noise in (1) and (2), the cost function in (25) becomes By the constraint (18), the terms in the parentheses of (61) become identically zero. Hence, the solution to (61) is the unbiased gain K k given by (46). It then follows that The UFIR filter is a deadbeat filter for deterministic systems.
If (18) is not applied, then the solution to (61) becomes which can also be obtained by setting the terms w and v in (45) to zero. We thus infer that The OFIR filter is a deadbeat filter for deterministic systems. Table 2 summarizes the gains for the UFIR, OFIR-EU (MVU FIR), and OFIR filters. Note that all these filter gains are given in the batch form, where the computational complexity is large when the estimation horizon is long. Therefore, corresponding iterative realization is required for a fast computation.

Estimation errors
Provided a correspondence between the OFIR, OFIR-EU (MVU FIR), and UFIR filter gains (Table 2), in this section, we proceed with an analysis of the estimation errors. We compare the MSEs of these filters and point out their common features and differences.

Mean square errors
The MSE J k at the estimator output can be defined as k|k , (64) where each of the mean square values can be decomposed via the squared bias and variance. Assuming that the actual x k is inherently unbiased, we write E{x k x T k } = Var(x k ) and E x k|kx T k|k = Bias 2 (x k|k ) + Var(x k|k ). We further decompose the estimatex k|k asx k|k = Bias(x k|k )+ x k|k , wherex k|k is a random part ofx k|k , find and finally transforme (64) to where the state variance Var (x k ) is specified by and, for unbiased estimate, we have Based upon (65), below we specify the MSEs for the above considered FIR filters.

MSE in the UFIR estimate
For the UFIR filter, the third term Var(x k|k ) on the righthand side of (65) can be transformed to Taking into account that W k,l and V k,l are mutually independent, the covariance Cov(x k ,x k|k ) can be obtained as Accordingly, the MSE in the UFIR filter becomes where K U k is given by (46). The MSE (70) was first studied in [18].

MSE in the OFIR-EU estimate
For the OFIR-EU filter, Var(x k|k ) and Cov(x k ,x k|k ) are given by, respectively, From (54) we have K OEU Next, substituting (66), (73) and (74) into (65) and rearranging the terms yield Finally, by invoking ϒ k−l given by (53), we transform (75) to in which J U k is provided by (70).

MSE in the OFIR estimate
We first notice that the OFIR filter gain K O k given by (45) can equivalently be rewritten as For this filter, the bias-dependent term becomes Now, by combining (65), (68), and (69), the MSE of the OFIR filter can be found to be The MSE (79) was first studied in [12,34]. If we further substitute K O k with (77), refer to (70), and rearrange the terms, we arrive at the final form The above-provided relations (70), (76), and (80) allow analyzing effect of the unbiasedness constraint on the OFIR-filtering estimates that we provide below.

Correspondence between the MSEs
A general relationship between the MSEs associated with different FIR filters is ascertained by the following theorem.
and it becomes an equality when the state-space model is deterministic.
Proof. The proof is given in [40] and we support it with a simple analysis. The UFIR filter is designed to obtain zero bias. Although the noise variance is reduced here as ∝ 1 N , the optimality is not guaranteed. Therefore, the MSE in UFIR filter generally exceeds those in two other filters. The MSE in the OFIR filter is minimal among other filters. The OFIR-EU filter minimizes MSE with the embedded unbiasedness. Its error is thus in between the UFIR and OFIR filters.

Applications
Theorem 3 states that the OFIR-EU and MVU FIR filters produce intermediate estimates between the OFIR and UFIR filters. In order to learn the effect of the embedded unbiasedness in more detail, we test the UFIR, OFIR-EU, and OFIR filters in line with the KF in different noise environments by a two-state polynomial model specified with The reader can also find some other comparisons of the KF and FIR filters in [16,18,34,41].

Accurate model-ideal case
In an ideal case, one may think that the model represents a process accurately and the noise statistics are known exactly. The goal then is to learn the effect of the horizon length N on the FIR estimates. We set the measurement noise variance as σ 2 v = 10, and the initial states as x 10 = 1 and x 20 = 0.01 / s.
We then compute the root MSE (RMSE) of the estimate by tr J k as a function of N. The results are illustrated in Fig. 1 for σ 2 w = 1 and in Fig. 2 Fig. 1 with N opt = 33 and Fig. 2 with N opt = 47). -Because the MSEs in the OFIR and OFIR-EU filters diminish with N, these filters are full-horizon [18].

Filtering with errors in the noise statistics
The noise statistics required by the KF are commonly not completely know to the engineer. In order to investigate the effect of the imprecisely defined noise covariances in the worst case, we introduce a correction coefficient p as p 2 Q and R / p 2 , vary p from 0.1 to 10, and plot the RMSE √ tr J k as shown in Fig. 3. Note that the MSE functions of optimal filters are inherently concave on p with a minimum at p = 1 and the MSE of the UFIR filter is p-invariant.
As expected, p = 1 makes the OFIR filter, OFIR-EU filter, and KF a bit more accurate than the UFIR filter. But, that is only within a narrow range of p (0.6 < p < 1.5 in Fig. 3) that the KF slightly outperforms the UFIR filter. Otherwise, the UFIR filter demonstrates smaller errors. Referring to practical difficulties in the determination of noise statistics [7], the latter can be considered as an important engineering advantage of the UFIR filter. Some other generalizations also emerge from Fig. 3: -The embedded unbiasedness makes the OFIR-EU filter p-invariant with p < 1. In this sense, the OFIR-EU is equal here to the UFIR filter, and this can be considered as a particular meaningful property of the approach proposed. -With p < 1, the KF is more sensitive to errors in the noise statistics than the FIR filters. -By p > 1, the MSEs in the KF, OFIR filter, and OFIR-EU filter grow and converge.
Overall, we conclude that the OFIR-EU filter inherits the robustness of the UFIR filte against the noise statistics and has better performance than the OFIR filter and KF.

Filtering with model uncertainties
To learn effect of the temporary model uncertainties on the filtering accuracy, in this section we set τ = 0.1 s when 160 k 180 and τ = 0.05 s otherwise. The noise variances are allowed to be σ 2 w1 = 1, σ 2 w2 = 1/s 2 , and σ 2 v = 10. The process is simulated at 400 subsequent points.
Typical filtering estimates are sketched in Fig. 4. As can be seen, the OFIR-EU filter (case p = 0.2) and the UFIR filter produce almost equal errors and demonstrate good robustness against the uncertainties. Just on the contrary, the KF demonstrates much worse robustness for any p 1.

Conclusions
Summarizing, we notice that the unbiasedness imbedded to the OFIR filter instills into it several useful properties. Unlike the OFIR filter, the OFIR-EU filter completely ignores the initial conditions. The OFIR-EU filter is equivalent to the MVU FIR filter. In terms of accuracy, the OFIR-EU filter is in between the UFIR and OFIR filters. Unlike in the UFIR filter which MSE is minimized by N opt , MSEs in the OFIR-EU and OFIR filters diminish with N and these filters are thus full-horizon. The Fig. 4 Instantaneous estimation errors caused by the temporary model uncertainties with p < 1 for the KF, UFIR filter, and OFIR-EU filter performance of OFIR-EU filter is developed by varying the horizon N around N opt or ranging the correction coefficient p around p = 1. Accordingly, the OFIR-EU filter in general demonstrates higher immunity against errors in the noise statistics and better robustness against temporary model uncertainties than the OFIR filter and KF.
Referring to the fact that optimal FIR filters are essentially the full-horizon filters but their batch forms are computationally inefficient, we now focus our attention on the fast iterative form for OFIR-EU filter and plan to report the results in near future. Endnote 1x k|k means the estimate at k via measurements from the past to k.
In the case of θ = 0 which is denoted as case (d), the derivative of ϕ i|d with respect to k i|d becomes where d = a , and yields Then K d can be found to be Finally, by observing that when F = U and L = U, and using θ as an indicating parameter of the constraint, matrices K a , K b , K c , and K d can be unified with where is specified by (23). An equivalent form of (100) is (22) and the proof is complete.