Prediction on widely factorizable signals

Abstract

This article extends the solutions to the prediction problem for factorizable real random signals to the class of improper complex-valued random signals. For that, the concept of widely factorizable signals is introduced and several real examples of signals having a widely factorizable correlation function are presented. A widely linear processing is considered in the design of both linear and nonlinear prediction algorithms which are computationally feasible from the practical standpoint. These algorithms are valid for stationary as well as nonstationary signals and they can be applied from only the knowledge of the second-order properties of the augmented vectors involved, it not being necessary to know if the signal satisfies a state-space model.

1 Introduction

In recent research, estimation theory becomes very relevant within the complex random signals field. In fact, although traditionally the treatment of this type of problem has consisted in mere extensions of the vectorial real-valued estimation algorithms to the complex plane, a complex formalism has a special value in the description of some physical systems in such diverse fields as communications, oceanography, meteorology and optics among others (see, for example, [1] and the references therein).

The classic processing with such signals, called strictly linear (SL), assumes that the signals involved are proper.^a However, this assertion is not always justified. The improper nature of some signals requires that the pseudocorrelation function must be taken into account in describing and characterizing their second order properties completely [2–4].

With this motivation, [2] introduces a new methodology called widely linear (WL) whose more notable characteristic is that it utilizes not only the observed signal but also its conjugate to obtain estimators with better behavior than the conventional ones in the sense of reducing the mean square estimation error. Moreover, this kind of processing has become very usual in the last decade for designing linear and nonlinear estimation algorithms from a discrete-time [1, 5–9] as well as a continuous-time perspective of the problem [10]. Specifically, focussing our attention on the discrete case, the recent books of Mandic and Goh [1] and Adali and Haykin [11] about WL adaptive systems can be considered as two reference texts in this area which provide a unified treatment of linear and nonlinear complex-valued adaptive filters.

On the other hand, knowledge of second-order statistics is a key assumption in solving estimation problems (see, for example, [12]). In practice this knowledge is available because second-order statistics of the problem have been measured experimentally or they are understandable enough from the physical mechanism [13]. In this framework, WL estimation algorithms for computing all types (filtering, prediction, and smoothing) of estimates have been devised in [8] for second-order stationary (SOS) signals, i.e., those signals with constant mean function and both correlation and pseudocorrelation functions only dependent on the difference of time instants. Although the WL estimation problem considered is very general, its applicability is limited to SOS signals.

In the real field, an alternative estimation methodology based on correlation information has been developed to solve very general linear and nonlinear estimation problems for the class of factorizable signals (see, for example, [14, 15]). This type of signal is characterized by having a factorizable kernel which is a very general condition valid for stationary and nonstationary signals. This formulation does not require postulating a dynamical model for the signal and thus, it is useful whenever the physical mechanism generating the signal of interest is not known or if it impossible to determine (for instance, if it does not satisfy a state-space model).

In this article, our objective is to extend this last methodology to the complex plane. For that, we introduce a new class of signals, called widely factorizable, whose correlation function of the augmented vector formed by the signal and its conjugate is a factorizable kernel. Then, a WL processing is employed in the development of both linear and nonlinear prediction algorithms for widely factorizable signals observed in the presence of noise. With this aim, in Section 2 widely factorizable signals are defined and also the basic notation and concepts about complex-valued signals are summarized. Next, the linear augmented complex prediction problem is addressed in Section 3 where a recursive algorithm is provided for computing the optimal WL prediction estimate as well as its associated error. This section also includes three numerical examples which show the enhancement of the proposed WL predictor in relation to the SL solution. Finally, by following a similar reasoning to the extended Kalman filter (EKF), the nonlinear augmented complex prediction problem is solved in Section 4 where a numerical example is also developed in order to compare the proposed WL algorithm with two conventional nonlinear augmented complex techniques, the WL EKF and the WL unscented Kalman filter (UKF) suggested in [1]. Furthermore, these solutions are also compared with the SL EKF and SL UKF.

2 Preliminaries

This section covers some of the more basic notions associated with the complex-valued random signals field and introduces a new class of signals, called widely factorizable. Moreover, the notation and hypotheses held throughout the article are also established in this section.

First of all, note that all vectors will be denoted by bold small letters and bold capital letters will be used for matrices. Also, row k of any matrix A(·) will be denoted by a_[k](·). Furthermore, 0 _p and 0 _{p × q} represent the p-vector and the p × q-matrix, respectively, whose elements are all zeros.

Unless indicated to the contrary, throughout this article we consider an improper complex-valued signal {s(t _i ), t _i ∈ T}, T = {t₁, t₂, ...}, with zero-mean and correlation function r _s (t _i , t _j ) = E[s(t _i )s* (t _j )], where the superscript '*' represents the complex conjugate.

The signal {s(t _i ), t _i ∈ T} is said to be factorizable if there exist two l-vectors α(t _i ) and β(t _i ) such that its correlation function r _s (t _i , t _j ) can be expressed in the form

$$r_s\left(t_i,t_j\right)=\left\{\begin{array}{c}{\mathit{\alpha }}^{\text{T}}\left(t_i\right){\mathit{\beta }}^{*}\left(t_j\right),t_i\ge t_j\\ {\mathit{\beta }}^{\text{T}}\left(t_i\right){\mathit{\alpha }}^{*}\left(t_j\right),t_i\le t_j\end{array}\right.$$

(1)

where the superscript 'T' denotes the transpose. Note that this type of signal is very general and includes both stationary and nonstationary signals. However, a new class of signals is possible by imposing the condition of factorizable kernel on the correlation function R _s (t _i , t _j ) = E[s(t _i )s^H(t _j )], with the superscript 'H' denoting the conjugate transpose, of the augmented vector s(t _i ) = [s(t _i ), s* (t _i )] ^T. This type of signal, called widely factorizable, is introduced next.

Definition 2.1 A signal {s(t _i ), t _i ∈ T} is said to be widely factorizable if and only if there exist two 2 × m-matrices A(t _i ) and B(t _i ) such that the correlation function R _s (t _i , t _j ) of the augmented vector s(t _i ) can be expressed in the form

$${\mathbf{R}}_{\mathbf{s}}\left(t_i,t_j\right)=\left\{\begin{array}{c}\mathbf{A}\left(t_i\right){\mathbf{B}}^H\left(t_j\right),t_i\ge t_j\\ \mathbf{B}\left(t_i\right){\mathbf{A}}^H\left(t_j\right),t_i\le t_j\end{array}\right.$$

(2)

where the superscript 'H' denotes the conjugate transpose.

Note that condition (2) implies (1), however condition (1) does not assure that the correlation function of the augmented vector satisfies (2). As a consequence, we have that all widely factorizable signals are also factorizable but the converse does not hold.

Some illustrative examples of widely factorizable signals are the following:

i) The rotation of a real factorizable zero-mean signal x(t _i ) by an independent random phase θ, s(t _i ) = e^{θ j}x(t _i ), with $\text{j}\text{ = }\sqrt{-1}$. Assume

$$r_x\left(t_i,t_j\right)=\left\{\begin{array}{c}{\mathit{a}}^{\mathrm{T}}\left(t_i\right)\mathit{b}\left(t_j\right),t_i\ge t_j\\ {\mathit{b}}^{\mathrm{T}}\left(t_i\right)\mathit{a}\left(t_j\right),t_i\le t_j\end{array}\right.$$

(3)

with a(t _i ) and b(t _i ) two real l-vectors. Thus,

$$\mathbf{A}\left(t_i\right)=\left(\begin{array}{cc}\hfill 0_l^{\mathrm{T}}\hfill & \hfill {\mathit{a}}^{\mathrm{T}}\left(t_i\right)\hfill \\ \hfill {\mathit{a}}^{\mathrm{T}}\left(t_i\right)\hfill & \hfill 0_l^{\mathrm{T}}\hfill \end{array}\right),{\mathbf{B}}^H\left(t_i\right)=\left(\begin{array}{cc}\hfill {\mathbf{\phi }}_{\mathbf{\theta }}\left(-2\right)\mathit{b}\left(t_i\right)\hfill & \hfill \mathit{b}\left(t_i\right)\hfill \\ \hfill \mathit{b}\left(t_i\right)\hfill & \hfill {\mathbf{\phi }}_{\mathbf{\theta }}\left(2\right)\mathit{b}\left(t_i\right)\hfill \end{array}\right)$$

(4)

where φ _θ (·) is the characteristic function of θ.

ii) The seismic ground acceleration can be represented by a uniformly modulated nonstationary process given by s(t _i ) = d(t _i )x(t _i ), where x(t _i ) is a stationary process with zero mean and known second-order statistics, and d(t _i ) is the time modulating function. Both x(t _i ) and d(t _i ) can be either real or complex-valued [16]. Thus, if d(t _i ) is complex and x(t _i ) is real and factorizable with correlation function given by (3) then, s(t _i ) is widely factorizable being

$$\mathbf{A}\left(t_i\right)=\left(\begin{array}{c}d\left(t_i\right){\mathit{a}}^{\text{T}}\left(t_i\right)\\ d^{*}\left(t_i\right){\mathit{a}}^{\text{T}}\left(t_i\right)\end{array}\right),{\mathbf{B}}^H\left(t_i\right)=\left(\mathit{b}\left(t_i\right)d^{*}\left(t_i\right)\mathit{b}\left(t_i\right)d\left(t_i\right)\right)$$

Common choices are $d\left(t_i\right)=e^{\text{j}2\mathbf{\pi }{\mathbf{\omega }}_0{t}_i}$ with ω₀ a constant frequency and x(t _i ) the Ornstein-Uhlenbeck process.

On the other hand, if both d(t _i ) and x(t _i ) are complex and x(t _i ) is also widely factorizable with

$${\mathbf{R}}_{\mathbf{x}}\left(t_i,t_j\right)\left\{\begin{array}{c}\mathbf{E}\left(t_i\right){\mathbf{F}}^{\text{H}}\left(t_j\right),t_j\ge t_j\\ \mathbf{F}\left(t_i\right){\mathbf{E}}^{\text{H}}\left(t_j\right),t_i\le t_j\end{array}\right.$$

then, s(t _i ) is widely factorizable with

$$\mathbf{A}\left(t_i\right)=\left(\begin{array}{cc}\hfill d\left(t_i\right)\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill d^{*}\left(t_i\right)\hfill \end{array}\right)\mathbf{E}\left(t_i\right),{\mathbf{B}}^{\text{H}}\left(t_i\right)={\mathbf{F}}^{\text{H}}\left(t_i\right)\left(\begin{array}{cc}\hfill d^{*}\left(t_i\right)\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill d\left(t_i\right)\hfill \end{array}\right)$$

iii) In electromagnetic theory, the time-varying position of the electric field vector can be represented as the following signal $s\left(t_i\right)=A{e}^{\mathrm{j}{w}_0{t}_i}+B{e}^{-\mathrm{j}{w}_0{t}_i}$ where the expressions for the random coefficients A and B can be found in [[17], p. 7]. Hence, s(t _i ) is widely factorizable with

$$\mathbf{A}\left(t_i\right)={\left(\begin{array}{c}{r}_{A}e\left(t_i\right)+r_{AB}^{*}{e}^{*}\left(t_i\right){\mathbf{\gamma }}_A^{*}{e}^{*}+{\mathbf{\gamma }}_{AB}^{*}e\left(t_i\right)\hfill \\ r_B{e}^{*}\left(t_i\right)+r_{AB}e\left(t_i\right){\mathbf{\gamma }}_B^{*}e\left(t_i\right)+{\mathbf{\gamma }}_{AB}^{*}{e}^{*}\left(t_i\right)\hfill \\ {\mathbf{\gamma }}_B{e}^{\text{*}}\left(t_i\right)+{\mathbf{\gamma }}_{AB}e\left(t_i\right)r_{B}e\left(t_i\right)+r_{AB}^{*}{e}^{*}\left(t_i\right)\hfill \\ {\mathbf{\gamma }}_{AB}{e}^{*}\left(t_i\right)+{\mathbf{\gamma }}_{A}e\left(t_i\right)r_A{e}^{*}\left(t_i\right)+r_{AB}e\left(t_i\right)\hfill \end{array}\right)}^{\text{T}}$$

(5)

$${\mathbf{B}}^H\left(t_i\right)={\left(\begin{array}{cc}\hfill e\left(t_i\right)\hfill & \hfill e^{*}\left(t_i\right)\hfill \\ \hfill 0\hfill & \hfill 0\hfill \end{array}\begin{array}{cc}\hfill 0\hfill & \hfill 0\hfill \\ \hfill e\left(t_i\right)\hfill & \hfill e^{*}\left(t_i\right)\hfill \end{array}\right)}^{\text{H}}$$

where $e\left(t_i\right)=e^{\text{j}{\mathbf{\omega }}_0{t}_i},r_A=E\left[A{A}^{*}\right],r_{AB}=E\left[A{B}^{*}\right],{\mathbf{\gamma }}_A=E\left[AA\right],{\mathbf{\gamma }}_B=E\left[BB\right]$, and γ _AB = E[AB].

iv) A signal widely used in many areas of signal processing is a linear frequency modulation or chirp with random phase. The chirp process can be expressed as $s\left(t_i\right)=e^{\text{j}\mathbf{\pi }\left(2\mathit{\alpha }{t}_i+\mathit{\beta }{t}_i^2+2\mathbf{\theta }\right)}$ where θ is the random phase, α determines the starting instantaneous frequency of the chirp and β is the chirp rate. Then, the chirp process is widely factorizable with

$$\mathbf{A}\left(t_i\right)=\left(\begin{array}{cc}\hfill c\left(t_i\right)\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill c^{*}\left(t_i\right)\hfill \end{array}\right)$$

$${\mathbf{B}}^{\text{H}}\left(t_i\right)=\left(\begin{array}{cc}\hfill \left(1-{\left|{\phi }_{\theta }\left(2\pi \right)\right|}^2\right)c^{*}\left(t_i\right)\hfill & \hfill \left({\phi }_{\theta }\left(4\pi \right)-{\phi }_{\theta }^2\left(2\pi \right)\right)c\left(t_i\right)\hfill \\ \hfill {\left({\phi }_{\theta }\left(4\pi \right)-{\phi }_{\theta }^2\left(2\pi \right)\right)}^{*}{c}^{*}\left(t_i\right)\hfill & \hfill \left(1-{\left|{\phi }_{\theta }\left(2\pi \right)\right|}^{2}c\left(t_i\right)\right)\hfill \end{array}\right)$$

with $c\left(t_i\right)=e^{\text{j}\mathbf{\pi }\left(2\mathit{\alpha }{t}_i+\mathit{\beta }{t}_i^2\right)}$ and φ _θ (·) the characteristic function of θ.

v) An application of the complex Ornstein-Uhlenbeck process is the description of the motion of the instantaneous axis of the Earth's rotation [18]. This motion has an 1 year period and if it is removed, there remains the so-called Chandler Wobble, which has a period of about 435 days (14 months). Kolmogorov proposed the complex stochastic process $s\left(t_i\right)=x\left(t_i\right)+\text{j}y\left(t_i\right)=\mathbf{\sigma }{e}^{\text{j}2\mathbf{\pi }{t}_i}+\mathbf{\xi }\left(t_i\right)$ to describe the Chandler Wobble, i.e., the motion of the pole, where x(t _i ) and y(t _i ) are the coordinates of the deviation of the instantaneous pole from the North Pole. In that model the first term is a periodical component, and the second term ξ(t _i ) is a complex Ornstein-Uhlenbeck process. It is not difficult to check that the signal s(t _i ) is proper and then,

$$\mathbf{A}\left(t_i\right)=\left(\begin{array}{cc}\hfill \frac{1}{\mathbf{\lambda }}{e}^{\left(\mathbf{\omega }\text{j{}\mathbf{\lambda }\right)t_i}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill \frac{1}{\mathbf{\lambda }}{e}^{-\left(\mathbf{\omega }\text{j - }\mathbf{\lambda }\right)t_i}\hfill \end{array}\right),{\mathbf{B}}^{\text{H}}\left(t_i\right)=\left(\begin{array}{cc}\hfill e^{\left(-\mathbf{\omega }\text{j}+\mathbf{\lambda }\right)t_i}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill e^{\left(\mathbf{\omega }\text{j}+\mathbf{\lambda }\right)t_i}\hfill \end{array}\right)$$

where λ > 0 is the drift parameter and ω ∈ ℝ is the period.

On the other hand, we provide a simple example of a factorizable signal which is not widely factorizable: a zero-mean complex-valued signal {s(t _i ), t _i = i/100, i = 1, ..., 100} with $r_s\left(t_i,t_j\right)=2e^{-\left|t_i-t_j\right|}$ and ${\rho }_s\left(t_i,t_j\right)=\text{j}0.5e^{t_i{t}_j}.$

In the following, we also assume that the signal of interest s(t _i ) is widely factorizable in the sense of Definition 2.1.

Finally, R _sy (t _i , t _j ) = E[s(t _i )y^H(t _j )] denotes the cross-correlation function between any two augmented signals s(t _i ) and y(t _i ), and r _{s y} (t _i , t _j ) = E[s(t _i )y^H(t _j )] represents the cross-correlation function between s(t _i ) and the augmented vector y(t _i ).

3 Linear augmented complex prediction

Assume that the signal s(t _i ) established in Section 2 is observed through the following linear equation:

$$y\left(t_i\right)=g\left(t_i\right)s\left(t_i\right)+v\left(t_i\right),t_1\le t_i\le t_n$$

(6)

where g(t _i ) is a deterministic complex-valued function and v(t _i ) is a doubly white noise^b correlated with s(t _i ). Moreover, the cross-correlation function and the augmented signal s(t _i ) and the augmented noise v(t _i ) is of the form

$${\mathbf{R}}_{\mathbf{s}\mathbf{v}}\left(t_i,t_j\right)=\left\{\begin{array}{c}\mathbf{C}\left(t_i\right){\mathbf{D}}^{\text{H}}\left(t_j\right),t_i\ge t_j\\ \mathbf{E}\left(t_i\right){\mathbf{F}}^{\text{H}}\left(t_j\right),t_i\le t_j\end{array}\right.$$

(7)

where C(t _i ), D(t _i ), E(t _i ), and F(t _i ) are matrices of dimensions 2 × l, 2 × l, 2 × l^', and 2 × l^', respectively.

We consider the problem of obtaining the optimal (in the sense of minimizing the WL mean square error) estimator^c of the signal s(t _k ) as a function of the information given by the observations {y(t₁), ..., y(t _n ), y*(t₁), ..., y*(t _n )}, for t _k ≥ t _n . It is known that such an estimator can be expressed as a linear function of the set of augmented observations {y(t₁), ..., y(t _n )} as follows [2]

$$\mathit{ŝ}\left(t_k|t_n\right)=\sum _{\text{j}=1}^n{\mathbf{h}}^{\text{T}}\left(t_k,t_j,t_n\right)\mathbf{y}\left(t_j\right),t_k\ge t_n$$

(8)

where the 2D vector h(t _k , t _j , t _n ), called the impulse response function, satisfies the equation

$${\text{r}}_{s\mathbf{y}}\left(t_k,t_j\right)=\sum _{i=1}^n{\mathbf{h}}^{\text{T}}\left(t_k,t_i,t_n\right)\mathbf{R}\left(t_i,t_j\right)+{\mathbf{h}}^{\text{T}}\left(t_k,t_j,t_n\right)\mathbf{\sum },t_1\le t_j\le t_n,t_k\ge t_n$$

(9)

where $\mathbf{R}\left(t_i,t_j\right)=\mathbf{G}\left(t_i\right){\mathbf{R}}_s\left(t_i,t_j\right){\mathbf{G}}^{\text{H}}\left(t_j\right)+\mathbf{G}\left(t_i\right){\mathbf{R}}_{\mathbf{s}\mathbf{v}}\left(t_i,t_j\right)+{\mathbf{R}}_{\mathbf{s}\mathbf{v}}^{\text{H}}\left(t_j,t_i\right){\mathbf{G}}^{\text{H}}\left(t_j\right)$, with G(t _i ) denoting the 2 × 2-diagonal matrix G(t _i ) = diag(g(t _i ), g*(t _i )), and E[v(t _i )v^H(t _i )] = Σ.

From (2) and (7), it is easy to check that r _{s y} (t _k , t _j ) and R(t _i , t _j ) can be written as follows:

$${\mathbf{r}}_{s\mathbf{y}}\left(t_k,t_j\right)=\left\{\begin{array}{c}{\mathit{\psi }}_{\left[1\right]}\left(t_k\right){\mathbf{\Gamma }}^{\text{H}}\left(t_j\right),t_k\ge t_j\\ {\mathbf{\pi }}_{\left[1\right]}\left(t_k\right){\mathbf{\Phi }}^{\text{H}}\left(t_j\right),t_k\le t_j\end{array}\right.$$

(10)

$$\mathbf{R}\left(t_i,t_j\right)=\left\{\begin{array}{c}\mathbf{\Phi }\left(t_i\right){\mathbf{\Gamma }}^{\text{H}}\left(t_j\right),t_i\ge t_j\\ \mathbf{\Gamma }\left(t_i\right){\mathbf{\Phi }}^{\text{H}}\left(t_j\right),t_i\le t_j\end{array}\right.$$

(11)

where ψ_[1](t _k ) is the first row of the 2 × q-matrix $\mathbf{\Psi }\left(t_i\right)=\left[\mathbf{A}\left(t_i\right),\mathbf{C}\left(t_i\right),0_{2\times l^{\mathbf{\prime }}}\right]$, π_[1](t _k ) is the first row of the 2×q-matrix Π(t _i ) = [B(t _i ), 0_2×l, E(t _i )], Φ(t _i ) = [G(t _i )A(t _i ), G(t _i ) C(t _i ), F(t _i )], and Γ(t _i ) = [G(t _i ) B(t _i ), D(t _i ), G(t _i ) E(t _i )] are also matrices of dimensions 2 × q, with q = m + l + l^' .

Although the problem is completely determined from the computation of the impulse response function by solving Equation (9), our aim here is to provide a recursive algorithm for its computation. Next, the recursive formulas for computing the estimator (8) and its associated error p(t _k |t _n ) = E[|s(t _k ) - ŝ(t _k |t _n )|²] are devised.

Theorem 3.1 The optimal WL estimate ŝ(t _k |t _n ) defined in (8) can be recursively computed as follows:

$$\mathit{ŝ}\left(t_k|t_n\right)={\mathit{\psi }}_{\left[1\right]}\left(t_k\right)\mathbf{ϵ}\left(t_n\right),t_k\ge t_n$$

(12)

where the q-vector ϵ (t _n ) is recursively computed from the expression

$$\begin{array}{c}\mathbf{ϵ}\left(t_n\right)=\mathbf{ϵ}\left(t_{n-1}\right)+\mathbf{J}\left(t_n,t_n\right)\left[\mathbf{y}\left(t_n\right)-\mathbf{\Phi }\left(t_n\right)\mathbf{ϵ}\left(t_{n-1}\right)\right]\hfill \\ \mathbf{ϵ}\left(t_0\right)=0_q\hfill \end{array}$$

with the q × 2-matrix J(t _n , t _n ) given by the equation

$$\mathbf{J}\left(t_n,t_n\right)=\left[{\mathbf{\Gamma }}^H\left(t_n\right)-\mathbf{Q}\left(t_{n-1}\right){\mathbf{\Phi }}^H\left(t_n\right)\right]{\mathbf{\Omega }}^{-1}\left(t_n\right)$$

(14)

with the 2 × 2-matrix Ω(t _n ) = Σ + [Γ(t _n ) - Φ(t _n ) Q (t_n-1)] Φ ^H (t _n ) and the q × q-matrix Q(t _n ) satisfying the recursive equation

$$\begin{array}{c}\mathbf{Q}\left(t_n\right)=\mathbf{Q}\left(t_{n-1}\right)+\mathbf{J}\left(t_n,t_n\right)\left[\mathbf{\Gamma }\left(t_n\right)-\mathbf{\Phi }\left(t_n\right)\mathbf{Q}\left(t_{n-1}\right)\right]\hfill \\ \mathbf{Q}\left(t_0\right)=0_{q\times q}\hfill \end{array}$$

(15)

Moreover, the associated error is given by the expression

$$p\left(t_k|t_n\right)=r_s\left(t_k,t_k\right)-{\mathit{\psi }}_{\left[1\right]}\left(t_k\right)\mathbf{Q}\left(t_n\right){\mathit{\psi }}_{\left[1\right]}^H\left(t_k\right),t_k\ge t_n$$

(16)

Proof. From (10) and (11), Equation (9) can be rewritten as

$${\mathbf{h}}^{\text{T}}\left(t_k,t_j,t_n\right)\mathbf{\sum }={\mathit{\psi }}_{\left[1\right]}\left(t_k\right){\mathbf{\Gamma }}^{\text{H}}\left(t_j\right)-{\displaystyle \mathbf{\sum }_{i=1}^n}{\mathbf{h}}^{\text{T}}\left(t_k,t_i,t_n\right)\mathbf{R}\left(t_i,t_j\right)$$

Then, if we introduce a function J(t _j , t _n ) satisfying the equation

$$\mathbf{J}\left(t_j,t_n\right)\sum ={\mathbf{\Gamma }}^{\text{H}}\left(t_j\right)-{\displaystyle \mathbf{\sum }_{i=1}^n}\mathbf{J}\left(t_i,t_n\right)\mathbf{R}\left(t_i,t_j\right)$$

(17)

we obtain that

$${\mathbf{h}}^{\text{T}}\left(t_k,t_j,t_n\right)={\mathit{\psi }}_{\left[1\right]}\left(t_k\right)\mathbf{J}\left(t_j,t_n\right)$$

(18)

and then, substituting (18) in (8), and defining the function

$$\begin{array}{c}\mathbf{ϵ}\left(t_n\right)=\sum _{i=1}^n\mathbf{J}\left(t_i,t_n\right)\mathbf{y}\left(t_i\right)\hfill \\ \mathbf{ϵ}\left(t_0\right)=0_q\hfill \end{array}$$

(19)

the Equation (12) for the optimal estimator is devised.

Now, subtracting the Equation (17) for t _n and t_{n- 1}and taking (11) into account, we can write

$$\left[\mathbf{J}\left(t_j,t_n\right)-\mathbf{J}\left(t_j,t_{n-1}\right)\right]\mathbf{\sum }=-\mathbf{J}\left(t_n,t_n\right)\mathbf{\Phi }\left(t_n\right){\mathbf{\Gamma }}^{\text{H}}\left(t_j\right)-{\displaystyle \mathbf{\sum }_{i=1}^{n-1}}\left[\mathbf{J}\left(t_i,t_n\right)-\mathbf{J}\left(t_i,t_{n-1}\right)\right]\mathbf{R}\left(t_i,t_j\right)$$

Thus, from (17), we have the relation

$$\mathbf{J}\left(t_j,t_n\right)-\mathbf{J}\left(t_j,t_{n-1}\right)=-\mathbf{J}\left(t_n,t_n\right)\mathbf{\Phi }\left(t_n\right)\mathbf{J}\left(t_j,t_{n-1}\right)$$

(20)

As a consequence, subtracting the Equation (19) for t _n and t_{n- 1}and using (20) in the resulting equation, the recursive expression (13) is obtained.

Next, we proceed to derive expression (14) for J(t _n , t _n ). By taking t _j = t _n in (17) and using (11), we have

$$\mathbf{J}\left(t_n,t_n\right)\mathbf{\sum }={\mathbf{\Gamma }}^{\text{H}}\left(t_n\right)-{\displaystyle \mathbf{\sum }_{i=1}^n}\mathbf{J}\left(t_i,t_n\right)\mathbf{\Gamma }\left(t_i\right){\mathbf{\Phi }}^H\left(t_n\right)=\left[{\mathbf{\Gamma }}^H\left(t_n\right)-\mathbf{Q}\left(t_n\right){\mathbf{\Phi }}^H\left(t_n\right)\right]$$

(21)

where we have introduced the q × q-matrix

$$\begin{array}{c}\mathbf{Q}\left(t_n\right)=\sum _{i=1}^n\mathbf{J}\left(t_i,t_n\right)\mathbf{\Gamma }\left(t_i\right)\hfill \\ \mathbf{Q}\left(t_0\right)=0_{q\times q}\hfill \end{array}$$

(22)

Moreover, if we subtract Q(t_{n- 1}) from Q(t _n ), and use (20) and (22) in the resulting expression, the recursive Equation (15) for Q(t _n ) is derived. Finally, using (15) in (21), it is easy to check that J(t _n , t _n ) satisfies the expression (14).

Finally, in order to derive expression (16) for the error p(t _k |t _n ) associated with the above estimate, we remark that, from the orthogonal projection lemma, this function can be expressed as

$$p\left(t_k|t_n\right)=r_s\left(t_k,t_k\right)-E\left[\mathit{ŝ}\left(t_k|t_n\right){\mathit{ŝ}}^{*}\left(t_k|t_n\right)\right]$$

Then, substituting (12) in the above equation and using (17) and (19), we check that

$$p\left(t_k|t_n\right)=r_s\left(t_k,t_k\right)-{\mathit{\psi }}_{\left[1\right]}\left(t_k\right)E\left[\mathbf{ϵ}\left(t_n\right){\mathbf{ϵ}}^{\text{H}}\left(t_n\right)\right]{\mathit{\psi }}_{\left[1\right]}^{\text{H}}\left(t_k\right)=r_s\left(t_k,t_k\right)-{\mathit{\psi }}_{\left[1\right]}\left(t_k\right)\sum _{i=1}^n\mathbf{J}\left(t_i,t_n\right)\mathbf{\Gamma }\left(t_i\right){\mathit{\psi }}_{\left[1\right]}^{\text{H}}\left(t_k\right)$$

As a consequence, from (22), (16) is obtained.

Remark 1 When {s(t _i ), t _i ∈ T} is a factorizable real-valued signal with correlation function of the form (1), and the observations of the signal verify the complex-valued linear Equation (6) with

$$r_{s\mathbf{v}}\left(t_i,t_j\right)=\left\{\begin{array}{c}{\mathbf{c}}^T\left(t_i\right){\mathbf{D}}^H\left(t_j\right),t_i\ge t_j\\ {\mathbf{e}}^T\left(t_i\right){\mathbf{F}}^H\left(t_j\right),t_i\le t_j\end{array}\right.$$

where c(t _i ) and e(t _i ) are vectors of dimensions l and l', respectively, and D(t _i ) and F(t _i ) are matrices of respective dimensions 2 ×l and 2 ×l^', we obtain that Algorithm 3.1 holds, replacing the involved 2 × q-matrices Ψ(t _i ), Π(t _i ) by the vectors ${\mathit{\psi }}^T\left(t_i\right)=\left[{\mathit{\alpha }}^T\left(t_i\right),{\mathbf{c}}^T\left(t_i\right),0_{l^{\mathbf{\prime }}}^T\right]$ and ${\mathbf{\pi }}^T\left(t_i\right)=\left[{\mathit{\beta }}^T\left(t_i\right),0_l^T,{\mathbf{e}}^T\left(t_i\right)\right]$, and taking the matrices Φ(t _i ) = [g(t _i ) α ^T (t _i ), g(t _i )c ^T (t _i ), F(t _i )] and Γ(t _i ) = [g(t _i )β^T(t _i ), D(t _i ), g(t _i )e ^T (t _i )], with g(t _i ) = [g(t _i ), g*(t _i )]^T .

Remark 2 The efficiency of Algorithm 3.1 is closely related to the dimensions l, l^', and m of the matrices involved in the factorizations (2) and (7). Indeed, the computational complexity of this algorithm is of order q, with q = m + l + l^', and thus, it involves a further complication in implementation and an increased computational burden as q grows. Since the factorization of the covariance is not unique then, the key question is in choosing the one which minimizes the dimension q. There exist simple cases, as illustrated previously, where the factorization is easily obtained. Nevertheless, in those more complex cases where this factorization is not trivial, one can use several methods available in the literature to get a factorization with minimum dimension (see, e.g., [19]).

3.1 Numerical examples

The advantages of the proposed Algorithm with respect to the SL solution are illustrated here through three numerical examples. The first one involves real correlation matrices and analyzes the effectiveness of the WL processing with respect to the SL one in terms of the impropriety degree of the observations. In the second example, complex correlation matrices are considered and the resulting WL and SL estimation errors are graphically compared. Finally, the third example shows a real application to seismic signal processing.

3.1.1 Example

Let {x(t _i ), t₁≤ t _i ≤ t₁₀₀}, with t _i = i/ 100, i = 1, ..., 100, be an Ornstein-Uhlenbeck process with correlation function

$$r_x\left(t_i,t_j\right)=\text{exp}\left(-|t_i-t_j|\right),t_1\le t_i,t_j\le t_{100}$$

which is transmitted over a channel that rotates it by a standard normal phase θ and adds a doubly white Gaussian noise v(t _i ) correlated with the signal with

$${\mathbf{R}}_{xv}\left(t_i,t_j\right)=\left(\begin{array}{cc}\hfill \frac{1}{25}{t}_i^2{t}_j\hfill & \hfill \frac{1}{25}{t}_i^2{t}_j\hfill \\ \hfill \frac{1}{25}{t}_i^2{t}_j\hfill & \hfill \frac{1}{25}{t}_i^2{t}_j\hfill \end{array}\right)$$

Thus, the signal of interest is s(t _i ) = e^{θ j}x(t _i ) and the observations y(t _i ) are of the form (6) with g(t _i ) = 1. Moreover, we assume that θ is independent of x(t _i ) and v(t _i ).

Note that the correlation matrix of the augmented signal s(t _i ) can be expressed in the form (2), where A and B are as in (4) with l = 1, $a\left(t_i\right)=e^{-t_i}$, $b\left(t_i\right)=e^{t_i}$, and φ _θ (-2) = φ _θ (2) = e^-2. Moreover, the cross-correlation matrix between s(t _i ) and v(t _i ) is of the form (7) with D(t _i ) = F(t _i ) = [t _i /25, t _i /25]^T and $\mathbf{C}\left(t_i\right)=\mathbf{E}\left(t_i\right)={\left[e^{-1/2}{t}_i^2,e^{-1/2}{t}_i^2\right]}^{\text{T}}$.

In this example, we consider the problem of computing the fixed-lead predictor ŝ(t_k+10|t _k ). As a measure for comparing the performance of the WL and SL fixed-lead predictors we use the one defined in [6], the mean square of the difference between both errors, for

$$\mathbf{\sum }=\left(\begin{array}{c}\hfill 2\tau \hfill \\ \hfill \tau 2\hfill \end{array}\right)$$

with τ varying within the interval [1, 2):

$$\frac{1}{100}\sum _{k=1}^{100}{\left({\tilde{p}}_{\tau }\left(t_{k+10}|t_k\right)-p_{\tau }\left(t_{k+10}|t_k\right)\right)}^2$$

(24)

with p _τ (t_k+10|t _k ) and ${\tilde{p}}_{\tau }\left(t_{k+10}|t_k\right)$ denoting the WL and SL fixed-lead prediction errors, respectively, for every value τ. The results obtained are displayed in Figure 1 which shows that, not only the WL fixed-lead predictor presents a better behavior than the SL fixed-lead predictor but also the difference between both errors (in the mean square sense) increases with τ , and hence, the WL technique becomes more effective.

Figure 1

Performance of fixed-lead predictors. Quadratic mean of the difference between the WL and SL fixed-lead prediction errors.

3.1.2 Example

Let {s(t _i ), t₁≤ t _i ≤ t₁₀₀}, with t _i = i/100, i = 1, ..., 100, be a signal of the form

$$s\left(t_i\right)=A{e}^{\text{j}{t}_i}+B{e}^{-\text{j}{t}_i}$$

where A and B are complex random variables. In this example, the signal is assumed to be widely stationary, that is E[AA] = E[BB] = E[AB*] = 0 (see [[17], p. 24]), and also we consider that E[AA*] = E[BB*] = 1 and E[AB] = -0.8. Thus, by using (5), the correlation matrix of the signal s(t _i ) can be expressed in the form (2). Moreover, we consider that the observations y(t _i ) are of the form (6) with g(t _i ) = 1 and where the noise ν(t _i ) is uncorrelated with the signal and its augmented variance matrix is

$$\mathbf{\sum }=\left(\begin{array}{c}21\\ 12\end{array}\right)$$

On the basis of the set of observations {y(t₁), y(t₂), ..., y(t₁₀₀)}, we consider the problem of computing the fixed-lead predictor ŝ(t_k+10|t _k ). Then, Algorithm 3.1 is used to obtain the WL fixed-lead prediction error p _τ (t_k+10|t _k ) which is compared with the SL fixed-lead prediction error ${\overline{p}}_{\tau }\left(t_{k+10}|t_k\right)$ in Figure 2. As could be expected, this figure shows that the WL fixed-lead predictor presents a better behavior than the SL fixed-lead predictor. Finally, Figure 3 depicts the performance measure given by (24) with τ ∈ [1, 2) and Σ as in (23). Again, the improved precision attained with the WL fixed-lead predictor with respect to the SL fixed-lead predictor is observed as τ increases.

Figure 2

Fixed-lead prediction errors. WL (solid line) and SL (dashed line) fixed-lead prediction errors.

Figure 3

Performance of fixed-lead predictors. Quadratic mean of the difference between the WL and SL fixed-lead prediction errors.

3.1.3 Example

As indicated in Section 2, uniformly modulated nonstationary processes are often used to model seismic records, especially acceleration records. The modulated nonstationary process is given by s(t _i ) = d(t _i )x(t _i ), where x(t _i ) is a stationary process with zero mean and known second-order statistics, and d(t _i ) is the time modulating function. A stochastic earthquake model commonly used for x(t _i ) is the Kanai-Tajimi process (see, e.g., [20]). It is well-known that the Kanai-Tajimi earthquake model is covariance equivalent with the subset of the ARMA(2,1) model corresponding to a unit value of the spring-dashpot input ratio [21]. For firm ground conditions, at moderate epicentral distance, Kanai and Tajimi have suggested specific values for the parameters in the equation of motion in continuous time whose corresponding discrete ARMA(2,1) model is [21]

$$x\left(t_i\right)-1.604x\left(t_{i-1}\right)+0.686x\left(t_{i-2}\right)=e\left(t_i\right)-0.767e\left(t_{i-1}\right)$$

with ${\sigma }_e^2=\text{ 39}.0\text{8}$ and Δt = t _i -t_{i- 1}= 0.02 s. This ARMA model has a factorizable correlation function given by (3). Moreover, we have chosen the modulating function $d\left(t_i\right)=e^{\text{j}{t}_i}$ and the augmented error covariance matrix of the form

$$\mathbf{\sum }=\left(\begin{array}{c}\hfill 1\tau \hfill \\ \hfill \tau 1\hfill \end{array}\right)$$

with τ ∈ [0, 1). Here we have studied the filtering problem. Figure 4 illustrates the enhancement of the WL filter in relation to the SL one by using the measure

Figure 4

Performance of filters. Quadratic mean of the difference between the WL and SL filtering errors.

$$\frac{1}{100}\sum _{k=1}^{100}{\left({\tilde{p}}_{\tau }\left(t_k|t_k\right)-p_{\tau }\left(t_k|t_k\right)\right)}^2$$

Similar to the previous examples, we observe the superiority of the WL estimate as τ increases.

4 Nonlinear augmented complex prediction

Given the same conditions on the signal established in Section 2, we suppose that the ob-servation process can be given by a nonlinear relation of the form

$$y\left(t_i\right)=z\left(s\left(t_i\right),t_i\right)+\nu \left(t_i\right),t_{\text{1}}\le t_i\le t_n$$

(25)

where z(·) is a complex-valued nonlinear function and the signal s(t _i ) is uncorrelated with the noise ν(t _i ). As in the linear case, the aim here is to estimate the signal s(t _k ) on the basis of the observation set {y(t₁), ..., y(t _n ), y*(t₁), ..., y*(t _n )}, with t _k ≥ t _n .

The solution to this problem is addressed by following a similar philosophy to the EKF [22]. Specifically, following the basic idea of the EKF the nonlinear function z(s(t _n ), t _n ) is linearized at each time instant by a first-order Taylor series expanded about the estimated signal ŝ(t _n |t_n-1)

$$z\left(s\left(t_n\right),t_n\right)\approx z\left(\mathit{ŝ}\left(t_n|t_{n-1}\right),t_n\right)+\frac{\partial z\left(s,t_n\right)}{\partial s}{|}_{s=\mathit{ŝ}\left(t_n|t_{n-1}\right)}\left(s\left(t_n\right)-\mathit{ŝ}\left(t_n|t_{n-1}\right)\right)$$

Consequently, we can proceed to approximate the nonlinear observation Equation (25) as shown by

$$\mathit{ȳ}\left(t_n\right)\approx g\left(t_n\right)s\left(t_n\right)+v\left(t_n\right)$$

(26)

where

$$\mathit{ȳ}\left(t_n\right)=y\left(t_n\right)-z\left(\mathit{ŝ}\left(t_n|t_{n-1}\right),t_n\right)+g\left(t_n\right)\mathit{ŝ}\left(t_n|t_{n-1}\right)$$

and

$$g\left(t_n\right)={\left.\frac{\partial z\left(s,t_n\right)}{\partial s}\right|}_{s=\mathit{ŝ}\left(t_n|t_{n-1}\right)}$$

(27)

and thus, s(t _k ) can be estimated in terms of the set of observations $\left\{\mathit{ȳ}\left(t_1\right),\dots ,\mathit{ȳ}\left(t_n\right),{\mathit{ȳ}}^{*}\left(t_1\right),\dots ,{\mathit{ȳ}}^{*}\left(t_n\right)\right\}$ from the relation (26). For that, the formulas given in Algorithm 3.1 for the WL predictor can be used. Note that in the case of the signal and observation noise being uncorrelated, Algorithm 3.1 holds with Ψ(t _i ) = A(t _i ), Π(t _i ) = B(t _i ), Φ(t _i ) = G(t _i )A(t _i ) and Γ(t _i ) = G(t _i )B(t _i ). Then, as in the EKF, in the resulting formulas we also use the linearized observation function (27) in place of the previous function g(t _n ) and the term G(t _n )A(t _n )ϵ(t_{n- 1}) is replaced by the vector

$$\mathbf{z}\left(\mathit{ŝ}\left(t_n|{t_{n-}}_{\text{1}}\right),t_n\right)={\left[z\left(\mathit{ŝ}\left(t_n|{t_{n-}}_{\text{1}}\right),t_n\right),z^{*}\left(\mathit{ŝ}\left(t_n|{t_{n-}}_{\text{1}}\right),t_n\right)\right]}^{\text{T}}$$

Next the formulas of the proposed Algorithm are summarized.

Theorem 4.1 A nonlinear augmented complex predictor of the signal s(t _k ) based on the set of nonlinear observations {y(t₁), ..., y(t _n ), y* (t₁), ..., y*(t _n )} of the form (25) can be determined through the equation

$$\mathit{ŝ}\left(t_k|t_n\right)={\mathbf{a}}_{\left[1\right]}\left(t_k\right)\mathbf{ϵ}\left(t_n\right)$$

where the m-vector ϵ (t _n ) is recursively computed from the expression

$$\begin{array}{c}\mathbf{ϵ}\left(t_n\right)=\mathbf{ϵ}\left(t_{n-1}\right)+\mathbf{J}\left(t_n,t_n\right)\left[\mathbf{y}\left(t_n\right)-\mathbf{z}\left(\mathit{ŝ}\left(t_n|t_{n-1}\right),t_n\right)\right]\hfill \\ \mathbf{ϵ}\left(t_0\right)=0_m\hfill \end{array}$$

with

$$\mathbf{J}\left(t_n,t_n\right)=\left[{\mathbf{B}}^H\left(t_n\right)-\mathbf{Q}\left(t_{n-1}\right){\mathbf{A}}^H\left(t_n\right)\right]{\mathbf{G}}^H\left(t_n\right){\mathbf{\Omega }}^{-1}\left(t_n\right)$$

where Ω(t _n ) = Σ + G(t _n ) [B(t _n ) - A(t _n )Q(t_{n- 1})] A ^H (t _n )G ^H (t _n ), G(t _n ) = diag(g(t _n ), g*(t _n )), with$g\left(t_n\right)=\partial z\left(s,t_n\right)/\partial s{|}_{s=\mathit{ŝ}\left(t_n|t_{n-1}\right)},$and Q(t _n ) satisfies the recursive equation

$$\begin{array}{c}\mathbf{Q}\left(t_n\right)=\mathbf{Q}\left(t_{n-1}\right)+\mathbf{J}\left(t_n,t_n\right)\mathbf{G}\left(t_n\right)\left[\mathbf{B}\left(t_n\right)-\mathbf{A}\left(t_n\right)\mathbf{Q}\left(t_{n-1}\right)\right]\hfill \\ \mathbf{Q}\left(t_0\right)=0_{m\times m}\hfill \end{array}$$

Remark 3 Similarly to Remark 1, if the signal {s(t _i ), t _i ∈ T} is a real-valued signal with factorizable kernel of the form (1) which is observed through a complex-valued nonlinear equation of the form (25), Algorithm 4.1 holds with a_[1](t _i ) = α^T(t _i ), g(t _i ) = [g(t _i ), g*(t _i )] ^T and replacing the 2 × q-matrices A(t _i ) and B(t _i ) by the vectors α (t _i ) and β (t _i ), respectively.

4.1 Numerical example

In this Example, the estimation of a real signal on the basis of a set of complex-valued observations is considered. Specifically, let {s(t _i ), t₁≤ t _i ≤ t₁₀₀} be a real Wiener process with unit variance parameter and the observation equation

$$y\left(t_i\right)=e^{s\left(t_i\right)\text{j}}+v\left(t_i\right),t_i=i/100,i=1,\dots ,100$$

where v(t _i ) = e^{θ j}u(t _i ), with θ a standard normal phase and u(t _i ) a white Gaussian noise with unit spectral height and uncorrelated with the signal s(t _i ).

In this case, the correlation function of the signal s(t _i ) can be expressed in the form (1) with α(t _i ) = 1 and β(t _i ) = t _i .

With the aim of examining the good behavior of the WL solution proposed in Algorithm 4.1, the estimation error of the WL filter is compared with the errors associated with SL and WL conventional Algorithms. Specifically, the standard EKF and UKF and the WL EKF and WL UKF proposed in [1] have been implemented. For that, we use the fact that the signal s(t _i ) obeys the state equation

$$s\left(t_i\right)=s\left(t_{i-1}\right)+w\left(t_i\right),t_i=i/100,i=1,\dots ,100$$

with w(t _i ) a centered Gaussian signal with variance parameter 10^-2.

On the other hand, for computing the estimation errors, Monte Carlo simulations have been performed. Figure 5 shows the results obtained with 5000 sample paths, confirming the better behavior of a WL processing in the nonlinear estimation problem. In fact, the dashed line represents the errors associated with the SL EKF and SL UKF (the differences between them are negligible) and the solid line depicts the errors associated with the WL UKF, WL EKF and the filter given in Algorithm 4.1 (again the differences between them are negligible). Obviously, the similar behavior shown here by the three WL filters has not to be repeated in other examples. As occurs in the standard estimation techniques, there is not a best nonlinear estimator either. In each application one has to pick the appropriate nonlinear estimation method. Really, in every particular case one has to choose the estimator which is found to best trade off various properties such as estimation accuracy, ease of implementation, numerical robustness, and computational burden [22]. Note that unlike UKF and EKF, Algorithm 4.1 does not require a state space model but only the knowledge of the second-order statistics of the processes involved.

Figure 5

Nonlinear filtering errors. Errors for the WL filtering Algorithms (solid line) versus the nonlinear standard methods (dashed line).

Endnotes

^aA zero-mean complex-valued signal {s(t _i ), t _i ∈ T}, T = {t₁, t₂, ..., } is said to be proper if the pseudocorrelation function, ρ _s (t _i , t _j ) = E [s(t _i )s(t _j )], is null for all t _i , t _j ∈ T. Otherwise, it is called improper. ^bv(t _i ) is said to be a doubly white noise if E[v(t _i )v* (t _j )] = σ₁δ _ij and E[v(t _i )v(t _j )] = σ₂δ _ij , with |σ₂| < σ₁ and δ _ij stands for the Kronecker delta function [3]. ^cA simple application of the Hilbert space projection theorem shows that WL estimation outperforms SL estimation for general complex-valued signals. Specifically, denote $\overline{\text{sp}}\left\{y\left(t_{\text{1}}\right),\dots ,y\left(t_n\right)\right\}$ and $\overline{\text{sp}}\left\{y\left(t_{\text{1}}\right),\dots ,y\left(t_n\right),y^{*}\left(t_{\text{1}}\right),\dots ,y^{*}\left(t_n\right)\right\}$ the closed spans of the following sets {y(t₁), ..., y(t _n )} and {y(t₁), ..., y(t _n ), y*(t₁), ..., y (t _n )}, respectively. Let $\tilde{s}\left(t_k|t_n\right)$ and ŝ(t _k |t _n ) be the projections of s(t _k ), t _k ≥ t _n , onto the spaces $\overline{\text{sp}}\left\{y\left(t_{\text{1}}\right),\dots ,y\left(t_n\right)\right\}$ and $\overline{\text{sp}}\left\{y\left(t_{\text{1}}\right),\dots ,y\left(t_n\right),y^{*}\left(t_{\text{1}}\right),\dots ,y^{*}\left(t_n\right)\right\}$ respectively. Thus, $\tilde{s}\left(t_k|t_n\right)$ is the SL estimate of s(t _k ) and ŝ(t _k |t _n ) is its WL estimate. Since $\overline{\text{sp}}\left\{y\left(t_{\text{1}}\right),\dots ,y\left(t_n\right)\right\}\subseteq \overline{\text{sp}}\left\{y\left(t_{\text{1}}\right),\dots ,y\left(t_n\right),y^{*}\left(t_{\text{1}}\right),\dots ,y^{*}\left(t_n\right)\right\}$, then, by the projection theorem, it follows that the mean square error of ŝ(t _k |t _n ) is smaller than that of ŝ(t _k |t _n ).