Particle rejuvenation of Rao-Blackwellized sequential Monte Carlo smoothers for conditionally linear and Gaussian models

This paper focuses on sequential Monte Carlo approximations of smoothing distributions in conditionally linear and Gaussian state spaces. To reduce Monte Carlo variance of smoothers, it is typical in these models to use Rao-Blackwellization: particle approximation is used to sample sequences of hidden regimes while the Gaussian states are explicitly integrated conditional on the sequence of regimes and observations, using variants of the Kalman filter/smoother. The first successful attempt to use Rao-Blackwellization for smoothing extends the Bryson-Frazier smoother for Gaussian linear state space models using the generalized two-filter formula together with Kalman filters/smoothers. More recently, a forward-backward decomposition of smoothing distributions mimicking the Rauch-Tung-Striebel smoother for the regimes combined with backward Kalman updates has been introduced. This paper investigates the benefit of introducing additional rejuvenation steps in all these algorithms to sample at each time instant new regimes conditional on the forward and backward particles. This defines particle-based approximations of the smoothing distributions whose support is not restricted to the set of particles sampled in the forward or backward filter. These procedures are applied to commodity markets which are described using a two-factor model based on the spot price and a convenience yield for crude oil data.

State space models are bivariate stochastic processes {(Y _i ,Z _i )} _i≥1 where the state sequence (Z _i ) _i≥1 is a Markov chain which is only partially observed through the sequence (Y _i ) _i≥1. Conditionally, on the state sequence (Z _i ) _i≥1, the observations are independent, and for all ℓ≥1, the conditional distribution of Y _ℓ given (Z _i ) _i≥1 depends on Z _ℓ only. These models are used in a large variety of disciplines such as financial econometrics, biology, signal processing, see [10], and the references therein. In general state space models, Bayesian filtering and smoothing problems, i.e., the computation of the posterior distributions of a sequence of states (Z _i ,…,Z _p ) for 1≤i≤p≤ℓ given observations (Y ₁,…,Y _ℓ ), are challenging tasks. Filtering refers to the estimation of the distributions of the hidden state Z _i given the observations (Y ₁,…,Y _i ) up to time i, while fixed-interval smoothing stands for the estimation of the distribution of sequence of states (Z _i ,…,Z _p ) given observations (Y ₁,…,Y _ℓ ) with 1≤i≤p<ℓ. When the state and observation models are linear and Gaussian, filtering can be solved explicitly using the Kalman filter [18]. Exact solutions of the fixed-horizon smoothing problem can be obtained using either the Rauch-Tung-Striebel smoother [22] or the Bryson-Frazier two-filter smoother [4]. This paper focuses on conditionally linear and Gaussian models (CLGM) given for i≥2 by:

where:

• (ε _i ) _i≥2 is a sequence of independent and identically distributed (i.i.d.) m-dimensional Gaussian vectors with zero mean and identity covariance.

• (a _i ) _i≥1 is a homogeneous Markov chain taking values in a finite space {1,…,J}, called regimes, with initial distribution π and transition matrix Q.

• (H _j )_1≤j≤J are m×m positive-definite matrices, (d _j )_1≤j≤J m-dimensional vectors, and (T _j )_1≤j≤J m×m positive-definite matrices.

• Z ₁ is a m-dimensional Gaussian random variable with mean μ ₁ and variance Σ ₁ independent of (ε _i ) _i≥2.

Let n be the number of observations. At each time step 1≤i≤n, the observation Y _i is given by:

where:

• (η _i ) _i≥1 is a i.i.d. sequence of p-dimensional Gaussian vectors, independent of (ε _i ) _i≥2 and Z ₁.

• (G _j )_1≤j≤J are p×p positive-definite matrices, (c _j )_1≤j≤J p-dimensional vectors and (B _j )_1≤j≤J p×m matrices.

CLGM play an important role in many applications; see [23] and the references therein for an up-to-date account. A crucial feature of these models is that, conditional on the regime sequence (a ₁,…,a _n ), both the state equation and the observation equation are linear and Gaussian, which implies that conditional on the sequence of regimes and on the observations, the filtering and the smoothing distributions of the continuous states (Z ₁,…,Z _n ) can be computed explicitly.

To exploit this specific structure, it has been suggested in the pioneering works of [6, 7] to solve the filtering problem by combining sequential Monte Carlo (SMC) methods to sample the regimes with the Kalman filter to compute the conditional distribution of the states sequence (Z _i )_1≤i≤n conditional on the regimes and on the observations. This is a specific instance of Rao-Blackwellized Monte Carlo filters, often referred to as the mixture Kalman filter. Improvements of these early filtering techniques have been introduced in [8, 26].

The use of Rao-Blackwellization to solve the smoothing problem has been proved to be more challenging and has received satisfactory solutions only recently. The first forward-backward smoother proposed in the literature [12] was not fully Rao-Blackwellized as it required to sample the hidden linear states in the backward pass. An alternative approach, based on the so-called structural approximation of the model suggested in an early paper by [19], was proposed by [2] to avoid to sample a continuous state in the backward pass. This approximation is rather ad hoc and the resulting smoother is not consistent when the number of particles goes to infinity. The inaccuracy introduced by the approximation might be difficult to control.

The first fully Rao-Blackwellized SMC smoother which should lead to consistent approximations when the number of particles grows to infinity was proposed by [3] and extends the Bryson-Frazier smoother for Gaussian linear state space models using the generalized two-filter formula with Rao-Blackwellization steps for the forward and the backward filters. This two-filter approach combines a forward filter with a backward information filter which are approximated numerically using SMC for the regime sequence and Kalman filtering techniques for the hidden linear states.

More recently, [20, 21, 24] introduced a Rao-Blackwellized smoother based on the forward-backward decomposition of the FFBS algorithm with Rao-Blackwellization steps both in the forward and backward time directions. The update of the smoothing distribution of the regime given the observations shares some striking similarities with the Rauch-Tung-Striebel smoothing procedure, which is at the heart of the FFBS procedure. The Rao-Blackwellization requires to update backward in time the smoothing distribution of the states given the regimes and the observations, which is achieved by using an à la Kalman backward update.

In this paper, we propose to improve the performance of the algorithms introduced in [3] and in [20, 21, 24] by using additional Rao-Blackwellization steps which allows to sample new particles in the backward pass. This approach may be seen as an extension of the ideas of [11] for Rao-Blackwellized smoothers. In [3], for all 1≤i≤n, the sampled forward and backward sequences are merged to approximate the posterior distribution of (a _i ,z _i ). This provides an approximation whose support is restricted to the particles produced at time i by the backward particle filter. As noted in ([13], Section 2.6), these two-filter smoothers are prone to suffer from degeneracy issues when the algorithm associates forward particles at time i−1 with backward particles at time i. We propose to approximate the marginal smoothing distribution of (a _i ,z _i ) by merging the sampled forward and backward trajectories at times i−1 and i+1 and integrating out all possible paths between time i−1 and time i and between time i and time i+1 instead of sampling random variables. Similarly, in the backward pass of [20, 21, 24], a regime \(\tilde {a}_{i}\) is sampled at time 1≤i≤n−1 using the particles produced by the forward filter at time i. In this case, particle rejuvenation may be introduced by using the forward weighted samples at time i−1 and extending these trajectories at time i with a Kalman filter for all possible values of the regime. Then, \(\tilde {a}_{i}\) is sampled in {1,…,J} using an appropriately adapted weight.

The paper is organized as follows. The algorithms introduced in [3] and in [20, 21, 24] as long as the proposed rejuvenation associated with each method are presented in Section 2. The performance of all these methods is illustrated in Section 3 with simulated data. In Section 4, an application to commodity markets is presented; the performance of our procedure is illustrated with crude oil data. A detailed derivation of the algorithms is provided in the “Appendix: Technical lemmas”.

This section details the sequential Monte Carlo algorithms which can be used to approximate the conditional distribution of the states (a ₁,…,a _n ) or the marginal distributions of (a _i ,z _i ) given the observations (Y ₁,…,Y _n ). For all m × m matrix let |A| be the determinant of A. If A is a positive-definite matrix, for all \(z\in \mathbb {R}^{m}\) define

where for any vector or matrix z, z ^′ denotes the transpose matrix of z. Let m(a _i ,z _i−1;z _i ) be the probability density of the conditional distribution of Z _i given (a _i ,Z _i−1) and g(a _i ,z _i ;y _i ) be the probability density of the conditional distribution of Y _i given (a _i ,Z _i ):

where

All the algorithms considered in this paper are based on forward-backward or two-filter decompositions of the smoothing distributions and share the same forward filter presented in Section 2.1.

Forward filter

The SMC approximation \(p^{N}_{}(a_{1:i},z_{i}|y_{1:i})\) of p(a _1:i ,z _i |y _1:i ) may be obtained using a standard Rao-Blackwellized algorithm. The procedure produces a sequence of trajectories \(\left (a^{k}_{1:i}\right)_{1\le k \le N}\) associated with normalized importance weights \(\left (\omega ^{k}_{i}\right)_{1\le k \le N} \left (\sum _{k=1}^{N} \omega ^{k}_{i} = 1\right)\) used to define the following approximation of p(a _1:i ,z _i |y _1:i ):

where δ is the Dirac delta function. In this equation, the conditional distribution of the hidden state z _i given the observations y _1:i and a trajectory \(a^{k}_{1:i}\) is a Gaussian distribution whose mean \(\mu ^{k}_{i}\) and variance \(P^{k}_{i}\) may be obtained by using the Kalman filter update.

Initialization

At time i=1, write, for all 1≤j≤J,

\(\left (a^{k}_{1}\right)_{1\le k \le N}\) are sampled independently in {1,…,J} with probabilities proportional to

Then, \(\mu _{1}^{k}\) and \(P_{1}^{k}\) are computed using a Kalman filter:

where for all positive integer p, I _p is the p×p identity matrix. Each particle \(a_{1}^{k}\) is associated with the importance weight \(\omega ^{k}_{1} = 1/N\).

Iterations

Several procedures may be used to extend the trajectories \(\left (a^{k}_{1:i-1}\right)_{1\le k \le N}\) at time i. For all sampled trajectories \(\left (a_{1:i-1}^{k}\right)_{1\le k \le N}\) and all 1≤j≤J, [6] used the incremental weights:

The conditional distribution of Y _i given \(a^{k}_{1:i-1}\), a _i , and Y _1:i−1 is a Gaussian distribution with mean \(c_{a_{i}}+B_{a_{i}}\mu ^{k}_{i|i-1}(a_{i})\) and variance \(B_{a_{i}}P^{k}_{i|i-1}(a_{i})B'_{a_{i}} + \overline {G}_{a_{i}}\) where

Therefore,

where

In [6], for all 1≤k≤N, an ancestral path is chosen with probabilities proportional to \(\left (\omega ^{k}_{i-1}\right)_{1\le k \le N}\). Then, the new regime \(a_{i}^{k}\) is sampled in {1,…,J} with probabilities proportional to \((\gamma _{i}^{j,k})_{1\le j\le J}\). A drawback of this method is that only ancestral paths that have been selected using the importance weights \(\left (\omega ^{k}_{i-1}\right)_{1\le k \le N}\) are extended at time i. Following [5], this may be improved by considering all the offsprings of all ancestral trajectories \(\left (a_{1:i-1}^{k}\right)_{1\le k \le N}\). Each ancestral path has J offsprings at time i, it is thus necessary to choose a given number of trajectories at time i (for instance N) among the NJ possible paths. To obtain the weight associated with each offspring, write the following approximation of p(a _1:i |y _1:i ) based on the weighted samples at time i−1:

Therefore, each ancestral trajectory of the form \(\left (a^{k}_{1:i-1},j\right)\), 1≤k≤N, 1≤j≤J, is associated with the normalized weight \(\tilde {\omega }^{j,k}_{i} \propto \omega ^{k}_{i-1}\gamma _{i}^{j,k}\). Several random selection schemes have been proposed to discard some of the possible offsprings to maintain an average number of N particles at each time step. Following [5], we might choose between the Kullback-Leibler optimal selection (KL-OS) or the chi-squared optimal selection (CS-OS) to associate a new weight to each of the NJ trajectories. If the new weight is 0, then the corresponding particle can be removed.

KL-OS:

λ is chosen as the solution of

For all 1≤j≤J and 1≤k≤N, if \(\tilde {\omega }^{j,k}_{i}\ge \lambda \) then the new weight \(\tilde {\Omega }^{j,k}_{i}\) is \(\tilde {\Omega }^{j,k}_{i}=\tilde {\omega }^{j,k}_{i}\) and if \(\tilde {\omega }^{j,k}_{i}< \lambda \):

CS-OS:

λ is chosen as the solution of

For all 1≤j≤J and 1≤k≤N, if \(\tilde {\omega }^{j,k}_{i}\ge \lambda \) then the new weight \(\tilde {\Omega }^{j,k}_{i}\) is \(\tilde {\Omega }^{j,k}_{i}=\tilde {\omega }^{j,k}_{i}\) and if \(\tilde {\omega }^{j,k}_{i}< \lambda \):

Then, in both cases, all particles such that \(\tilde {\Omega }^{j,k}_{i} = 0\) are discarded and for all the other trajectories defined as an ancestral path \(\left (a^{k}_{1:i-1}\right)\) extended by \(a^{k}_{i} = j\), the new corresponding weight ω in (5) is given by the normalized weight \(\tilde {\Omega }^{j,k}_{i}\). In the numerical sections of this paper, the Kullback-Leibler optimal selection (KL-OS) scheme is used.

FFBS-based algorithms

FFBS algorithms of [20, 21, 24]

Lindsten et al. [20, 21, 24] proposed a Rao-Blackwellized procedure to sample the regime backward in time following the same steps as in the forward filtering backward smoothing algorithm [7, 15]. The algorithm relies on the decomposition given, for all 1≤i≤n−1, by:

This decomposition is similar to the Rauch-Tung-Striebel decomposition of the filtering distribution. The first factor on the right hand side of the previous equation is nevertheless more difficult to handle because it itself relies on all the observations. As noted by [24], this term can be computed recursively by considering the following decomposition:

The second factor in the last equation may be approximated using the ancestral trajectories \(\left (a^{k}_{1:i}\right)_{1\le k \le N}\) and the associated importance weights \(\left (\omega ^{k}_{i}\right)_{1\le k \le N}\) produced by the forward filter. Therefore, p(a _1:i |a _i+1:n ,y _1:n ) may be approximated by:

Then, a trajectory \(\tilde {a}_{1:n}\) approximatively distributed according to p(a _1:n |y _1:n ) may be sampled following these steps:

• Set \(\tilde {a}_{n}= a_{n}^{k}\) with probabilities proportional to \(\left (\omega _{n}^{k}\right)_{1\le k \le N}\).

• For all 1≤i≤n−1, set \(\tilde {a}_{i} = a_{i}^{k}\) with probabilities proportional to \(\left (\tilde {\omega }_{i|n}^{k}\right)_{1\le k \le N}\).

This algorithm requires to compute the quantity \(p\left (y_{i+1:n},a_{i+1:n}|a^{k}_{1:i},y_{1:i}\right)\). This predictive quantity is available analytically using Kalman filtering techniques. However, this has to be done for each trajectory \(\left (a^{k}_{1:i}\right)_{1\le k \le N}\), which leads to an algorithm with a prohibitive computational complexity. Lindsten et al. [21] proposed a procedure computationally less intensive by conditioning with respect to z _i and then marginalizing with respect to this variable:

This is similar to the two-filter decomposition of the smoothing distribution, see Section 2.3. By [21],

where the proportionality is with respect to (a _i ,z _i ) and

where the proportionality is with respect to z _i . These quantities may be computed recursively backward in time with:

Then, for 1≤i≤n−1, define \(m_{i+1} = \widehat {\lambda }_{i+1} - \widehat {\Omega }_{i+1}d_{a_{i+1}}\) and \(M_{i+1} = H_{a_{i+1}}'\widehat {\Omega }_{i+1}H_{a_{i+1}} + I_{\mathsf {m}}\) and write

As p(y _i:n ,a _i+1:n |z _i ,a _i )=p(y _i |z _i ,a _i )p(y _i+1:n ,a _i+1:n |z _i ,a _i ),

Then, by (9),

where the proportionality is with respect to \(a^{k}_{1:i}\) and

where \(P_{i}^{k} = \Gamma _{i}^{k}(\Gamma _{i}^{k})'\). Therefore,

If \(\left (\tilde {a}^{k}_{1:n}\right)_{1\le k \le \tilde {N}}\) are independent copies of \(\tilde {a}_{1:n}\), the SMC approximation of [21] of the joint smoothing distribution of the regime is:

Particle rejuvenation of FFBS algorithms

The crucial step of the FFBS algorithm is the decomposition (8) which allows to extend a backward trajectory \(\tilde {a}_{i+1:n}\) by choosing a particle in the set \(\left (a_{i}^{k}\right)_{1\le k \le N}\) produced by the forward filter (and discarding the states \(a^{k}_{1:i-1}\)). An improved version of this FFBS algorithm which is not constrained to sample states in the support \(\left (a_{i}^{k}\right)_{1\le k \le N}\) may be defined for all 2≤i≤n−1 by writing:

Replacing p(a _1:i−1,z _i−1|y _1:i−1) in the integral by the particle approximation obtained during the forward pass and using Kalman filtering techniques for each trajectory \(\left (a^{k}_{1:i-1}\right)_{1\le k\le N}\) and each a _i ∈{1,…,J} yields:

where

On the other hand, for all 1≤k≤N, \(p\left (y_{i+1:n},a_{i+1:n}|a^{k}_{1:i-1},a_{i},y_{1:i}\right)\) is computed as in (10) with all possible values a _i ∈{1,…,J} and not only the regime of the filtering pass \(\left (a_{i}^{k}\right)_{1\le k \le N}\). This means that a Kalman filter must be used for each trajectory \(a^{k}_{1:i-1}\) which may be extended by a _i ∈{1,…,J}. Denote by \(\mu _{i|i-1}^{k}(a_{i})\) and \(P_{i|i-1}^{k}(a_{i}),\) the mean and covariance matrix of the law of z _i given \(\left (a^{k}_{1:i-1},a_{i}\right)\) obtained as in (6) and (7). Then,

where the proportionality is with respect to \(\left (a^{k}_{1:i-1},a_{i}\right)\) and

where \(\Gamma _{i|i-1}^{k}(a_{i})\) is defined as \(P_{i|i-1}^{k}(a_{i}) = \Gamma _{i|i-1}^{k}(a_{i}) \left (\Gamma _{i|i-1}^{k}(a_{i})\right)'\). The distribution p(a _1:i |a _i+1:n ,y _1:n ) is then approximated by :

By integrating over all possible paths, a _1:i−1, \(\tilde {a}_{i}\) is sampled in {1,…,J}. This particle rejuvenation step allows to explore states which are not in the support of the particles produced by the forward filter and improves the accuracy and the variance of the original FFBS algorithm, see Section 3 for numerical illustrations.

Another modification of the FFBS algorithm based on a Markov chain Monte Carlo (MCMC) sampling step was introduced in ([21], Section 5.2). Instead of sampling from (12), ([21], Section 5.2) proposed to draw a forward path a _1:i−1 in \((a^{k}_{1:i-1})_{1\le k \le N}\) and a sate a _i in {1,…,J} according to:

where \(\left (\widetilde {\vartheta }^{k}_{i-1}\right)_{1\le k \le N}\) are adjustment multipliers and \(\widetilde {q}\left (a_{i}|a^{k}_{1:i-1},a_{i+1:n},y_{1:n}\right)\) is a proposal kernel chosen by the user. This means that an ancestral path \(a^{\star }_{1:i-1}\) is sampled in \(\left (a^{k}_{1:i-1}\right)_{1\le k \le N}\) with weights \(\left (\widetilde {\vartheta }^{k}_{i-1}\right)_{1\le k \le N}\) and \(a^{\star }_{i}\) is sampled from \(\widetilde {q}(\cdot |a^{\star }_{1:i-1},a_{i+1:n},y_{1:n})\). Then, the proposed sequence \(a^{\star }_{1:i}\) is accepted or rejected using the usual Metropolis-Hastings acceptance ratio. The choice of MCMC rejuvenation has interesting practical consequences as the computation of the acceptance ratio only requires to compute the posterior probability (11) for the proposed sequence \(a^{\star }_{1:i}\) while our technique is based on the computation of (11) for all combinations of sequences \(\left (a^{k}_{1:i-1}\right)_{1\le k \le N}\) and states a _i ∈{1,…,J}. Sampling from (12) is computationally more intensive, especially when N is large, but our method is based on a direct approximation of p(a _1:i |a _i+1:n ,y _1:n ) based on \(\left (a^{k}_{1:i-1}\right)_{1\le k \le N}\) and a _i+1:n instead of approximate MCMC draws.

Rao-Blackwellized two-filter smoother

Rao-Blackwellized two-filter smoother of [3]

Contrary to the previous methods, two-filter-based smoothers are designed to compute approximations of marginal smoothing distributions (usually the posterior distribution of one or two consecutive regimes given all the observations). Briers et al. [3] introduced the following decomposition of the smoothing distributions for all 2≤i≤n:

The first term on the right hand side may be approximated using the forward filter by noting that

In the forward pass described in Section 2.1, a set of possible sequences of regimes \(a_{1:i-1}^{k}\) associated with importance weights \(\omega _{i-1}^{k}\), 1≤k≤N are sampled to approximate p(a _i−1,z _i−1|y _1:i−1). This provides a normalized approximation p ^N(a _i ,z _i |y _1:i−1) of p(a _i ,z _i |y _1:i−1). Define

Then,

As the function (a _i ,z _i )↦p(y _i:n |a _i ,z _i ) is not a probability density function, approximating the second term of (13) using SMC samples is not straightforward. The backward filter uses artificial densities to introduce a surrogate target density function which may be approximated recursively using SMC methods. Then, the forward and backward weighted samples are combined using (13) to approximate p(a _i ,z _i |y _1:n ). Following [3], for any probability densities \((\gamma ^{}_{i})_{1\le i \le n}\), define the following joint probability densities:

and, for all 1≤i≤n−1,

Note that this choice differs slightly from [3] where it is advocated to set \(\gamma ^{}_{i}\) as the product of two independent densities \(\gamma _{i}^{a}(a_{i})\) and \(\gamma _{i}^{z}(z_{i})\). As the accuracy of the algorithm relies heavily on a proper tuning of this artificial density, a more general choice of \(\gamma ^{}_{i}\) is considered in this paper. By Lemma 1, these probability densities may be used to approximate the quantities p(y _i:n |a _i ,z _i ), 1≤i≤n, in (13).

Lemma 1

For all 1≤i≤n−1,

Proof

The proof is postponed to “Appendix: Technical lemmas”. □

By definition of \(\tilde {p}_{i}\) for all 1≤i≤n,

This yields

A set of weighted trajectories \(\left (\tilde {a}^{\ell }_{i:n}\right)_{1\le \ell \le N}\) with importance weights \(\left (\tilde {\omega }^{\ell }_{i}\right)_{1\le \ell \le N}\), 1≤i≤n, may then be sampled recursively backward in time to produce a SMC approximation of \(\tilde {p}_{}(a_{i:n}|y_{i:n})\) as follows.

• For 1≤ℓ≤N, sample \(\tilde {a}^{j}_{n}\sim \tilde {q}_{n}(\cdot)\) and set:

  $$\tilde{\omega}^{\ell}_{n} \propto \frac{\int\gamma^{}_{n}\left(\tilde{a}^{\ell}_{n},z'\right)g_{}\left(\tilde{a}^{\ell}_{n},z';y_{n}\right)\mathrm{d} z'}{\tilde{q}_{n}\left(\tilde{a}^{\ell}_{n}\right)}\,. $$

• For all 1≤i≤n−1, resample the set \(\left (\tilde {a}^{\ell }_{i+1:n}\right)_{1\le j\le N}\) using the normalized weights \(\left (\tilde {\omega }^{\ell }_{i+1}\right)_{1\le j \le N}\). Then, for 1≤ℓ≤N, sample \(\tilde {a}^{j}_{i}\sim \tilde {q}_{i}\left (\tilde {a}^{\ell }_{i+1:n},\cdot \right)\) and set:

  $${}\tilde{\omega}^{\ell}_{i} \propto \frac{Q\left(\tilde{a}^{\ell}_{i},\tilde{a}^{\ell}_{i+1}\right)\int \gamma^{}_{i}\left(\tilde{a}^{\ell}_{i},z'\right)p_{}\left(y_{i:n}|\tilde{a}^{\ell}_{i:n},z'\right)\mathrm{d} z'}{\tilde{q}_{i}\left(\tilde{a}^{\ell}_{i+1:n},\tilde{a}^{\ell}_{i}\right)\int \gamma^{}_{i+1}\left(\tilde{a}^{\ell}_{i+1},z'\right)p_{}\left(y_{i+1:n}|\tilde{a}^{\ell}_{i+1:n},z'\right)\mathrm{d} z'}\,. $$

To obtain uniformly weighted samples at each time step, in the numerical experiments we use

By (15) and (16),

which suggests the following particle approximation \(p^{N}_{}(y_{i:n}|a_{i},z_{i})\) of p(y _i:n |a _i ,z _i ):

The conditional likelihood of the observations given the sequence of states p(y _i:n |a _i:n ,z _i ) can be computed explicitly using a Gaussian backward smoother; these computations are summarized in Lemma 2. In the numerical experiments, \(\gamma ^{}_{i}(a_{i},z_{i})\) is set as a mixture of Gaussian distributions. Note that for such a choice, the integral \(\int \gamma ^{}_{i}(a_{i},z')p_{}(y_{i:n}|a_{i:n},z')\mathrm {d} z'\) may be computed explicitly, see Lemma 3. Then, combining (17) and (14) with (13) provides an approximation of p(a _i ,z _i |y _1:n ) by merging the forward particles \(\left (a^{k}_{i-1}\right)_{1\le k \le N}\) with the backward particles \(\left (\tilde {a}^{k}_{i+1}\right)_{1\le k \le N}\), the support of this SMC approximation of p(a _i ,z _i |y _1:n ) being \(\left (\tilde {a}^{k}_{i+1}\right)_{1\le k \le N}\).

As noted in ([13], Section 2.6), two-filter smoothers are prone to suffer from degeneracy issues when the algorithm associates forward particles at time i−1 with backward particles at time i. The authors illustrate this issue in the case where the hidden state is an AR(2) process. To overcome the weakness of such standard two-filter approaches the particle rejuvenation proposed in Section 2.3.2 follows the idea introduced in [13] where new particles at time i are sampled conditional on \((a^{k}_{1:i-1})_{1\le k \le N}\) and on \(\left (\tilde {a}^{k}_{i+1:n}\right)_{1\le k \le N}\) and appropriately weighted. This allows to produce new particles at time i and to obtain a SMC approximation of p(a _i ,z _i |y _1:n ) whose support is not restricted to \(\left (\tilde {a}^{k}_{i+1}\right)_{1\le k \le N}\). Section 2.3.2 exploits this idea in the specific case of linear and Gaussian models where explicit computations allows to produce an approximation using \(\left (a^{k}_{1:i-1}\right)_{1\le k \le N}\) and \(\left (\tilde {a}^{k}_{i+1:n}\right)_{1\le k \le N}\) with support {1,…,J} and without any additional sampling steps.

Particle rejuvenation of two-filter based algorithms

For 2≤i≤n−1, particle rejuvenation relies on the explicit marginalization:

where \(\psi ^{n}_{i}(a_{i-1:i+1},z_{i-1:i+1})\) is the smoothing distribution of the hidden regimes and states between time indices i−1 and i+1. Note that the EM algorithm requires the approximation of p(a _i−1,z _i−1,a _i−1,z _i−1|y _1:n ) in the E-step, this may be obtained following the same steps by marginalizing explicitly the linear states at time i−2 and i+1. Intermediate computations follow the same steps as for the approximation of p(a _i ,z _i |y _1:n ). First, \(\psi ^{n}_{i}\) may be decomposed as follows:

where the proportionality is with respect to (a _i−1:i+1,z _i−1:i+1). Then, by (18), the smoothing distribution p(a _i ,z _i |y _1:n ) may be written as

where m and g are defined in (3) and (4) and

The backward pass described in Section 2.3.1 produces a sequence of states \(\tilde {a}_{i+1:n}^{\ell }\) associated with importance weights \(\tilde {\omega }_{i+1}^{\ell }\), 1≤ℓ≤N which are used to approximate p(y _i+1:n |a _i+1,z _i+1). Plugging this approximation into (20) provides an approximation \(t^{N}_{i}(a_{i},z_{i},y_{i+1:n})\) of t _i (a _i ,z _i ,y _i+1:n ) integrating over all possible choices (a _i+1,z _i+1). These steps are then combined to form a non normalized SMC approximation of p(a _i ,z _i |y _1:n ) using (19). The normalization of the SMC approximation of p(a _i ,z _i |y _1:n ) is obtained by integrating over the states a _i ,z _i , when p(a _i ,z _i |y _1:i−1) and t _i (a _i ,z _i ,y _i+1:n ) are replaced by p ^N(a _i ,z _i |y _1:i−1) and \(t^{N}_{i}(a_{i},z_{i},y_{i+1:n})\) in (19). Our procedure allows to construct sequence of regimes with non-degenerated importance weights in the combination step. This procedure improves significantly [3] where no marginalization of p(a _i ,z _i |y _1:n ) over the states at times i−1 and i+1 is performed and where the proposed forward and backward paths are directly merged. This method often leads to importance weights which are close to be numerically degenerated. By Lemma 2, the SMC approximation \(p^{N}_{}(y_{i:n}|a_{i},z_{i})\) of p(y _i:n |a _i ,z _i ) is then given by:

where \(\left (\tilde {P}^{\ell }_{i}\right)^{-1} {:=} \tilde {P}_{i}^{-1}\left (\tilde {a}^{\ell }_{i:n}\right)\), \(\tilde {\nu }^{\ell }_{i}{:=} \tilde {\nu }_{i}\left (\tilde {a}^{\ell }_{i:n}\right)\) and \(\tilde {c}^{\ell }_{i} {:=} \tilde {c}^{\ell }_{i}\left (\tilde {a}^{\ell }_{i:n}\right)\) are defined in Lemma 2. Define

Then, by (20), the SMC approximation \(t^{N}_{i}(a_{i},z_{i},y_{i+1:n})\) of t _i (a _i ,z _i ,y _i+1:n ) is given by:

where

In the numerical experiments, \(\gamma ^{}_{i}(a_{i},z_{i})\) is set as a mixture of Gaussian distributions. As explained in Section 2.3.1, the integral \(\int _{z_{i+1}} \gamma ^{}_{i+1}\left (\tilde {a}^{\ell }_{i+1}, z\right)\tilde {p}\left (y_{i+1:n}|\tilde {a}^{\ell }_{i+1:n},z\right) \mathrm {d} z\) may be computed explicitly, see Lemma 3.

This section highlights the improvements brought by the additional Rao-Blackwellization steps for the two-filter and the FFBS approximations of the marginal smoothing distributions in the case where the number of states is J=2. The transition matrix Q is such that the probability of switching from one regime to the other is small, as expected for the WTI crude oil data, see Section 4. First, the algorithms are applied to a simple one-dimensional model with:

The original FFBS algorithm of [21] and the FFBS algorithm with rejuvenation proposed in this paper are used with \(N = \tilde {N} = 25\). For comparable computational costs, the two-filter method of [3] and the method with rejuvenation are run with N=100. The artificial distributions are chosen as \(\gamma ^{}_{i} (a_{i}, z_{i}) = p^{N}(a_{i},z_{i}|y_{1:i-1})\) where p ^N(a _i ,z _i |y _1:i−1) is defined by (14). All these algorithms are compared to the estimation obtained with the proposed FFBS algorithm with rejuvenation and 5000 particles considered as a benchmark value. Figure 1 displays the mean estimation error over 100 independent Monte Carlo runs. The estimation error is defined as the absolute difference between the benchmark value and the estimations given by all algorithms.

Posterior probabilities estimation error for all algorithms

Full size image

In addition, Fig. 2 displays the empirical variance of the estimation for each algorithm. Figures 1 and 2 illustrate that in both cases the additional rejuvenation step improves the accuracy and the variability of SMC smoothers. In addition, even with a sharp choice for the artificial distributions γ _i , 1≤i≤n, FFBS-based methods outperform two-filter-based smoothers for this model.

Empirical variances of the estimation of \(\mathbb {P}(a_{k}=1|Y_{1:n})\) for all algorithms

Full size image

Model

Modeling commodity prices is a crucial step to valuate contingent claims related to energy markets and to optimize storage or extraction strategies. In [14, 25], the authors proposed a model where the spot price of a commodity (S _t ,t≥0) depends on a second factor (δ _t ,t≥0), referred to as the instantaneous convenience yield. This factor plays the role of dividends in equity markets and models the benefit of holding the physical commodity or the storage and maintenance costs required to keep the commodity. In this model, this convenience yield is described as an Ornstein-Uhlenbeck process:

where the parameters (r,σ,κ,α,η,ρ) are constant and \(\left (\left (W_{t}^{1},W_{t}^{2}\right), t \geq 0\right)\) are standard Brownian motions. This model appears to be too restrictive as energy markets are not likely to revert to a single equilibrium value. This assumption is relaxed using Markov switching models to allow several possible regimes for the spot price and the convenience yield. Following [1], the spot price and convenience yield are described in this paper as:

where (a _t ) _t≥0 is a finite state space Markov process. This model allows to exhibit fundamental features of commodity future prices, which typically display different regimes of volatility and/or convenience yield. A two-regime model is already sufficient to produce stylized effects such as contango (increase of future prices) and backwardation (decrease of future prices). Assuming that the switching rate between regimes is negligible compared to the inverse of the discretization period, the discretized version of the spot price and convenience yield Z _i =(lnS _i ,δ _i ) (the sampling period is taken to be 1) is modeled as a CLGM. The explicit integration of this SDE detailed in Lemma 4 yields the following discrete time model for (Z _i ) _i≥2:

where (with \(\overline {H}_{a_{i-1}} {:=} H_{a_{i-1}}H'_{a_{i-1}}\) and τ=t _i −t _i−1):

The observations are Wednesday future contracts of the West Texas Intermediate crude oil (WTI) traded in the Chicago Mercantile Exchange (CME) from 11 January 1995 to 13 November 2013. The contracts are numbered F ₁,F ₂,…,F ₃₆ where F ₁ (or front month) is the earliest delivery future contract, F ₂ is the second earliest delivery future contract and so on. Among these 36 contracts, the four future contracts F ₁,F ₄,F ₆,F ₁₃ are used since their trading volumes and their impacts on the term structures are the most important (F ₁ is the most liquid contract, F ₁₃ characterizes the gap between prices over a one year period, F ₄ and F ₆ are intermediate future contracts that are mostly traded). As in [1], we consider that each future contract has a fixed time to maturity F ₁,F ₄,F ₆,F ₁₃ have time to maturity 4, 16, 26, and 56 weeks. Our time series contains n=975 weekly data with 534 in backwardation and 441 in contango (the backwardation effect is more frequent with crude oil data). At each time t _i =i τ, with τ=0.0192, the observations of the p=4 future prices are \(Y_{i} {:=} \left (\ln \left (F^{(market)}_{i\tau t,m_{1}}\right), \ldots, \ln \left (F^{(market)}_{i\tau t,m_{\mathsf {p}}}\right)\right)'\), where \(F_{t_{i},m}\) is the future price at t _i for a maturity m weeks. A closed form solution for \(F_{t_{i},m}\) may be written:

where \(\mathsf {B}_{0} = \left (\begin {array}{cc}1 & 0 \end {array}\right)\)and B _m =B _m−1 T so that \( \mathsf {B}_{m}= \left (\begin {array}{cc} 1 & -\left (1-\mathrm {e}^{-\kappa m \tau t}\right)/\kappa \end {array}\right)\)and for all 1≤j≤J, A ₀(j)=0, and

Therefore, the observations of the logfuture prices are given, for all 1≤i≤n, by

where η _i is a standard multivariate Gaussian random variable and

The model depends on the parameters:

The aim of this section is to estimate θ and the posterior probabilities \(\mathbb {P}(a_{k}=j|Y_{1:n})\), 1≤k≤n, 1≤j≤J. Given the observations Y _1:n , the EM algorithm introduced in [9] maximizes the incomplete data log-likelihood \(\theta \mapsto \ell _{\theta }^{n}\) defined by

where the complete data likelihood p _θ is given by

Denote by \(\mathbb {E}_{\theta }\left [\cdot \middle |Y_{1:n}\right ]\) the conditional expectation given Y _1:n when the parameter value is set to θ. The EM algorithm iteratively builds a sequence {θ _p } _p≥0 of parameter estimates following the two steps:

1. 1.

   E-step: compute \(\theta \mapsto Q(\theta,\theta _{p}){:=} \mathbb {E}_{\theta _{p}}\left [\log p_{\theta }(a_{1:n},Z_{1:n},Y_{1:n})\middle |Y_{1:n}\right ]\) ;

2. 2.

   M-step: choose θ _p+1 as a maximizer of θ↦Q(θ,θ _p ).

All the conditional expectations involved in Q(θ,θ _p ) are approximated using our two-filter algorithm with rejuvenation to define the SMC approximation θ↦Q ^N(θ,θ _p ) of θ↦Q(θ,θ _p ). As the function θ↦Q ^N(θ,θ _p ) cannot be maximized analytically, the M-step is performed numerically using the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) introduced in [17]. This derivative-free optimization procedure is known to perform well in complex multimodal optimization settings, see e.g. [16].

Numerical results

The initial transition probability in CMA-ES is chosen as Q(1,1)=0.98, Q(2,2)=0.97, and π ₁=π ₂=0.5 where the number 1 represents the backwardation regime and 2 represents the contango regime. The other parameters are initialized as shown in Table 1.

The number of particles is set to N=100, τ=1/52. The interest rate is r=0.0296 as in [1]. The initial guess for the mean and variance of the initial state are

The CMA-ES algorithm is used with an initial standard deviation for the parameters σ _cmaes =0.005, a number of selected search points μ _cmaes =20 and a population size λ _cmaes =100. The algorithm is stopped after 10000 iterations.

In Gibson-Schwartz model [14], a stronger backwardation effect implies a greater value for α for the same values of the other parameters. For the CME WTI Crude Oil, backwardation effect is more frequent than contango effect so that α ₁ should be greater than α ₂. Therefore, this condition is imposed for all simulations in the CMA-ES algorithm. The results after 2500 iterations of the EM algorithm are given in Table 2. The estimated values and standard deviations are obtained with 50 independent runs of the algorithm.

As expected, we obtain σ ₁≥σ ₂, α ₁≥α ₂, η ₁≥η ₂ and ρ ₁≥ρ ₂ at convergence of the EM algorithm. Moreover, Q(1,1)>Q(2,2) corresponds to the prediction that we did from the data description. The fact that σ ₁≥σ ₂ and α ₁≥α ₂ indicates the first regime (backwardation) characterized by a higher value in both volatility and equilibrium level of convenience yield, and the second regime (contango) characterized by a lower value in both volatility and equilibrium convenience yield level. This in accordance with the theory of storage that the volatility of the commodity spot price is high when the inventory is low, and the convenience yield is all the higher as inventory is low.

Figure 3 compares the evolution of future 1M (the nearest contracts) to the term structure observed from CME WTI crude oil, defined as the difference of future 13M and future 1M (to avoid seasonality). The figure shows that it is not necessary to have an inverse relationship between the price of the nearest contract and the term structure. But when a significant drop in the price of the nearest contract occurs, the term structure increases (i.e. in contango).

Log-price (red line) and slope of future curves (blue line)

Full size image

The correlation between the spot price and the convenience yield is positive and high in both two regimes. This is an accordance to what as been observed in most commodity market, see [14]. The slope of future curve decreases in function of maturity.

Figures 4 and 5 display the estimated posterior probabilities of the regimes and the observed future slope. When the future curve is in backwardation (resp. contango), the model is expected to be in the first regime (resp. second regime), except for the period where the slope of the future curve is too small and in the period from December 2008 to April 2009 (beginning of the crisis).

Posterior probability (triangle black) and slope of future curves (blue line)

Full size image

Posterior probability (triangle black) and slope of future curves (blue line)

Full size image

This paper presents Rao-Blackwellized sequential Monte Carlo methods to approximate smoothing distributions in conditionally linear and Gaussian state spaces in a common unifying framework. It also provides different techniques that could be used in the forward filtering pass to improve significantly the usual mixture Kalman filter. The filtering distributions are approximated at each time step by considering all possible offsprings of all ancestral trajectories before discarding degenerated paths instead of resampling the ancestral paths before propagating them at the next time step. The paper investigates the benefit of additional Rao-Blackwellization steps to sample new regimes at each time step conditional on the forward and backward particles. This rejuvenation step uses explicit integration of the hidden linear states before merging the forward and backward filters for two-filter based algorithms or before sampling new states backward in time for FFBS based methods. The paper displays some Monte Carlo experiments with simulated data to illustrate that this additional rejuvenation step improves the performance of the smoothing algorithms with no substantial additional computational costs. They are also applied to commodity markets using WTI crude oil data.

Lemmas 1, 2, and 3 are close to ([3], Proposition 5, Proposition 6). The proofs are detailed in this appendix for completeness.

Proof of Lemma 1

For all 1≤i≤n−1,

which concludes the proof of (15). To prove (16) write,

Therefore,

and the proof is completed upon noting that

□

Lemma 2

For all 1≤i≤n,

where

and, for all 1≤i≤n−1,

with

Proof

The result is proved by backward induction. (24), (25) and (26) follow directly from (2). Assume that for a given 1≤i≤n−1, p(y _i+1:n |a _i+1:n ,z _i+1) is given by (23). Write

with

Let Δ _i+1 and δ _i+1 be given by:

Then, \(\overline {H}_{a_{i+1}}^{-1} + \tilde {P}_{i+1}^{-1}(a_{i+1:n}) = \left (H_{a_{i+1}}\Delta _{i+1}(a_{i+1:n})H'_{a_{i+1}}\right)^{-1}\) and (27), (28) and (29) follows from

□

Lemma 3

For all 1≤i≤n,

where ϕ _μ,Σ is the probability density function of a m dimensional Gaussian random variable with mean μ and variance matrix Σ and

and where c _i , \(\tilde {p}_{i}\) and ν _i are given in Lemma 2.

Proof

By Lemma 2,

The proof is completed noting that

□

Lemma 4

Let (X _t ,δ _t ) _t≥0 be solutions to the following SDE:

\(\left (W_{t}^{1}\right)_{t\ge 0}\) and \(\left (W_{t}^{2}\right)_{t\ge 0}\) are standard Brownian motions such that \(\mathrm {d} \langle W_{t}^{1},W_{t}^{2} \rangle = \rho \mathrm {d} t\). Then, for all t≥0 and h>0,

where ε is a standard 2-dimensional Gaussian random variable and (with \(\overline {H}_{h}{:=} H_{h}'H_{h}\)),

Proof

For all t≥0,

and, as (δ _t )_0≤t≤T is an Ornstein-Uhlenbeck process,

Then,

Defining \(\tilde {W}^{1}_{t} {:=} - (\eta /\kappa) \int _{0}^{t} (1-\mathrm {e}^{-\kappa (t-s)}) \mathrm {d} W^{2}_{s}+\sigma W_{t}^{1}\) and \(\tilde {W}_{t}^{2}{:=} \int _{0}^{t} \eta \mathrm {e}^{\kappa (s-t)}\mathrm {d} W^{2}_{s}\), this yields:

The proof is concluded upon noting that \(\tilde {W}^{1}_{t}\) and \(\tilde {W}^{2}_{t}\) are centered Gaussian random variables such that:

• \( \mathrm{V}\mathrm{a}\mathrm{r}\left[{\overset{\sim }{W}}_t^1\right]={\sigma}^2t+{\eta}^2\left\{t+\left(1-{e}^{-2\kappa t}\right)/\left(2\kappa \right)\right.\left.-2\left(1-{e}^{-\kappa t}\right)/\kappa \right\}/{\kappa}^2-2\rho \eta \sigma \left\{t-\left(1-{e}^{-\kappa t}\right)/\kappa \right\}/\kappa \kern0.3em , \)

• \( \mathrm{V}\mathrm{a}\mathrm{r}\left[{\overset{\sim }{W}}_t^2\right]={\eta}^2\left(1-{\mathrm{e}}^{-2\kappa t}\right)/\left(2\kappa \right)\kern0.3em , \)

• \( \mathrm{C}\mathrm{o}\mathrm{v}\left[{\overset{\sim }{W}}_t^1,{\overset{\sim }{W}}_t^2\right]=\left(\rho \eta \sigma -{\eta}^2/\kappa \right)\left(1-{e}^{-\kappa t}\right)/\kappa +{\eta}^2\left(1-{e}^{-2\kappa t}\right)/\left(2{\kappa}^2\right)\kern0.3em . \)

□

Funding

This work has been developed during a 3-year Ph.D. at Télécom ParisTech and Lunalogic.

Affiliations

1. LTCI, CNRS and Télécom ParisTech, 46 rue Barrault, Paris Cedex 13, 75634, France

   • Ngoc Minh Nguyen

2. Laboratoire de Mathématiques d’Orsay, Univ. Paris-Sud, CNRS, Université Paris-Saclay, Orsay, 91405, France

   • Sylvain Le Corff

3. Centre de Mathématiques Appliquées, UMR 7641, Ecole Polytechnique, Palaiseau, France

   • Éric Moulines

Contributions

All the authors have contributed to the conception of the algorithms and to the redaction of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sylvain Le Corff.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Reprints and Permissions