 Research
 Open Access
 Published:
Online sequential Monte Carlo smoother for partially observed diffusion processes
EURASIP Journal on Advances in Signal Processing volume 2018, Article number: 9 (2018)
Abstract
This paper introduces a new algorithm to approximate smoothed additive functionals of partially observed diffusion processes. This method relies on a new sequential Monte Carlo method which allows to compute such approximations online, i.e., as the observations are received, and with a computational complexity growing linearly with the number of Monte Carlo samples. The original algorithm cannot be used in the case of partially observed stochastic differential equations since the transition density of the latent data is usually unknown. We prove that it may be extended to partially observed continuous processes by replacing this unknown quantity by an unbiased estimator obtained for instance using general Poisson estimators. This estimator is proved to be consistent and its performance are illustrated using data from two models.
Introduction
This paper introduces a new algorithm to solve the smoothing problem for hidden Markov models (HMMs) whose hidden state is a solution to a stochastic differential equation (SDE). These models are referred to as partially observed diffusion (POD) processes in [27]. The hidden state process (X_{ t })_{t≥0} is assumed to be a solution to a SDE, and the only information available is given by noisy observations (Y_{ k })_{0≤k≤n} of the states (X_{ k })_{0≤k≤n} (where X_{ k } stands for \(X_{t_{k}}\)) at some discrete time points (t_{ k })_{0≤k≤n}. The bivariate stochastic process {(X_{ k },Y_{ k })}_{0≤k≤n} is a statespace model such that conditional on the state sequence (X_{ k })_{0≤k≤n} the observations (Y_{ k })_{0≤k≤n} are independent and for all 0≤ℓ≤n the conditional distribution of Y_{ ℓ } given {X_{ k }}_{0≤k≤n} depends on X_{ ℓ } only.
Statistical inference for HMMs often requires to solve Bayesian filtering and smoothing problems, i.e., the computation of the posterior distributions of sequences of hidden states given observations. The filtering problem refers to the estimation, for each 0≤k≤n, of the distributions of the hidden state X_{ k } given the observations (Y_{0},…,Y_{ k }). Smoothing stands for the estimation of the distributions of the sequence of states (X_{ k },…,X_{ p }) given observations (Y_{0},…,Y_{ ℓ }) with 0≤k≤p≤ℓ≤n. These posterior distributions are crucial to compute maximum likelihood estimators of unknown parameters using the observations (Y_{0},…,Y_{ n }) only. For instance, the Estep of the EM algorithm introduced in [9] boils down to the computation of a conditional expectation of an additive functional of the hidden states given all the observations up to time n. Similarly, by Fisher’s identity, recursive maximum likelihood estimates may be computed using the gradient of the log likelihood which can be written as the conditional expectation of an additive functional of the hidden states. See [7, Chapters 10 and 11], [19, 23, 24, 31] for further references on the use of these smoothed expectations of additive functionals applied to maximum likelihood parameter inference in latent data models.
However, in most cases, the exact computation of these expectations is usually not possible explicitly. Sequential Monte Carlo (SMC) methods are popular algorithms to approximate smoothing distributions with random particles associated with importance weights. [17, 22] introduced the first particle filters and smoothers for statespace models by combining importance sampling steps to propagate particles with resampling steps to duplicate or discard particles according to their importance weights. In the case of HMMs, approximations of the smoothing distributions may be obtained using the forward filtering backward smoothing algorithm (FFBS) and the forward filtering backward simulation algorithm (FFBSi) developed respectively in [11, 18, 22], and [16]. Both algorithms require first a forward pass which produces a set of particles and weights approximating the sequence of filtering distributions up to time n. Then, a backward pass is performed to compute new weights (FFBS) or sample trajectories (FFBSi) in order to approximate the smoothing distributions. Recently, [28] proposed a new SMC algorithm, the particlebased rapid incremental smoother (PaRIS), to approximate onthefly (i.e., using the observations as they are received) smoothed expectations of additive functionals. Unlike the FFBS algorithm, the complexity of this algorithm grows only linearly with the number of particles N and contrary to the FFBSi algorithm, no backward pass is required. One of the best features of PaRIS algorithm is that it may be implemented online, using the observations (Y_{ k })_{k≥0} as they are received, without any increasing storage requirements.
Unfortunately, these methods cannot be applied directly to POD processes since some elementary quantities, such as transition densities of the hidden states, are not available explicitly. In the context of SDEs, discretization procedures may be used to approximate transition densities. For instance, the classical EulerMaruyama method, the Ozaki discretization which proposes a linear approximation of the drift coefficient between two observations [29, 32], or Gaussianbased approximations using Taylor expansions of the conditional mean and variance of an observation given the observation at the previous time step, [20, 21, 33]. Other approaches based on Hermite polynomials expansion were also introduced by [1–3] and were extended in several directions recently, see [25] and all the references on the approximation of transition densities therein. However, even the most recent discretizationbased approximations of the transition densities induce a systematic bias in the approximation of the transition densities, see for instance [8].
To overcome this difficulty, [13] proposed to solve the filtering problem by combining SMC methods with an unbiased estimate of the transition densities based on the generalized Poisson estimator (GPE). In this case, only the Monte Carlo error has to be controlled as there is no Taylor expansion to approximate unknown transition densities, i.e., no discretization scheme is used. The only solution to solve the smoothing problem for POD processes using SMC methods without any discretization procedure has been proposed in [27] and extends the fixedlag smoother of [26]. Using forgetting properties of the hidden chain, the algorithm improves the performance of [13] to approximate smoothing distributions but at the cost of a bias, this time due to the fixed lag approximation, that does not vanish as the number of particles grows to infinity.
In this paper, we propose to use SMC methods to obtain consistent approximations of smoothing expectations of POD processes by extending the PaRIS algorithm. The proposed algorithm allows to approximate smoothed expectations of additive functionals online, with a complexity growing only linearly with the number of particles and without any discretization procedure or Taylor expansion of the transition densities. The crucial and simple result (Lemma 1) of the application of the PaRIS algorithm to POD processes is that the acceptance rejection mechanism introduced in [10] ensuring the linear complexity of the procedure is still correct when the transition densities are replaced by unbiased estimates. The usual FFBS and FFBSi algorithms may not extend this easily since they both require the computation of weights defined as ratios involving the transition densities, thus replacing these unknown quantities by unbiased estimates does not lead to unbiased estimators of the weights. The linear version of the FFBSi algorithm proposed in [10] could be extended in a similar way as PaRIS algorithm but it would still require a backward pass and would not be an online smoother. The proposed generalized random version of PaRIS algorithm, hereafter named GRand PaRIS algorithm, may not only be applied to POD processes but also to any general statespace model where the transition density of the hidden chain may be approximated using a positive and unbiased estimator.
Section 2 describes the model and the smoothing quantities to be estimated. Section 3 provides the algorithm to approximate smoothed additive functionals using unbiased estimates of the transition density of the hidden states. This section also details the application of this algorithm when the transition density are approximated using a GPE. In Section 4, classical convergence results for SMC smoothers are extended to the setting of this paper and illustrated with numerical experiments in Section 5. All proofs are postponed to Appendix.
Model and framework
Let (X_{ t })_{t≥0} be defined as a weak solution to the following SDE in \(\mathbb {R}^{d}\):
where (W_{ t })_{t≥0} is a standard Brownian motion on \(\mathbb {R}^{d}\), \(\alpha : \mathbb {R}^{d}\to \mathbb {R}^{d}\), and \(\Gamma : \mathbb {R}^{d}\to \mathbb {R}^{d\times d}\). The solution to (1) is supposed to be partially observed at times t_{0}=0,…,t_{ n } through an observation process (Y_{ k })_{0≤k≤n} in \(\left (\mathbb {R}^{m}\right)^{n+1}\). In the following, for all 0≤k≤n, the state \(X_{t_{k}}\) at time k is referred to as X_{ k }. For all 0≤k≤n, the distribution of Y_{ k } given X_{ k } has a density with respect to a reference measure λ on \(\mathbb {R}^{m}\) given by g(X_{ k },·). For the sake of simplicity, the shorthand notation g_{ k }(X_{ k }) for g(X_{ k },Y_{ k }) is used. The distribution of X_{0} has a density with respect to a reference measure μ on \(\mathbb {R}^{d}\) given by χ. For all 0≤k≤n−1, the conditional distribution of X_{k+1} given X_{ k } has a density q_{ k }(X_{ k },·) with respect to μ.
For all 0≤k≤k^{′}≤n, the joint smoothing distributions of the hidden states are defined, for all measurable function h on \((\mathbb {R}^{d})^{k'k + 1}\), by:
and ϕ_{ k }=ϕ_{k:kk} denotes the filtering distributions. The aim of this paper is to approximate expectations of the form:
when \(\{h_{k}\}_{k=0}^{n1}\) are given functions on \(\mathbb {R}^{d}\times \mathbb {R}^{d}\). Smoothed additive functionals as (2) are crucial for maximum likelihood inference of latent data models. These quantities appear naturally when computing the Fisher score in hidden Markov models or the intermediate quantity of the expectation maximization algorithm (see Section 5). They are also pivotal to design online expectation maximizationbased algorithms which motivates the method introduced in this paper that does not require growing storage and can process observations online.
The algorithm proposed in this paper is based on sequential Monte Carlo methods which offer a flexible framework to approximate such distributions with weighted empirical measures associated with random samples. At each time step, the samples are moved randomly in \(\mathbb {R}^{d}\) and associated with importance weights. In general situations, the computation of these importance weights involve the unknown transition density of the process (1). The solution introduced in Section 3 requires an unbiased estimator of these unknown transition densities. Moreover, this estimator must be almost surely positive and upper bounded. Statistical inference of stochastic differential equations is an active area of research, and several solutions have been proposed to design unbiased estimates of these transition densities. Those estimators require different assumptions on the model (1), we provide below several solutions that can be investigated.
General Poisson estimators This paper focuses mainly on GPEs which have been widely used recently and applied in a variety of disciplines. These estimators require that the diffusion coefficient Γ is constant and equal to the identity matrix, see [13]. They may be applied to reducible SDE for which there exists an invertible and infinitely differentiable function η such that the process {Z_{ t }=η(X_{ t })}_{t≥0} satisfies the SDE z_{0}=η(x_{0}) and
By Ito’s formula, it is straightforward to show that, in the case of a reducible diffusion, the Jacobian matrix of η satisfies
and, in the case d=1,
In the case of a scalar diffusion, this Lamperti transform is given by
In the general case, [3, Proposition 1] shows that when Γ is nonsingular, the SDE is reducible if and only if, for all 1≤i,j,k≤d,
In the case of a diagonal matrix Γ (4) is equivalent to assume that Γ is such that for each 1≤i≤d, Γ_{ i,i } depends on x_{ i } only. [3] notes that the reducibility condition (4) holds also for some nondiagonal matrices Γ. This is true in particular in the case d=2 for stochastic volatility models where σ is of the form:
GPEs consider that the process (X_{ t })_{t≥0} satisfies the SDE (1) with Γ being the identity matrix, i.e., we consider a diffusion after the application of the Lamperti transform. In addition, designing GPEs also requires that

i)
α is of the form α(x)=∇_{ x }A(x) where \(A: \mathbb {R}^{d} \to \mathbb {R}\) is a twice continuously differentiable function ;

ii)
the function x↦(∥α(x)∥^{2}+△A(x))/2 is lower bounded where △ is the Laplace operator.
Assumption (i) is somewhat restrictive as it requires α to derive from a scalar potential, however, it has natural applications in many fields such as movement ecology, see [15]. Assumption (ii) is a technical condition which ensures that exact sampling of processes solution to (1) using acceptance rejection methods, see for instance [4, 5, 13]. In addition to provide an unbiased estimate of the transition density, the GPE ensure that this estimate is almost surely positive. Moreover, as detailed below, under additional conditions, a GPE that is almost surely upper bounded can be defined.
Continuous importance samplingbased estimators In the case the previous assumptions are not fulfilled, in particular assumption (i), alternatives to GPEs are given by continuous importance sampling procedures for SDE. In [34], for each 0≤k≤n−1, the transition density between t_{ k } and t_{k+1} is expressed as an infinite expansion obtained using the Kolmogorov backward operator associated with (1). This analytical expression of the transition density is not tractable and is estimated by updating random samples at random times between t_{ k } and t_{k+1} using tractable proposal distributions (for instance, based on an Euler discretization of the original SDE). Then, these samples are associated with random weights to ensure that the proposed estimator is unbiased. More recently, [14] extended the discrete time importance sampling estimator by introducing updates at random times associated with a renewal process. The random samples are weighted using the Kolmogorov forward operator associated with the SDE which relies on the first two order derivatives of the drift and diffusion coefficients (and is therefore tractable).
The unbiasedness of these procedures and the controls of the variability of the estimates require moments assumptions and Holder type conditions on the parameters of the SDE (1). Their efficiency require a fair amount of tuning as they highly depend on the proposal densities used to obtain the Monte Carlo samples and the point processes generating the underlying random times. In addition to unbiasedness, the proposed algorithm in this work requires that the estimator of the transition density is almost surely positive and upper bounded. This implies additional assumptions on the SDE depending on the chosen estimate and could lead to interesting perspectives.
The generalized random PaRIS algorithm
The algorithm is based on the following link between the filtering and smoothing distributions for additive functionals, see [28]:
The approximation of (5) requires first to approximate the sequence of filtering distributions. Sequential Monte Carlo methods provide an efficient and simple solution to obtain these approximations using sets of particles \(\left \{\xi ^{\ell }_{k}\right \}_{\ell =1}^{N}\) associated with weights \(\left \{\omega ^{\ell }_{k}\right \}_{\ell =1}^{N}\), 0≤k≤n.
At time k=0, N particles \(\left \{\xi ^{\ell }_{0}\right \}_{\ell =1}^{N}\) are sampled independently according to \(\xi ^{\ell }_{0} \sim \eta _{0}\), where η_{0} is a probability density with respect to μ. Then, \(\xi ^{\ell }_{0}\) is associated with the importance weights \(\omega _{0}^{\ell } = \chi \left (\xi ^{\ell }_{0}\right)g_{0} \left (\xi ^{\ell }_{0}\right)/\eta _{0}\left (\xi ^{\ell }_{0}\right)\). For any bounded and measurable function h defined on \(\mathbb {R}^{d}\), the expectation ϕ_{0}[h] is approximated by
Then, for 1≤k≤n, using \(\left \{\left (\xi ^{\ell }_{k1},\omega ^{\ell }_{k1}\right)\right \}_{\ell =1}^{N}\), the auxiliary particle filter of [30] samples pairs \(\left \{\left (I^{\ell }_{k},\xi ^{\ell }_{k}\right)\right \}_{\ell =1}^{N}\) of indices and particles using an instrumental transition density p_{k−1} on \(\mathbb {R}^{d}\times \mathbb {R}^{d}\) and an adjustment multiplier function 𝜗_{ k } on \(\mathbb {R}^{d}\). Each new particle \(\xi ^{\ell }_{k}\) and weight \(\omega ^{\ell }_{k}\) at time k are computed following these steps:


choose a particle index \(I^{\ell }_{k}\) at time k−1 in {1,…,N} with probabilities proportional to \(\omega _{k1}^{j} \vartheta _{k} \left (\xi ^{j}_{k1}\right)\), for j in {1,…,N} ;


sample \(\xi ^{\ell }_{k}\) using this chosen particle according to \(\xi ^{\ell }_{k} \sim p_{k1}\left (\xi ^{I^{\ell }_{k}}_{k1},\cdot \right)\) ;


associate the particle \(\xi ^{\ell }_{k}\) with the importance weight:
$$ \omega^{\ell}_{k} := \frac{q_{k1}\left(\xi_{k1}^{I^{\ell}_{k}},\xi^{\ell}_{k}\right)g_{k}\left(\xi^{\ell}_{k}\right)}{\vartheta_{k}\left(\xi^{I^{\ell}_{k}}_{k1}\right) p_{k1} \left(\xi_{k1}^{I^{\ell}_{k}},\xi^{\ell}_{k}\right)}\;. $$(6)
The expectation ϕ_{ k }[h] is approximated by
The most simple choice for p_{k−1} and 𝜗_{ k } is the bootstrap filter proposed by [17] which sets p_{k−1}=q_{k−1} and for all \(x\in \mathbb {R}^{d}\), 𝜗(x)=1. In the case of POD processes, q_{k−1} is unknown but it can be replaced by any approximation to sample the particles as any choice of p_{k−1} can be made. The approximation can be obtained using a discretization scheme such as Euler method or a Poissonbased approximation as detailed below. A more appealing choice is the fully adapted particle filter which sets for all \(x,x'\in \mathbb {R}^{d}\), p_{k−1}(x,x^{′})∝q_{k−1}(x,x^{′})g_{ k }(x^{′}) and for all \(x\in \mathbb {R}^{d}\), \(\vartheta (x) = \int q_{k1}\left (x,x'\right)g_{k}\left (x'\right)\mu \left (\mathrm {d} x'\right)\). Here, again q_{k−1} has to be replaced by an approximation. In Section 5, it is replaced by the Gaussian approximation provided by a Euler scheme which leads to a Gaussian proposal density p_{k−1} as the observation model is linear and Gaussian.
The PaRIS algorithm uses the same decomposition as the FFBS algorithm introduced in [12] and the FFBSi algorithm proposed by [16] to approximate smoothing distributions. It combines both the forwardonly version of the FFBS algorithm with the sampling mechanism of the FFBSi algorithm. It does not produce an approximation of the smoothing distributions but of the smoothed expectation of a fixed additive functional and thus may be used to approximate (2). Its crucial property is that it does not require a backward pass, the smoothed expectation is computed onthefly with the particle filter and no storage of the particles or weights is needed.
PaRIS algorithm relies on the following fundamental property of T_{ k }[H_{ k }] when H_{ k } is as in (2):
Therefore, [28] introduces sufficient statistics \(\tau ^{i}_{k}\) (starting with \(\tau ^{i}_{0} = 0\), 1≤i≤N), approximating \(T_{k}\left [H_{k}\right ]\left (\xi ^{i}_{k}\right)\), for 1≤i≤N and 0≤k≤n. First, replacing ϕ_{k−1} by \(\phi ^{N}_{k1}\) in the last equation leads to the following approximation of \(T_{k}\left [H_{k}\right ]\left (\xi ^{i}_{k}\right)\):
where
Computing exactly these approximations would lead to a complexity growing quadratically with N because of the normalizing constant in (8). Therefore, PaRIS algorithm sample particles in the set \(\left \{\xi ^{j}_{k1}\right \}_{j=1}^{N}\) with probabilities \(\Lambda _{k}^{N}(i,\cdot)\) to approximate the expectation (7) and produce \(\tau ^{i}_{k}\). Choosing \(\tilde {N}\ge 1\), at each time step 0≤k≤n−1 these statistics are updated according to the following steps:

(i)
Run one step of a particle filter to produce \(\left \{\left (\xi ^{\ell }_{k}, \omega ^{\ell }_{k}\right)\right \}\) for 1≤ℓ≤N.

(ii)
For all 1≤i≤N, sample independently \(J_{k}^{i,\ell }\) in {1,…,N} for \(1\le \ell \le \widetilde N\) with probabilities \(\Lambda _{k}^{N}(i,\cdot)\), given by (8).

(iii)
Set
$$\tau^{i}_{k+1} := \frac{1}{\widetilde{N}} \sum^{\widetilde{N}}_{\ell=1} \left\{ \tau^{J_{k}^{i,\ell}}_{k} + h_{k} \left(\xi^{J_{k}^{i,\ell}}_{k}, \xi^{i}_{k+1}\right) \right\}\;. $$
Then, (2) is approximated by
It is clear from steps (i) to (iii) that each time a new observation Y_{n+1} is received, the quantities \(\left (\tau _{n+1}^{i}\right)_{1\le i \le N}\) can be updated only using Y_{n+1}, \(\left (\tau _{n}^{i}\right)_{1\le i \le N}\) and the particle filter at time n. This means that storage requirements do not increase when processing additional data.
As proved in [28], the algorithm is asymptotically consistent (as N goes to infinity) for any precision parameter \(\tilde N\). However, there is a significant qualitative difference between the cases \(\tilde {N} = 1\) and \(\tilde {N} \geq 2\). As for the FFBSi algorithm, when there exists σ_{+} such that 0<q_{ k }<σ_{+}, PaRIS algorithm may be implemented with \(\mathcal {O}(N)\) complexity using the acceptreject mechanism of [10].
In general situations, PaRIS algorithm cannot be used for stochastic differential equations as q_{ k } is unknown. Therefore, the computation of the importance weights \(\omega _{k}^{\ell }\) and of the acceptance ratio of [10] is not tractable. Following [13, 27], filtering weights can be approximated by replacing \({q_{k}}\left (\xi ^{\ell }_{k},\xi _{k+1}^{i}\right)\) by an unbiased estimator \(\widehat {q}_{k}\left (\xi ^{\ell }_{k},\xi _{k+1}^{i};\zeta _{k}\right)\), where ζ_{ k } is a random variable in \(\mathbb {R}^{q}\) such that
where for all 0≤k≤n,
Practical choices for ζ_{ k } are discussed below, see for instance (14) which presents the choice made for the implementation of such estimators in our context. In the case where q_{ k } is unknown, the filtering weights in (6) then become
In Algorithm 1, M independent copies \(\left (\zeta ^{m}_{k1}\right)_{1\le m \le M}\) of ζ_{k−1} are sampled and the empirical mean of the associated estimates of the transition density are used to compute \(\widehat {\omega }^{\ell }_{k}\) instead of a single realization. Therefore, to obtain a generalized random version of PaRIS algorithm, we only need to be able to sample from the discrete probability distribution \(\Lambda _{k}^{N}(i,\cdot)\) in the case of POD processes.
Consider the following assumption: for all 0≤k≤n−1, there exists a random variable \(\hat {\sigma }^{k}_{+}\) measurable with respect to \(\mathcal {G}_{k+1}^{N}\) such that,
Lemma 1
Assume that (10) holds for some 0≤k≤n−1. For all 1≤i≤N, define the random variable \(J_{k}^{i}\) as follows:
Then, the conditional probability distribution given \(\mathcal {G}_{k+1}^{N}\) of \(J_{k}^{i}\) is \(\Lambda _{k}^{N}(i,\cdot)\).
Proof
See Appendix. □
Note that Lemma 1 still holds if assumption (10) is relaxed and replaced by
It is worth noting that under assumptions (10) or (11), the linear complexity property of PaRIS algorithm is ensured. The following assumption can also be considered. For all 1≤i≤N,
If only assumption (12) holds, the algorithm has a quadratic complexity. The bound of (10) is uniform (it does not depend on the particles) and can be used for every particle 1≤i≤N. However, this bound can be large (with respect to the simulated set of particles) for the algorithm of Lemma 1. The bound of (12) requires N computations per particle (therefore, N^{2} computations). However, it is clear that this second bound is sharper that the one of (10) for the acceptance rejection procedure and may lead to a computationally more efficient algorithm.
Bounded estimator ofq_{ k } using GPEs For \(x, y \in \mathbb {R}^{d}\), by Girsanov and Ito’s formulas, the transition density q_{ k }(x,y) of (1) satisfies, with Δ_{ k }=t_{k+1}−t_{ k },
where \(\mathbb {W}^{x,y,\Delta _{k}}\) is the law of Brownian bridge starting at x at 0 and hitting y at Δ_{ k }, \(\phantom {\dot {i}\!}(\mathsf {w}_{t})_{0\leq t \leq \Delta _{k}}\) is such a Brownian bridge, \(\varphi _{\Delta _{k}}(x,y)\) is the p.d.f. of a normal distribution with mean x and variance Δ_{ k }, evaluated at y and \(\phi :\mathbb {R}^{d}\to \mathbb {R}\) is defined as
Assume that there exist random variables L_{ w } and U_{ w } such that for all 0≤s≤Δ_{ k }, L_{ w }≤ϕ(w_{ s })≤U_{ w }. The performance of the estimator depends on the choice of L_{ w } and U_{ w } which is specific to the SDE. In the case of the models analyzed in Section 5, these bounds are discussed in [13] for the SINE model and in [27] for the loggrowth model. Note that in the case where ϕ is not upper bounded, [5] proposed the EA3 algorithm. This layered Brownian bridge construction first samples random variables to determine in which layer the Brownian bridge lies before simulating the bridge conditional on the event that it belongs to the layer. By continuity of ϕ, L_{ w }, and U_{ w } can be computed easily.
Let κ be a random variable taking values in \(\mathbb {N}\) with distribution μ and (U_{ j })_{1≤j≤κ} be independent uniform random variables on [0,Δ_{ k }] and ζ_{ k }={κ,w,U_{1},…,U_{ κ }}. As shown in [13], a positive unbiased estimator is given by
Interesting choices of μ are discussed in [13], and we focus here on the so called GPE1, where μ is a Poisson distribution with intensity (U_{ w }−L_{ w })Δ_{ k }. In that case, the estimator (13) becomes
On the r.h.s. of (14), the product over κ elements is bounded by 1. Therefore, a sufficient condition to satisfy one of the assumptions (10)–(12) is that the function
is upper bounded almost surely by \(\hat {\sigma }^{k}_{+}\). In particular, if L_{ w } is bounded below almost surely, (14) always satisfies assumption (12) and Algorithm 1 can be used. This condition is always satisfied for models in the domains required for the applications of exact algorithms EA1, EA2, and EA3 defined in [6].
When (10) or (11) holds, it can be nonetheless of practical interest to choose the bounds \(\hat {\sigma }^{k,i}_{+}\), 1≤i≤N, corresponding to (12). Indeed, this might increase significantly the acceptance rate of the algorithm, and therefore reduce the number of draws of the random variable ζ_{ k }, which has a much higher cost than the computation of \(\rho _{\Delta _{k}}\), as it requires simulations of Brownian bridges. Moreover, this option allows to avoid numerical optimization if no analytical expression of \(\hat {\sigma }_{+}^{k}\) is available. In practice, this seems more efficient in terms of computational time when N has moderate values.
Convergence results
Consider the following assumptions.

H1 (i) For all k≥0 and all \(x\in \mathbb {R}^{d}\), g_{ k }(x)>0.

(ii) \(\underset {k\geq 0}{\sup }g_{k}_{\infty } < \infty \).
Assumption H1 only involves the marginal likelihood g_{ k } of the observations and does not depend on the unbiased estimation of the transition density. In the case where the observations are given as in the Section 5, this assumption holds as soon as the variance of the observation is bounded away from zero.

H2 \(\underset {k\geq 1}{\sup }\vartheta _{k}_{\infty } < \infty \), \(\underset {k\geq 1}{\sup }p_{k}_{\infty } < \infty \), and \(\underset {k\geq 1}{\sup }\widehat {\omega }_{k}_{\infty } < \infty \), where
$$\begin{array}{*{20}l} \widehat{\omega}_{0}(x) &= \frac{\chi(x)g_{0}(x)}{\eta_{0}(x)} \quad\text{and for}\; k\ge1,\\ \widehat{\omega}_{k}\left(x,x';z\right) &= \frac{\widehat{q}_{k1}\left(x,x';z\right)g_{k}(x')}{\vartheta_{k}(x) p_{k1} (x,x')}\;. \end{array} $$
Assumption H2 depends on the algorithm used to estimate the transition densities and on the tuning parameters of the SMC filter. The most common choice is 𝜗_{ k }=1 so that under H1, the only requirement is to control \(\widehat {q}_{k1}\) and p_{k−1}. For instance, in the case of the GPE1, as explained in Section 3, H2 is satisfied if ϕ is upper bounded (as for the EA1).
Lemma 2
For all 0≤k≤n−1, the random variables \(\left \{\widehat {\omega }_{k+1}^{i}\tau _{k+1}^{i}\right \}_{i=1}^{N}\) are independent conditionally on \(\mathcal {F}_{k}^{N}\) and
Proof
See Appendix □
Proposition 1
Assume that H1 and H2 hold and that for all 1≤k≤n, osc(h_{ k })<+∞. For all 0≤k≤n and all \(\widetilde {N}\ge 1\), there exist b_{ k },c_{ k }>0 such that for all N≥1 and all \(\varepsilon \in \mathbb {R}_{+}^{\star }\),
Proof
See Appendix □
Numerical experiments
This section investigates the performance of the proposed algorithm with the sine and loggrowth models (Fig. 1). In both cases, the proposal distribution p_{ k } is chosen as the following approximation of the optimal filter (or the fully adapted particle filter in the terminology of [30]):
where \(\tilde {q}_{k1}(x_{k1},x_{k})\) is the p.d.f. of Gaussian distibution with mean α(x_{k−1})Δ_{ k } and variance Δ_{ k }I_{ d }, i.e., the Euler approximation of Eq. (1). As the observation model is linear and Gaussian, the proposal distribution is therefore Gaussian with explicit mean and variance.
In order to evaluate the performance of the proposed algorithm, the following strategy has been chosen. We compare the estimation of the EM intermediate quantity with the one obtained by the fixed lag method of [27], for different values of the lag (namely, 1,2,5,10,50). The particle approximation of \(\mathcal {Q}(\theta,\theta)\) for each model is computed using each algorithm, see Fig. 2 for the SINE model and Fig. 3 for the loggrowth model. This estimation is performed 200 times to obtain the estimates \(\widehat {Q}_{1},\dots,\widehat {Q}_{200}\), using \(\tilde {N}=2\) particles for PaRIS algorithm, and M=30 replications for the Monte Carlo approximation \(\widehat q_{k}\) of each q_{ k }. Moreover, the E step requires the computation of a quantity such as (2) with h_{ k }= logg_{ k }+ logq_{ k }. logq_{ k } is not available explicitly and is approximated using the unbiased estimator proposed in [27, Appendix B] based on 30 independent Monte Carlo simulations. In order to obtain a reference value for our study, the intermediate quantity of the EM algorithm is also estimated 30 times using the GRand PaRIS algorithm with N = 5000 particles, the reference value is then computed as the arithmetic mean of these 30 estimations, and denoted by \(\widehat {Q}_{\star }\). Figures 2 and 3 display this estimate for an example with one simulated data set. The GRand Paris algorithm is performed using N = 400 particles in both cases, the fixed lag technique using N = 1600 so that both estimations require similar computational times, resulting a fair comparison. On a personal computer^{Footnote 1}, for the parameters mentioned above, it takes around 25 s to perform each E step.
The SINE model
The performance of the GRand PaRIS algorithm are first highlighted using the SINE model, where (X_{ t })_{t≥0} is supposed to be the solution to
This simple model has no explicit transition density, however, GPEs may be computed by simulating Brownian bridges. The process solution to (15) is observed regularly at times t_{0}=0,…,t_{100}=50 through the observation process (Y_{ k })_{0≤k≤100}:
where the (ε_{ k })_{0≤k≤100} are i.i.d. \(\mathcal {N}\left (0, \sigma ^{2}_{\text {obs}}\right)\), the resulting set of model parameters is θ=(μ,σ_{obs}). In the example displayed in Fig. 1, we set μ=0 and σ_{obs}=1.
In the case of the SINE model, the estimator \(\widehat {q}_{k}\) defined by Eq. (14) satisfies both (10) and (11). The corresponding bound \(\widehat {\sigma }_{+}^{k}\) can be obtained using numerical optimization. If that bound is chosen, the GRand PaRIS algorithm has linear complexity in the number of particles. As an alternative, it is worth noting here that the bounds \(\widehat {\sigma }_{+}^{k,i}\), 1≤i≤N, defined by (12) can also be used. This method has a quadratic cost in the number of particles but provides the optimal bound for the algorithm of Lemma 1. This may reduce significantly the expected time before acceptance, in particular when the time step Δ_{ k } is large. In the experiment configuration presented here, both bounds resulted in an equivalent computational time.
This same experiment was reproduced on 100 different simulated data sets. For each simulation s, the empirical absolute relative bias arb_{ s } and the empirical absolute coefficient of variation acv_{ s } are computed as
where \(m\left (\widehat {Q}^{s}\right)\) and \(\sigma \left (\widehat {Q}^{s}\right)\) are the empirical mean and standard deviation of the sequence \(Q_{1}^{s},\dots,Q_{200}^{s}\). For each estimation method, the resulting distributions of arb_{1},…,arb_{100} and acv_{1},…,acv_{100} are shown in Figs. 4 and 5.
The GRand PaRIS algorithm outperforms the fixedlag methods for any value of the lag as the bias is the lowest (it is already negligible for N = 400) and with a lower variance than fixed lag estimates with negligible bias (i.e., in this case, lags larger than 10). Small lags lead to strongly biased estimates for the fixedlag method, and unbiased estimates are at the cost of a large variance. It is worth noting here that the lag for which the bias is small is model dependent.
Generalized EM procedure The performance of our algorithm is also assessed in the case where θ and the variance \(\sigma ^{2}_{\text {obs}}\) are unknown and estimated using a generalized EM algorithm. The study is done using a data set with n = 200 observations simulated with μ = 0 an d \( \sigma ^{2}_{\text {obs}}~=~1\). The GRand PaRIS algorithm is used to perform the E step, with the same settings as before for N, \(\tilde {N}\), and M. As there is no closed form solution to compute the M step of the EM algorithm and propose new parameter estimates, we use a generalized EM procedure: given the current estimation \(\theta ^{(k)}:= \left (\mu ^{(k)}, \sigma _{\text {obs}}^{(k)} \right)\), the function \(\mathcal {Q}\left (\cdot, \theta ^{(k)}\right)\) is approximated for 50 new candidates θ_{1},…,θ_{50} chosen by the user. The new estimate is set as
This procedure has the nice property of using the same particle filter and the same retrospective sampling of Lemma 1 for all candidates, avoiding to repeat this time consuming procedure. The number of candidates and the way to choose them is problem dependent and then left to the user. In our case, we sampled candidates using Gaussian distributions around the current estimate θ^{(k)}, decreasing the variance when k increases. Figures 6 and 7 illustrate the performance of the estimation for 12 different initializations of μ (resp. σ_{obs}) uniformly chosen in ]−π,π[ (resp. in ]0,6[), illustrating a convergence after only a few iterations of the EM procedure.
Loggrowth model
Following [6] and [28], the performance of the proposed algorithm are also illustrated with the loggrowth model (Fig. 8) defined by
In order to use the exact algorithms of [6] and the GPE of [13], we consider (16) after the Lamperti transform, i.e., the process defined by X_{ t } = η(Z_{ t }), with η(z): = − log(z)/σ, which satisfies the following SDE:
In this case, the conditions of the exact Algorithm 2 defined in [6] are satisfied, as for any \(m \in \mathbb {R}\) there exists U_{ m } such that for all x≥m, ψ(x):=α^{2}(x)+α^{′}(x)≤U_{ m }. Moreover, ψ is lower bounded uniformly by L. Then, GPE estimators may be computed by simulating the minimum of a Brownian bridge, and simulating Bessel bridges conditionally to this minimum, as proposed by [6].
The process solution to (17) is observed regularly at times t_{0}=0,…,t_{50}=100 through the observation process (Y_{ k })_{0≤k≤50} defined as
where the (ε_{ k })_{0≤k≤50} are i.i.d. \( \mathcal {N}\left (0,\sigma ^{2}_{obs}\right)\). The parameters are given by
In the case of the loggrowth model, the estimator \(\widehat {q}_{k}(\cdot)\) defined by Eq. (14) satisfies (11), leading to a GRand PaRIS algorithm with linear complexity in the number of particles. However, the remarks about the bound \(\widehat {\sigma }_{+}^{k}\) made for the SINE model above still hold in this case. The intermediate quantity of the EM algorithm is evaluated as for the SINE model, see Figs. 3, 9, and 10.
The results for the fixedlag technique are similar to the ones presented in [27, Figure 1] using the same model. For small lags, the variance of the estimates is small, but the estimation is highly biased. The bias rapidly decreases as the lag increases, together with a great increase of variance. Again, the GRand PaRIS algorithm outperforms the fixed lag smoother as it shows a similar (vanishing) bias as the fixed lag for the largest lag and a smaller variance than the fixed lags estimates with negligible bias.
Note that in this case, the Lamperti transform to obtain a diffusion with a unitary diffusion term depends on σ. The process (X_{ t })_{t≥0} is a function of σ and is not directly observed if σ is unknown, which prevents a direct use of an EM algorithm to estimate σ. Following [6, Section 8.2], this may be overcome with a twostep transformation of the process (Z_{ t })_{t≥0}.
Conclusions
This paper presents a new online SMC smoother for partially observed differential equations. This algorithm relies on an acceptancerejection procedure inspired from the recent PaRIS algorithm. The main result of the article for practical applications is that the mechanism of this procedure remains valid when the transition density is approximated by a an unbiased positive estimator. The proposed procedure therefore extends the PaRIS algorithm to HMMs whose transition density is unknown and can be unbiasedly approximated. The GRand PaRIS algorithm outperforms the existing fixed lag smoother for POD processes of [27], as it does not introduce any intrinsic and nonvanishing bias. In addition, numerical simulations highlight a better variance using data from two different models. It can be implemented for the class of models for which exact algorithms of [6] are valid, with a linear complexity in N in the best cases, or at worse in N^{2}.
Appendix
Proofs
Proof of Lemma 1
Let τ be the first time draws are accepted in the acceptreject mechanism. For all ℓ≥1, write
Let h be a function defined on {1,…,N},
which concludes the proof. □
Proof of Lemma 2
The independence is ensured by the mechanism of SMC methods. By (9),
Note that by Lemma 1,
Since \(\tau ^{i}_{k+1}\) and ζ_{ k } are independent conditionally to \(\mathcal {G}_{k+1}^{N}\):
Moreover, conditionally to \(\mathcal {F}_{k}^{N}\), the probability density function of \(\left (\xi _{k+1}^{i},I_{k+1}^{i}\right)\) is given by
Therefore, this yields
which concludes the proof. □
Proof of Proposition 1
The results is proved by induction. At time k = 0, the result holds using that for all 1≤i≤N, \(\rho _{0}^{i} = 0\) and the convention T_{0}[h_{0}] = 0. In addition, \(\phi _{0}^{N}\) is a standard importance sampler estimator of ϕ_{0} with \(\widehat \omega _{0}^{i}\le \widehat {\omega }_{0}_{\infty }\) so that for any bounded function h on X,
Assume the results holds for k≥0 and that 𝜗_{k+1} = 1 for simplicity. Write
where \(a_{N} = N^{1}\sum _{i=1}^{N} \widehat {\omega }_{k+1}^{i} \left (\!\tau _{k+1}^{i}  \phi _{k+1}\left [T_{k+1}[h_{k+1}]\right ]\right)\) and \(b_{N} =N^{1}\sum _{i=1}^{N} \widehat {\omega }_{k+1}^{i}\). By Lemma 2, the random variables \(\{\widehat {\omega }_{k+1}^{i}\tau _{k+1}^{i}\}_{i=1}^{N}\) are independent conditionally on \(\mathcal {F}_{k}^{N}\) and by H2,
Therefore, by Hoeffding inequality,
On the other hand,
where
By [28, Lemma 11], ϕ_{ k }[Υ_{ k }] = 0 which implies by the induction assumption that
Then,
Similarly, as \(b_{N} \le \widehat {\omega }_{k}_{\infty }\), by Hoeffding inequality,
Note that
By the induction assumption,
The proof is completed using Lemma 3. □
Lemma 3
Assume that a_{ N }, b_{ N }, and b are random variables defined on the same probability space such that there exist positive constants β, B, C, and M satisfying

(i)
a_{ N }/b_{ N }≤M, \(\mathbb {P}\)a.s. and b≥β, \(\mathbb {P}\)a.s.,

(ii)
For all ε>0 and all N≥1, \(\mathbb {P}\left [b_{N}b>\epsilon \right ]\leq B \exp \left (C N \epsilon ^{2}\right)\),

(iii)
For all ε>0 and all N≥1, \(\mathbb {P} \left [ a_{N}>\epsilon \right ]\leq B \exp \left (C N \left (\epsilon /M\right)^{2}\right)\).
Then,
Proof
See [10]. □
Notes
 1.
(i76600U CPU @ 2.60GHz)
References
 1
Y AitSahalia, Transition densities for interest rate and other nonlinear diffusions. J. Financ. 54:, 1361–1395 (1999).
 2
Y AitSahalia, Maximum likelihood estimation of discretely sampled diffusions: a closedform approximation approach. Econometrica. 70:, 223–262 (2002).
 3
Y AitSahalia, Closedform likelihood expansions for multivariate diffusions. Ann. Stat. 36:, 906–937 (2008).
 4
A Beskos, O Papaspiliopoulos, GO Roberts, Retrospective exact simulation of diffusion sample paths with applications. Bernoulli. 12(6), 1077:1098 (2006).
 5
A Beskos, O Papaspiliopoulos, GO Roberts, A factorisation of diffusion measure and finite sample path constructions. Methodol. Comput. Appl. Probab. 10(1), 85–104 (2008).
 6
A Beskos, O Papaspiliopoulos, GO Roberts, P Fearnhead, Exact and computationally efficient likelihoodbased estimation for discretely observed diffusion processes (with discusion). J. Roy. Statist. Soc. Ser. B. 68(3), 333–382 (2006).
 7
O Cappé, E Moulines, T Rydén, Inference in hidden Markov models (SpringerVerlag, New York, 2005).
 8
P Del Moral, J Jacod, P Protter, The Monte Carlo method for filtering with discretetime observations. Probab. Theory Relat. Fields. 120:, 346–368 (2001).
 9
AP Dempster, NM Laird, DB Rubin, Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. B. 39(1), 1–38 (1977). (with discussion).
 10
R Douc, A Garivier, E Moulines, J Olsson, Sequential Monte Carlo smoothing for general state space hidden Markov models. Ann. Appl. Probab. 21(6), 2109–2145 (2011).
 11
A Doucet, S Godsill, C Andrieu, On sequential MonteCarlo sampling methods for Bayesian filtering. Stat. Comput. 10:, 197–208 (2000).
 12
A Doucet, S Godsill, C Andrieu, On sequential montecarlo sampling methods for bayesian filtering. Stat. Comput. 10:, 197–208 (2000).
 13
P Fearnhead, O Papaspiliopoulos, GO Roberts, Particle filters for partially observed diffusions. J. Roy. Statist. Soc. Ser. B. 70(4), 755–777 (2008).
 14
P Fearnhead, K Latuszynski, GO Roberts, G Sermaidis, Continuoustime importance sampling: Monte Carlo methods which avoid timediscretisation error (2017). Technical report.
 15
P Gloaguen, MP Étienne, S Le Corff, Stochastic differential equation based on a multimodal potential to model movement data in ecology. To appear in the Journal of the Royal Statistical Society: Series C. http://onlinelibrary.wiley.com/doi/10.1111/rssc.12251/abstract.
 16
SJ Godsill, A Doucet, M West, Monte Carlo smoothing for nonlinear time series. J. Am. Stat. Assoc. 50:, 438–449 (2004).
 17
N Gordon, D Salmond, AF Smith, Novel approach to nonlinear/nonGaussian bayesian state estimation. IEE Proc. F. Radar Sig. Process. 140:, 107–113 (1993).
 18
M Hürzeler, HR Künsch, Monte Carlo approximations for general statespace models. J. Comput. Graph. Stat. 7:, 175–193 (1998).
 19
N Kantas, A Doucet, SS Singh, J Maciejowski, N Chopin, On particle methods for parameter estimation in statespace models. Stat. Sci. 30(3), 328–351 (2015).
 20
M Kessler, Estimation of an ergodic diffusion from discrete observations. Scand. J. Stat. 24(2), 211–229 (1997).
 21
M Kessler, A Lindner, M Sorensen, Statistical methods for stochastic differential equations (CRC Press, Boca Raton, 2012).
 22
G Kitagawa, MonteCarlo filter and smoother for nonGaussian nonlinear state space models. J. Comput. Graph. Stat. 1:, 1–25 (1996).
 23
S Le Corff, G Fort, Convergence of a particlebased approximation of the block online Expectation Maximization algorithm. ACM Trans. Model. Comput. Simul. 23(1), 2 (2013).
 24
S Le Corff, G Fort, Online expectation maximization based algorithms for inference in hidden Markov models. Electron. J. Stat. 7:, 763–792 (2013).
 25
C Li, Maximumlikelihood estimation for diffusion processes via closedform density expansions. Ann. Stat. 41(3), 1350–1380 (2013).
 26
J Olsson, O Cappe, R Douc, E Moulines, Sequential monte carlo smoothing with application to parameter estimation in nonlinear state space models. Bernoulli. 14(1), 155–179 (2008).
 27
J Olsson, J Strojby, Particlebased likelihood inference in partially observed diffusion processes using generalised Poisson estimators. Electron. J. Stat. 5:, 1090–1122 (2011).
 28
J Olsson, J Westerborn, Efficient particlebased online smoothing in general hidden Markov models: the PaRIS algorithm. Bernoulli. 3:, 1951–1996 (2017).
 29
T Ozaki, A bridge between nonlinear time series models and nonlinear stochastic dynamical systems: a local linearization approach. Stat. Sin. 2:, 1130–135 (1992).
 30
MK Pitt, N Shephard, Filtering via simulation: Auxiliary particle filters. J. Am. Stat. Assoc. 94(446), 590–599 (1999).
 31
G Poyiadjis, A Doucet, SS Singh, Particle approximations of the score and observed information matrix in state space models with application to parameter estimation. Biometrika. 98:, 65–80 (2011).
 32
I Shoji, T Ozaki, Estimation for nonlinear stochastic differential equations by a local linearization method 1. Stoch. Anal. Appl. 16(4), 733–752 (1998).
 33
M Uchida, N Yoshida, Adaptive estimation of an ergodic diffusion process based on sampled data. Stoch. Process. Appl. 122(8), 2885–2924 (2012).
 34
W Wagner, Unbiased Monte Carlo estimators for functionals of weak solutions of stochastic differential equations. Stochast. Stochast. Rep. 28:, 1–20 (1989).
Funding
This work has been developed during a 1year postdoc funded by ParisSaclay Center for Data Science.
Competing interests
The authors declare that they have no competing interests.
Author information
Affiliations
Contributions
All the authors have contributed to the conception of the algorithms, the analysis of the proposed estimator, and to the redaction of the manuscript. PG provided the simulations displayed in the final version. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Gloaguen, P., Étienne, MP. & Le Corff, S. Online sequential Monte Carlo smoother for partially observed diffusion processes. EURASIP J. Adv. Signal Process. 2018, 9 (2018). https://doi.org/10.1186/s1363401805303
Received:
Accepted:
Published:
Keywords
 Stochastic differential equations
 Smoothing
 Sequential Monte Carlo Methods
 Online estimation