A Robust Approach for Clock Offset Estimation in Wireless Sensor Networks

Abstract

The maximum likelihood estimators (MLEs) for the clock phase offset assuming a two-way message exchange mechanism between the nodes of a wireless sensor network were recently derived assuming Gaussian and exponential network delays. However, the MLE performs poorly in the presence of non-Gaussian or nonexponential network delay distributions. Currently, there is a need to develop clock synchronization algorithms that are robust to the distribution of network delays. This paper proposes a clock offset estimator based on the composite particle filter (CPF) to cope with the possible asymmetries and non-Gaussianity of the network delay distributions. Also, a variant of the CPF approach based on the bootstrap sampling (BS) is shown to exhibit good performance in the presence of reduced number of observations. Computer simulations illustrate that the basic CPF and its BS-based variant present superior performance than MLE under general random network delay distributions such as asymmetric Gaussian, exponential, Gamma, Weibull as well as various mixtures.

1. Introduction

Wireless sensor networks (WSNs) have been recently proposed for observing and monitoring various aspects of the physical world. In WSNs, the basic operation is data fusion, where data from multiple sensors are integrated together to form a single meaningful result [1]. The fusion of individual sensor readings is possible only by exchanging messages that are timestamped by each sensor's local clock. This mandates the need for a common notion of time among the sensors. Such a common notion of time is achieved through the clock synchronization task. In WSNs, clock synchronization is an important research area [2].

The two-way message exchange mechanism used in the Network Time Protocol (NTP) [3] and Timing Synch Protocol for Sensor Networks (TPSN) [4] is adopted herein as the clock synchronization approach between two nodes of the WSN. Due to the presence of nondeterministic and unbounded message delays, messages can get delayed arbitrarily, which makes the synchronization very difficult in WSNs [2]. The most commonly widely used models to capture the non-deterministic delay distributions in WSNs are Gaussian, exponential, Gamma, and Weibull probability density functions (pdfs) [5–7]. However, in general, it is difficult to determine which delay model should be adopted in a given WSN at a certain time instant. Recently, [8] studied the performance of maximum likelihood estimators corresponding to symmetric Gaussian (GML) and exponential (EML) network delay distributions. Preliminary computer simulations [8] illustrated the fact that GML and EML are not robust to asymmetries or uncertainties in the network delay distributions.

In [9], the inference of general state-space models characterized by nonlinear process and observation equations is addressed via the concept of Gaussian sum particle filter (GSPF) which approximates the filtering and predictive distributions by weighted Gaussian mixtures, that is, banks of Gaussian particle filters (GPFs). With non-Gaussian noise approximated by Gaussian mixtures, the non-Gaussian noise models are approximated by banks of Gaussian noise models. However, in wireless sensor networks, the process and observation equations are linear functions. Therefore, we extend the use of a new filter, the composite particle filter (CPF), to encompass linear and additive non-Gaussian noise models. For a linear state-space model with additive non-Gaussian noise, CPF approximates the posterior distributions as Gaussian mixtures using banks of parallel Kalman filters (KFs). The main contribution of this paper is a novel clock offset estimation method, called the composite particle filter (CPF), which is shown to be robust to the unknown distribution of network delays. The CPF approximates the filtering and predictive distributions by using weighted Gaussian mixtures and is basically implemented via banks of Kalman filters (KFs) instead of Gaussian Particle filters (GPFs) [9]. Thus, CPF appears as a variation of the Gaussian sum particle filter (GSPF) [9], fit for estimation of linear models perturbed by non-Gaussian random noise components.

There is another method capable of coping with nonlinear processes and observation equations, and non-Gaussian noise models, which is called the Gaussian mixture sigma point particle filter (GMSPPF) [10]. Both GSPF and GMSSPF approximate the filtering and predictive distributions by weighted Gaussian mixtures. The big difference consists in the integrating sub-techniques. Specifically, the GMSPPF combines the Particle filter and Sigma Point filter with a Gaussian Mixture Model (GMM) whose features are estimated via the Expectation-Maximization (EM) algorithm, while the GSPF integrates the Gaussian Particle filter with GMM via a Gaussian sum filtering approach. In [11], the Gaussian mixture Kalman particle filter (GMKPF) was proposed, which is a slightly changed version of the GMSPPF and obtained by replacing the sigma point Kalman filter (SPKF) with a KF. The CPF proposed in this paper and the GMKPF [11] present a different updating mechanism of GMM parameters such as the weights. Also, the bootstrapping sampling technique is shown herein to be an effective mechanism to improve the performance of clock estimation schemes.

As explained in [4, 12], energy conservation is a very important concern. Reference [13] pointed out that much less power is consumed in processing data than transmitting it. In fact, [13] showed that the energy required for a sensor node to transmit 1 Kbit over 100 meters (3 Joules) in a particular WSN was equivalent to the energy required to execute 3 millions of instructions. Therefore, the possibility of trading off computational power for more savings in energy consumption appears as a very feasible approach. Thus, one way to reduce the amount of energy spent on signal transmissions and implicitly on achieving clock synchronization is the usage of possibly more sophisticated signal processing algorithms with the goal of achieving more accurate clock offset estimates especially in operational regimes characterized by a reduced number of observations and unknown network delay distributions. In case that message exchange errors occur, a node will not retransmit the message to a neighbor node but will resample the observation data from the original observation data using the bootstrap sampling (BS) approach [14, 15]. The BS can be implemented by constructing a number of resamples of the observed data, each of which being obtained by random sampling with replacement from the original observed data. Notice that a node will then estimate the clock offset from the resampled observation data by using the CPF. The computer simulations highlight that the CPF with BS achieves better performances in various random delay models, and it aims at reducing the number of message exchanges. Therefore, the CPF with BS leads to less power consumption relative to the CPF, GML, and EML.

It is interesting also to remark that the clock synchronization literature for wireless sensor networks is quite scarce in terms of contributions addressing the robustness or improving the mean square error (MSE) performance of existing state-of-the-art clock synchronization algorithms in the presence of message errors, unknown and possibly time-varying network delay distributions, or reduced number of observations (data measurements). Thus far, it appears that only very few preliminary and straightforward applications of standard Kalman filtering or general adaptive signal processing techniques have been recently reported (see [16–18]) to improve the MSE performance of protocols such as RBS [19] or TPSN [4]. However, no attempts have been made to address the problem of building clock synchronization algorithms that are robust to the unknown distribution of random network delays, message errors, or presence of reduced number of observations. This paper aims to answer these questions from the light of a composite particle filtering technique.

The rest of this paper is organized as follows. Section 2 introduces the state-space model that will be used throughout the paper and a description of problem formulation. Section 3 provides a description of the CPF and BS-based CPF approaches for estimating the clock offset in wireless sensor networks. The results of computer simulations are given in Section 4. Finally, Section 5 provides concluding remarks.

2. Problem Modeling and Objectives

The two-way timing message exchange protocol is a recently proposed clock synchronization approach for WSNs [4, 5, 8]. Under this protocol, the synchronization between two generic nodes A and B is achieved by transmitting timing messages in both directions. The message exchanges between nodes A and B are organized in terms of cycles, and during each cycle a message exchange occurs in each direction. For example, during the th cycle, the Node A sends its time reading to Node B, which records the time of arrival of the message sent by node A as , according to its own time scale. Similarly, a timing message exchange is performed from Node B to Node A. At time node B transmits back to node A the timing information and . According to Node A's clock, the message transmitted by Node B arrives at node A at time . Therefore, at the end of the th cycle, node A has access to all the time information , , that prove to be sufficient for estimating the clock phase offset and deterministic propagation delay.

Indeed, following [5, 8], the time differences corresponding to the th up and down link delay observations of the th timing message exchange can be expressed respectively as and , respectively. The fixed value denotes the clock offset between the two nodes, stands for the (deterministic) propagation delay, and and model the variable portions of delay and might assume any distribution such as Gaussian, exponential, Gamma, Weibull, or a mixture of two distributions. For notational convenience, we adopt the notation to denote the unknown clock phase offset. Given the observation samples , our goal is to find the minimum mean square estimator of the unknown clock offset , which is given by

(1)

where denotes the set of observed samples up to time , . Since the clock offset value is constant, the clock offset is assumed to obey a Gauss-Markov dynamic channel model of the form

(2)

where represents the state transition scalar value (if is vector, is matrix) for the clock offset. The noise vector is modeled as a Gaussian random variable with zero mean and covariance . Notice also that the vector observation model follows from the observed data and takes the expression

(3)

where the observation noise vector may assume any probability density function (pdf). Hence, it turns out that our initial problem is now casted as the estimation problem of a Gauss-Markov model with unknown state (see (2) and (3)).

3. Composite Particle Filtering and Bootstrap Sampling Approach

The maximum likelihood clock offset estimator was reported in [8] for the two-way timing message exchange protocol such as TPSN and NTP under the assumption of Gaussian or exponential delay models. Herein we will derive a CPF for clock phase offset estimation assuming a general unknown distribution of network delays and then compare the CPF and existing maximum likelihood clock offset estimators that were derived for Gaussian (GML) and exponential (EML) delay models. Under the Bayesian framework, an emerging powerful technique for obtaining the posterior, predictive, and filtering probability density functions is referred to as the particle filtering (PF) (see, e.g., [20, 21]). The PF technique allows for a complete representation of the state posterior distribution, which approximates , by stochastic samples generated using a sequential importance sampling strategy. The most common employed PF strategy is to sample from the transition prior distribution due to its simplicity. Since the prior importance sampling distribution employs no information from observations in proposing new samples, its use is often ineffective and leads to poor filtering performance. To overcome these challenges, we will derive an extension of GSPF [9] applicable for linear non-Gaussian models.

In the GSPF, the filtering and predictive distributions are recursively represented as finite Gaussian mixtures (GMs) using Gaussian Particle Filtering (GPF) [22]. One set of methods approximates the mixture components of the predictive and filtering distributions as Gaussian. The approximation can be implemented by the GSPF, resulting in a parallel bank of GPFs. Since Gaussian mixture models are increasingly used for modeling non-Gaussian densities [23, 24], herein we plan extending the use of the GSPF to linear non-Gaussian models. The resulting new approach will be referred to as CPF. Notice that in the measurement and time-update equations of CPF, the updated mean and covariance of each mix and follow from the KF. The CPF is implemented by means of parallel KFs, and the weights are adjusted according to the given update equations. Notice also that the CPF approach comes out of the utilization of another filtering technique (KF) producing a filtering probability density function used as importance function (IF) for the particle filtering.

We next describe briefly the general framework of CPF methods. Assume at time , the posterior distribution is approximated as closely as desired by a Gaussian mixture model (GMM) of the following form [25]:

(4)

where stands for the number of mixing components, denote the mixing weights, and is the normal distribution of RV with mean and covariance . The transition prior is modeled as

(5)

where denote the mixing weights, is the number of Gaussian mixing components for modeling the transition prior. The time update stage that the previous observations and state used to predict the current state is encompassed by the predictive distribution which can be approximated as follows:

(6)

The predicted and updated Gaussian component means and covariances are calculated using the KF. As in the KF, the integral on the right is approximated by a Gaussian. Then, the predictive distribution can be approximated as

(7)

where the parameters of the mixture are obtained according to KF

(8)

for appropriate , and and .

In the measurement update stage, the operation of acting on new observations to improve on previously "predictive" states can be approximated as

(9)

where is the normalizing constant. Therefore, the updated filtering distribution is approximated as

(10)

The reader is directed to [9] for more detailed explanations of the GSPF algorithm. Finally, notice that the conditional mean state estimate and the corresponding error covariance can be calculated, respectively, as

(11)

Next, we introduce another clock estimation scheme obtained through the integration of the CPF technique with the bootstrap sampling (BS) approach. The reader is directed to [14, 15] for more detailed explanations about bootstrap sampling. In order to provide a consistent amount of observation data in the presence of errors during timing message transmissions, new sampled observation data from the original observation data are generated via the BS. Then, the clock offset is estimated based on the CPF. Notice that even in the presence of corrupted or lost data packets, BS can create additional samples to the original sample set, by drawing at random with replacement from , and without being necessary for additional retransmissions. Each of the bootstrap samples is considered as new data. Based on the additional sampled observation data, we can then approximate the clock offset by using the CPF. The major steps of the CPF approach with bootstrap sampling are summarized by the following pseudocode

Algorithm: CPF with BS

() Conduct the experiment to obtain the random sample and calculate the estimate from the sample .

() Construct the empirical distribution , which puts equal mass at each observation .

() From , draw a sample , operation called bootstrap resampling.

() From the bootstrap resample , estimate the clock offset by CPF.

The calculation of the computational cost of CPF is very complex, compared to GML and EML. In general, the computational cost of CPF is a function of the number of particles and the number of measurements. However, GML and EML are a function of the number of measurements and do not use the particle filtering method. Hence, it is difficult explicitly to compare the complexities of GML/EML and CPF. However, we will use the big notation to express the computational complexities of GML/EML and CPF in terms of flops by evaluating only the most computationally demanding steps. Letting , , and denote the dimension of the state vector , the number of particles, and the number of GMM, respectively, the GML is approximately , and the CPF is approximately which is the maximum complexity and occurs in the posterior pdf step. This shows that the CPF is approximately times slower than GML in an application with , , and .

4. Simulation Results

In this section, extensive computer simulation results are presented to illustrate the performance of the CPF, CPF with BS, GML [8], and EML [8] approaches for estimating the clock offset in wireless sensor networks, assuming a variety of random network delay models such as asymmetric Gaussian, exponential, Gamma, and Weibull as well as a mixture of Gamma and Weibull, respectively. These computer simulations and numerous other simulations not shown herein due to space limitations corroborate the conclusion that the proposed method can be widely and flexibly applied for any delay distribution. The stationary process assumes the constant variance , while the number of particles and GMMs are set to 100 and 3, respectively. The bootstrap samples are twice the number of measurements.

Figures 1, 2, 3 and 4 show the MSE (Mean Square Error) of the estimators when the network delay distributions are asymmetric Gaussian, exponential, Gamma, and Weibull pdfs, respectively. The subscript attached to the distribution parameters are used to differentiate the parameters of the uplink distribution with respect to those of the downlink distribution. For example, for an asymmetric Gaussian delay model, () and () denote the uplink and downlink variances, respectively. The MSE curves are plotted against the number of observations ranging from 5 to 25. Note that the CPF performs much better with over MSE-reduction when compared to the GML or EML. Notice also that the CPF with BS exhibits the best performance in the presence of reduced number of observation data. Notice also the MSE of GML achieves better performance than EML in asymmetric Gaussian delay models, while EML assumes superior performance relative to GML in asymmetric exponential, Gamma, and Weibull delay models. The reason for this is that Gamma and Weibull delay models are closer to the exponential distribution than the Gaussian distribution.

Figure 1

MSEs of clock offset estimators for asymmetric Gaussian random delays .

Figure 2

MSEs of clock offset estimators for asymmetric Exponential random delays .

Figure 3

MSEs of clock offset estimators for Gamma random delays .

Figure 4

MSEs of clock offset estimators for Weibull random delays .

To further quantify the robustness of the estimators, we studied the performance of the CPF with BS, CPF, GML, and EML under more general random delay models obtained by mixing two arbitrary distributions. For example, in Figure 5, we mix uniformly the Gamma with the Weibull delay model, each distribution accounting for 50% of samples. This means that if 10 observations are observed, 5 observations are Gamma, and the remaining ones are Weibull distributed. From Figure 5, we observe that CPF clearly outperforms the GML and EML. Notice that additional simulations not shown herein and performed assuming different mixture models for the network delays such as mixtures of Gaussian and exponential, Gaussian and Gamma, exponential and Gamma, and exponential and Weibull corroborate the same conclusion, namely, the fact that CPF outperforms both the GML and EML no matter what distribution model is assumed for the network delays.

Figure 5

MSEs of clock offset estimators for mixing of Gamma and Weibull [ and ].

Figures 6, 7, 8, 9, and 10 depict the MSEs versus the number of observation data in the case when 2 message exchange errors occur with uniform distribution for the scenarios assumed by Figures 1–5. From the Figures 6–9, we observe that CPF with BS clearly outperforms CPF, GML, and EML. From Figure 10, we observe that unlike the Figures 6–9, CPF with BS exhibits the best performance in the presence of a reduced number of observations. These simulation results corroborate the general conclusion that the CPF with BS and CPF are reliable methods in the presence of a reduced number of samples.

Figure 6

MSEs of clock offset estimators for asymmetric Gaussian random delay [ ] and 2 message exchange errors.

Figure 7

MSEs of clock offset estimators for asymmetric Exponential random delay [ ] and 2 exchange message errors occur.

Figure 8

MSEs of clock offset estimators for Gamma random delay [ ], and 2 exchange message errors occur.

Figure 9

MSEs of clock offset estimators for Weibull random delay [ and ] and 2 exchange message errors occur.

Figure 10

MSEs of clock offset estimators for mixing of Gamma [ and ] and Weibull [ and ] and 2 exchange message errors occur.

5. Conclusions and Future Work

This paper provided novel methods such as CPF and BS for estimating the clock offset in wireless sensor networks. The benefits are in terms of improved performance and applicability to any random delay models such as asymmetric Gaussian, exponential, Gamma, and Weibull, as well as mixtures of these delay models. In addition, the proposed CPF approaches are robust to the presence of a small number of observations, message exchange errors, and unknown network delay distributions. Also, the proposed iterative clock phase estimation algorithms can track time-varying clock phase offsets, which represents a notable improvement relative to the existing state-of-the-art GML and EML estimators. Possible disadvantages of the proposed composite particle filtering-based approaches are the facts that they present high computational complexity and require good initializations; analytical closed form expressions do not seem to exist for the clock estimators and the computation of the lower bound performance bounds appears difficult due to the non-Gaussian nature of involved distributions. In addition, the CPF with BS and CPF achieve excellent performance compared to GML and EML in environments which manifest in message exchange errors and time-varying network delay distributions.