- Research Article
- Open Access
Linear High-Order Distributed Average Consensus Algorithm in Wireless Sensor Networks
EURASIP Journal on Advances in Signal Processing volume 2010, Article number: 373604 (2010)
This paper presents a linear high-order distributed average consensus (DAC) algorithm for wireless sensor networks. The average consensus property and the convergence rate of the high-order DAC algorithm are analyzed. In particular, the convergence rate is determined by the spectral radius of a network topology-dependent matrix. Numerical results indicate that this simple linear high-order DAC algorithm can accelerate the convergence without additional communication overhead and reconfiguration of network topology.
The distributed average consensus (DAC) algorithm aims to provide distributed nodes in a network agreement on a common measurement, known at any one node as the local state information. As such, it has many relevant applications in wireless sensor networks [1, 2], for example, moving-object acquisition and tracking, habitat monitoring, reconnaissance, and surveillance. In the DAC approach, average consensus can be sufficiently reached within a connected network by averaging pair-wise local state information at network nodes. In , Olfati-Saber et al. established a theoretical framework for the analysis of consensus-based algorithms.
In this paper, we study a simple approach to improve the convergence rate of DAC algorithms in wireless sensor networks. The author of  demonstrates that the convergence rate of DAC can be increased by using the "small-world" phenomenon. This technique, however, needs to redesign the network topology based on "random rewiring". In , an extrapolation-based DAC approach is proposed; it utilizes a scalar epsilon algorithm to accelerate the convergence rate without extra communication cost. However, numerical results show that mean square error does not decrease monotonically with respect to iteration time, which may not be desirable in practical applications. In , the authors extend the concept of average consensus to a higher dimension one via the spatial point of view, where nodes are spatially grouped into two disjoint sets: leaders and sensors. Specifically, it is demonstrated that under appropriate conditions, the sensors' states converge to a linear combination of the leaders' states. Furthermore, multi-objective optimization (MOP) and Pareto optimality are utilized to solve the learning problem, where the goal is to minimize the error between the convergence state and the desired estimate subject to a targeted convergence rate. In , the authors introduce the concept of nonlinear DAC algorithm, where standard linear addition is replaced by a sine operation during local state update. The convergence rate of this nonlinear DAC algorithm is shown to be faster under appropriate weight designs.
In this paper, we apply the principles of high-order consensus to the distributed computation problem in wireless sensor networks. This simple linear high-order DAC requires no additional communication overhead and no reconfiguration of the network topology. Instead, it utilizes gathered data from earlier iterations to accelerate consensus. We study here the convergence property and convergence rate of the high-order DAC algorithm and show that its convergence rate is determined by the spectral radius of a network topology-dependent matrix. Moreover, numerical results indicate that the convergence rate can be greatly improved by storing and using past data.
This paper is outlined as follows. Section 2 provides background and system model for the high-order DAC algorithm. Section 3 discusses convergence analysis for this scheme. Simulation results are presented in Section 4, and conclusions are provided in Section 5.
2. Background and System Model
2.1. Linear High-Order DAC Algorithm
We assume a synchronized, time-invariant connected network. In each iteration of the -th order DAC algorithm, each node transmits a data packet to its neighbor which contains the local state information. Each node then processes and decodes the received message from its neighbors. After retrieving the state information, each node updates its local state using the weighted average of the current state between itself and its neighboring nodes as well as stored state information from the previous iterations of the algorithm.
The update rule of the -th order DAC algorithm at each node is given as
where is the local state at node during iteration ; is the set of neighboring nodes that can communicate reliably with node ; is a constant step size; are predefined constants with and ; is a forgetting factor, such that . We assume initial conditions of the -th order DAC algorithm are , where is initial local state information for node . It is worth mentioning that when , the high-order DAC algorithm reduces to the (conventional) first-order DAC algorithm.
This linear high-order DAC algorithm can be regarded as a generalized version of DAC algorithm; it requires no additional communication cost and no reconfiguration of network topology. Compared to the conventional first-order DAC algorithm, with negligible increase in memory size and computation load in each sensor node, the convergence rate can be greatly improved with appropriate algorithm design. In , the authors propose an average consensus algorithm with improved convergence rate by considering a convex combination of conventional operation and linear predication. In particular, a special case of one step predication is presented for detailed analysis. We note that the major difference between the DAC algorithm in  and our proposed scheme is that we utilize stored state difference for high-order updating and show that optimal convergence rate can be significantly improved by this simple extension. Furthermore, we present explicitly the optimal convergence rate of second-order DAC algorithm in Section 3.2.
2.2. Network Model and Some Preliminaries
In the following, we model the wireless sensor network as an undirected graph (The convergence properties presented here can be easily extended for a directed graph. We omit this extension here.), consisting of a set of nodes and a set of edges . Each edge is denoted as where and are two nodes connected by edge . We assume that the presence of an edge indicates that nodes and can communicate with each other reliably. We assume here a connected graph, that is, there exists a path connecting any pair of distinct nodes.
Given this network model, we denote as the adjacency matrix of such that if and otherwise. Next, let be the graph Laplacian matrix of which is defined as , where is the degree matrix of , and . Given this matrix , we have and , where and . Additionally, is a symmetric positive semidefinite matrix. And for a connected graph, the rank of is and its eigenvalues can be arranged in increasing order as .
Let us define . The -th order DAC algorithm in (1) thus evolves as
with the initial conditions , where and denotes the identity matrix.
3. Convergence Analysis of High-Order DAC Algorithm
3.1. Average Consensus Property of High-Order DAC Algorithm
Before we investigate the convergence property of the high-order DAC algorithm, we define two matrices
where , and denotes the all-zero matrix. Then we have the following lemma:
The eigenvalues of agree with those of except that is replaced by .
Let us define two vectors and . It is easy to check that and are left and right eigenvectors of corresponding to , respectively, that is, and . Additionally, , . In order to obtain the eigenvalues of , we have 
The above equation is valid because
Thus, the eigenvalues of are and This completes the proof.
The average consensus property of the -th order DAC algorithm in wireless sensor networks is stated in the following theorem.
Consider the -th order DAC algorithm in (2) in a time-invariant, connected, undirected wireless sensor network, with initial conditions . When , an average consensus is achieved asymptotically, or equivalently,
where denotes the spectral radius of a matrix.
Let us define . Then, the -th order DAC algorithm in (2) can be rewritten as which implies that To calculate the eigenvalues of , we have 
Thus, the eigenvalues of should satisfy the following equation:
Note that there are roots corresponding to one . For a time invariant and connected network, has only one eigenvalue, . From (8), when , the eigenvalues of satisfy . Then, for this , has only two distinct eigenvalues, (with algebraic multiplicity 1) and (with algebraic multiplicity ). Additionally, it is easy to show that the algebraic multiplicity of eigenvalue is equal to 1. Based on Lemma 1, we know that the eigenvalues of agree with those of except that is replaced by . Since , we see that the eigenvalues of stay inside the unit circle except for . Thus, we have
where is the Jordan form matrix corresponding to eigenvalues . Thus, we have Then, which indicates
This completes the proof.
According to Theorem 1, we see that when this linear high-order DAC algorithm is employed in an undirected wireless sensor network, average consensus can be achieved asymptotically. We also note that our proposed high-order DAC algorithm relies heavily on local state information exchange between two or more nodes in the networks. Noisy links  and packet drop failures  will certainly affect the performance of our proposed high-order DAC algorithm. We will investigate these important issues in the future.
3.2. Convergence Rate for High-Order DAC Algorithm
One of the most important measures of any distributed, iterative algorithm is its convergence rate. As we show next, the convergence rate of the high-order DAC algorithm is determined by the spectral radius of , which is similar to the first-order DAC algorithm .
Let us define the average consensus value in each iteration as . In the high-order DAC algorithm, this value remains invariant during each iteration since
We now define the disagreement vector as , which indicates the difference between the updated local state and the average state of the network nodes. Then, the evolution of the disagreement vector is obtained as.
Given this dynamic of the disagreement vector, we note.
For the -th order DAC algorithm in (2) in a time invariant, connected, undirected wireless sensor network, with initial conditions and , an average consensus is exponentially reached in the following form:
where denotes the norm of a vector.
Let us define the error vector as which can be obtained from , where , and denotes the Kronecker product.
Based on this definition, we see that the error vector results in the following evolution:
The above equation is valid because , and . Then, we have
which is equivalent to (13). This completes the proof.
Let us define the convergence region to satisfy , that is,
Based on Lemma 2, we see that the convergence rate for the -th order DAC algorithm in wireless sensor networks is determined by the spectral radius of , which depends solely on the network topology. Furthermore, we note that there may exist possible choices of and to achieve the optimal convergence rate of the high-order DAC algorithm. To see this, we formulated the following spectral radius minimization problem to find the optimal and for the high-order DAC algorithm, that is,
From (17), we see that the optimal convergence rate of our proposed high-order DAC algorithm depends solely on the eigenvalues of Lapacian matrix. Let us define the minimal spectral radius of as , and the optimal convergence rate as . When , the optimal convergence rate of second-order DAC algorithm can be obtained as 
Recall that in the first-order DAC algorithm, we have 
Clearly, we see that . In the case when , we note that, in general, the closed-form solution for this optimization problem is hard to find due to the fact that high-order polynomial equations are involved in calculating the eigenvalues of . For example, when and , we need to find the roots of the following cubic equation to obtain the eigenvalues of :
In practical applications, since the optimal and depend only on the network topology, a numerical solution can be obtained offline based on node deployment, and all design parameters can be flooded to the sensor nodes before they run the distributed algorithm. As we will show in the simulations, the optimal convergence rate can be greatly improved by this linear high-order DAC algorithm.
4. Simulation Results
In the following, we simulate networks in which the initial local state information of node is equally spaced (trends similar to the ones noted below were observed when initial local state information between nodes were arbitrary (e.g., when they were uniformly distributed over ). We use this fixed local state assumption here for comparison purposes) in , where . For the sake of simplicity, we only consider and for the higher-order DAC approach. In the simulations, we denote our proposed DAC algorithm as best constant (BC) high-order DAC algorithm and choose two types of ad hoc weights as comparison: maximum degree (MD) and metropolis hasting (MH) weights . Furthermore, we assume and study the following two network topologies:
Fixed network with 6 nodes as shown in Figure 1(a).
Random network with 16 nodes. The 16 nodes were randomly generated with uniform distribution over a unit square; two nodes were assumed connected if the distance between them was less than , a predefined threshold. One realization of such a network is shown in Figure 1(b).
Figure 2 shows the optimal convergence rates for the DAC algorithms with various weights in random networks with 16 nodes as a function of . The results are based on 1000 realizations of the random network where we excluded disconnected networks. From the plots, we note that the first-order BC DAC algorithm outperforms the first-order MH and MD DAC algorithms. Furthermore, we see that the optimal convergence rate increases as increases. However, we also observe that the fourth-order DAC algorithm has negligible improvement compared to the third-order algorithm. Based on this, we restrict our examination of higher-order DAC algorithm to in the subsequent results.
In addition to the results shown here, we ran this simulation setup for various realizations of random networks, assuming a large number of nodes. Figure 3 shows the convergence rate comparison for DAC algorithms with various weights when . As expected, we see that the results show a similar trend, that is, the optimal convergence rate of DAC algorithm increases as increases.
In Figure 4, we compare the convergence rates of the third-order DAC algorithm with the first- and second-order DAC algorithms for both the random and fixed network topologies. Specifically, we plot the mean square error (defined as ). In simulating random networks, we average out results over 1000 network realizations and assume , that is, network nodes are well connected with one another. As expected, we see that the third-order DAC algorithm converges faster than the first- and second-order DAC algorithms for both network scenarios.
In this paper, we present a linear high-order DAC algorithm to address the distributed computation problem in wireless sensor networks. Interestingly, the high-order DAC algorithm can be regarded as a spatial-temporal processing technique, where nodes in the network represent the spatial advantage, the high-order processing represents the temporal advantage, and the optimal convergence rate can be viewed as the diversity gain. In the future, we intend to investigate the effects of fading, link failure, and other practical conditions when utilizing the DAC algorithm in wireless sensor networks.
Olfati-Saber R, Fax JA, Murray RM: Consensus and cooperation in networked multi-agent systems. Proceedings of the IEEE 2007, 95(1):215-233.
Xiao L, Boyd S: Fast linear iterations for distributed averaging. Proceedings of the 42nd IEEE Conference on Decision and Control, December 2003 5: 4997-5002.
Olfati-Saber R: Ultrafast consensus in small-world networks. Proceedings of the American Control Conference (ACC '05), June 2005 4: 2371-2378.
Kokiopoulou E, Frossard P: Accelerating distributed consensus using extrapolation. IEEE Signal Processing Letters 2007, 14(10):665-668.
Khan UA, Kar S, Moura JMF: Higher dimensional consensus: learning in large-scale networks. IEEE Transactions on Signal Processing 2010, 58(5):2836-2849.
Khan UA, Kar S, Moura JMF: Distributed average consensus: beyond the realm of linearity. Proceedings of the 43rd IEEE Asilomar Conference on Signals, Systems and Computers, November 2009
Oreshkin BN, Aysal TC, Coates MJ: Distributed average consensus with increased convergence rate. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '08), April 2008 2285-2288.
Horn RA, Johnson CR: Matrix Analysis. Cambridge University Press, Cambridge, UK; 1985.
Meyer CD: Matrix Analysis and Applied Linear Algebra. SIAM; 2001.
Xiao L, Boyd S, Kim S-J: Distributed average consensus with least-mean-square deviation. Journal of Parallel and Distributed Computing 2007, 67(1):33-46. 10.1016/j.jpdc.2006.08.010
Hatano Y, Mesbahi M: Agreement over random networks. IEEE Transactions on Automatic Control 2005, 50(11):1867-1872.
Xiong G, Kishore S: Discrete-time second-order distributed consensus time synchronization algorithm for wireless sensor networks. EURASIP Journal on Wireless Communications and Networking 2009, 2009:-12.
Xiao L, Boyd S, Lall S: A scheme for robust distributed sensor fusion based on average consensus. Proceedings of the 4th International Symposium on Information Processing in Sensor Networks (IPSN '05), April 2005 63-70.
About this article
Cite this article
Xiong, G., Kishore, S. Linear High-Order Distributed Average Consensus Algorithm in Wireless Sensor Networks. EURASIP J. Adv. Signal Process. 2010, 373604 (2010). https://doi.org/10.1155/2010/373604
- Convergence Rate
- Wireless Sensor Network
- Network Topology
- Spectral Radius
- Random Network