Linear high-order distributed average consensus algorithm in wireless sensor networks

This paper presents a linear high-order distributed average consensus (DAC) algorithmfor wireless sensor networks. The average consensus property and convergence rate of the highorder DAC algorithm are analyzed. In particular, the convergence rate is determined by the spectral radius of a network topology dependent matrix. Numerical results indicate that this simple linear high-order DAC algorithm can accelerate the convergence without additional communication overhead and reconfiguration of network topology.


Introduction
The distributed average consensus (DAC) algorithm aims to provide distributed nodes in a network agreement on a common measurement, known at any one node as the local state information.As such, it has many relevant applications in wireless sensor networks [1,2], for example, movingobject acquisition and tracking, habitat monitoring, reconnaissance, and surveillance.In the DAC approach, average consensus can be sufficiently reached within a connected network by averaging pair-wise local state information at network nodes.In [1], Olfati-Saber et al. established a theoretical framework for the analysis of consensus-based algorithms.
In this paper, we study a simple approach to improve the convergence rate of DAC algorithms in wireless sensor networks.The author of [3] demonstrates that the convergence rate of DAC can be increased by using the "smallworld" phenomenon.This technique, however, needs to redesign the network topology based on "random rewiring".In [4], an extrapolation-based DAC approach is proposed; it utilizes a scalar epsilon algorithm to accelerate the convergence rate without extra communication cost.However, numerical results show that mean square error does not decrease monotonically with respect to iteration time, which may not be desirable in practical applications.In [5], the authors extend the concept of average consensus to a higher dimension one via the spatial point of view, where nodes are spatially grouped into two disjoint sets: leaders and sensors.Specifically, it is demonstrated that under appropriate conditions, the sensors' states converge to a linear combination of the leaders' states.Furthermore, multiobjective optimization (MOP) and Pareto optimality are utilized to solve the learning problem, where the goal is to minimize the error between the convergence state and the desired estimate subject to a targeted convergence rate.In [6], the authors introduce the concept of nonlinear DAC algorithm, where standard linear addition is replaced by a sine operation during local state update.The convergence rate of this nonlinear DAC algorithm is shown to be faster under appropriate weight designs.
In this paper, we apply the principles of high-order consensus to the distributed computation problem in wireless sensor networks.This simple linear high-order DAC requires no additional communication overhead and no reconfiguration of the network topology.Instead, it utilizes gathered data from earlier iterations to accelerate consensus.We study here the convergence property and convergence rate of the high-order DAC algorithm and show that its convergence rate is determined by the spectral radius of a network topology-dependent matrix.Moreover, numerical results indicate that the convergence rate can be greatly improved by storing and using past data.
This paper is outlined as follows.Section 2 provides background and system model for the high-order DAC algorithm.Section 3 discusses convergence analysis for this scheme.Simulation results are presented in Section 4, and conclusions are provided in Section 5.

Background and System Model
2.1.Linear High-Order DAC Algorithm.We assume a synchronized, time-invariant connected network.In each iteration of the M-th order DAC algorithm, each node transmits a data packet to its neighbor which contains the local state information.Each node then processes and decodes the received message from its neighbors.After retrieving the state information, each node updates its local state using the weighted average of the current state between itself and its neighboring nodes as well as stored state information from the M − 1 previous iterations of the algorithm.
The update rule of the M-th order DAC algorithm at each node i is given as where x i (k) is the local state at node i during iteration k; N i is the set of neighboring nodes that can communicate reliably with node i; ε is a constant step size; c m are predefined constants with c 0 = 1 and c m / = 0(m > 0); γ is a forgetting factor, such that |γ| < 1.We assume initial conditions of the M-th order DAC algorithm are where θ i is initial local state information for node i.It is worth mentioning that when γ = 0, the highorder DAC algorithm reduces to the (conventional) firstorder DAC algorithm.
This linear high-order DAC algorithm can be regarded as a generalized version of DAC algorithm; it requires no additional communication cost and no reconfiguration of network topology.Compared to the conventional first-order DAC algorithm, with negligible increase in memory size and computation load in each sensor node, the convergence rate can be greatly improved with appropriate algorithm design.In [7], the authors propose an average consensus algorithm with improved convergence rate by considering a convex combination of conventional operation and linear predication.In particular, a special case of one step predication is presented for detailed analysis.We note that the major difference between the DAC algorithm in [7] and our proposed scheme is that we utilize stored state difference for high-order updating and show that optimal convergence rate can be significantly improved by this simple extension.Furthermore, we present explicitly the optimal convergence rate of second-order DAC algorithm in Section 3.2.

Network Model and Some Preliminaries.
In the following, we model the wireless sensor network as an undirected graph ( The convergence properties presented here can be easily extended for a directed graph.We omit this extension here.)G = (V, E ), consisting of a set of N nodes V = {1, 2, . . ., N} and a set of edges E .Each edge is denoted as e = (i, j) ∈ E where i ∈ V and j ∈ V are two nodes connected by edge e.We assume that the presence of an edge (i, j) indicates that nodes i and j can communicate with each other reliably.We assume here a connected graph, that is, there exists a path connecting any pair of distinct nodes.
Given this network model, we denote A = [a i j ] as the adjacency matrix of G such that a i j = 1 if (i, j) ∈ E and a i j = 0 otherwise.Next, let L be the graph Laplacian matrix of G which is defined as and d Given this matrix L, we have L1 = 0 and 1 T L = 0 T , where 1 = [1, 1, . . ., 1] T and 0 = [0, 0, . . ., 0] T .Additionally, L is a symmetric positive semidefinite matrix.And for a connected graph, the rank of L is N − 1 and its eigenvalues can be arranged in increasing order as 0 The M-th order DAC algorithm in (1) thus evolves as

Convergence Analysis of High-Order DAC Algorithm
3.1.Average Consensus Property of High-Order DAC Algorithm.Before we investigate the convergence property of the high-order DAC algorithm, we define two MN × MN matrices where K = (1/N)11 T , and 0 N×N denotes the N × N all-zero matrix.Then we have the following lemma: It is easy to check that h l and h r are left and right eigenvectors of H corresponding to In order to obtain the eigenvalues of H − J, we have [9] det(H The above equation is valid because (5) Thus, the eigenvalues of H − J are λ 1 (H − J) = 0 and λ i (H − J) = λ i (H), i = 2, . . ., MN.This completes the proof.
The average consensus property of the M-th order DAC algorithm in wireless sensor networks is stated in the following theorem.
where ρ(•) denotes the spectral radius of a matrix.
Then, the M-th order DAC algorithm in (2) can be rewritten as ψ(k) = Hψ(k − 1), which implies that ψ(k) = H k ψ(0).To calculate the eigenvalues of H, we have [9] det(H − λI MN ) Thus, the eigenvalues of H should satisfy the following equation: Note that there are M roots corresponding to one λ i (L).For a time invariant and connected network, L has only one eigenvalue, λ 1 (L) = 0. From (8), when λ 1 (L) = 0, the eigenvalues of H satisfy f (λ) = λ M − λ M−1 = 0.Then, for this λ 1 (L) = 0, H has only two distinct eigenvalues, λ 1 (H) = 1 (with algebraic multiplicity 1) and λ 2 (H) = 0 (with algebraic multiplicity M − 1).Additionally, it is easy to show that the algebraic multiplicity of eigenvalue λ(H) = 1 is equal to 1. Based on Lemma 1, we know that the eigenvalues of H − J agree with those of H except that λ 1 (H) = 1 is replaced by λ 1 (H − J) = 0. Since ρ(H − J) < 1, we see that the eigenvalues of H stay inside the unit circle except for λ 1 (H) = 1.Thus, we have where Λ is the Jordan form matrix corresponding to eigenvalues λ i (H) / = 1 [9].Thus, we have lim This completes the proof.
According to Theorem 1, we see that when this linear high-order DAC algorithm is employed in an undirected wireless sensor network, average consensus can be achieved asymptotically.We also note that our proposed high-order DAC algorithm relies heavily on local state information exchange between two or more nodes in the networks.Noisy links [10] and packet drop failures [11] will certainly affect the performance of our proposed high-order DAC algorithm.We will investigate these important issues in the future.

Convergence Rate for
High-Order DAC Algorithm.One of the most important measures of any distributed, iterative algorithm is its convergence rate.As we show next, the convergence rate of the high-order DAC algorithm is determined by the spectral radius of H − J, which is similar to the firstorder DAC algorithm [1].
Let us define the average consensus value in each iteration as m(k) = (1/N)1 T x(k).In the high-order DAC algorithm, this value remains invariant during each iteration since We now define the disagreement vector as δ(k) = x(k) − m(k)1, which indicates the difference between the updated local state and the average state of the network nodes.Then, the evolution of the disagreement vector is obtained as.
Given this dynamic of the disagreement vector, we note.
where • denotes the 2 norm of a vector.
Proof.Let us define the error vector as e(k) where J 1 = I M ⊗ K, and ⊗ denotes the Kronecker product.
Based on this definition, we see that the error vector results in the following evolution: The above equation is valid because (H − J)J 1 = 0 MN×MN , and J 1 H = J.Then, we have which is equivalent to (13).This completes the proof.

Let us define the convergence region R to satisfy ρ(H
Based on Lemma 2, we see that the convergence rate for the M-th order DAC algorithm in wireless sensor networks is determined by the spectral radius of H − J, which depends solely on the network topology.Furthermore, we note that there may exist possible choices of ε and γ to achieve the optimal convergence rate of the high-order DAC algorithm.
To see this, we formulated the following spectral radius minimization problem to find the optimal ε and γ for the high-order DAC algorithm, that is, From ( 17), we see that the optimal convergence rate of our proposed high-order DAC algorithm depends solely on the eigenvalues of Lapacian matrix.Let us define the minimal spectral radius of H − J as α opt = min{ρ(H − J)}, and the optimal convergence rate as ν opt = − log(α opt ).When M = 2, the optimal convergence rate of second-order DAC algorithm can be obtained as [12] Recall that in the first-order DAC algorithm, we have [2] Clearly, we see that ν opt,SO ≥ ν opt,FO .In the case when M ≥ 3, we note that, in general, the closed-form solution for this optimization problem is hard to find due to the fact that high-order polynomial equations are involved in calculating the eigenvalues of H − J.For example, when M = 3 and c 1 = 1, c 2 = 1, we need to find the roots of the following cubic equation to obtain the eigenvalues of H − J: In practical applications, since the optimal ε and γ depend only on the network topology, a numerical solution can be obtained offline based on node deployment, and all design parameters can be flooded to the sensor nodes before they run the distributed algorithm.As we will show in the simulations, the optimal convergence rate can be greatly improved by this linear high-order DAC algorithm.

Simulation Results
In the following, we simulate networks in which the initial local state information of node i is equally spaced ( trends similar to the ones noted below were observed when initial local state information between nodes were arbitrary (e.g., when they were uniformly distributed over [−β, β]).We use this fixed local state assumption here for comparison purposes) in [−β, β], where β = 500.For the sake of simplicity, we only consider M = 3 and M = 4 for the higherorder DAC approach.In the simulations, we denote our proposed DAC algorithm as best constant (BC) high-order DAC algorithm and choose two types of ad hoc weights as comparison: maximum degree (MD) and metropolis hasting (MH) weights [13].Furthermore, we assume c 1 = 1, c 2 = 1, c 3 = 1/6 and study the following two network topologies:  Case 2. Random network with 16 nodes.The 16 nodes were randomly generated with uniform distribution over a unit square; two nodes were assumed connected if the distance between them was less than η, a predefined threshold.One realization of such a network is shown in Figure 1(b).
Figure 2 shows the optimal convergence rates for the DAC algorithms with various weights in random networks with 16 nodes as a function of η.The results are based on 1000 realizations of the random network where we excluded disconnected networks.From the plots, we note that the first-order BC DAC algorithm outperforms the first-order MH and MD DAC algorithms.Furthermore, we see that the optimal convergence rate increases as M increases.However, we also observe that the fourth-order DAC algorithm has negligible improvement compared to the third-order algorithm.Based on this, we restrict our examination of higher-order DAC algorithm to M = 3 in the subsequent results.
In addition to the results shown here, we ran this simulation setup for various realizations of random networks, assuming a large number of nodes.Figure 3 shows the convergence rate comparison for DAC algorithms with various weights when N = 256.As expected, we see that the results show a similar trend, that is, the optimal convergence rate of DAC algorithm increases as M increases.
In Figure 4, we compare the convergence rates of the third-order DAC algorithm with the first-and second-order DAC algorithms for both the random and fixed network  topologies.Specifically, we plot the mean square error (defined as (1/N) δ(k) 2 ).In simulating random networks, we average out results over 1000 network realizations and assume η = 0.9, that is, network nodes are well connected with one another.As expected, we see that the third-order DAC algorithm converges faster than the first-and secondorder DAC algorithms for both network scenarios.

Conclusions
In this paper, we present a linear high-order DAC algorithm to address the distributed computation problem in wireless sensor networks.Interestingly, the high-order DAC algorithm can be regarded as a spatial-temporal processing technique, where nodes in the network represent the spatial advantage, the high-order processing represents the temporal advantage, and the optimal convergence rate can be viewed as the diversity gain.In the future, we intend to investigate the effects of fading, link failure, and other practical conditions when utilizing the DAC algorithm in wireless sensor networks.

Figure 2 :
Figure 2: Convergence rate comparison of DAC algorithms with various weights in random networks versus distance threshold when N = 16.

Figure 3 :Case 1 .
Figure 3: Convergence rate comparison of DAC algorithms with various weights in random networks versus distance threshold when N = 256.
Iteration time index k Mean square error BC: first-order DAC algorithms BC: second-order DAC algorithms BC: third-order DAC algorithms (b) Random network with 16 secondary users