- Research
- Open Access
Variable forgetting factor mechanisms for diffusion recursive least squares algorithm in sensor networks
- Ling Zhang^{1},
- Yunlong Cai^{1}Email author,
- Chunguang Li^{1} and
- Rodrigo C. de Lamare^{2}
https://doi.org/10.1186/s13634-017-0490-z
© The Author(s) 2017
- Received: 9 November 2016
- Accepted: 6 July 2017
- Published: 15 August 2017
Abstract
In this work, we present low-complexity variable forgetting factor (VFF) techniques for diffusion recursive least squares (DRLS) algorithms. Particularly, we propose low-complexity VFF-DRLS algorithms for distributed parameter and spectrum estimation in sensor networks. For the proposed algorithms, they can adjust the forgetting factor automatically according to the posteriori error signal. We develop detailed analyses in terms of mean and mean square performance for the proposed algorithms and derive mathematical expressions for the mean square deviation (MSD) and the excess mean square error (EMSE). The simulation results show that the proposed low-complexity VFF-DRLS algorithms achieve superior performance to the existing DRLS algorithm with fixed forgetting factor when applied to scenarios of distributed parameter and spectrum estimation. Besides, the simulation results also demonstrate a good match for our proposed analytical expressions.
Keywords
- Sensor networks
- Distributed parameter estimation
- Distributed spectrum estimation
- Diffusion recursive least-squares
- Variable forgetting factor
1 Introduction
Distributed estimation is commonly utilized for distributed data processing over sensor networks, which exhibits increased robustness, flexibility, and system efficiency compared to centralized processing. Owing to these merits, distributed estimation has received more and more attention and been widely used in applications ranging from environmental monitoring [1], medical data collecting for healthcare [2], animal tracking in agriculture [1], monitoring physical phenomena [3], localizing moving mobile terminals [4, 5] to national security. Particularly, distributed estimation technique relies on the cooperation among geographically spread sensor nodes to process locally collected data. With different cooperation strategies employed, distributed estimation algorithms can be classified into the incremental type and the diffusion type. Note that we consider the diffusion cooperation strategy in this paper since the incremental strategy requires the definition of a path through the network and may be not suitable for large networks or dynamic configurations [6, 7]. Many distributed estimation algorithms with the diffusion strategy have been put forward recently, such as diffusion least-mean squares (LMS) [8, 9], diffusion sparse LMS [10–12], variable step size diffusion LMS (VSS-DLMS) [13, 14], diffusion recursive least squares (RLS) [6, 7], distributed sparse RLS [15], distributed sparse total least squares (LS) [16], diffusion information theoretic learning (ITL) [17], and the diffusion-based algorithm for distributed censor regression [18]. Among assorted distributed estimation algorithms, the RLS-based algorithms achieve superior performance to the LMS-based ones by inheriting the advantages of fast convergence and low steady-state misadjustment from the RLS technique. Thus, the distributed estimation algorithms based on the diffusion strategy and the RLS adaptive technique are investigated in this paper.
However, the existing RLS-based distributed estimation algorithms provide a fixed forgetting factor, which has some drawbacks. With a fixed forgetting factor, the algorithm fails to keep up with real-time variations in environment, such as variations in sensor network topology. Moreover, it is expected to adjust the forgetting factors automatically according to the estimation errors rather than choose appropriate values for them through simulations. There have been several studies on variable forgetting factor (VFF) methods. Specifically, the classic gradient-based VFF (GVFF) mechanism was proposed in [19], and most of the existing VFF mechanisms are extensions of this method [20–24]. Nevertheless, the GVFF mechanism requires a large amount of computation. In order to reduce the computational complexity, the improved low-complexity VFF mechanisms have been reported in [25, 26]. To the best of our knowledge, the existing VFF mechanisms are mostly employed in a centralized context and have not been considered in the field of distributed estimation yet.
In this work, the previously reported VFF mechanisms [25, 26] are employed to the diffusion RLS algorithms for distributed signal processing applications, by simplifying the inverse relation between the forgetting factor and the adaptation component to provide lower computational complexity. The resulting algorithms are referred to as low-complexity time-averaged VFF diffusion RLS (LTVFF-DRLS) algorithm and low-complexity correlated time-averaged VFF diffusion RLS (LCTVFF-DRLS) algorithm, respectively. Compared with the GVFF mechanisms, the proposed LTVFF and LCTVFF mechanisms can reduce the computational complexity significantly [25, 26]. Then, we carry out the analysis for the proposed algorithms in terms of the mean and mean square error performance. Finally, we provide simulation results to verify the effectiveness of the proposed algorithms when applied in distributed parameter estimation and distributed spectrum estimation.
- 1)
We propose the low-complexity VFF-DRLS algorithms for distributed estimation in sensor networks. To the best of our knowledge, the VFF mechanisms have not been considered in the distributed estimation algorithms yet.
- 2)
We study the mean and mean square performance for the proposed algorithms in a general case, and provide the transient analysis for a specialized case. Specifically, for the general case, in terms of the mean performance, we show that the mean value of the weight error vector approaches zero as iteration numbers go to infinity, which implies the asymptotical convergence of the proposed algorithms; from the perspective of mean square performance, we derive the mathematical expressions for the steady-state MSD and EMSE values. In the specialized case, we study the transient analysis by focusing on the learning curve and prove that the proposed algorithms are convergent and the convergence rate is related to the varying forgetting factors.
- 3)
We perform simulations to evaluate the performance of the proposed algorithms when applied to distributed parameter estimation and distributed spectrum estimation tasks. The simulation results indicate that the proposed algorithms exhibit remarkable improvements in convergence and steady-state performance when compared with the DRLS algorithm that has a fixed forgetting factor. Besides, effectiveness of our analytical expressions for calculating the steady-state MSD and EMSE is verified by the simulation results. In addition, we also provided detailed simulation results regarding the choice of the parameters in the proposed algorithms to help with the parameter selection in practice.
This paper is organized as follows. Section 2 provides the system model for the distributed estimation over sensor networks. Besides, the DRLS algorithm with the fixed forgetting factor is described briefly. In Section 3, two low-complexity VFF mechanisms are presented, followed by the analyses for the variable forgetting factor in terms of steady-state statistical properties. Besides, the proposed LTVFF-DRLS algorithm and the LCTVFF-DRLS algorithm are presented. In the last part of this section, the computational complexity of the VFF mechanisms as well as the proposed algorithms is analyzed. In Section 4, detailed analyses based on mean and mean-square performance for the proposed algorithms are carried out and analytical expressions to compute MSD and EMSE are derived. In addition, transient analysis for a specialized case is provided in the last part of Section 4. In Section 5, simulation results are presented for distributed parameter estimation and distributed spectrum estimation. Section 6 draws the conclusions.
Notation: Boldface letters are used for vectors or matrices, while normal font for scalar quantities. Matrices are denoted by capital letters and small letters are used for vectors. We use the operator row {·} to denote a row vector, col {·} to denote a column vector, and diag {·} to denote a diagonal matrix. The operator E[·] stands for the expectation of some quantity, and Tr {·} represents the trace of a matrix. We use (·)^{ T } and (·)^{−1} to denote the transpose and inverse operator, respectively, and (·)^{∗} for complex conjugate-transposition. We also use the symbol I _{ n } to represent an identity matrix of size n and \(\mathbf {\mathbb {I}}\) to denote a vector of appropriate size with all elements equal to one.
2 System model and diffusion-based DRLS algorithm
In this section, we first illustrate the system model for the distributed estimation over sensor networks. Following this, we review the conventional DRLS algorithm with the fixed forgetting factor briefly.
2.1 System model
where w ^{ o } is the unknown optimal weight vector of size M×1, and v _{ k,i } is zero-mean additive white Gaussian noise with variance \({\sigma }_{v,k}^{2}\). Particularly, we assume that the noise variance has been determined in advance somehow. We also assume that the noise samples v _{ k,i }, k=1,2,…,N, i=1,2,…, are independent of each other as well as the input vectors u _{ k,i }. We aim to estimate the unknown optimal weight vector w ^{ o } in a distributed manner. That is, each sensor node k obtains a local estimate w _{ k,i } of size M×1 to approach the optimal weight vector w ^{ o } as much as possible. To this end, each node k not only uses its local measurement d _{ k,i } and input vector u _{ k,i } but also cooperates with its closest neighbors for updating its local estimate w _{ k,i }. Specifically, by cooperation, each node k has access to its neighbors’ data {d _{ l,i },u _{ l,i }} and estimates w _{ l,i } at each time instant i where \(l\in \mathcal {N}_{k}\), and then, each node k fuses all the available information to update its local estimate ψ _{ k,i }.
Besides, we define \(\mathbf {\mathcal {R}}_{v,i}=E[\mathbf {\mathcal {V}}_{i}\mathbf {\mathcal {V}}_{i}^{*}]\).
2.2 Brief review of diffusion-based DRLS algorithm
In this part, we give a brief introduction to the diffusion-based DRLS algorithm [6, 7].
Note that the notation \(\|\mathbf {a}\|_{\boldsymbol {\Sigma }}^{2}=\mathbf {a}^{*}\boldsymbol {\Sigma }\mathbf {a}\) represents the weighted vector norm of any positive definite Hermitian matrix Σ. Besides, the matrix Π _{ i } is given by Π _{ i }=λ ^{ i+1} Π where 0≪λ<1 representing the forgetting factor and Π=δ ^{−1} I _{ M } with δ>0. Furthermore, the matrix \(\boldsymbol {\mathcal {W}}_{k,i}\) can be expressed as \(\boldsymbol {\mathcal {W}}_{k,i}=\boldsymbol {\mathcal {R}}_{v,i}^{-1}\boldsymbol {\Lambda }_{i}\text {diag}\{\mathbf {C}_{k},\mathbf {C}_{k},\ldots,\mathbf {C}_{k}\}\), where Λ _{ i }=diag{I _{ N },λ I _{ N },…,λ ^{ i } I _{ N }} and C _{ k } is a diagonal matrix. It is worth noting that the main diagonal elements of the matrix C _{ k } is composed of the kth column of matrix C. Particularly, the matrix C is the adaptation matrix for the diffusion-based DRLS algorithm and satisfies \(\mathbf {\mathbb {I}}^{T}\mathbf {C}=\mathbf {\mathbb {I}}\) and \(\mathbf {C}\mathbf {\mathbb {I}}=\mathbf {\mathbb {I}}\) [6]. Also, note that the matrix C is a doubly stochastic matrix, that is, both a left stochastic matrix and a right stochastic matrix.
However, the closed-form solution in (7) is obtained via calculating the inversion of matrices, which requires large computation. Instead, the diffusion-based DRLS algorithm provides a recursive approach to solve (6), which can be implemented by the following two steps.
where A _{ l,k } denotes the (l,k)th element of the matrix A. Particularly, the matrix A is the combination matrix for the diffusion-based DRLS algorithm and is chosen such that \(\mathbf {\mathbb {I}}^{T}\mathbf {A}=\mathbf {\mathbb {I}}\) [6].
Note that the steps (9)–(13) constitute the diffusion-based DRLS algorithm [6, 7].
3 Low-complexity variable forgetting factor mechanisms
In this section, we introduce the LTVFF mechanism and the LCTVFF mechanism that are employed by our proposed algorithms. Particularly, the analyses for the variable forgetting factor in terms of the steady-state properties of the first-order statistics are presented, and the LTVFF-DRLS algorithm that employs the LTVFF mechanism as well as the LCTVFF-DRLS algorithm that applies the LCTVFF mechanism are proposed. In the last part of this section, we analyze the computational complexity for these two VFF mechanisms as well as the proposed algorithms.
3.1 LTVFF mechanism
where the quantity ζ _{ k }(i) is related to the estimation errors and varies in an inverse way to the forgetting factor, which is referred to as the adaptation component. The operator \([\cdot ]_{\lambda _{-}}^{\lambda _{+}}\) denotes the truncation of the forgetting factor to the limits of the range [λ _{+},λ _{−}].
That is to say, in the LTVFF mechanism, the adaptation component is updated based on the instantaneous estimation error.
The LTVFF mechanism is given by (14) and (15). The value of the forgetting factor λ _{ k }(i) is controlled by the parameters α and β. Particularly, the effects of α and β on the performance of our proposed algorithms are investigated in Section 5. As can be seen from (14) and (15), large estimation errors will cause an increase in the adaptation component ζ _{ k }(i), which yields a smaller forgetting factor and provides a faster tracking speed. Conversely, small estimation errors will lead to the decrease of the adaptation component ζ _{ k }(i), and thus, the forgetting factor λ _{ k }(i) will be increased to yield smaller steady-state misadjustment.
LTVFF-DRLS and LCTVFF-DRLS algorithms
LTVFF-DRLS Algorithm | LCTVFF-DRLS Algorithm | ||
---|---|---|---|
1 | For each node k=1,2,…,N do | 1 | For each node k=1,2,…,N do |
2 | Initialize w _{ k,−1}=0, P _{ k,−1}=Π ^{−1}. | 2 | Initialize w _{ k,−1}=0, P _{ k,−1}=Π ^{−1}. |
3 | For time instant i=1,2,… do | 3 | For time instant i=1,2,… do |
4 | \(\lambda _{k}(i)=[1-\zeta _{k}(i)]_{\lambda _{-}}^{\lambda _{+}}\). | 4 | \(\lambda _{k}(i)=[1-\zeta _{k}(i)]_{\lambda _{-}}^{\lambda _{+}}\). |
5 | ζ _{ k }(i)=α ζ _{ k }(i−1)+β|e _{ k }(i)|^{2}. | 5 | ζ _{ k }(i)=α ζ _{ k }(i−1)+β|ρ _{ k }(i)|^{2}. |
6 | ρ _{ k }(i)=γ ρ _{ k }(i−1)+(1−γ)|e _{ k }(i−1)||e _{ k }(i)|. | ||
6 | Set ψ _{ k,i }=w _{ k,i−1}, \(\mathbf {P}_{k,i}=\lambda ^{-1}_{k}(i)\mathbf {P}_{k,i-1}\). | 7 | Set ψ _{ k,i }=w _{ k,i−1}, \(\mathbf {P}_{k,i}=\lambda ^{-1}_{k}(i)\mathbf {P}_{k,i-1}\). |
7 | For \(l\in \mathcal {N}_{k}\) do | 8 | For \(l\in \mathcal {N}_{k}\) do |
8 | \(\boldsymbol {\psi }_{k,i}{\longleftarrow }\boldsymbol {\psi }_{k,i}+ \frac {C_{l,k}\mathbf {P}_{k,i}\mathbf {u}_{l,i}[d_{l,i}-\mathbf {u}_{l,i}^{*}\boldsymbol {\psi }_{k,i}]} {\sigma _{v,l}^{2}+C_{l,k}\mathbf {u}_{l,i}^{*}\mathbf {P}_{k,i}\mathbf {u}_{l,i}}\). | 9 | \(\boldsymbol {\psi }_{k,i}{\longleftarrow }\boldsymbol {\psi }_{k,i}+ \frac {C_{l,k}\mathbf {P}_{k,i}\mathbf {u}_{l,i}[d_{l,i}-\mathbf {u}_{l,i}^{*}\boldsymbol {\psi }_{k,i}]} {\sigma _{v,l}^{2}+C_{l,k}\mathbf {u}_{l,i}^{*}\mathbf {P}_{k,i}\mathbf {u}_{l,i}}\). |
9 | \(\mathbf {P}_{k,i}{\longleftarrow }\mathbf {P}_{k,i}- \frac {C_{l,k}\mathbf {P}_{k,i}\mathbf {u}_{l,i}\mathbf {u}_{l,i}^{*}\mathbf {P}_{k,i}}{\sigma _{v,l}^{2}+C_{l,k}\mathbf {u}_{l,i}^{*}\mathbf {P}_{k,i}\mathbf {u}_{l,i}}\). | 10 | \(\mathbf {P}_{k,i}{\longleftarrow }\mathbf {P}_{k,i}- \frac {C_{l,k}\mathbf {P}_{k,i}\mathbf {u}_{l,i}\mathbf {u}_{l,i}^{*}\mathbf {P}_{k,i}}{\sigma _{v,l}^{2}+C_{l,k}\mathbf {u}_{l,i}^{*}\mathbf {P}_{k,i}\mathbf {u}_{l,i}}\). |
10 | End | 11 | End |
11 | Generate the final estimate \(\mathbf {w}_{k,i}=\sum \limits _{l\in \mathcal {N}_{k}}A_{l,k}\boldsymbol {\psi }_{l,i}\). | 12 | Generate the final estimate \(\mathbf {w}_{k,i}=\sum \limits _{l\in \mathcal {N}_{k}}A_{l,k}\boldsymbol {\psi }_{l,i}\). |
12 | End | 13 | End |
13 | End | 14 | End |
3.2 LCTVFF mechanism
where 0<γ<1 and γ is slightly smaller than 1. Note that the LCTVFF mechanism is given by (14), (25), and (26).
By employing the LCTVFF mechanism to the diffusion-based DRLS algorithm, we propose the LCTVFF-DRLS algorithm, which is presented in the right column of Table 1.
3.3 Computational complexity
Computational complexity of the DRLS algorithm
Number of operations for each node at each iteration | ||
---|---|---|
Multiplications | Additions | |
DRLS (fixed forgetting factor) | M ^{2}+N ^{2}(4M ^{2}+5M+1)+M | 4N ^{2} M ^{2}+M−1 |
Additional computational complexity of the analyzed VFF mechanisms
Number of operations for each node at each iteration | ||
---|---|---|
Multiplications | Additions | |
GVFF | M ^{2}+N ^{2}(9M ^{2}+4M)+2M+1 | M ^{2}+N ^{2}(9M ^{2}−3M−1)+2M−2 |
LTVFF | 3 | 2 |
LCTVFF | 6 | 3 |
4 Performance analysis
In this section, we carry out the analyses in terms of mean and mean square error performance for the proposed LTVFF-DRLS and LCTVFF-DRLS algorithms. In particular, we derive mathematical expressions to describe the steady-state behavior based on MSD and EMSE. In addition, we also perform transient analysis in a specialized case for the proposed algorithms in the last part of this section. To proceed with the analysis, we first introduce several assumptions, which have been widely adopted in the analysis for the RLS-type algorithms and have been verified by simulations [7, 27].
Assumption 1
Assumption 2
For the RLS-type algorithms with the fixed forgetting factor, we have the ergodicity assumption for P _{ k,i } [6, 7, 27], that is, the time average of a sequence of random variables can be replaced by its expected value so as to make the analysis for the performance of these algorithms tractable. Similarly, for the RLS-type algorithms with variable forgetting factors, we still have the ergodicity assumption:
Assumption 3
The derivation is presented in Appendix B. Since \(\lim \limits _{i\to \infty }E\left [\mathbf {P}_{k,i}^{-1}\right ]\) is independent of i, we can denote it by \(\mathbf {P}_{k}^{-1}\). Moreover, based on the ergodicity assumption, it is also common in the analysis of the performance of the RLS-type algorithms to replace the random matrix P _{ k,i } by P _{ k } when i is large enough.
4.1 Mean performance
Each element in the product \(\prod \limits _{j=N_{i}+1}^{i}E[\mathbf {F}(j)]\) can be viewed as a polynomial of F _{1,1}(i),F _{1,2}(i),⋯, with an order of i−N _{ i }+1. When i→∞, each element of this product approaches zero since F _{1,1}(i),F _{1,2}(i),⋯ are all smaller than unity. Now, assuming that all the elements of \(E[\widetilde {\mathbf {W}}_{N_{i}}]\) are bounded in absolute value by some finite constant, therefore, all the elements of \(E[\widetilde {\mathbf {W}}_{i}]\) converge to zero when i→∞. As a result, we can conclude that the proposed LTVFF-DRLS and LCTVFF-DRLS algorithms converge asymptotically when i→∞.
4.2 Mean-square error and deviation performances
where e _{ k } is a column vector of length N with unity for the kth element and zero for the others. Next, we write the Euclidean norm of the weight error vector \(\widetilde {\mathbf {w}}_{k,i}\), that is, \(\|\widetilde {\mathbf {w}}_{k,i}\|^{2}\), or equivalently, \(Tr\{\widetilde {\mathbf {w}}_{k,i}\widetilde {\mathbf {w}}_{k,i}^{*}\}\).
Expressions (76) and (78) describe the steady-state behavior of the proposed LTVFF-DRLS and LCTVFF-DRLS algorithms. By comparing the expressions (76) and (78) with the analytical results in [7], it is clear that the fixed matrix λ ^{2} A in the expressions for the conventional DRLS algorithms has been replaced by the matrix F(i) in the expressions (76) and (78), which is weighted by the matrix Λ _{ i }. Since Λ _{ i } varies from one iteration to the next, F(i) varies for each iteration as well, which improves the tracking performance of the resulting algorithms. Furthermore, since all the elements in F(i) are bounded by zero and unity, the values of the steady-state MSD and EMSE given by (76) and (78) are both very small values when i is large enough. Thus, we can verify that the proposed LTVFF-DRLS and LCTVFF-DRLS algorithms both converge in the mean-square sense.
4.3 Transient analysis under spatial invariance assumption
In addition, to facilitate analysis, we assume that all elements of the adaptation matrix C are equal to \(\frac {1}{N}\).
where we use the representation \(\|\mathbf {t}\|_{\mathbf {A}}^{2}=\mathbf {t}^{*}\mathbf {A}\mathbf {t}\) in the last equality.
This recursive equation is stable and convergent if \(E[\boldsymbol {\mathcal {F}}_{i}]\) is stable [31].
Particularly, the quantity \(\boldsymbol {\mathcal {F}}_{i}\) has a spectral radius smaller than unity and thus is stable. This can be proved as follows: If we replace each element in Λ _{ i } by its upper bound λ _{+}, then we have \(\boldsymbol {\mathcal {F}}_{i}\) replacced by \(\lambda _{+}^{2}\mathbf {A}\otimes \mathbf {A}\). Note that A satisfies \(\mathbf {\mathbb {I}}^{T}\mathbf {A}=\mathbf {\mathbb {I}}\), and then, we can readily verify that each column of A⊗A sums up to unity. Hence, the quantity \(\lambda _{+}^{2}\mathbf {A}\otimes \mathbf {A}\) has the spectral radius \(\lambda _{+}^{2}\) that is smaller than one. Given that each element in Λ _{ i } does not exceed λ _{+}, the spectral radius of \(\boldsymbol {\mathcal {F}}_{i}\) is smaller than \(\lambda _{+}^{2}\) and surely is smaller than unity. Therefore, for this specialized case, it can be verified theoretically that the proposed LTVFF-DRLS and LCTVFF-DRLS algorithms are convergent in terms of the learning curve and the convergence rate is related to the varying forgetting factors.
Also note that, since the convergence performance of the adaptive algorithms does not depend on the outside environment but rely on the network topology and the design of algorithms, the analytical results in this specialized case also apply to the general case.
5 Simulation results
In this section, we present the simulation results for the proposed LTVFF-DRLS and LCTVFF-DRLS algorithms when applied in two applications, that is, distributed parameter estimation and distributed spectrum estimation over sensor networks.
5.1 Distributed parameter estimation
In this part, we evaluate the performance of the proposed LTVFF-DRLS and LCTVFF-DRLS algorithms when applied to distributed parameter estimation in comparison with the DRLS algorithm with the fixed forgetting factor and the GVFF-DRLS algorithm. In addition, we also verify the effectiveness of the proposed analytical expressions in (76) and (78) based on simulations.
5.1.1 Effects of α, β, and γ
As can be seen from Figs. 3 and 4 for both the LTVFF and LCTVFF mechanisms, the optimal choice of α and β is not unique. Specifically, different pairs of α and β can yield the same steady-state MSD value. For example, for the LTVFF mechanism, the pairs α=0.91,β=0.0015, α=0.89,β=0.002, and α=0.87,β=0.0025 provide almost the same steady-state MSD performance. For the LCTVFF mechanism, when γ=0.95, the pairs α=0.93,β=0.0025, α=0.90,β=0.005, α=0.85,β=0.0075, and α=0.80,β=0.01 yield almost the same steady-state MSD value. In addition, it can also be observed that with the decreasing of α and β, the steady-state performance degrades. Furthermore, the result in Fig. 5 reveals that the steady-state MSD performance of the LCTVFF mechanism does not change so much as γ varies for different pairs of α and β.
5.1.2 MSD and EMSE performance
LTVFF-1 | α=0.91,β=0.0015 |
λ _{0}=0.995,λ _{+}=0.9998,λ _{−}=0.980 | |
LTVFF-2 | α=0.91,β=0.0015 |
λ _{0}=0.950,λ _{+}=0.9998,λ _{−}=0.950 | |
LCTVFF-1 | α=0.95,β=0.005,γ=0.95 |
λ _{0}=0.995,λ _{+}=0.9998,,λ _{−}=0.950 | |
LCTVFF-2 | α=0.95,β=0.005,γ=0.95 |
λ _{0}=0.950,λ _{+}=0.9998,,λ _{−}=0.950 | |
GVFF-1 | λ _{0}=0.995,μ=0.005,λ _{+}=0.9998,λ _{−}=0.990 |
GVFF-2 | λ _{0}=0.950,μ=0.005,λ _{+}=0.9998,λ _{−}=0.950 |
Fixed-1 | λ=0.998 |
Fixed-2 | λ=0.995 |
Fixed-3 | λ=0.950 |
Next, we elaborate the numerical stability of the proposed LTVFF-DRLS and LCTVFF-DRLS algorithms. Through tuning the parameters α, β, γ, λ _{+}, λ _{+} to different values, the proposed LTVFF-DRLS and LCTVFF-DRLS algorithms can have different convergence speed and steady-state performance, but their MSD and EMSE curves always decrease to the steady-state. Indeed, after a number of experiments, we have not encountered the case where they diverge. Hence, the proposed LTVFF and LCTVFF mechanisms do not make the numerical stability of the DRLS algorithm worse. Besides, the simulation results in Fig. 14 show that, after switching some nodes in the network, the proposed LTVFF-DRLS and LCTVFF-DRLS algorithms still achieve superior performance to the conventional DRLS algorithm, and they exhibit smoother MSD curves at the time of switching nodes, especially the LCTVFF-DRLS algorithm. This further verifies that the proposed algorithms improve instead of impair the numerical stability of the DRLS algorithm by keeping better tracking of the variations.
5.2 Distributed spectrum estimation
In this part, we extend the proposed LTVFF-DRLS and LCTVFF-DRLS algorithms to the application of distributed spectrum estimation, for which we focus on estimating the parameter w _{0} that is relevant to the unknown spectrum of a transmitted signal s. First of all, we characterize the system model of distributed spectrum estimation.
where \(\phantom {\dot {i}\!}\mathbf {b}_{0}(f)=\text {col}\{b_{1}(f),b_{2}(f),\ldots,b_{N_{b}}(f)\}\) is the vector of basis functions [33, 34], \(\phantom {\dot {i}\!}\mathbf {w}_{0}=\text {col}\{{w}_{01},{w}_{02},\ldots,{w}_{0N_{b}}\}\) is the expansion parameter to be estimated and represents the power that transmits the signal s over each basis, and N _{ b } is the number of basis functions.
where \(\mathbf {b}_{k,i}(f)=\left [|H_{k}(f,i)|^{2}b_{m}(f)\right ]_{m=1}^{N_{b}}\in \mathbb {R}^{N_{b}}\) and \(\sigma _{r,k}^{2}\) denotes the receiver noise power at node k.
where \(\mathbf {d}_{k,i}=\left [d_{k,i}^{f_{j}}\right ]_{j=1}^{N_{c}}\in \mathbb {R}^{N_{c}}\), \(\mathbf {B}_{k,i}=\left [\mathbf {b}^{T}_{k,i}(f_{j})\right ]_{j=1}^{N_{c}}\in \mathbb {R}^{N_{c}{\times }N_{b}}\), with N _{ c }>N _{ b }, and \(\mathbf {v}_{k,i}=\left [v_{k,i}^{j}\right ]_{j=1}^{N_{c}}\in \mathbb {R}^{N_{c}}\).
Next, we carry out simulations to show the performance of the proposed algorithms when applied to distributed spectrum estimation. We consider a sensor network composed of N=20 nodes in order to estimate the unknown expansion parameter w _{0}. We use N _{ b }=50 non-overlapping rectangular basis functions with amplitude equal to one to approximate the PSD of the unknown spectrum. The nodes can scan N _{ c }=100 frequencies over the frequency axis, which is normalized between 0 and 1. In particular, we assume that only 8 entries of w _{0} are non-zero, which implies that the unknown spectrum is transmitted over 8 basis functions. Thus, the sparsity ratio equals to 8/50. We set the power transmitted over each basis function to be 0.7 and the variance of the sampling noise to be 0.004.
6 Conclusions
In this paper, we have proposed two low-complexity VFF-DRLS algorithms for distributed estimation including the LTVFF-DRLS and LCTVFF-DRLS algorithms. For the LTVFF-DRLS algorithm, the forgetting factor is adjusted by the time-averaged cost function, while for the LCTVFF-DRLS algorithm, the forgetting factor is adjusted by the time-averaged of the correlation of two successive estimation errors. We also have investigated the computational complexity of the low-complexity VFF mechanisms as well as the proposed VFF-DRLS algorithms. In addition, we have carried out the convergence and steady-state analysis for the proposed algorithms. Moreover, we also have derived analytical expressions for the steady-state MSD and EMSE. The simulation results have shown the superiority of the proposed algorithms to the conventional DRLS and GVFF-DRLS algorithms in applications of distributed parameter estimation and distributed spectrum estimation and have verified the effectiveness of our proposed analytical expressions for the steady-state MSD and EMSE.
7 Appendices
7.1 A: Proof of the uncorrelation of ρ _{ k }(i−1) and |e _{ k }(i)|^{2} in the steady state
That is, we can conclude that ρ _{ k }(i−1) and |e _{ k }(i)|^{2} are uncorrelated in the steady state.
7.2 B: Proof of (39)
where the values of λ _{ k }(∞) is given in (24) for the LTVFF mechanism and in (35) for the LCTVFF mechanism, respectively. Hence, we obtain (39). Note that, by setting appropriate truncation bounds for λ _{ k }(i), the steady-state forgetting factor value will not be influenced by the truncation. Hence, the result (39) always holds true despite the truncation employed to the VFF mechanisms. Indeed, the truncation mechanism only plays a role during the process of converging. Once the algorithms reach the steady state, the values of the forgetting factor are not affected by the truncation mechanism any longer.
Declarations
Funding
This work was supported in part by the National Natural Science Foundation of China under Grant 61471319, the Scientific Research Project of Zhejiang Provincial Education Department under Grant Y201122655, and the Fundamental Research Funds for the Central Universities.
Authors’ contributions
YC and RCdL proposed the original idea. LZ carried out the experiment. In addition, LZ and YC wrote the paper. CL and RCdL supervised and reviewed the manuscript. All authors read and approved the final manuscript.
Competing interests
The authors declare that they have no competing interests
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
- P Corke, T Wark, R Jurdak, W Hu, P Valencia, D Moore, Environmental wireless sensor networks. Proc. IEEE.98(11), 1903–1917 (2010).View ArticleGoogle Scholar
- JG Ko, C Lu, MB Srivastava, JA Stankovic, A Terzis, M Welsh, Wireless sensor networks for healthcare. Proc. IEEE.98(11), 1947–1960 (2010).View ArticleGoogle Scholar
- R Abdolee, B Champagne, AH Sayed, in Proc. IEEE Statistical Signal Processing Workshop. Diffusion LMS for Source and Process Estimation in Sensor Networks (IEEEAnn Arbor, 2012).Google Scholar
- R Abdolee, B Champagne, AH Sayed, in Proc. IEEE ICASSP. Diffusion LMS Localization and Tracking Algorithm for Wireless Cellular Networks (IEEEVancouver, 2013).Google Scholar
- R Abdolee, B Champagne, AH Sayed, Diffusion adaptation over multi-agent networks with wireless link impairments. IEEE Trans. Mob. Comput. 15(6), 1362–1376 (2016).View ArticleGoogle Scholar
- FS Cattiveli, CG Lopes, AH Sayed, in Proc. IEEE Workshop Signal Process. Advances Wireless Commun. (SPAWC). A Diffusion RLS Scheme for Distributed Estimation over Adaptive Networks (IEEEHelsinki, 2007), pp. 1–5.Google Scholar
- FS Cattiveli, CG Lopes, AH Sayed, Diffusion recursive least-squares for distributed estimation over adaptive networks. IEEE Trans. Signal Process.56(5), 1865–1877 (2008).MathSciNetView ArticleGoogle Scholar
- FS Cattiveli, AH Sayed, Diffusion LMS strategies for distributed estimation. IEEE Trans. Signal Process.58(3), 1035–1048 (2010).MathSciNetView ArticleGoogle Scholar
- CG Lopes, AH Sayed, Diffusion least-mean squares over distributed networks: formulation and performance analysis. IEEE Trans. Signal Process.56(7), 3122–3136 (2008).MathSciNetView ArticleGoogle Scholar
- Y Liu, C Li, Z Zhang, Diffusion sparse least-mean squares over networks. IEEE Trans. Signal Process.60(8), 4480–4485 (2012).MathSciNetView ArticleGoogle Scholar
- S Xu, RC de Lamare, HV Poor, in Proc. IEEE ICASSP. Adaptive link selection strategies for distributed estimation in diffusion wireless networks (IEEEVancouver, 2013).Google Scholar
- S Xu, RC de Lamare, HV Poor, Distributed compressed estimation based on compressive sensing. IEEE Signal Process. Lett.22(9), 1311–1315 (2015).View ArticleGoogle Scholar
- MOB Saeed, A Zerguine, SA Zummo, A variable step-size strategy for distributed estimation over adaptive networks. EURASIP J. Adv Signal Process. 2013(1), 1–14 (2013).View ArticleGoogle Scholar
- H Lee, S Kim, J Lee, W Song, A variable step-size diffusion LMS algorithm for distributed estimation. IEEE Trans. Signal Process.63(7), 1808–1820 (2015).MathSciNetView ArticleGoogle Scholar
- Z Liu, Y Liu, C Li, Distributed sparse recursive least-squares over networks. IEEE Trans. Signal Process.62(6), 1386–1395 (2014).MathSciNetView ArticleGoogle Scholar
- S Huang, C Li, Distributed sparse total least-squares over networks. IEEE Trans. Signal Process.63(11), 2986–2998 (2015).MathSciNetView ArticleGoogle Scholar
- C Li, P Shen, Y Liu, Z Zhang, Diffusion information theoretic learning for distributed estimation over network. IEEE Trans. Signal Process.61(16), 4011–4024 (2013).MathSciNetView ArticleGoogle Scholar
- Z Liu, C Li, Y Liu, Distributed censored regression over networks. IEEE Trans. Signal Process.63(20), 5437–5449 (2015).MathSciNetView ArticleGoogle Scholar
- S Haykin, Adaptive Filter Theory, 4th edn (Prentic-Hall, Englewood cliffs, 2000).MATHGoogle Scholar
- S Leung, CF So, Gradient-based variable forgetting factor RLS algorithm in time-varying environments. IEEE Trans. Signal Process. 53(8), 3141–3150 (2005).MathSciNetView ArticleGoogle Scholar
- CF So, SH Leung, Variable forgetting factor RLS algorithm based on dynamic equation of gradient of mean square error. Electron. Lett.37(3), 202–203 (2011).View ArticleGoogle Scholar
- S Song, J Lim, S Baek, K Sung, Gauss Newton variable forgetting factor recursive least squares for time varying parameter tracking. Electron. Lett.36(11), 988–990 (2000).View ArticleGoogle Scholar
- S Song, J Lim, SJ Baek, K Sung, Variable forgetting factor linear least squares algorithm for frequency selective fading channel estimation. IEEE Trans. Vehi. Techonol. 51(3), 613–616 (2002).View ArticleGoogle Scholar
- F Albu, in Proc. of ICARCV 2012. Improved Variable Forgetting Factor Recursive Least Square Algorithm (IEEEGuangzhou, 2012).Google Scholar
- Y Cai, RC de Lamare, M Zhao, J Zhong, Low-complexity variable forgetting factor mechanisms for blind adaptive constrained constant modulus algorithms. IEEE Trans. Signal Process.60(8), 3988–4002 (2012).MathSciNetView ArticleGoogle Scholar
- L Qiu, Y Cai, M Zhao, Low-complexity variable forgetting factor mechanisms for adaptive linearly constrained minimum variance beamforming algorithms. IET Signal Process. 9(2), 154–165 (2015).View ArticleGoogle Scholar
- R Arablouei, K Dogancay, S Werner, Y Huang, Adaptive distributed estimation based on recursive least-squares and partial diffusion. IET Signal Process. 62(14), 1198–1208 (2014).MathSciNetGoogle Scholar
- DS Tracy, RP Singh, A new matrix product and its applications in partitioned matrix differentiation. Statistica Neerlandica. 51(3), 639–652 (2003).Google Scholar
- H Shin, AH Sayed, in Proc. IEEE ICASSP. Transient Behavior of Affine Projection Algorithms (IEEEHong Kong, 2003).Google Scholar
- JH Husoy, MSE Abadi, in IEEE MELECON 2004. A Common Framework for Transient Analysis of Adaptive Filters (IEEEDubrovnik, 2004).Google Scholar
- AH Sayed, Adaptive filters (Wiley, 2011).Google Scholar
- JA Bazerque, GB Giannakis, Distributed spectrum sensing for cognitive radio networks by exploiting sparsity. IEEE Trans. Signal Process.58(3), 1847–1862 (2010).MathSciNetView ArticleGoogle Scholar
- S Chen, DL Donoho, MA Saunders, Atomic decomposition by basis pursuit. SIAM J. Sci Comput. 20:, 33–61 (1998).MathSciNetView ArticleMATHGoogle Scholar
- Y Zakharov, T Tozer, J Adlard, Polynomial splines-approximation of Clarke’s model. IEEE Trans. Signal Process.52(5), 1198–128 (2004).MathSciNetView ArticleGoogle Scholar
- PD Lorenzo, S Barbarossa, A Sayed, Distributed spectrum estimation for small cell networks based on sparse diffusion adaptation. IEEE Signal Process. Lett.20(123), 1261–1265 (2013).View ArticleGoogle Scholar
- ID Schizas, G Mateos, GB Giannakis, Distributed LMS for consensus-based in-network adaptive processing. IEEE Trans. Signal Process.57(6), 2365–2382 (2009).MathSciNetView ArticleGoogle Scholar