 Research
 Open Access
 Published:
Hierarchical coordinated antijamming channel access in clustering networks: a multileader multifollower Stackelberg game approach
EURASIP Journal on Advances in Signal Processing volume 2021, Article number: 42 (2021)
Abstract
This paper mainly investigates the multiuser coordinated antijamming problem in clustering communication networks. In such kinds of networks, there exist multiple clusters and multiple users who communicate with their receivers simultaneously. Besides, a malicious jammer persistently attacks channels with wideband and dynamic changing jamming signals. To cope with these challenges brought by the largescale clustering network and the dynamic wideband jamming, a hierarchical coordinated antijamming approach is proposed, and a multileader multifollower Stackelberg game is introduced to model the antijamming problem. In detail, cluster heads act as leaders, and select available frequency bands to avoid jamming attacks, while users in each cluster act as followers and select corresponding channels distributedly and independently. Moreover, it is proved that there exist multiple Stackelberg equilibriums (SEs) in the proposed game. To obtain SEs, a hierarchical coordinated antijamming channel access (HCACA) algorithm is designed. Simulation results illustrate that the proposed approach is effective to cope with the dynamic wideband jamming attacks. Furthermore, it is also depicted that the proposed approach outperforms the distributed antijamming comparative approach in terms of convergence speed.
Introduction
With the development of wireless communication technologies, the number of wireless communication devices increases explosively, causing severe shortage of spectrum resources. Thus, how to coordinate the internal interference is a vital problem that remains to be solved. In addition, due to the advance of the softwaredefined radio, it is convenient for malicious users (e.g., malicious jammer) to carry out jamming attacks with lowcost and wideband noise signals; hence, there exists more serious security threaten for communication devices. In a word, it is an important issue to “coordinate the internal interference” and “resist external malicious attacks” simultaneously in wireless communication networks [1–4].
Focusing on the internal coordination and external confrontation problem for effective spectrum resource utilization, game theory [5–14] is a feasible tool to model and analysis the internal and external competitive or cooperative relationships among different decisionmakers (i.e., users or malicious jammer). For example, as for internal coordination, authors in [6] reviewed the decisiontheoretic solutions for channel access strategies in the opportunistic spectrum access system and presented some effective game models with respect to internal interference avoidance. Besides, authors in [7] reviewed the applications of repeated games in wireless networks, while authors in [8] formulated users’ competition for channel access using a noncooperative game and proved that the proposed game model is an ordinal potential game where there exists at least one pure strategy Nash equilibrium (NE).
While considering the existence of malicious jamming attacks, the Stackelberg game [15–27] is a suitable game model to formulate the competitive interactions between the userside and the jammerside as it can well depict the sequential decisionmaking relationship between two countermeasure sides. For example, Yang et al. formulated the antijamming power control problem using the Stackelberg game [15, 16]. In detail, the user was the leader, and the smart jammer acted as the follower of the game. Besides, the utility functions of the user and the jammer were designed respectively. Authors in [17] considered the antijamming power control problem with observation error and derived the Stackelberg equilibrium (SE) of the proposed Stackelberg game. In addition, the authors in [18] formulated the competition between one user and one jamming as a Bayesian Stackelberg game and took the incomplete information into consideration. In [19], the authors investigated the discrete antijamming power control problem, and solved the problem using the Stackelberg game and Qlearning algorithm. Considering the multiuser antijamming scenario, in [20], a oneleader multifollower Bayesian Stackelberg game was formulated, and the influence of observation error as well as mobility of unmanned aerial vehicles (UAVs) were investigated. Moreover, authors in [21] investigated the communication confrontation scenario, and modeled the interactions in the confrontation scenario as a Stackelberg game, with each player seeking for maximizing their utility respectively.
However, there exist some main challenges to extend the above game approaches to the multiuser coordinated channel access scenario in the clustering networks with dynamic wideband jamming attacks. (i) The increase of network scale causes more severe internal conflicts, thus it is important to coordinate the cochannel interference as well as to reduce the convergence time and complexity of the channel access algorithm. (ii) Considering the existence of the dynamic wideband attacker, it is harder for users to adapt to the dynamic change of the jamming attack and to avoid the wideband jamming signals which take up more spectrum resources than the single tone jamming signals.
To solve these challenges, and to fully take advantage of the clustering network, a multileader multifollower Stackelberg game is formulated to model the competitive interactions between cluster heads and cluster users. Moreover, to accommodate the dynamic changing of the jamming attacks, the idea of jamming utilization is also introduced. To sum up, the main contributions of this paper are as follows:

The malicious wideband jamming signals are “utilized” by cluster heads as coordination signals [28] to guide channel access of users, which means for different sensed jamming signals, different antijamming channel access strategies are adopted by users. The application of coordination signals helps users to adapt to the dynamic changing of jamming attacks.

A multileader multifollower Stackelberg game is formulated, where cluster heads act as leaders, and select available frequency bands to avoid jamming attacks. While users in each cluster act as followers, and select corresponding channels distributedly and independently.

The proposed Stackelberg game is decomposed into multileader subgame from the coarse granularity and the multifollower subgame from the fine granularity. It is proved that the proposed game has at least one SE for each jamming attack.

To obtain SEs, a hierarchical coordinated antijamming channel access (HCACA) algorithm is designed. In addition, simulation results illustrate that the proposed approach is effective to cope with the dynamic wideband jamming attacks, and it accelerates the convergence of the clustering network compared with the distributed learning scheme.
Note that some existing works investigated the antijamming problem using gametheoretic framework [20, 29]. The main differences are summarized as follows: as mentioned in Section 1, the authors in [20]proposed a oneleader multifollower Bayesian Stackelberg game, and the closedform of the SE was derived. However, work [20] is not suitable for multiuser scenarios with a large scale, and the proposed approach can not adapt to the dynamic changing characteristic of jamming attacks. Besides, in [29], the authors investigated the distributed coordinated antijamming scenario. However, the proposed approach may not be welladapted to the wideband jamming in the clustering network. Thus, different from these two works, we fully take advantage of the clustering network in this work, and then propose a hierarchical coordinated antijamming channel access approach that is effective with the property of fast convergence.
The rest of this paper is organized as follows. In Section 2, methods are introduced briefly. In Section 3, the system model and problem formulation are presented. In Section 4, the multileader multifollower Stackelberg game for hierarchical coordinated antijamming channel access is formulated. While in Section 5, the hierarchical coordinated antijamming channel access algorithm is proposed. In Section 6, simulation results and discussions are illustrated. Finally, the conclusion is made in Section 7.
Methods
The aim of this study is to solve the multiuser coordinated antijamming channel access problem in clustering communication networks. In detail, we propose a hierarchical coordinated antijamming approach, and formulate the antijamming problem as a multileader multifollower Stackelberg game. Besides, we prove that there exist multiple Stackelberg Equilibriums (SEs) in the proposed game, and we then design a hierarchical coordinated antijamming channel access (HCACA) algorithm to obtain NEs of the game.
The paper presents a “game & learning” structure, where the proposed game is used to model the interactions among different decisionmakers, and the proposed learning algorithm is to find the optimal or nearly optimal solutions of the game model. The theoretical analysis is supported by the experimental evaluation with the randomly generated clustering network under different dynamic wideband jamming patterns, while the simulations have been carried out with Matlab R2019a.
System model and problem formulation
System model
In this paper, we consider a multiuser clustering network that is under the attack of a malicious jammer. As shown in Fig. 1, the network is divided into H clusters according to the geographical location of communication devices [30], and the cluster set is \({\mathcal {H}}{\mathrm { = }}\left \{ {1,...,h,...,H} \right \}\). Besides, the neighbor set of cluster h is denoted as \({{\mathcal {J}}_{h}}\). In each cluster, there exist multiple transmitterreceiver pairs, and each transmitterreceiver pair is assumed to be one user. Denote M_{h} as the number of users in cluster h, and the user set is \({{\mathcal {M}}_{h}}\). In cluster h, the kth user is denoted as m_{h,k}, thus we have \({m_{h,k}} \in {{\mathcal {M}}_{h}}\). There exist A orthogonal channels for users to access, and the channel set is \({\mathcal {A}}\). To disturb the legitimate transmissions of users, a malicious jammer incessantly sends highpower noise signals to multiple channels.
In each cluster, there exists a cluster head who can exchange information with cluster users. For example, each cluster head can send available frequency band information to its governed users. After obtaining such frequency information, cluster users are able to select their transmission channels distributedly. Denote the transmission channel of user k in cluster h as a_{h,k}, and the transmission throughput of this user is expressed as follows:
where \({B_{{a_{h,k}}}}\) is the channel bandwidth, P_{h,k} is the transmission power of the kth user in cluster h, and G_{h,k} is the channel gain of the transmission link. Assume that wireless communication channels undergo block fading, and the Rayleigh fading channel model is considered [31]. Thus, \({G_{h,k}} = d_{h,k}^{ \alpha }{\varepsilon _{h,k}}\), where d_{h,k} denotes the transmission distance, α is the pathloss factor, and ε_{h,k} is the instantaneous random component. Besides, \(N_{a_{h,k}}\) is the noise power of the channel. In this paper, users adopt multiple access technique that is based on multichannel slotted ALOHA, and only one transmission is allowed in a slot. To show the influence of the intracluster interference, intercluster interference and external malicious jamming, a cumulative jamminginterference indicator function Δ(a_{h,k},a_{−(h,k)},A_{j}) is designed [32], where a_{−(h,k)} denotes the channel access strategy combination of all users except the kth user in cluster h, and A_{j} is the jamming channel combination of the malicious jammer. In detail, Δ(a_{h,k},a_{−(h,k)},A_{j}) is expressed as follows:
where \({{\mathcal {M}}_{h}}\backslash k\) denotes all users in cluster h except user k, a_{h,i} is the channel access strategy of user i in the cluster, while a_{g,i} is the channel access strategy of user i in the neighboring cluster g. Note that there exist three indicator functions δ(a_{h,k},a_{h,i}),δ(a_{h,k},a_{g,i}) and δ(a_{h,k},A_{j}), which show the intracluster interference, the intercluster interference and the external malicious jamming respectively. Besides, these indicator functions can be denoted as follows:
where \({P_{y}}d_{y \to h,k}^{ \alpha } > {\tau _{\mathrm {{threshold}}}}\) represents that the received interference/jamming signal power from user/jammer y is higher than the threshold τ_{threshold}. The above indicator function illustrates that if the channel access strategies of users are conflicting, they will suffer from cochannel interference when the interference threshold is reached. While if the channel access strategy of one user is overlapped with the attack strategy of the malicious jammer, this user will suffer from malicious jamming as the jammer will send highpower noise signals to those attacking channels. Note that the jammer is able to carry out wideband jamming attacks, the above relationship turns to a_{h,k}∈A_{j} for the wideband jamming case.
Problem formulation
Denote the channel set obtained by the cluster h as \({\mathcal {A}}_{h}\), thus we have \({a_{h,k}} \in {{\mathcal {A}}_{h}}\). Besides, the optimization objective of the clustering network is:
where \(\mathbf {a}_{h}^ * {\mathrm { = }}\left ({a_{h,1}^ *,...,a_{h,{\mathcal {M}}_{h}}^ *} \right),\forall h \in {\mathcal {H}}\) is all users’ channel access strategies in the cluster h. The above optimization objective indicates that each user is willing to find a optimal channel access strategy to maximize the sum of the throughput of the clustering network. The first constraint condition limits the range of jamming channels, while the second constraint means that each user either selects one channel or keeps silent at each time slot.
However, there exist some challenges to solve the above optimization objective. (i) The optimization objective is a nonconvex problem, which is difficult to be solved directly in distributed clustering networks. (ii) Due to the dynamic and wideband characteristics of malicious jamming, cluster users should coordinate internal interference and avoid external jamming attacks. (iii) With the increase of the network scale, the computation complexity increases correspondingly, and thus, a lowcomplexity channel access approach is preferred to enhance the convergence speed of the network. To cope with these three challenges, the idea of “game and learning” is introduced, in which a multileader multifollower Stackelberg game is formulated to analyze the relationship among users, and a hierarchical coordinated antijamming channel access algorithm is designed to obtain the game equilibriums.
Multileader multifollower Stackelberg game for hierarchical coordinated antijamming channel access
Stackelberg game formulation
To solve the hierarchical coordinated antijamming channel access problem in the clustering network, a multileader multifollower Stackelberg game is formulated firstly, where cluster heads act as multiple leaders of the game, while cluster members are followers. Besides, the proposed game model can be decomposed into multileader subgame and multifollower subgame, as shown in Fig. 2.
Considering the influence of the malicious, wideband and dynamic jamming attacks, each cluster competes with the jammer to obtain available spectrum bands for its cluster members. Due to the dynamic characteristic of the jammer, the band selection strategies of each cluster need to be adjusted to adapt to the changing of the attacking strategies. After avoiding the malicious jamming bands, each cluster sends available band information to its cluster members. Then, users in the cluster access these available channels distributedly to coordinate the intracluster and intercluster conflicts. Mathematically, the game model is expressed as follows:
where \({\mathcal {H}}\) is the cluster set, \({\left \{ {{{\mathcal {M}}_{h}}} \right \}_{h \in {\mathcal {H}}}}\) is the user set of each cluster, and \({\mathcal {J}}\) represents the jammer. \({\mathcal {A}}\) is the channel set of the clustering network, \({\left \{ {{{\mathcal {A}}_{h}}} \right \}_{h \in {\mathcal {H}}}}\) is the channel set of each cluster \(h \in {\mathcal {H}}\), and U_{h} is the utility function of cluster head h. u_{h,}_{k} is the utility function of the user k in the cluster h, which is the same as the throughput R_{h,k} shown in Eq. 1. In addition, \({{\mathcal {A}}_{j}}\) is the attack strategy set of the wideband jammer.
After the Stackelberg game formulation, the definition of Stackelberg equilibrium (SE) is given, which is shown as follows:
Definition 1
In the proposed multileader multifollower Stackelberg game, SE delivers the optimal strategy combination of cluster heads and users for each jamming strategy \(A_{j} \in {{\mathcal {A}}_{j}}\). For multiple leaders, there exists strategy combination \(\left ({{{A}}_{1}^ *, \cdots {{A}}_{H}^ *} \right)\), which maximizes the cumulative utility of the cluster heads. While for multiple followers, strategy combination \({\mathrm { }}\left \{ {\left ({a_{1,1}^ *, \cdots a_{1,{{\mathcal {M}}_{1}}}^ *} \right)...\left ({a_{H,1}^ *, \cdots a_{H,{{\mathcal {M}}_{H}}}^ *} \right)} \right \}\) is the bestresponse strategy combination with respect to leaders’ strategies [33]. Here, strategy combination \(\left \{ {\left ({A_{1}^{*}, \cdots A_{H}^{*}} \right),\left ({a_{1,1}^{*}, \cdots a_{1,{M_{1}}}^{*}} \right)...\left ({a_{H,1}^{*}, \cdots a_{H,{M_{H}}}^{*}} \right)} \right \}\) is the SE of the game, which satisfies the following conditions:
In Eq. 6, A−h∗ denotes all the cluster heads’ optimal strategy combination except cluster head h, and \(a_{ \left ({h,k} \right)}^ *\) is all users’ bestresponse strategy combination except user k in cluster h.
According to the hierarchical characteristic of the multileader multifollower Stackelberg game, the optimization objective P_{1} is rewritten as:
Note that there are multiple coupling relationships in the clustering network, and there also exist competitive relationships between different clusters and the malicious jammer, thus the above game formulation is hard to be solved directly using traditional optimization approaches. In this paper, the game is decomposed into two subgames from two different granularities, that is, the multileader(cluster head) subgame antijamming spectrum sharing from the coarse granularity, and the multifollower (cluster member) distributed channel access from the fine granularity.
Multileader subgame from the coarse granularity
The objective of the multileader subgame is to analyze and model the competitive interactions between different cluster heads and the jammer. Generally, this subgame is formulated as:
where CH_{h} is the cluster head of the cluster h, \(A_{h} \subseteq {{\mathcal {A}}_{h}}\) is one of the band selection strategy of the cluster h, \(A_{j} \subseteq {{\mathcal {A}}_{j}}\) is one of the jamming strategy, and U_{h} is the utility function of corresponding cluster head, which is expressed as follows:
where e∈A_{h} is a specific channel in spectrum band \({{{A}}_{h}},\left  {\mathcal {A}} \right \) is the cardinality of channel set \(\mathcal {A}\). Moreover, the jamming indicator function δ(e,A_{j}) is expressed as follows:
Equation 10 means that the utility of the cluster head h is the difference between the cardinality of channel set \(\mathcal {A}\) and the jamming degree of current spectrum band strategy A_{h}. Hence, the optimization objective of the multileader subgame is:
The above optimization objective means that each cluster head tries to find the optimal spectrum band that minimizes the cumulative jamming degrees. Then, we give the definition of NE, and we then prove that there exists at least one pure strategy NE in the multileader subgame.
Definition 2
The strategy combination \(\left ({{{A}}_{1}^ *, \cdots {{A}}_{H}^ *} \right)\) is the pure strategy NE of the game if and only if no cluster head could enhance its utility via unilaterally changing its spectrum band selection strategy [34]:
The above equation shows that for a specific jamming strategy A_{j}, the pure strategy NE improves the total network utility via unilaterally optimizing each cluster head’s utility.
Theorem 1
For a specific jamming strategy A_{j}, there exists at least one pure strategy NE \(\left ({{{A}}_{1}^ *, \cdots {{A}}_{H}^ *} \right)\) with the following conditions:
Equation 13 depicts that the NE strategy of each cluster head changes with the variation of the jamming strategies.
Proof
The proof by contradiction is applied for the multileader subgame [29]. Assume that strategy combination \(\left ({{{A}}_{1}^ *, \cdots {{A}}_{H}^ *} \right)\) is not the NE, which means \(\exists h \in {\mathcal {H}}\) that satisfies \({U_{h}}\left ({A_{h}^{*},{A_{j}}} \right) < {U_{h}}\left ({{A_{h}},{A_{j}}} \right)\). Actually, unilaterally changing from \(A_{h}^{*}\) to A_{h} means that the cluster head h selects some channels that are jammed. Then, \(\sum \limits _{e \in {{{A}}_{h}}} {\delta \left ({e,{{ A}_{j}}} \right)}\) increases, and U_{h} decreases correspondingly. The above case depicts that the unilateral change does not satisfy \({U_{h}}\left ({A_{h}^{*},{A_{j}}} \right) < {U_{h}}\left ({{A_{h}},{A_{j}}} \right)\), and it is contradictory with the previous assumption. Thus, the proof of the above theorem has been completed. □
Multifollower subgame from the fine granularity
The objective of the multifollower subgame is to analyze and model the intracluster and intercluster competitive interactions among different cluster users. Note that the multileader subgame is to guide the coarsegrained spectrum band selection of different cluster heads, and the multifollower subgame is to guide the finegrained channel access of different users. Mathematically, the multifollower subgame is expressed as follows:
where u_{h,k}=R_{h,k} denotes the throughput of the user k in the cluster h. In the multifollower subgame, only interference coordination needs to be considered after the multifollower subgame reaching NE for jamming strategy A_{j}.
Thus, the objective of the finegrained multifollower subgame is:
The above optimization objective means that each user tries to find the optimal channel access strategy that maximizes the cumulative throughput of the clustering network.
Theorem 2
For a specifical NE strategy \(\left ({{{A}}_{1}^ *, \cdots {{A}}_{H}^ *} \right)\) when the jammer attacks band A_{j}, there exists at least one pure strategy NE \({\mathrm { }}\left \{ {\left ({a_{1,1}^ *, \cdots a_{1,{{\mathcal {M}}_{1}}}^ *} \right)...\left ({a_{H,1}^ *, \cdots a_{H,{{\mathcal {M}}_{H}}}^ *} \right)} \right \}\) with the following condition:
a_{h,k}=0 means that the user keeps silent when strategy 0 is selected. The first line of Eq. 16 means that each user selects one channel from the optimal spectrum band of the cluster it belongs, or keeps silent. The second line depicts that neighboring users choose different channels to access when reaching NE. The third line illustrates that users avoid those jamming channels.
Proof
Similarly, the proof by contradiction is also applied for the multifollower subgame. Assume that the strategy combination \({\mathrm { }}\left \{ {\left ({a_{1,1}^ *, \cdots a_{1,{{\mathcal {M}}_{1}}}^ *} \right)...\left ({a_{H,1}^ *, \cdots a_{H,{{\mathcal {M}}_{H}}}^ *} \right)}\right \}\) is not one NE of the multifollower subgame, which means \(\exists h \in {\mathcal {H}},\exists k \in {{\mathcal {M}}_{k}}\) that satisfies \({u_{h,k}}\left ({a_{h,k}^ *,a_{ \left ({h,k} \right)}^ *,{A_{j}}} \right) < {u_{h,k}}\left ({{a_{h,k}},{a_{ \left ({h,k} \right)}},{A_{j}}} \right)\). Then, the throughput of the user k in cluster h can be rewritten as:
However, note that unilateral change from channel selection \(a_{h,k}^{*}\) to a_{h,k} causes the throughput of current user down to zero, as the conflict degree increases when breaking the equilibrium state. Thus, the above case means that \({u_{h,k}}\left ({a_{h,k}^ *,a_{ \left ({h,k} \right)}^ *,{A_{j}}} \right) < {u_{h,k}}\left ({{a_{h,k}},{a_{ \left ({h,k} \right)}},{A_{j}}} \right)\) can not be satisfied, and the above hypothesis is not valid. To sum up, the conclusion of Theorem 2 holds. □
Theorem 3
For a specific jamming strategy A_{j}, there exists at least one SE in the proposed multileader multifollower Stackelberg game.
Proof
In the above subsections, we proved that there exist at least one NE strategy combination \(\left ({{{A}}_{1}^ *, \cdots {{A}}_{H}^ *} \right)\) for the leaders’ subgame, which maximizes the cumulative utility of all cluster heads, chooses optimal spectrum bands that are not jammed by the malicious jammer. In response to leaders’ strategies, the follower subgame shows that users can reach at least one NE strategy combination \({\mathrm { }}\left \{ {\left ({a_{1,1}^ *, \cdots a_{1,{{\mathcal {M}}_{1}}}^ *} \right)...\left ({a_{H,1}^ *, \cdots a_{H,{{\mathcal {M}}_{H}}}^ *} \right)}\right \}\) which maximize the network throughput of cluster users. Combining Theorem 1, Theorem 2 and the definition of SE, we conclude that the proposed game has at least one SE. □
Figure 3 shows the operation process to obtain SE of the multileader multifollower Stackelberg game. After observing the jamming strategy A_{j}, cluster heads (leaders) firstly adjust their spectrum band strategies to avoid the jamming attacks. Then, cluster users (followers) distributedly access channels after obtaining the spectrum band information sent by its cluster head. After that, cluster heads continue adjusting their band strategies until reaching the NE of the leader subgame. When cluster users obtain the NE band strategy of their cluster heads, they adjust their channel access strategies until reaching the bestresponse strategy (which is also the NE of the follower subgame). In the process of continuous adjustment of cluster heads and cluster users, the system gradually reaches the SE for the current jamming strategy. A_{j} While with the dynamic changing of jamming attacks, different SEs can be achieved via the Stackelberg game iterations.
Hierarchical coordinated antijamming channel access algorithm
In this section, a hierarchical coordinated antijamming channel access (HCACA) algorithm is designed to obtain SEs of the multileader multifollower Stackelberg game.
Slot structure
Before introducing the algorithm, we first design a slot structure for the operation of the hierarchical coordinated antijamming channel access, as shown in Fig. 4. At each time slot, actions are taken respectively for cluster heads and cluster users. Firstly, each cluster head observes the jamming signal, and maintains a corresponding strategy table for every jamming strategy. Secondly, each cluster head selects a spectrum band according to its strategy table. Then, all these cluster heads update their band strategies to avoid the current jamming signal. After the implementation of cluster heads, spectrum band strategies are sent to their cluster users, which means the first action for cluster users is the available channel acquisition. Each user will also maintain a specific strategy table for every jamming strategy to keep consistent with its cluster head, and it chooses one channel from its access table, which is randomly generated according to the information of the available channel. Once accessing the channel, each user transmission its data on the selected channel. When the transmission has been completed, users start updating their access strategies to avoid intercluster and intracluster conflicts among users.
Algorithm description
With the proposed slot structure, cluster heads and users update their spectrum band strategies and channel access strategies respectively using the hierarchical coordinated antijamming channel access (HCACA) algorithm. The detail of the HCACA algorithm description is shown in Algorithm 1.
The key steps of the proposed algorithm are the strategy updating processes of cluster heads and users, as shown in Eq. ?? and ??. In detail, when the cluster head h updates its strategy, it judges whether \({A_{h}}\left ({{C_{{A_{j}}}}} \right)=0\). If \({A_{h}}\left ({{C_{{A_{j}}}}} \right)=0\), then the cluster head chooses one spectrum band A^{′} to sense [35]. If the sensing band is available (not jammed), then it sets \({A_{h}}\left ({{C_{{A_{j}}}}} \right)=A'\). Otherwise, it keeps \({A_{h}}\left ({{C_{{A_{j}}}}} \right)=0\) if the sensed band is being jammed. Similarly, when user k in cluster h updates its strategy, it also judges whether a_{n,k}(A_{j})=0, and the sensing process of users are similar to cluster heads. However, if a_{n,k}(A_{j})≠0, the current user will transmit on the selected channel, and keep strategy a_{n,k}(A_{j}) unchanged for successful transmission. However, if conflicts occurs when transmitting, the user keeps its strategy with probability 1−P_{back}, or sets a_{n,k}(A_{j})=0 with probability P_{back}.
Complexity analysis
Motivated by [29], we analyze the complexity of the proposed HCACA algorithm, which is consisted of computational complexity and storage size.
First, we give the computation complexity under the fixed jamming. Note that each cluster head should observe the jamming signal A_{j}, and the corresponding complexity is \({\mathcal {O}}\left ({J{o_{1}}} \right)\). Generally, (Jo_{1}) is a small constant which is related to the observation process. Hence, the computation complexity for jamming observation is \(\left  {\mathcal {H}} \right {\mathcal {O}}\left ({J{o_{1}}} \right)\).
After the spectrum band selection, each cluster head needs to update their band strategy table ATH_{h} according to Eq. ??, and the computation complexity is \({\mathcal {O}}\left ({{B_{{\mathrm {up1}}}}} \right)\), where B_{up1} is a small constant related to the band updating process. Hence, the total computation complexity for band updating is \(\left  {\mathcal {H}} \right {\mathcal {O}}\left ({{B_{{\mathrm {up1}}}}} \right)\).
Following the updating process, each cluster head sends the band selection strategies to its cluster users, and then each user updates its channel strategy table according to Eq. ??, and the computation complexity is denoted as \({\mathcal {O}}\left ({{B_{{\mathrm {up2}}}}} \right)\). Thus, the total computation complexity for channel updating is \({\mathcal {O}}\left ({\sum \limits _{h \in {\mathcal {H}}} {\left  {{{\mathcal {M}}_{h}}} \right } {B_{{\mathrm {up2}}}}} \right)\). Assume that the convergence iterations is T_{1} for the fixed jamming, and the total computation complexity is expressed as:
Similarly, the computation complexity for the sweep jamming case is:
where \({\mathcal {A}}_{\mathrm {{sweep}}}\) is the strategy set of the sweep jammer.
While the computation complexity for the random jamming case is:
where \({\mathcal {A}}_{\mathrm {{random}}}\) is the strategy set of the sweep jammer. Note that the random jammer randomly selects one strategy from its jamming strategy at each time, thus average O(A_{random}^{2}T_{1}) iterations will be necessary for the convergence of the proposed HCACA algorithm (According to [28] Lemma 3).
Moreover, cluster heads and cluster users should store their band strategy tables and channel strategy tables respectively. Thus, the storage size is:
where \({{\mathcal {A}}_{j}}\) denotes different jamming strategy sets for different jamming patterns (1, \({{{\mathcal {A}}_{\mathrm {{sweep}}}}}\) or \({{{\mathcal {A}}_{\mathrm {{random}}}}}\)), St_{1} is a small constant which is related to the size of band strategy table for one jamming strategy, and St_{2} is a small constant with respect to the size of channel strategy table for one jamming strategy.
It is summarized that the computational complexity at each slot is a small constant, and the storage size is not too large as is mentioned above. In a word, the proposed HCACA algorithm has low complexity, and the approach can be implemented using typical technologies.
Simulation results and discussions
In this section, the simulations are conducted, and then some discussions with respect to simulation results are made.
Setting of parameters
First, the details of parameters are given, as shown in Table 1. Here, users randomly locate in an area of 12 km × 12 km, and different users may belong to different clusters. A typical scenario of the clustering network is shown in Fig. 5, where there exist 4 clusters and 24 users. In each cluster, there is a cluster head and 6 users (transmitterreceiver pairs). Users are influenced by the intracluster interference and intercluster interference if they choose the same channel to transmit data. It is assumed that the bandwidth of each channel is 2 MHz, the transmission power of each user is 0.1 W, the pathloss factor is assumed to be 3, and the background noise power is − 100 dBm. Besides, the interference distance is set to be 2000 m, which is positively related to the interference threshold τ_{thres}. Besides, the time is slotted, and the duration of one time slot is 10 ms. In detail, the duration of jamming observation, spectrum band selection, strategy updating of the cluster head, available channel acquisition, channel access and strategy updating of users are set to be 0.5 ms, and the duration of data transmission is 7 ms. Furthermore, the clustering network is under the attack of a malicious jammer, and three different wideband jamming patterns are considered, i.e., the fixed jamming, the sweep jamming and the random jamming, as shown in Fig. 6.
Convergence analysis
Secondly, the convergence performance of the proposed HCACA algorithm is depicted in this subsection. Here, simulations are conducted under these three jamming patterns mentioned above. In detail, Fig. 7 shows the convergence process of the network throughput with fixed, sweep and random wideband jamming (At each time slot, the jammer chooses 5 continuous channels to attack). It can be concluded that the proposed HCACA algorithm converges with the fastest speed under fixed jamming, as it is much easier for cluster heads to avoid the fixed jamming, and for users to coordinate the internal conflicts. Besides, the convergence time increases with the increase of the dynamic characteristic of the jammer as we can see that the proposed algorithm converges slower under the random jamming than under the sweep jamming. Note that in the initial stage, the network throughput fluctuates acutely under the sweep jamming and the random jamming, as the jamming state varies as time goes by. While with the updating of the strategy tables, the network throughput converges gradually. In a word, it depicts that the proposed algorithm is welladapted to the dynamic and wideband jamming environment. Moreover, the existence of channel fading also leads to slight fluctuations in network throughput; however, this phenomenon does not affect the convergence process of the HCACA algorithm.
Performance comparison
To further investigate the performance of the proposed approach, four different algorithms are compared in this subsection, the hierarchical coordinated antijamming algorithm HCACA, the distributed antijamming learning algorithm (DAL), the coordinated learning algorithm proposed in [36], and the random selection algorithm [37]. Here, users in the DAL algorithm first utilize the jamming signal as the coordination signal, and distributedly update their channel access strategies according to Eq. ??. While users in the coordinated learning algorithm adopt random integers as coordination signals, and avoid collision via distributed learning. Moreover, users in the random selection algorithm randomly select a channel from their channel set to access.
Figures 8 and 9 show the performance comparison under the sweep wideband jamming and the random wideband jamming. As shown in Fig. 8, it is concluded that the proposed HCACA outperforms other comparative algorithms and achieves the highest throughput with the fastest convergence speed under the sweep wideband jamming. Note that the proposed DAL algorithm also performs well compared with the coordinated learning algorithm and the random selection algorithm, which shows the advantage of jamming signal utilization. Besides, due to that the coordinated algorithm adopts random integers as coordination signals, it can not adapt to the dynamic wideband jamming well. Thus, the throughput with coordination learning fluctuates and does not converge to a stable state. Similarly, Fig. 9 depicts that the proposed HCACA algorithm has good adaptive capacity under random jamming. In detail, the proposed HCACA algorithm adapts to the stochastic behavior of the random jamming and converges gradually. The proposed DAL algorithm also performs well thanks to the observation of jamming signals. While the coordination learning algorithm and the random selection algorithm are hard to accommodate the random jamming and obtain relatively low network throughput than proposed algorithms.
To analyze the advantage in convergence speed of the proposed HCACA algorithm and DAL algorithm. Here we count the convergence time of 100 times Monte Carlo experiments, and then take the average value when reaching the equilibrium strategy combination of the network (for the HCACA algorithm, different SEs are achieved, while for the DAL algorithm, NEs are achieved for different jamming attacks). As shown in Fig. 10, the proposed hierarchical coordinated antijamming approach with Stackelberg game formulation converges more faster than the distributed antijamming learning approach under three different wideband jamming attacks. The reason is that the proposed HCACA approach avoids jamming attacks via the cluster heads, and it reduces the strategy space greatly. In addition, it is also illustrated that with the increase of the dynamic and random characteristic of the jamming attacks, the average iterations increases accordingly, however, the proposed hierarchical antijamming approach still outperforms the distributed antijamming approach in terms of convergence speed.
Conclusion
This paper investigated the multiuser coordinated antijamming problem in clustering communication networks. A hierarchical coordinated antijamming approach was proposed, and a multileader multifollower Stackelberg game was introduced to model the antijamming problem. In detail, cluster heads acted as leaders, and selected available frequency bands to avoid jamming attacks. While users in each cluster acted as followers, and selected corresponding channels distributedly and independently. Moreover, it was proved that there exist multiple Stackelberg equilibriums (SEs) in the proposed game. To obtain SEs, a hierarchical coordinated antijamming channel access (HCACA) algorithm was designed. Simulation results illustrated that the proposed approach is effective to cope with the dynamic wideband jamming attacks. Furthermore, it was also depicted that the proposed approach outperforms the distributed antijamming comparative approach in terms of convergence speed.
Availability of data and materials
Not applicable.
Abbreviations
 ‐:

A summary of main notations is given in Table 2.
References
 1
X Lu, L Xiao, C Dai, H Dai, UAVaided cellular communications with deep reinforcement learning against jamming. IEEE Wirel. Commun.27:, 48–53 (2020).
 2
L Xiao, X Lu, T Xu, et al., Reinforcement learning based mobile offloading for edge computing against jamming and interference. IEEE Trans. Commun.68:, 6114–6126 (2020).
 3
X Wang, J Wang, et al., Dynamic spectrum antijamming communications: Challenges and opportunities. IEEE Commun. Mag.58:, 79–85 (2020).
 4
Q Wu, T Ruan, et al., A unified cognitive learning framework for adapting to dynamic environment and tasks. arXiv preprint arXiv:2106.00501 (2021). https://arxiv.org/abs/2106.00501.
 5
Z Han, D Niyato, W Saad, T Başar, A Hjørungnes. Game theory in wireless and communication networks: theory, models, and applications (Cambridge university press, 2012).
 6
Y Xu, A Anpalagan, Q Wu, et al., Decisiontheoretic distributed channel selection for opportunistic spectrum access: Strategies, challenges and solutions. IEEE Commun. Surv. Tutor.15:, 1689–1713 (2013).
 7
D T. Hoang, X Lu, D Niyato, P Wang, D I. Kim, Z Han, Applications of repeated games in wireless networks: A survey. IEEE Commun. Surv. Tutor.17:, 2102–2135 (2015).
 8
J Zheng, H Zhang, Y Cai, et al., Gametheoretic multichannel multiaccess in energy harvesting wireless sensor networks. IEEE Sensors J.16:, 4587–4594 (2016).
 9
Y Xu, J Wang, Q Wu, J Zheng, et al., Dynamic spectrum access in timevarying environment: Distributed learning beyond expectation optimization. IEEE Trans. Commun.65:, 5305–5318 (2017).
 10
C Yang, J Li, P Semasinghe, E Hossain, S M. Perlaza, Distributed interference and energyaware power control for ultradense D2D networks: A mean field game. IEEE Trans. Wirel. Commun.16:, 1205–1217 (2017).
 11
Y Zhang, C Jiang, J Wang, Z Han, et al., Coalition formation game based access point selection for lteu and wifi coexistence. IEEE Trans. Ind. Informat.14:, 2653–2665 (2018).
 12
Y Hu, A Sanjab, W Saad, Dynamic psychological game theory for secure Internet of battlefield things (IoBT) systems. IEEE Internet Things J.6:, 3712–3726 (2019).
 13
J Moura, D Hutchison, Game theory for multiaccess edge computing: survey, use cases, and future trends. IEEE Commun. Surv. Tutor.21:, 260–288 (2019).
 14
J Cao, T Peng, Z Qi, et al., Interference management in ultradense networks: A usercentric coalition formation game approach. IEEE Trans. Veh. Technol.67:, 5188–5202 (2018).
 15
D Yang, J Zhang, X Fang, et al., Optimal transmission power control in the presence of a smart jammer. IEEE GLOBECOM, 5506–5511 (2012).
 16
D Yang, G Xue, J Zhang, et al., Coping with a smart jammer in wireless networks: A Stackelberg game approach. IEEE Trans. Wirel. Commun.12:, 4038–4047 (2013).
 17
L Xiao, T Chen, J Liu, H Dai, Antijamming transmission Stackelberg game with observation errors. IEEE Commun. Lett.19:, 949–952 (2015).
 18
L Jia, F Yao, Y Sun, et al., Bayesian Stackelberg game for antijamming transmission with incomplete information. IEEE Commun. Lett.20:, 1991–1994 (2016).
 19
L Jia, F Yao, Y Sun, et al., A hierarchical learning solution for antijamming Stackelberg game with discrete power strategies. IEEE Wirel. Commun. Lett.6:, 818–821 (2017).
 20
Y Xu, G Ren, Jin Chen, et al., A oneleader multifollower BayesianStackelberg game for antijamming transmission in UAV communication networks. IEEE Access. 6:, 21697–21709 (2018).
 21
N Qi, W Wang, M Xiao, et al., A learningbased spectrum access Stackelberg game: Friendly jammerassisted communication confrontation. IEEE Trans. Veh. Technol.70:, 700–713 (2021).
 22
Y Li, L Xiao, J Liu, et al., Power control Stackelberg game in cooperative antijamming communications. GAMENETS, 1–6 (2014).
 23
Y Yuan, T Yang, H Feng, et al., An iterative matchingStackelberg game model for channelpower allocation in D2D underlaid cellular networks. IEEE Trans. Wirel. Commun.17:, 7456–7471 (2018).
 24
D. N Duong, S. A Madhukumar, D Niyato, Stackelberg bayesian game for power allocation in twotier networks. IEEE Trans. Veh. Technol., 2341–2354 (2016).
 25
Z Su, N Qi, Z Du, et al., Guarding legal communication with smart jammer: Stackelberg game based power control analysis. China Commun.18:, 126–136 (2021).
 26
K. I Ahmed, O. A Fapojuwo, Stackelberg equilibria of an antijamming game in cooperative cognitive radio networks. IEEE Trans. Cogn. Commun. Netw.4:, 121–134 (2018).
 27
C Han, L Huo, X Tong, et al., Spatial antijamming scheme for internet of satellites based on the deep reinforcement learning and Stackelberg game. IEEE Trans. Veh. Technol.69:, 5331–5342 (2020).
 28
L Cigler, B Faltings, in 10th International Conference on Autonomous Agents and Multiagent Systems, 2. Reaching correlated equilibria through multiagent learning (Taipei, 2011), pp. 509–516.
 29
Y Xu, Y Xu, X Dong, et al., Convert harm into benefit: A coordinationlearning based dynamic spectrum antijamming approach. IEEE Trans. Veh. Technol.69:, 8–13032 (2020).
 30
P Neamatollahi, S Abrishami, M Naghibzadeh, et al., Hierarchical clusteringtask scheduling policy in clusterbased wireless sensor networks. IEEE Trans. Ind. Informat.14:, 1876–1886 (2018).
 31
C Singhal, S De, in Advances in Wireless Technologies and Telecommunication (AWTT). Resource allocation in nextgeneration broadband wireless access networks (IGI GlobalHershey, 2017).
 32
Y Xu, J Wang, Q Wu, A Anpalagan, et al., Opportunistic spectrum access in unknown dynamic environment: A gametheoretic stochastic learning solution. IEEE Trans. Wirel. Commun.11:, 1380–1391 (2012).
 33
Y Sun, J Wang, F Sun, et al., EnergyAware joint user scheduling and power control for twotier femtocell networks: A hierarchical game approach. IEEE Syst. J.12:, 2533–2544 (2018).
 34
K Yao, J Wang, Y Xu, et al., Selforganizing slot access for neighboring cooperation in UAV swarms. IEEE Trans. Wirel. Commun.19:, 2800–2812 (2020).
 35
J Wang, G Ding, Q Wu, et al., Spatialtemporal spectrum hole discovery: A hybrid spectrum sensing and geolocation database framework. Chinese Sci. Bull.59:, 1896–1902 (2014).
 36
L Wang, K Wu, M Hamdi, et al., Attachmentlearning for multichannel allocation in distributed ofdmabased networks. IEEE Trans. Wirel. Commun.12:, 1712–1721 (2013).
 37
D Shen, O. V Li, Stabilized multichannel aloha for wireless OFDM networks. IEEE GLOBECOM. 1:, 701–705 (2002).
Acknowledgements
The authors would like to thank Dr. Ximing Wang for discussions on the system model.
Funding
This work was supported by the National Natural Science Foundation of China under 62071488 and 61771488.
Author information
Affiliations
Contributions
Authors’ contributions
Yifan Xu, Jin Chen conceived the system model. Yifan Xu, Zhibin Feng and Kailing Yao completed the gametheoretic analysis of this paper. Yifan Xu, Guoxin Li, Fei Song and Gui Fang contributed to the algorithm design. Yifan Xu also performed the simulations with Zhibin Feng. Besides, Yifan Xu, Jin Chen and Zhibin Feng wrote the manuscript. The authors read and approved the final manuscript.
Authors’ information
Not applicable.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Xu, Y., Chen, J., Feng, Z. et al. Hierarchical coordinated antijamming channel access in clustering networks: a multileader multifollower Stackelberg game approach. EURASIP J. Adv. Signal Process. 2021, 42 (2021). https://doi.org/10.1186/s13634021007640
Received:
Accepted:
Published:
Keywords
 Multiuser coordinated antijamming
 Stackelberg game
 Channel access
 Dynamic wideband jamming