In this section, the system Model, improved APF algorithm, CS Scheduling and UAV decision are introducted respectively.

### 2.1 System model

The mission of dynamic target collaborative search with UAV swarm can be described as: there is a task area \(D\in {{L}^{2}}\) which has \({{N}_{\mathrm{T}}}\) dynamic moving targets and \({{N}_{p}}\) unknown threats. According to the prior information, the target information including position, speed and direction is partially known. There are also a swarm of UAVs connected through a cloud server to accomplish the mission. The UAVs in the swarm are isomorphic, which namely each UAV has the same functional and performance constraints.

As shown in Fig. 1, the mission area is discretized and grided and the length and width of each grid are \({{L}_{x}}\) , \({{L}_{y}}\) respectively. The five-pointed shape represents the target and the filled circle represents the threat. The detect performance of UAV is modeled as round and the detect radius is *R*. The targets and threats that appear in the detection range of the UAV can be detected by the UAV. Assume that the maximum turning angle of the UAV is \({{\varphi }_{\mathrm{max}}}\), hence the possible positions of the UAV at the next moment are marked in dark.

The purpose of collaborative search of UAV swarm is to find as many targets as possible under constrained conditions. The target search benefit \({{J}_{\mathrm{T}}}\) can be defined as:

$$\begin{aligned} {{J}_{\mathrm{T}}}(k)=\sum \limits _{i=1}^{{{N}_{\mathrm{T}}}}{{{a}_{ik}}{{v}_{i}}} \end{aligned}$$

(1)

where \({{N}_{\mathrm{T}}}\) is the total number of targets. \(a_{ik}\) indicates whether the target is detected. \({{v}_{i}}\) indicates the value of the target detected by the UAV. Define area search benefit \(J_E\) as:

$$\begin{aligned} {{J}_{E}}(k)=\sum \limits _{m=1}^{{{N}_{x}}}{\sum \limits _{n=1}^{{ {N}_{y}}}{gri{{d}_{(m,n)}}(k)/({{N}_{x}}\times {{N}_{y}})}} \end{aligned}$$

(2)

where the value of \(gri{{d}_{(m,n)}}\) can be 0 and 1. If the \(gri{{d}_{(m,n)}}\) has been searched, \(gri{{d}_{(m,n)}}(k)\) is 1, otherwise it is 0. The area search benefit is expressed as the search coverage of the mission area.

The constraints are:

$$\begin{aligned} \left\{ \begin{aligned}&{{C}_{\mathrm{c}}}:{{d}_{\min }}-{{d}_{ij}}(k)\le 0,(i,j=1,2,\ldots ,{{N}_{V}},i\ne j) \\&{{\text {C}}_{\mathrm{b}}}:0\le x,y\le {{N}_{x}},{{N}_{y}} \\&{{C}_{\mathrm{m}}}:\Delta {{\varphi }_{i}}(k)-{{\varphi }_{\max }}\le 0,(i=1,2,\ldots ,{{N}_{V}}) \\&{{C}_{\mathrm{r}}}:{{R}_{l}}-{{d}_{il}}(k)\le 0,(i=1,2,\ldots ,{{N}_{V}};l=1,2,\ldots ,{{N}_{p}}) \\ \end{aligned} \right. \end{aligned}$$

(3)

where \({{C}_{\mathrm{m}}}\) is the maximum turning radius constraint of the UAV, which is determined by the maneuverability. \({{\varphi }_{\max }}\) is the turning radius of the UAV at the moment k. \({{C}_{\mathrm{c}}}\) is the minimum safe distance constraint between the UAVs. The UAV needs to maintain a reasonable safe distance to avoid collisions, and \({{d}_{\mathrm{min}}}\) is the minimum safe distance between the UAVs, \({{d}_{ij}}(k)\) is the distance between the *i*-th UAV and *j*-th UAV at the moment *k*. \({{\text {C}}_{\mathrm{b}}}\) is the boundary constraint, (*x*, *y*) is the coordinates of the UAV in the mission area, and the UAV cannot fly out of the mission area during the execution of the mission. \({{C}_{\mathrm{r}}}\) is obstacle avoidance constraints and \({{\text {R}}_{l}}\) is the threat radius, \({{d}_{il}}(k)\) is the distance between the *i*-th UAV and the *l*-th threat.

For multi-UAV mission planning, the optimization model is:

$$\begin{aligned} & U_{i}^{*} (k) = \arg \mathop {\max }\limits_{{U_{i} (k)}} {\mkern 1mu} (\omega _{1} J_{{T_{i} }} (X_{i} ,\tilde{X}_{i} {\mkern 1mu} ) + \omega _{2} J_{{E_{i} }} (X_{i} ,\tilde{X}_{i} {\mkern 1mu} )) \\ & \tilde{X}_{i} {\mkern 1mu} = \{ X_{j} |j \in N_{i}^{C} \} \\ & {\text{s.t.}}\quad C_{i} \le 0,i = 1,2, \ldots ,N_{V} \\ \end{aligned}$$

(4)

where \({{\omega }_{1}}\) and \({{\omega }_{2}}\) respectively represent the weight coefficient of the target search benefit and area search benefit. The value of the coefficient is determined according to the specific combat mission, when \({{\omega }_{2}}\) is 0, this is a pure target mission, such as subsequent continuous monitoring, attack, etc., when \({{\omega }_{1}}\) is 0, this is an scan search mission. \({{X}_{i}}\) and \(\overset{\sim }{{{{X}_{i}}}}\,\) respectively represent the status of the UAV and its communicable UAVs, \({{J}_{{\mathrm{T}_{i}}}}({{X}_{i}},\overset{\sim }{{{{X}_{i}}}}\,)\) and \({{J}_{{{E}_{i}}}}({{X}_{i}},\overset{\sim }{{{{X}_{i}}}}\,)\) respectively are the target search benefit and area search benefit are the above-mentioned various constraints.

### 2.2 Improved APF algorithm

The basic idea of the artificial potential field [28, 29] is to regard the movement of the UAV in the task space as a force movement in the virtual force field. The target point forms a attraction field, which generates gravity and attracts the target to fly towards it. The no-fly zone or threat forms a repulsive force field, which generates a repulsive force to push the target away from the threat, and the UAV moves under the combined force of gravity and repulsive force. This section will introduce the search algorithm based on the improved artificial potential field (IAPF), which mainly includes the target attraction field, the search repulsion field and other repulsive field.

#### 2.2.1 Target attraction field

Based on the APF method, the target will generate a gravitational field which can attract UAV. The basic gravitational field has a linear relationship with the distance between UAV’s current position and the target. In a multi-target mission, this gravitational field will cause the UAV to be oscillate between targets. For this reason, the target attraction field is redefined as follows:

$$\begin{aligned} {{F}_{\mathrm{att}}}(i,j)=\left\{ \begin{array}{ll} {{k}_{i,{\mathrm{att}}}}\cdot {{{{{\hat{\mathbf {v}}}}}}_{{\mathbf {j}}}}, &{}\quad {{d}_{i,j}}\le {{L}_{i,{\mathrm{uc}}}} \\ {{k}_{i,{\mathrm{att}}}}\cdot \frac{{{L}^{2}}_{i,{\mathrm{uc}}}}{{{d}^{2}}_{i,j}}\cdot {{{{{\hat{\mathbf {x}}} }}}_{\mathbf {i,j}}},&{}\quad {{L}_{i,{\mathrm{uc}}}}<{{d}_{i,j}}\le {{L}_{i,{\mathrm{max}}}} \\ \mathbf {0},&{}\quad {{d}_{i,j}}>{{L}_{i,{\mathrm{max}}}} \\ \end{array} \right. \end{aligned}$$

(5)

where \({{k}_{i,{\mathrm{att}}}}\) is the magnitude of the target attraction field, which is proportional to the value of the *i*-th target. \({{{{\hat{\mathbf {v}}}}}_{{\mathbf {j}}}}\) is the unit vector of the movement direction of the *j*-th UAV. \({{d}_{i,j}}\) is the distance between the *i*-th target and the *j*-th UAV. is the field range brought about by the uncertainty of the target, which can be set as UAV detect radius *R*. In this range, the direction of the target attraction field received by the UAV is determined by its own motion direction, and the field’s amplitude is fixed. When the UAV enters this range, it does not need to change the flight direction, and it can continue to search for the possible location of the target. \({{{{\hat{\mathbf {x}}}}}_{\mathbf {i,j}}}\) is the unit vector of the direction from the UAV to the target. \({{L}_{i,\max }}\) is the maximum range of the target attraction field, which can be set as:

$$\begin{aligned} {{L}_{i,{{\mathrm{max}}}}}={N}_{{\mathrm{max}}}/{N}_{{{\mathrm{T,known}}}}+2R \end{aligned}$$

(6)

where \({{N}_{\mathrm{max}}}\) is the maximum number of iterations. \({N}_{{\mathrm{T,known}}}\) is the number of known targets. Setting the maximum range can not only reduce the computational complexity effectively, but also help the UAV focusing on the possible unknown targets. The action of the target attraction field is shown in Fig. 2. It can be seen that when the possible location of the target is within the detection range of the UAV, the UAV can maintain the current flight direction and complete the search. Meanwhile, it also avoids that the UAV receives the opposite force when it just flies away from the target point, forcing it to turn around. When the distance exceeds the maximum range, the UAV will search for other targets at this time. In other cases, the UAV receives a gravitational force in the direction of the potential target, so that the gravitational force drives the UAV to fly toward the target. Compared with the algorithm using target probability map, the method in this paper does not need to store and update the probability map, which reduces the computational complexity while ensuring the search efficiency.

#### 2.2.2 Search repulsive field

In order to avoid repeated searches in a short time interval, the related work [24] is based on the ant colony algorithm, which reduces the pheromone in the searched path according to the rules to increase the probability of the UAVs flying to the undetected area. Each iteration needs to update the pheromone of the entire task area, which also has the problem of high computational complexity. Based on the artificial potential field algorithm, this paper designs a search repulsion field, so that the UAV will leave a series of search repulsion fields on its search path. Futher, in order to adapt to the uncertainty of the moving target, the search repulsion field decays with time. This method reduces the amount of calculation while ensuring search efficiency. Define the *k*-th search repulsion field center position left by *j*-th UAV as:

$$\begin{aligned} Lo{{c}_{\mathrm{srp}}}(j,k)=\frac{1}{{{N}_{\mathrm{srp}}}}\sum \limits _{i=0}^{{ {N}_{\mathrm{srp}}}-1}{Lo{{c}_{\mathrm{uav}}}(j,k\cdot }{{N}_{\mathrm{srp}}}+i) \end{aligned}$$

(7)

where \({{N}_{\mathrm{srp}}}\) is the time interval constant of the search repulsive filed, which can be set as \(\sqrt{\frac{{{N}_{x}}+{{N}_{y}}}{2}}\). \(Lo{{c}_{\mathrm{uav}}}(j,k)\) indicates the position of the *j*-th UAV at time k. Define search repulsive field between the *i*-th search repulsive field and *j*-th UAV as:

$$\begin{aligned} {{F}_{\mathrm{srp}}}(i,j,t)=\left\{ \begin{array}{ll} {{k}_{\mathrm{srp}}}{{e}^{-\beta t}}{{{{{\hat{\mathbf {v}}}}}}_{{\mathbf {j}}}}, &{}\quad {{d}_{i,j}}\le {{L}_{\mathrm{srp1}}} \\ {{k}_{\mathrm{srp}}}\frac{{{L}^{2}}_{\mathrm{srp1}}}{{{d}^{2}}_{i,j}}{{e}^{-\beta t}}{{{{{\hat{\mathbf {x}}}}}}_{\mathbf {i,j}}}, &{}\quad {{L}_{\mathrm{srp1}}}<{{d}_{i,j}}\le {{L}_{\mathrm{srp2}}} \\ \mathbf {0},&{}\quad {{d}_{i,j}}>{{L}_{\mathrm{srp2}}} \\ \end{array} \right. \end{aligned}$$

(8)

where \({{k}_{\mathrm{srp}}}\) is the search repulsion field constant. \({{L}_{\mathrm{srp1}}}\) and \({{L}_{\mathrm{srp2}}}\) is the uncertainty range and the maximum range of action respectively. \(\beta\) is the time decay factor, which represents the uncertainty of the environment. The larger \(\beta\) is, the more likely the UAV will perform a second search in the same area.

#### 2.2.3 Other repulsive field

According to reference [24], the threat repulsive field between *i*-th threat and *j*-th UAV is designed as:

$$\begin{aligned} {{F}_{\mathrm{trp}}}(i,j)=\left\{ \begin{array}{ll} {{k}_{\mathrm{trp}}}\left( \frac{1}{d_{i,j}^{2}}-\frac{1}{{{({{d}_{\max ,i}}-{{d}_{0}})}^{2}}}\right) {{{{{\hat{\mathbf {x}}}}}}_{\mathbf {i,j}}}, &{}\quad {{d}_{i,j}}\le {{d}_{\max ,i}} \\ \mathbf {0},&{}\quad {{d}_{i,j}}>{{d}_{\max ,i}} \\ \end{array} \right. \end{aligned}$$

(9)

where \({{k}_{\mathrm{trp}}}\) is the threat repulsive field constant, \({{d}_{\max ,i}}\) is the radius of action of *i*-th threat, *d*_{0} is the minimum safe distance. The repulsive field between the *i*-th and *j*-th UAVs is designed as:

$$\begin{aligned} {{F}_{{\rm urp}}}=\frac{b\cdot {{e}^{\frac{{{d}_{i,j}}}{c}}}}{c\cdot {{({{e}^{\frac{{{d}_{i,j}}}{c}}}-{{e}^{\frac{{{d}_{\min }}}{c}}})}^{2}}}\cdot {{{{\hat{\mathbf {x}}}}}_{\mathbf {i,j}}} \end{aligned}$$

(10)

where *b* and *c* are determined. The magnitude of the repulsion between the UAVs depends only on the distance.

### 2.3 CS scheduling

In the first stage of mission, the improved artificial potential field algorithm can help the UAV to quickly search for the targets with prior knowledge (such as the start location). Meanwhile, the target search benefit \({{J}_{\mathrm{T}}}\) increase faster. In the later stage, when most of targets with prior knowledge have been detected, the search efficiency of UAV swarm is reduced. In order to solve this problem, a layered scheduling algorithm determined by the cloud server is added.

#### 2.3.1 Partitioning

According to the idea of the reference [25], the task area is partitioning. Divide the entire mission area into \(N \times N\) sub-areas. The length and width of each sub-area are *L*/*N*.

#### 2.3.2 Search map

CS managers sub-regions by storing search graphs in order to make assignment decisions. Define the basic search map as:

$$\begin{aligned} s\_map(i,j)=l{{a}_{i,j}} \end{aligned}$$

(11)

where \(l{{a}_{i,j}}\)=1, if *grid*(*i*, *j*) has been covered, otherwise \(l{{a}_{i,j}}=0\).

Here, considering dynamic targets, some area may need to be searched twice. After each iteration, the searched area needs to be decayed by time. Meanwhile, in order to distinguish the searched area from the unsearched area, it is necessary to set infimum for the searched area. Moreover, there are no-fly zones in the area, and the search map of the detected no-fly zone is set as a constant. This value is recommended to be between infimum and 1, which has been adapted to the UAV’s search for the area. In addition, in order to solve the problem of repeated assignment of the same location, the future flight target point of the UAV is also regarded as the detected area. In summary, the improved search map is defined as Algorithm 1, where \(\tau\) is the attenuation coefficient and is used to characterize the dynamic environment. \(\gamma\) is the threat search map constant. \(\mathrm{UAV}_m\) is the *m*-th UAV in the CS. \({UAV}_m\_Detect\_Range\) is the \({UAV}_m\)’s detect range and \({UAV}_m\_Assign\_Detect\_Range\) is the detect range when \({UAV}_m\) arrives at the designated place. \(Threat_m\) is the *m*-th known threat of CS. \(Threat_m\_Range\) is the range of \(Threat_m\). CS obtains the global search coverage of the UAV swarm by storing and updating the search map. Therefore, CS can start from the overall situation and help the UAV swarm to jump out of the local optimum, thereby improving the overall coverage.

#### 2.3.3 Sub-area priority

According to the priority of each sub-area, CS selects the closer and more idle UAV for the sub-area need to be detected. Define the search value of *k*-th sub-area\(part\_valu{{e}_{k}}\) as the sum of the search map values of the sub-area:

$$\begin{aligned} part\_valu{{e}_{k}}=\sum {s\_map(i,j),\quad \text {grid(i,j) in par}{{\text {t}}_{\text {k}}}} \end{aligned}$$

(12)

The smaller the\(part\_valu{{e}_{k}}\), the more the sub-area needs to be detected. The idle state of each UAV is defined as the amplitude of its potential field force. The more force the UAV receives, the busier the UAV. Above all, define the *k*-th sub-area priority as:

$$\begin{aligned} priority\_valu{{e}_{k}}={{w}_{p0}}\cdot part\_valu{{e}_{k}}+\min ({{w}_{p1}}*{{d}_{i,k}}+{{w}_{p2}}*{{F}_{i}}) \end{aligned}$$

(13)

where \({{w}_{p0}}\), \({{w}_{p1}}\), \({{w}_{p2}}\) are the search map, distance and field coefficient respectively. \({{d}_{i,k}}\) is the distance of *i*-th UAV and *k*-th sub-area’s center. \({{F}_{i}}\) is represents the magnitude of the potential field force of the *i*-th UAV. CS traverses all sub-area to get the minimum value of the priority. It should be noted that the priority also implies the assigned UAV.

#### 2.3.4 CS control mechanisms

The CS control mechanism includes two aspects. One is that in the initial stage of the mission, in order to maximize the use of the prior information of the targets, the UAVs search targets by the IAPF algorithm. Meanwhile, the CS does not participate in the control of the UAVs and is in an inactive state. When most of the known targets are found, CS participates in the control of the UAVs and is active. We call it the CS activation mechanism and define \(CS\_ACT\) as the activated state of CS. When the UAV arrives at the center of the designated sub-area during the CS scheduling, it needs to search the sub-area autonomously to improve the search efficiency of the sub-area. Uninterrupted CS scheduling will cause the UAV to fly back and forth between the center points of the sub-region, while ignoring the search for other locations in the sub-region. Therefore, CS needs a cool-down mechanism for the scheduling of each UAV. In the cool-down time, the UAV’s state is unready and can not be controlled by CS.

#### 2.3.5 Flow chart of CS

Based on the above design, the flow of the CS Scheduling is shown in Fig. 3. In order to make full use of the target prior information, when the number of undetected targets with prior information is less than or equal to one, \(CS\_ACT\) = 1. Define the ready statue of UAVs as that the UAV has not been assigned and has not been assigned during the previous iterations. The purpose of setting the assigned cool-down period is to effectively search the sub-areas after the assigned UAV goes to the target sub-areas. When the number of UAV in the ready state is more than zero, CS get the minimum priority of sub-areas and set the assigned *k*-th UAV \(CS\_flag\) true.

### 2.4 Hybrid-layered APF

In summary, this paper proposes the hybrid-layered APF method, including distributed artificial potential field algorithm and centralized layered algorithm scheduled by CS. Under the hybrid control architecture, the UAV will not only conduct self-organized search, but also be scheduled by CS. Under the condition of CS, the UAV will fly as far as possible to the center of the *k*-th sub-area that needs to be detected command and meeting the constraints. Assuming the current grid is \(s_i\), the transition rule when \(CS\_FLAG\) is ture are designed as:

$$\begin{aligned} {{\text {s}}_{j}}=\underset{j\in \Omega }{{\arg \min }}\,({{d}_{jk}}) \end{aligned}$$

(14)

where \(s_j\) is the next grid. \(\Omega\) is the candidate grid set that meeting the constraints. \(d_{jk}\) is the distance between \(s_j\) and the center of *k*-th sub-area.

When the UAV reaches the center of the designated sub-area or finds that the center of the sub-area is within the no-fly zone, \(CS\_flag\) is set to 0. In this situation, the UAV will enter the cool-down state and will not be assigned by CS in the next \(N_{cool}\) iterations. Meanwhile, the UAV will make decisions based on the IAPF.

When \(CS\_flag\) is 0, the UAV get the total field of current grid and surrounding grid. When the field direction of the surrounding grid of the current grid is consistent, it means there may be targets or no-fly zones around. At this point, the UAV will make decisions based on the direction of the field. Otherwise, the UAV will keep the current flight direction. The UAV decision under IAPF is designed as:

$$\begin{aligned} {{\text {s}}_{j}}=\underset{j\in \Omega }{{\arg \min }}\, ({{\theta }_{\text {j}}}),std({{F}_{i}})<{{\varphi }_{\max }}/2 \end{aligned}$$

(15)

where \({\theta }_j\) is the angle between the path from the current grid to the candidate grid \(s_j\) and the direction of the current grid potential field. \(std(F_i)\) is the standard deviation of the field direction among the current grid and the surrounding grids.