Multi-sensor tracking with partly overlapping FoV using detection field of probability modeling and the GLMB filter

In this paper, we consider multi-sensor with partly overlapping field of view (FoV) in the labeled random finite set (L-RFS) framework. This is different from most existing multi-sensor tracking algorithms, where the sensors are assumed to have the same FoV. We describe the partly overlapping FoV by modeling probability field of detection for individual sensors in whole observation area and can be seen as the same range of FoV. We consider all these using generalized labeled multi-Bernoulli filter in labeled RFS framework. Besides, we also propose a measurement-driven target birth model. Finally, the effectiveness of the proposed algorithm is verified by experiments.


Introduction
The aim of target tracking of far distance using radar sensor is to infer the states of targets from a set of measurements, which are received by the sensor. It is widely used in the military field or civilian areas, where the single sensor has been studied a lots in current literatures, whether in association-based approaches such as joint probability data association [1,2] and multiple hypothesis tracker [3], or in random finite sets-based approach. The RFS approach provides an elegant Bayesian formulation for multi-target tracking problem. The typical RFS multi-target tracking (RFS-MTT) filters include the probability hypothesis density (PHD) filter [4], Gaussian mixture (GM-PHD) filter [5], the cardinalized PHD (CPHD) filter [6,7], and the (cardinality balanced) multi-Bernoulli (CBMeMBer) filter [8,9]. In multitarget tracking, a recent break-through is a closed form solution to the Bayes multitarget filter which can also output target tracks [10,11]. The most important work is the generalized labeled multi-Bernoulli (GLMB) filter. It is with the first multitarget conjugate prior [10,11] the multitarget conjugate prior with respect to the standard multitarget likelihood function, or Vo-Vo prior. Additionally, this multi-target prior is also closed under the Chapman-Kolmogorov equation for the standard multi-target transition density [10,11].
• Dividing method: References [19][20][21] used the known radar FoV to divide the multi-target density and then fused the divided terms to represent the multi-target density of overlapping FoV. However, the target information of non-overlapping FoV was ignored in [19]. • Clustering method: Using a clustering method, i.e., finding estimated matches from different sensors for the same target according to different measures, and performing fusion only on the selected matches. Reference [22] calculated the Mahalanobis distance between PHD Gaussian components of different sensors, and fused the Gaussian components with shorter Mahalanobis distance through the GA method, realizing the distributed GA fusion of PHD filters in partly overlapping FoV. Reference [23] achieved the fusion of CPHD filters in partly overlapping fields of view by using a method similar to that in [22]. Reference [24] matched the Bernoulli components between multi-Bernoulli filters through Mahalanobis distance and realized distributed AA fusion of multi-Bernoulli filters in limited FoV. For details about GA and AA fusion, see [25,26]. Reference [27] used Optimal Subpattern Assignment (OSPA) distance as a measure to implement a method similar to that in [22]. Reference [28] used the method based on the highest a posteriori density distance measure to realize the PHD filter fusion of the limited FoV. • Other methods: Reference [29] proposed a track-to-track fusion method in which the information contents of posteriors are combined. It is proved that using Cauchy Schwarz divergence can better realize LMB filter fusion with limited FoV. A distributed network of sensors with limited field of views is proposed under labeled RFS frameworks [27]. Reference [30] proposed a dual-term node- The target birth intensity (TBI) is widely assumed to have a constant amplitude that must be determined in advance, implying that the intensity of emerging targets will be the same in all FoV. However, this is not always desirable. In actual tracking scenarios, the size of TBI is usually unknown and changes over time [31][32][33]. In [34], the target birth probability is adaptively performed in the pre-processing step, combined with the current measurements to correct the preset of the target birth probability, the proposed filter can really adapt to the target birth situation and achieve better tracking accuracy. In [35], a magnitude-adaptive TBI approach has been developed for RFS-based Bayesian filters , which adapts the TBI magnitude online with respect to the newest observations in exchange for very little additional computation. Reference [36] modelled the timevarying spatial distribution of target births as a dynamic density map with adaptive grid points, which is capable of estimating the unknown, dynamically changing birth process.
In this paper, we model the partly overlapping detection field of probability and provide multi-sensor measurement driven birth model. Based on these, we adopt the labeled random finite set to describe the target states and estimate their states by using the generalized labeled multi-Bernoulli (GLMB) filter. The preliminary results of this paper are published in a conference version [37], and this article is a complete one.
The structure of this paper is organized as follows. The Sect. 2 shows some theoretical basis for the labeled random finite set, including the GLMB filter. The Sect. 3 proposes the models of the detection of probability for the sensors. Section 3 considers the multisensor GLMB filter with partly overlapping field. The simulation is given in Sect. 4, and Sect. 5 concludes this paper.

The state and measurement RFSs
In category of RFS, the states and measurements of multi-target can be seen as a set, in which each element belongs to a random vector and the cardinality of the set is finite and random. More specifically, the multi-target state RFS can be modeled by [8]: where S k|k−1 (x) , B k|k−1 (x) , and Ŵ k are the target surviving, spawned, and birth RFSs. Let these RFSs are mutually independent. Then, the probability density of the multi-target state RFS can be gotten by [8]: is the probability densities of spontaneous birth RFS Ŵ k . The equation describes all actions of target motion, birth and death. It should be noted that the spawning case is ignored in the paper.
The multi-target is observed by various sensors and multiple measurements may be received. Assume at time k, a target may be detected and produces a measurement z k with probability P D (x k ) or missed and gives empty set {∅} with probability 1 − P D (x k ) .
(1) In a word, the received measurements z k , with probability of detection P D (x k ) and likelihood function g(z k |x k ) , involve all the information of the target state RFS X k . The target measurement RFS plus clutter or false alarms RFS K k are expressed in the following: The K k is Poisson RFS with intensity v K (·) . Its probability distribution is derived by: Under the assumption of mutual independence between � k (X k ) and K k , the probability density ϕ k (Z k |X k ) is given by [8]: where π � k ,k (W |X k ) is the probability density of target-generated measurement RFS and π(Z k − W ) for clutter.

The labeled random finite set
Based on traditional RFS, a variable of label is added to the above state element x [10], generated a labeled RFS. It is defined on a product space X × L , where L is a label space.
In practical applications, a labeled random vector is represented by an unique vector in a discrete countable space. For example, the label is defined on space (k, i) T , where k is time stamp and i is the count of this stamp. Some densities of the labeled RFS version are shown in the following.

The labeled Poisson RFS
A Poisson distribution is defined on the positive integer space N {0, 1, · · · , n, · · · } and given by where is Poisson intensity. For a RFS X = {x 1 , · · · , x n } , the elements

Its Poisson distribution is given by
where is Poisson intensity. For a RFS X = {x 1 , · · · , x n } , the elements x i ∈ X are i.i.d with density v(x)/N , where N = v(x)dx . Its Poisson distribution is given by A labeled Poisson RFS X is expressed by X = {(x 1 , ℓ 1 ), · · · , (x n , ℓ n )} with the following density function: ., e − n /n!, = �v, 1�.

Labeled multi-Bernoulli RFS
For a fixed number of independent Bernoulli RFS X (i) with parameters of existing probability r (i) and density p (i) (·) , their union belongs to a multi-Bernoulli RFS and with the following probability density For a labeled multi-Bernoulli RFS X with non-empty parameter set {r (ζ ) , p (ζ ) , ζ ∈ �} , its probability distribution is given by [38] where function α(·) is a 1-1 map shown as: α : � → L . It should be noted that the distribution is not a multi-Bernoulli distribution. For simplicity, an alternative form of the labeled multi-Bernoulli distribution is given by [10]: where �(X) is an indictor function to guarantee the distinct of individual labels. �(X; ·) is defined by

Generalized labeled multi-Bernoulli density
The generalized labeled multi-Bernoulli density is defined by [10] where c is a discrete index set, ω (c) is a weighted coefficient and dependent on the state label L (X) . p (c) p (c) (x, ℓ) is a distribution function for track with label ℓ . Thus, the exponential function [p (c) ] X is the factorial of all tracks. Moreover, it has been shown that the labeled Poisson RFS and labeled multi-Bernoulli are two special cases of GLMB RFSs [10].

The measurement models
Let F s,k be the observation area for the sensor s. That is, only the targets in the field F s,k may be observed. Assume that the state of sensor s is y s,k ∈ R n y , where R n y is the n y -dimensional Euclidean space. The measurement model can be modeled by: The measurement model for sensor s is given by: where P (·) is a projection from certain state space R n x to position space R 2 , i.e., P : R n x → R 2 for target state.

The models of multi-sensor probability of detection
In general, radar is a feasible sensor for tracking targets over long distances. Its field of view can be regarded as a fan-shaped area in polar coordinates, as shown in Fig. 1. It is defined by: where A s,k is the FoV of the sensor s, R + is the space of positive real numbers. Accordingly, the corresponding Euclidean space position can be gotten by a projection q : A s,k → R 2 . Thus, the FoV area in Euclidean space is defined by F s,k = {q(a s,k )| for all a s,k ∈ A s,k } . The model of probability of detection for sensor s is given by where P D is a constant, || · || 2 is 2-norm. This model implies that a target may be detected with a certain probability if it locates in area F s,k of sensor s, and with probability of detection zero otherwise.

Target birth models of multi-sensor measurement driven
In most existing algorithms, the FoV of a sensor is assumed to cover all the track region and there is no any dead zone. Hence, the target birth model is assumed to be prior and known. But for sensors with partly overlapping FoV, a sensor scans only part of the surveillance region. Hence, it is impossible to detect a target when the target is born outside the FoV. In order to build the target birth model, we refer the measurement-driven birth model given in [38][39][40] and extend it to the multi-sensor case.
A multi-Bernoulli RFS X is a union of a fixed number of independent Bernoulli RFSs X (i) with existence probability r (i) B ∈ (0, 1) and probability density p (i) [41]. So, the probability density of a multi-Bernoulli RFS can be abbreviated as: Assume that the form of the birth model with generalized labeled multi-Bernoulli (GLMB) is as follows: where The existence probability can be initialized by the following equation: where where �(I, S) is the space of multi-sensor association map. ϑ is a multi-sensor association map (defined next subsection).

Multi-sensor association map
In the GLMB filter, the association map plays a crucial role in the RFS multi-objective likelihood function.
Definition 1 [10]: The association map is a mapping: θ : L → {0, 1, · · · , |Z|} . If θ(i) = θ i ′ > 0 , it means i = i ′ , the set represents the associated map space, and its subset I can be represented by �(I) . The association diagram describes the correspondence between the trajectory and the measurement, the trajectory l produces a measure of z θ (l) ∈ Z , and the undetected trajectory is denoted by 0.
Definition 2 [42]: The multi-sensor association map is a mapping: ϑ : is the rth element in vector θ (i) . The set �(I, S) represents the space of multi-sensor associated map, and its subset I can be represented by �(I, S).
Under the independence of all sensors, the multi-sensor association map ϑ can be expressed as where ϑ T s can be seen as components of ϑ . Assume that multi-sensor measurements can be represented by Z k Z 1,k ∪ · · · ∪ Z S,k , multi-sensor posterior probability: where the integral is defined as: Assume that the multi-sensor are independent, then multi-sensor likelihood function is given by: where

GLMB estimation algorithm for multi-sensor with partly overlapping FoV
The basic principle of the multi-sensor tracking algorithm is to distribute the multi-sensor reasonably. At different times, the detection results of the individual sensors may be different. At each moment, we need to judge the target, to determine the target in which the sensor within the scope of observation. The specific multi-sensor GLMB estimation algorithm is as follows: if ϑ s (l) = 0

Update step
If the multi-sensor multi-target a priori density function is the form of generalized labeled multi-Bernoulli, then the multi-sensor multi-target posterior probability density is also the form of generalized labeled multi-Bernoulli: where ϑ represents the current multi-sensor association map. The association parameters are defined as follows: where �Z k (x, l; ϑ) is the likelihood function in the case of multi-sensor; assume that the sensors are independent of each other, it can be simplified as: ψ Z 1,k · · · ψ Z S,k ( : ; ϑ s ) , suppose that the likelihood function is a Gaussian distribution,P s,d (x k , l) = P s,d , g(z s,k |x k , l) = N z s,k ; H s,k , R s,k . H s,d and R s,d are the observation matrix and measurement noise covariance for sensor s. Assume that probability density p (ξ ) (·, ℓ) of target ℓ follows a Gaussian mixture distribution: Then we can get the following multi-sensor cost matrix: where z s k,j represents the jth measurement of the sth sensor, the update history (ξ , ϑ) and η (ξ ,ϑ) Z,k of the multi-sensor association map are as follows: (27) π (X k |Z k ) = �(X k ) (I,ξ )∈F(L)×� ϑ∈�(I,S) Although the multi-sensor labeled multi-Bernoulli filter has the same representation as the single-sensor, there are many differences between the multi-sensor and the singlesensor for the specific parameters: The above formula gives the recursive process of the measurement Z 1 to Z m of the multisensor association map.

Predict step
The predict step is the same as the case of single sensor. That is, if the multi-sensor multi-target a priori density function is the form of the generalized labeled multi-Bernoulli, then the multi-objective prediction is: where In this step, the multi-sensor has the same form as the single-sensor.  s (x, l) is the density of the survival target obtained from the prior density p (ξ ) (·, l). f (x|·, l) represents the probability density of the survival target. Algorithm 1 provides pseudocode for a multi-sensor multi-target tracking algorithm with partly overlapping fields of view.

Experiments and discussions
To verify the effectiveness and robustness of the proposed algorithm, we set up two scenarios. Experiment 1: high detection probability and small number of targets; Experiment 2: low detection probability and a larger number of targets. As the number of targets increases, more targets are born or die out outside the radar field of view, which poses a challenge to the trajectory tracking capability of the proposed algorithm. The reduction in detection probability also has a direct impact on tracking accuracy. Only when a target enters into the surveillance region of a sensor. It can be observed. The fields of the four sensors are shown in Fig. 2. From the figure, the fan-shaped areas are shown by two dashed lines and three fixed sensors with partly overlapping fields. Only a target entering into a sensor's field can be observed.

Experiment 1: high detection rate scenario and tracking a small number of targets
The target tracks are given in Fig. 3. It can be seen that the initial flying heights in y coordinate are 80 km, 30 km, 60 km and 10 km, respectively. The target 1 enters the observation range of the first sensor 1 around k = 20 . After moving about 250 km distance, the target 1 moves out of the field of sensor 1 and the measurement cannot be derived simultaneously. About 400 km on the horizontal axis, the first target enters into the observation range of sensor 2 and is observed. Also after around 600 km, target 1 moves out of the field of sensor 2. At about 700 km on the horizontal axis, it enters the observation range of sensor 3 and is observed until it dies within the field of sensor 3. Targets 2 and 3 have the same fields. It can be seen from the figure the proposed algorithm can detect and track the three targets correctly in each sensor field. Figure 4 shows blind region or has been died. Therefore, the estimated number of targets and the true value are not coincident in this time interval. Only in the observation region, the target can be detected. Outside the field, nothing can be found. Therefore, it is meaningful to get the true number of detected targets in the observation region, which is plotted in the bottom sub-figure of Fig. 5. It can be seen from the sub-figure that the number of targets can be estimated efficiently.
Here we use the optimal subpattern assignment (OSPA) metric to evaluate the tracking performance [43]: The parameters c = 50, p = 1 and the OSPA error is shown in Fig. 6. It shows that the OSPA error, OSPA location error and cardinality error, respectively. It seems there is a large error. In fact, if considering the detected region (outside blind area), the corresponding OSPA error given in Fig. 7 is much better. This shows that in the observation area of all sensors, the targets are detected and tracked well and the number of targets is accurately estimated.

Experiment 2: low detection rate scenario and tracking more targets
In this scenario, we will verify that the proposed algorithm can correctly track a large number of targets under more complex low detection probability conditions. The initial position of the sensor and other parameters remains unchanged; only the detection probability of each sensor is modified to 0.80. At the same time, we increase the number of targets to 10, whose motion model is still the CV model. The initial state and survival time of these targets are given by: The tracks for 10 targets are given in Fig. 8. As can be seen from the figure, the initial flight altitude of the target is 90 km, 70 km, 40 km and 10 km. Throughout the 200 s of tracking, the position estimate of target was fairly accurate. It shows that even in the case of low detection probability, the algorithm proposed in this paper can still track a large number of targets correctly. However, in Figs. 8 and 9, we can find that when the target enters the field of view of the sensor, especially when multiple targets overlap, there will be a comparatively large error in the position estimation. At low detection probability, when multi-target are in close proximity, the measurements obtained by the sensors are incomplete and the relative positions of these measurements are very close together. This will pose a great challenge to the filter update process. And this is often seen in sensor tracking with limited field of view. Similar to Scenario 1, the estimated number of 10 targets is given in Fig. 10. Targets can only be correctly estimated when they appear in the sensor field of view. For example, between 1 and 40 s, 3 targets actually survive. But it was not until about the 10th second that two targets entered the field of view of Sensor 1. A third target enters the field of view of Sensor 1 around the 30th second. By this time, the number of targets detected by the sensor is 3.
The OSPA error of target location estimation at low detection probability is given in Fig. 11. The parameters c = 50, p = 1 . When the blind area is considered, the OSPA error of the position estimation is larger. But when the blind area is not considered, as shown in Fig. 12, the OSPA error, OSPA position error and OSPA cardinality error are quite small. This illustrates that in scenarios with low detection probability, the target can be well tracked when it is within the field of view of the sensor and can be satisfied with practical tracking tasks.

Conclusions
In this paper, we consider the target tracking problem of multi-sensor partly overlapping FoV. We establish a model to describe partly overlapping FoV using probability field of detection (PFoD). Then, a multi-sensor multi-target tracking algorithm based on generalized labeled multi-Bernoulli (GLMB) filter is proposed. Finally, the proposed algorithm is verified by using three sensors with partly overlapping fields. Experiments show that the effective of the algorithm in detecting and tracking multi-target.