Joint target tracking and identification in complex traffic scene

Abstract

In practical road traffic scene, targets usually face high ground clutter, high and variable motion, high nonlinearity, which lead to targets tracking or identification challenging. What’s more, tracking and identification are usually interdependent in reality, and thus it is promising to solve them jointly. In this paper, we propose a novel joint tracking and identification (JTI) scheme to handle such problems involving coupled tracking and identification, i.e., JTI problems. Specifically, we formulate the JTI problem in complex traffic scene using a hybrid system. Then, by exploiting the generalized Bayes risk for JTI, we derive analytical estimator and decider for the coupling of tracking and identification in complex road targets motion. Furthermore, an unscented Kalman filter-expected mode augmentation-based estimation strategy is creatively developed to improve both estimation and decision performance. In additions, a joint performance evaluation metric is presented to assess the performance the joint of the proposed JTI scheme. Finally, two simulation examples under different traffic scenarios demonstrate that the proposed JTI approach outperforms the traditional tracking-then-identification and identification-then-tracking methods in joint performance.

1 Introduction

With the rapid development of intelligent transportation, vehicles tracking and identification, as two fundamental tasks of traffic monitoring, have recently attracted great interests from both industry and academia [1,2,3,4,5]. Specifically, the vehicles tracking aims to estimate the target state (e.g., position, velocity, acceleration, etc.), while identification aims to identify which class the target belongs to [6, 7], such as car, bus, tanker, and ambulance. In practical intelligent transportation systems, however, tracking and identification are inherently coupled and affect each other. Furthermore, in practical complex traffic scene, tracking and identification become more challenging due to the complicated and changeable mobility, high nonlinearity, large ground clutter interference and so on.

To address the above challenges, various schemes are designed to improve the tracking and identification performance in complex traffic scene. So far, there are four kinds of methods to handle the problems involving both tracking and identification [8]: (a) separate tracking and identification [6, 9], where tracking and identification are handled independently without considering their couplings at all; (b) identification-then-tracking, in which identification is made first without considering tracking and then tracking is made based on this identification without considering the possible identification error; (c) tracking-then-identification [10, 11], in which tracking is done first, and then, identification is based on it; (d) density-based method, which is beyond the scope of this paper (it is for point inference).

However, the above existing methods cannot work well [8] because the internal relationship between tracking and identification is not fully explored. Specifically, tracking can provide state information for different target classes, while identification helps tracking by selecting appropriate identity-dependent kinematic models. Therefore, a joint tracking and identification (JTI) approach is promising to improve tracking and identification performance jointly. Essentially, JTI is a joint decision and estimation (JDE) problem [12] with dual goals—decision and estimation, which are coupled. For such problems, an integrated JDE paradigm is proposed [12] which can fully utilize the coupling between decision and estimation and finally achieve superior joint performance [13,14,15,16,17]. Within this JDE framework, a conditional JDE (CJDE) approach was proposed in [8] which has simple calculation and superior joint performance.

Although CJDE is superior for solving joint problems, it cannot be applied to JTI in complex traffic scene directly due to the particularities and complexities of the problem. Among many difficulties of JTI in complex traffic scene, this paper considers two typical and common types: complicated mobility and high nonlinearity. To capture the complicated motion and obtain satisfactory estimation performance, multiple-model (MM) approach is usually used for tracking [18], which contains fixed-structure MM (FSMM) and variable-structure MM (VSMM) [1, 19, 20]. In the former, a large number of models are needed to improve the estimation performance. However, usages of extensive models increase the computational burden considerably; furthermore, the performance will deteriorate since too many models may cause excessive “competition” from the “unnecessary” models. Therefore, we adopt the VSMM estimation method since it can utilize the online mode information and is superior to a FSMM algorithm in both performance and computation for complicated real-world problems [19, 21].

For nonlinear target tracking, there are generally density-based and point estimation-based methods. The frontier has high computational complexity [22, 23] by approximating the posterior distribution, while the latter is simpler and is thus adequate for practical applications [24, 25]. Within the nonlinear point estimation, there are extended Kalman filter (EKF) [26], unscented Kalman filter (UKF) [27] using the deterministic sampling to compute the moments, the quadrature KF (QKF) [28], the cubature KF [29], and so on. For identification, studies mainly focus on video-based methods, which have been proven to be efficient in road target information processing [30, 31]. However, under complex environmental and illumination conditions, identification using these methods is difficult and may even fail.

In view of the above, solving JTI in complex traffic scene within the JDE framework faces the following difficulties. First, appropriate models are required for problem formulation. The models are expected to describe practical target state evolution objectively and incorporate the target identity and the coupling between state and identity effectively and also can be mathematically tackled easily. Second, tracking and identification solution concerning their couplings and practical complexities is required. Specifically, it is desired that on the one hand, the complex mobility and high nonlinearity can be appropriately utilized to ensure the fitness of JTI solution to reality; on the other hand, the coupling between tracking and identification should be fully utilized.

Motivated by the above, this paper proposes a novel scheme for the JTI problem in complex traffic scene. First, we present a hybrid system containing a dynamic model and a measurement model, which can describe the complex mobility and also the coupling between tracking and identification. Based on this model, we focus on solving the JTI problem within the JDE framework.

We present a generalized Bayes risk for JTI, which unifies tracking and identification. Then, we derive a joint solution by minimizing this JTI risk. Specifically, for estimation, we propose a new expected-mode augmentation (EMA) [20] estimation strategy, named UKF-EMA strategy. Here, UKF is adopted due to its superior performance and also adaptability to the JDE framework; EMA is utilized due to its superiority in handling complex motion. For decision, a decider is provided by incorporating the effect of estimation on decision. Generally, this JTI solution with explicit form fully exploits the coupling between target state and identity and also utilizes the characteristics in complex traffic scene. Furthermore, a joint performance metric is provided, which considers both tracking and identification errors. Simulation results verify that the proposed JTI approach outperforms traditional two-step methods in joint performance.

More specifically, the contributions of this work are summarized as follows:

• We propose a hybrid system for practical JTI problems in complex traffic scene. In this system, both the complicated target state evolution and the coupling between tracking and identification are incorporated.

• We propose a novel and tractable JTI approach for JTI problems in complex traffic scene. A joint risk is first presented, which unifies estimation and decision errors. Then, we derive an analytical JTI solution containing an estimator and a decider with their couplings being accounted for. Specifically, a UKF-EMA estimation strategy is creatively proposed due to its nice properties. Finally, we present an efficient JTI algorithm.

• We examine the performance of the proposed JTI approach in practical complex traffic scene, where motion at a road corner and a crossroads is representatively considered. The results verify that the proposed JTI approach can utilize the coupling between tracking and identification and finally outperforms the traditional methods in joint performance.

This paper is organized as follows. Section 2 formulates the JTI problem in complex traffic scene. Section 3 proposes an applicable JTI approach by considering the characteristics of JTI problems and also the coupling between tracking and identification. Also presented is a joint performance metric. Section 4 presents simulation results and analyses. Section 5 concludes the paper.

2 Problem formulation

2.1 Problem description

Figure 1 illustrates two typical JTI problems in complex traffic scene. A target with multiple possible classes moves on the road, where different classes have different dynamics. Specifically, in Fig. 1a, a car and a bus are moving at the corner of the road with different mobility, i.e., different turn rates. In Fig. 1b, a car and a bus are moving at the crossroads with different motion modes. Here, we aim to jointly identify and track the target using multiple sensor data under complex traffic scenes (intersection, high and variable motions, large ground clutter interference, etc.).

The tracking and identification are highly coupled in this problem. Accurate tracking provides the target’s location and motion information, which promotes identification. Correct identification benefits learn more behaviors of targets, which helps tracking. Therefore, this is essentially a JTI problem and good solutions require solving both tracking and identification problems jointly.

Fig. 1

JTI problems in complex traffic scene

2.2 Modeling

Let $x_{k}$ denote the target state (position, velocity, acceleration, etc.) at time k, and $c_{i}$ denotes the target class i, which belongs to the possible class set $\{1,\ldots ,N\}$. In the JTI problem, tracking is to obtain the state estimate ${\hat{x}}_{k}$ while identification is to determine the target identity $c_{i}$. Therefore, our goal is to obtain $\{ {\hat{x}}_{k},c_{i}\}$ jointly.

As is analyzed above, a hybrid system is expected to take both the complex target motion and the coupling between tracking and identification into consideration. Therefore, we propose the following hybrid system for JTI in complex traffic scene. For target class $c_{i}$, the state evolution and measurement models are given by

$$\begin{aligned} x_{k+1}&=f_{k}^{i}(x_{k})+w_{k}^{i} \\ z_{k}&=h_{k}^{i}(x_{k})+v_{k}^{i} \end{aligned}$$

(1)

where k is time index; $f_{k}^{i}(\cdot )$ and $h_{k}^{i}(\cdot )$ are the state transition function and measurement function, respectively, which can be either linear or nonlinear. $w_{k}^{i}$ and $v_{k}^{i}$ are zero-mean Gaussian white process and measurement noises with covariance matrixes $Q_{k}$ and $R_{k}$, respectively. Note that the superscript i denotes target class i. Therefore, different target classes have different system model (1).

Remark 1

Equation (1) describes the motion model of class $c_{i}$, i.e., target class is related to the motion model. Meanwhile, the motion model basically describes the evolution of target state. Therefore, the motion model relates the target state and class. Based on these, the relationship between the target state and class is as follows: Target classes differ from each other in motion models.

The state transition function $f_{k}^{i}(x_{k})$ and measurement function $h_{k}^{i}(x_{k})$ not only describe the transition of state and measurement, but also provide sufficient flexibility. For example, $f_{k}^{i}(x_{k})$ and $h_{k}^{i}(x_{k})$ can be either linear or nonlinear, time invariant or time-varying, etc. This fits the practical traffic scene in which the target motion may be complicated and changeable over time.

Remark 2

As road target JTI in complex traffic scene is typically a JDE problem, good solutions require solving tracking and identification jointly. In the following, we first review the existing JDE approach. Then, as the main part of this paper, we propose an applicable JTI approach to solve the JTI problem in complex traffic scene.

3 Methods

3.1 Motivation

For JDE problems, [12] proposed an integrated JDE framework based on a new generalized Bayes risk, as follows:

$$\begin{aligned} {\bar{R}}=\sum \nolimits _{i,j}\left( \alpha _{ij}c_{ij}+\beta _{ij}E\left[ {\bar{C}}(x,{\hat{x}} )|D^{i},H^{j}\right] \right) P\left\{ D^{i},H^{j}\right\} \end{aligned}$$

(2)

where $D^{i}$ and $H^{j}$ are the ith decision and the jth hypothesis, respectively; x is the true target state with ${\hat{x}}$ being its estimate; ${\bar{C}}(x,{\hat{x}})$ is the cost of estimating x by ${\hat{x}}$; $c_{ij}$ is the cost of deciding on $D^{i}$ but $H^{j}$ is true, and $E[\bar{C }(x,{\hat{x}})|D^{i},H^{j}]$ is the corresponding expected estimation cost; and $\alpha _{ij}$ and $\beta _{ij}$ are weight factors. This joint framework is optimal in the joint performance by accounting for the coupling between decision and estimation. Within this framework, we develop a conditional JDE (CJDE) approach by introducing the online data [8]. CJDE inherits the theoretical advantages of JDE but has much simpler calculation.

Although CJDE has many advantages for problems involving coupled decision and estimation like JTI, it cannot be directly applied to JTI in complex traffic scene. That is because in such problems, targets usually face high maneuverability and high nonlinearity, which are not considered in the original CJDE approach. Due to these complexities, the JTI solution in complex traffic scene is difficult to be obtained.

Therefore, great efforts are needed to overcome these difficulties so as to achieve a joint solution. Specifically, appropriate estimation strategy satisfying practical traffic scene is required, which can not only bring superior estimation performance but also be easily integrated into the JDE framework. Besides, the coupling between estimation and decision needs further exploration so as to improve the joint performance.

In the following, we propose an applicable JTI approach by accounting for the characteristics of JTI in complex traffic scene and also the adaptability to the JDE framework.

3.2 JTI solution in complex traffic scene

We propose the following CJDE risk for the JTI problem:

$$\begin{aligned} R^{c}(z)\triangleq \sum \nolimits _{i}\sum \nolimits _{j}\left( \alpha _{ij}c_{ij}+\beta _{ij}\xi _{ij}(z)\right) P\left\{ D^{i},H^{j}|z\right\} \end{aligned}$$

(3)

where $H^{j},D^{i},\alpha _{ij}$ and $\beta _{ij}$ are the same as in the JDE risk (2), z is the online data, and

$$\begin{aligned} \xi _{ij}(z)\triangleq E\left[ C(x,{\hat{x}})|D^{i},H^{j},z\right] \end{aligned}$$

(4)

is the expected estimation cost when $H^{j}$ is true but $D^{i}$ is decided, in which $C(x,{\hat{x}})$ is the cost of estimating x by ${\hat{x}}$.

To obtain the JTI estimation and decision results, we need to minimize the above CJDE risk.

3.2.1 Estimator

Suppose the decision $D^{i}$ is given and the estimation cost $C(x,{\hat{x}})$ has the quadratic form, i.e., $C(x,{\hat{x}})={\tilde{x}}^{\prime }{\tilde{x}}$ with ${\tilde{x}}=x-{\hat{x}}$. Then, the optimal JTI estimation which minimizes the JTI risk $R^{c}(z)$ is the following generalized posterior mean:

$$\begin{aligned} {\check{x}}^{(i)}=\sum \nolimits _{j}{\hat{x}}^{(j)}{\bar{P}}_{i}\{H^{j}|z\} \end{aligned}$$

(5)

where ${\hat{x}}^{(j)}$ is the state estimate under hypothesis $H^{j}$. $\bar{P }_{i}\{H^{j}|z\}$ is the generalized posterior probability, given by

$$\begin{aligned} {\bar{P}}_{i}\{H^{j}|z\}=\frac{\beta _{ij}P\{H^{j}|z\}}{\sum \nolimits _{l}\beta _{il}P\{H^{l}|z\}} \end{aligned}$$

(6)

where $P\{H^{j}|z\}$ is the posterior hypothesis probability of $H^{j}$.

3.2.2 Decider

According to the Bayes decision rule, the optimal decision is to minimize the decision risk, i.e., the decision candidate which has the smallest Bayes risk. Thus, with given expected estimation cost $\xi _{ij}(z)$, the optimal JTI decider D is to choose the one whose posterior cost is the smallest:

$$\begin{aligned} D=D^{i},\text { if }{\mathbf {C}}^{i}(z)\le {\mathbf {C}}^{l}(z),\forall l \end{aligned}$$

(7)

in which the posterior cost

$$\begin{aligned} {\mathbf {C}}^{i}(z)=\sum \nolimits _{j}\left( \alpha _{ij}c_{ij}+\beta _{ij}\xi _{ij}(z)\right) P\left\{ H^{j}|z\right\} \end{aligned}$$

It can be seen that in order to obtain the JTI decider D, the key is to determine the posterior cost ${\mathbf {C}}^{i}(z)$. Specifically, as $\alpha _{ij},c_{ij},\beta _{ij}$ are design parameters which are already given, it is the expected estimation cost $\xi _{ij}(z)$ and the posterior hypothesis probability $P\{H^{j}|z\}$ that affect ${\mathbf {C}}^{i}(z)$. Thus, in the following, we focus on determining $\xi _{ij}(z)$ and $P\{H^{j}|z\}$.

For $\xi _{ij}(z)$, with the linear Gaussian assumption and estimation cost $C(x,{\hat{x}})={\tilde{x}}^{\prime }{\tilde{x}}$, we can get that

$$\begin{aligned} \xi _{ij}(z)&= E\left[ {\tilde{x}}^{\prime }{\tilde{x}}|D^{i},H^{j},z\right] \\&= \mathrm {mse}\left( {\hat{x}}^{(j)}|H^{j},z\right) +\left( {\hat{x}}^{(j)}-{\check{x}} ^{(i)}\right) ^{\prime }\cdot \left( {\hat{x}}^{(j)}-{\check{x}}^{(i)}\right) \end{aligned}$$

(8)

where ${\hat{x}}^{(j)}$ is the state estimate under hypothesis $H^{j}$, ${\check{x}}^{(i)}$ is the estimate under decision $D^{i}$, and $\mathrm {mse}( {\hat{x}}^{(j)}|H^{j},z)$ is the estimation mean square error (mse) under $H^{j}$.

For the posterior hypothesis probability $P\{H^{j}|z\}$, following the Bayes rule, we can get:

$$\begin{aligned} P\left\{ H^{j}|z\right\} =f\left( z|H^{j}\right) P\left\{ H^{j}\right\} \end{aligned}$$

(9)

where $P\{H^{j}\}$ is the prior probability and $f(z|H^{j})$ is the measurement likelihood of $H^{j}$.

So far, the JTI solution $\{{\check{x}},D\}$ containing an estimator (5) and a decider (7) is presented. This joint solution has an analytical form, which makes it more practicable. More importantly, the coupling between decision and estimation is fully taken into account.

However, when it comes to practical JTI in complex traffic scene, the concrete decider and estimator $\{{\check{x}},D\}$ are difficult to be determined mainly because of the complicated motion patterns, e.g., high mobility, variability, nonlinearity. In the following part, we will strive to determine the concrete joint solution by considering the peculiarities of JTI in complex traffic scene.

3.3 Determination of JTI tracker and identifier in complex traffic scene

To get full insight of the JTI solution, we conduct a detailed analysis. The JTI estimator is weighed sum of the hypothesis-conditioned estimate ${\hat{x}} ^{(j)}$, where the weight factor is related to the hypothesis probability $P\{H^{j}|z\}(j=1,2,\ldots ,N)$. The JTI decider is to choose the decision candidate with the smallest posterior cost, which is mainly determined by the hypothesis-conditioned estimate ${\hat{x}}^{(j)}$ and the hypothesis probability $P\{H^{j}|z\}$.

In view of the above, the core of obtaining the JTI solution is to determine the hypothesis-conditioned estimate ${\hat{x}}^{(j)}$ and the corresponding hypothesis probability $P\{H^{j}|z\}$. Therefore, appropriate estimation strategy is needed, which should satisfy two basic requirements:

(1) It has accurate estimation performance for both linear and nonlinear systems;

(2) Through this estimation, the hypothesis probability can be easily obtained.

With these requirements, in order to derive the JTI solution, we focus on determining ${\hat{x}}^{(j)}$ and $P\{H^{j}|z\}(j=1,2,\ldots ,N)$ in the following parts.

3.3.1 Determination of ${\hat{x}}^{(j)}$

Determination of each hypothesis-conditioned estimate

For estimation, it has been demonstrated that variable-structure multiple model (VSMM) has superior performance and low computational complexity. Note that tracking and identification in complex scene are a difficult problem due to the complicated and changeable target motion. Therefore, VSMM is very suitable for this problem.

The essential issue of the VSMM approach is model set adaptation (MSA). Many MSA methods have been proposed, among which expected-mode augmentation (EMA) is widely used and extensively researched. In the EMA approach, the original set of models is augmented by a variable set of models intended to match the expected value of the unknown true mode. Specifically, the newly activated models are generated adaptively in real time which are probabilistically weighted sums of mode estimates over the model set.

By combining the variable-structure interacting-multiple model (VSIMM) with the EMA approach, we propose to use the EMA-VSIMM algorithm in this paper. Specifically, EMA-VSIMM algorithm consists of six steps: (1) probability prediction; (2) MSA using EMA approach; (3) interaction/mixing of the estimates; (4) filtering in each filters; (5) probability update; and (6) estimate fusion. Compared to fixed-structure MM method, the adaptive model set in this algorithm is obtained using the EMA algorithm. More details about EMA-VSIMM can be found in [32].

Fig. 2

Estimation strategy of JTI in complex traffic scene

Based on the above, we propose the following estimation strategy for JTI, as illustrated in Fig. 2. Assume that there are two possible target classes ($H^{1}$ and $H^{2}$), and under each class, there are M models composing the EMA filter, i.e., M is the total number of models in the EMA filter (for any target class).

In the lower layer, each basic filter (e.g., KF, UKF, etc.) runs and outputs the model-based state estimate ${\hat{x}}_{jq}$ and the corresponding model probability $\mu _{jq}$, where $j=1,2$ and $q=1,\ldots ,M$. Here, the subscript j is the variable denoting the jth hypothesis (i.e., $H^{j}$), while q is the variable denoting the qth model. Then, under each hypothesis $H^{j}$, through the EMA estimation process, the state estimate ${\hat{x}}^{(j)}$ and the hypothesis probability $P\{H^{j}\}$ ($j=1,2$) can be obtained.

In the upper layer, JTI approach runs based on (${\hat{x}}^{(1)},P\{H^{1}|z\}$) $,({\hat{x}}^{(2)},P\{H^{2}|z\})$, which are output by the EMA estimator under hypothesis $H^{1}$ and $H^{2}$, respectively. Finally, the JTI solution (${\hat{x}},D$) can be obtained by Eqs. (5) and (7).

Determination of each model-based estimator

Considering the above two requirements for estimation, a new estimation strategy is required. For linear case, Kalman filter can be applied easily since it is optimal in minimum mean square error (MMSE) sense, and it can also output the analytical estimation result and the corresponding model probability. However, this paper considers nonlinear case, which is much more common in practical traffic scene.

For nonlinear case, we propose to use UKF as it satisfies the requirements mentioned earlier in Sect. 3.3:

(a) It has satisfactory estimation performance and low calculation for handling the nonlinear estimation problem;

(b) It can output the required posterior hypothesis probability.

Suppose under hypothesis $H^{j}(j=1,2,\ldots ,N)$, there are totally M models in the model set for EMA, and $m_{jq}$ denotes the qth ($q=1,2,\ldots ,M$) model in the model set given $H^{j}$. Then, as shown in Fig. 2, every basic filter (based on model $m_{jq}$) is UKF. For each $m_{jq}$($q=1,2,\ldots ,M$), the estimation process based on UKF is as follows.

Generally, UKF is based on the UT conversion, whose basic idea can be described as: For nonlinear conversion $y=f(x)$, x is the n-dimensional state vector with ${\bar{x}}$ being its mean and P being its variance. We can get $2n+1$ Sigma points X with the corresponding weight $\omega$ to compute the statistics of y. Specifically, one cycle of the unscented Kalman filter is as follows:

(1) Given ${\hat{x}}_{k-1|k-1},P_{k-1|k-1}$, compute the one-step predict state ${\hat{x}}_{k|k-1}$ and the predict error covariance matrix $P_{k|k-1}$.

(a) Compute the $\sigma$ point $\xi _{k-1|k-1}^{(i)},i=1,2,\ldots ,2n$, that is,

$$\begin{aligned} \left\{ \begin{array}{l} \xi _{k-1|k-1}^{(0)}={\hat{x}}_{k-1|k-1} \\ \xi _{k-1|k-1}^{(i)}={\hat{x}}_{k-1|k-1}+(\sqrt{(n+\lambda )P_{k-1|k-1}})_{i}, \\ i=1,2,\ldots ,n \\ \xi _{k-1|k-1}^{(i)}={\hat{x}}_{k-1|k-1}-(\sqrt{(n+\lambda )P_{k-1|k-1}})_{i}, \\ i=n+1,n+2,\ldots ,2n \end{array} \right. \end{aligned}$$

(b) Calculate the $\sigma$ point of $\xi _{k|k-1}^{(i)},i=1,2,\ldots ,2n$ propagating through the state evolution function, that is,

$$\begin{aligned} \left\{ \begin{array}{l} \xi _{k|k-1}^{(i)}=f_{k}(\xi _{k-1|k-1}^{(i)}),i=1,2,\ldots ,2n \\ {\hat{x}}_{k|k-1}=\sum \limits _{i=0}^{2n}\omega _{i}^{(m)}\xi _{k|k-1}^{(i)} \\ P_{k|k-1}=\sum \limits _{i=0}^{2n}\omega _{i}^{(c)}(\xi _{k|k-1}^{(i)}-{\hat{x}} _{k|k-1})(\xi _{k|k-1}^{(i)}-{\hat{x}}_{k|k-1})^{T} \\ +Q_{k-1} \end{array} \right. \end{aligned}$$

(2) Obtain the propagation of the $\sigma$ point ${\hat{x}}_{k|k-1},P_{k|k-1}$ through the measurement equation using UT.

(a) Calculate the propagation of $\sigma$ point ${\hat{x}}_{k|k-1},P_{k|k-1}$ through the measurement equation to $x_{k}$, i.e.,

$$\begin{aligned} \left\{ \begin{array}{l} \xi _{k}^{(0)}={\hat{x}}_{k|k-1} \\ \xi _{k}^{(i)}={\hat{x}}_{k|k-1}+(\sqrt{(n+\lambda )P_{k-1|k-1}})_{i} \\ i=1,2,\ldots ,n \\ \xi _{k}^{(i)}={\hat{x}}_{k|k-1}-(\sqrt{(n+\lambda )P_{k-1|k-1}})_{i}, \\ i=n+1,n+2,\ldots ,2n \end{array} \right. \end{aligned}$$

(b) Compute the one-step predict of the output, i.e.,

$$\begin{aligned} \left\{ \begin{array}{l} \zeta _{k|k-1}^{(i)}=h_{k}(\xi _{k}^{(i)}),i=1,2,\ldots ,2n \\ {\hat{z}}_{k|k-1}=\sum \limits _{i=0}^{2n}\omega _{i}^{(m)}\zeta _{k|k-1}^{(i)} \\ P_{{\bar{z}}_{k}}=\sum \limits _{i=0}^{2n}\omega _{i}^{(c)}(\zeta _{k|k-1}^{(i)}- {\hat{z}}_{k|k-1})(\zeta _{k|k-1}^{(i)}-{\hat{z}}_{k|k-1})^{T}+R_{k} \\ P_{{\widetilde{x}}_{k}{\widetilde{z}}_{k}}=\sum \limits _{i=0}^{2n}\omega _{i}^{(c)}(\xi _{k}^{(i)}-{\hat{x}}_{k|k-1})(\xi _{k}^{(i)}-{\hat{x}} _{k|k-1})^{T} \end{array} \right. \end{aligned}$$

(3) After obtaining the new measurement $z_{k}$, update the following quantities:

$$\begin{aligned} {\hat{x}}_{k|k}={\hat{x}}_{k|k-1}+K_{k}(z_{k}-{\hat{z}}_{k|k-1}) \end{aligned}$$

(10)

$$\begin{aligned} K_{k}=P_{{\widetilde{x}}_{k}{\widetilde{z}}_{k}}P_{{\bar{z}}_{k}}^{-1} \end{aligned}$$

$$\begin{aligned} P_{k|k}=P_{k|k-1}-K_{k}P_{{\bar{z}}_{k}}K_{k}^{T} \end{aligned}$$

(11)

where $K_{k}$ is the filter gain.

Based on the above steps, each model $m_{jq}(q=1,2,\ldots ,M)-$based state estimate and the corresponding estimation MSE can be obtained by (10) and (11), respectively.

Besides, the probability of each model $m_{jq}(q=1,2,\ldots ,M)$ can be determined as follows. Under the Gaussian assumption, the probability of each model is calculated by

$$\begin{aligned} \mu _{jq}(k)&= P(m_{jq}(k)|z^{k})=\frac{L_{jq}(k)\cdot \mu _{jq}(k|k-1)}{ \sum \limits _{p}L_{jp}(k)\cdot \mu _{jp}(k|k-1)} \end{aligned}$$

(12)

$$\begin{aligned} L_{jq}(k)&= \frac{1}{(2\pi )^{m/2}|S_{jq}(k)|^{1/2}} e^{-r_{jq}^{T}(k)|S_{jq}(k)|^{1/2}r_{jq}(k)/2} \end{aligned}$$

(13)

where $L_{jq}(k)$ is the likelihood of the model $m_{jq}$; $r_{jq}(k)$ and $S_{jq}(k)$ are its residual and covariance, respectively, which can be given by UKF as follows:

$$\begin{aligned} r_{jq}(k)=z_{k}-{\hat{z}}_{k|k-1},S_{jq}(k)=P_{{\bar{z}}_{k}} \end{aligned}$$

The probability prediction is given by

$$\begin{aligned} \mu _{jq}(k|k-1)=P\left( m_{jq}(k)|z^{k-1}\right) =\sum \limits _{p}\pi _{pq}\mu _{jp}(k-1) \end{aligned}$$

where $\mu _{jq}(k|k-1)$ is the predicted probability from time $k-1$ to k , $\mu _{jp}(k-1)$ is the probability of the pth ($p=1,2,\ldots ,M$) model at time $k-1$, and $\pi _{pq}$ is the (p, q)th element of the transition probability matrix (TPM) for EMA. Note that this likelihood $L_{jq}(k)$ is the basis of computing the hypothesis probability, which further plays important role in obtaining the JTI solution.

3.3.2 Determination of $P\{H^{j}|z\}$

In the following, we focus on determining the posterior probability of hypothesis $H^{j}$, i.e., $P\{H^{j}|z\},$where $j=1,2,\ldots ,N$. According to the Bayesian rule, the probability of $H^{j}$ is given by:

$$\begin{aligned} P\big \{H^{j}|z\big \}=f\big \{z|H^{j}\big \}P\big \{H^{j}\big \} \end{aligned}$$

(14)

in which $P\{H^{j}\}$ is the prior probability and $f\{z|H^{j}\}$ is the likelihood of $H^{j}$. Therefore, the key is to obtain $f\{z|H^{j}\}$.

Since the EMA estimation method is adopted, the likelihood $f\{z|H^{j}\}$ is the total likelihood of all models given hypothesis $H^{j}$, i.e.,

$$\begin{aligned} f\big \{z|H^{j}\big \}=\sum _{q=1}^{M}f\big \{z|m_{jq},H^{j}\big \}P\big \{m_{jq}|H^{j}\big \} \end{aligned}$$

in which $f\{z|m_{jq},H^{j}\}$ denotes the model likelihood of $m_{jq}$ given hypothesis $H^{j}$, and $P\{m_{jq}|H^{j}\}$ means the model probability of $m_{jq}$ under hypothesis $H^{j}$.

Specifically, under hypothesis $H^{j},$the model likelihood of $m_{jq}$ is

$$\begin{aligned} f\big \{z|m_{jq},H^{j}\big \}=L_{jq}(k) \end{aligned}$$

where $L_{jq}(k)$ is given in (13).

The model probability of $m_{jq}$ is

$$\begin{aligned} P\big \{m_{jq}|H^{j}\big \}=\mu _{jq}(k) \end{aligned}$$

where $\mu _{jq}(k)$ is given in (12).

Remark 3

Based on the above, both the hypothesis-conditioned estimate ${\hat{x}}^{(j)}$ and the posterior hypothesis probability $P\{H^{j}|z\}(j=1,2,\ldots ,N)$ can be obtained, which are critical in JTI tracker and identifier.

Remark 4

To make it more clear, we state the JTI tracking and identification results again.

First, for tracking, the JTI tracking solution is given in (5), which is a weighted sum of the hypothesis-conditioned estimate ${\hat{x}}^{(j)}(j=1,2,\ldots ,N)$ with the weight being closely related to the hypothesis probability $P\{H^{j}|z\}$. Based on these, we can obtain the final JTI tracking result ${\check{x}}^{(i)}.$ (Suppose decision is $D^{i}.$)

Second, for identification, the JTI identification solution is given in (7), where the key is the expected estimation cost $\xi _{ij}(z)$ and the posterior hypothesis probability $P\{H^{j}|z\}$. Specifically, for the former $\xi _{ij}(z)=\mathrm {mse}({\hat{x}}^{(j)}|H^{j},z)+({\hat{x}}^{(j)}- {\check{x}}^{(i)})^{\prime }(\cdot )$ [given in (8)], the key is to obtain $\mathrm {mse}({\hat{x}}^{(j)}|H^{j},z),{\hat{x}}^{(j)}$, and ${\check{x}}^{(i)}$. Among these, $\mathrm {mse}({\hat{x}}^{(j)}|H^{j},z)$ and ${\hat{x}}^{(j)}$ can be determined by EMA under hypothesis $H^{j}$, and ${\check{x}}^{(i)}$ can be determined by (5). For the latter $P\{H^{j}|z\}$, the detailed calculation is given in (14).

3.4 A JTI algorithm in complex traffic scene

Based on the above JTI tracking and identification results, we propose the following JTI algorithm at time k.

1. Initialization. Under each hypothesis $H^{j}$ ($j=1,2,\ldots ,N$ ), calculate the hypothesis-conditioned estimate ${\hat{x}}_{k-1}^{(j)}$, the corresponding MSE $P_{k-1}^{(j)}$, and the hypothesis probability $P\{H^{j}|z^{k-1}\}$ at time $k-1$.

2. One-step prediction. Following the UKF-EMA-based estimation method, calculate the one-step predicted state estimate ${\hat{x}}_{k|k-1}^{(j)}$ and MSE $P_{k|k-1}^{(j)}$.

3. Update. When data $z_{k}$ comes, update ${\hat{x}}_{k|k}^{(j)}$ and ${\hat{P}}_{k|k}^{(j)}$. Based on these, calculate ${\check{x}}_{k|k}^{(i)}$ ($i=1,2,\ldots ,N$) according to (5).

4. Further calculation. Calculate the expected estimation cost $\xi _{ij}(z^{k})$ by (8) and the posterior cost ${\mathbf {C}}^{i}(z^{k})$. Then, JTI decision is $D_{k}^{i}$ , if ${\mathbf {C}}^{i}(z^{k})\le {\mathbf {C}}^{l}(z^{k}),\forall l$.

5. Output Output the constrained JTI solution for time k: $D_{k}=D_{k}^{i}$ in step 4 and ${\hat{x}}_{k}={\check{x}}_{k}^{(i)}$ in step 3.

Remark 5

The complexity of the proposed JTI algorithm is analyzed as follows.

(a) The tracking and identification results can be obtained jointly without iteration. The above algorithm steps show that to achieve the dual goals (tracking and identification), no iteration is required. Once new data come, after simple implementation of steps 1, 2, and 3, we can achieve the dual goals simultaneously.

(b) All elements are obtained by point estimation without any density estimation, which makes it easy in implementation. Specifically, the hypothesis-conditioned estimate ${\hat{x}}_{k}^{(j)},$the estimation MSE $P_{k}^{(j)}$, the hypothesis probability $P\{H^{j}|z^{k}\}$, the expected estimation cost $\xi _{ij}(z^{k})$, the posterior cost ${\mathbf {C}} ^{i}(z^{k})$, the finally JTI tracking result ${\check{x}}_{k}^{(i)},$ and identification result $D_{k}^{i}$ are all obtained by point estimation. In other words, the proposed UKF-EMA strategy does not involve any density estimation.

In summary, the proposed JTI algorithm has low implementation complexity due to its point estimation basis. Note that this paper considers tracking and identification with high maneuverability and high nonlinearity, while the traditional methods for such problems usually adopt density estimation-based method, e.g., particle filter, random finite set methods. From this point of view, this paper has superiority in calculation complexity.

3.5 Joint performance evaluation metric

The traditional performance evaluation of JDE problems is that the decision performance and the estimation performance are evaluated separately using their own metrics, where the correct-decision rate is usually used for decision performance evaluation, while mean square error is used to evaluate the estimation performance [33, 34]. For JDE problems, however, they are comprehensive and may even fail to compare different algorithms. Considering this, reference [8, 35] points out that decision and estimation performance should be evaluated jointly rather than separately.

To evaluate the joint performance of JTI in complex traffic scene, we adopt the following joint performance measure (JPM), as proposed in [17]:

$$\begin{aligned} \lambda _{k}=d_{k}^{i}(H^{i},{\hat{D}}_{k})+\gamma d_{k}^{t}(x_{k},{\hat{x}}_{k}) \end{aligned}$$

(15)

in which $d_{k}^{i}(H^{i},{\hat{D}}_{k})$ and $d_{k}^{t}(x_{k},{\hat{x}}_{k})$ are the cost for identification and tracking, respectively. Specifically, if decision is correct ($c_{i}={\hat{D}}_{k}$), $d_{k}^{i}(H^{i},{\hat{D}}_{k})=0$; otherwise, $d_{k}^{i}(H^{i},{\hat{D}}_{k})=1$. $d_{k}^{t}(x_{k},{\hat{x}}_{k})$ is the normalized estimation cost, which is defined in detailed in [15]. $\gamma$ is the weight factor, which can adjust the relative weight of tracking and identification cost.

4 Simulation and discussion

This section presents two typical JTI problems in complex traffic scene. Performance evaluation metrics are root-mean-square error (RMSE), probability of correct classification (PC), and JPM. The compared methods are the traditional identification-then-tracking (I-then-T), tracking-then-identification (T-then-I), and our proposed JTI method.

Specifically, in I-then-T, the optimal Bayes decision is made first based on the posterior hypothesis probability, and then, estimation is obtained based on this decision. In T-then-I, the minimum mean square error (MMSE) optimal estimation is obtained first, and then, decision is made based on the ratio of current measurement likelihoods conditioned on ${\hat{x}}_{k|k-1}$ and $H^{j}$ [13].

Suppose a vehicle moves in complex traffic scene, whose class may be $c_{1}$ or $c_{2}$. We want to identify the target class and track its state jointly using all available data. Here, classes differ in dynamic behaviors, which is reasonable since targets in different classes usually have different behaviors in reality. For example, for a car and a truck, a car usually has larger maneuverability than a truck. For model-based tracking or classification, such dynamic behaviors are described by motion models.

Two examples simulate two different scenarios. Example 1 considers a constant turning motion at a corner of a road, while Example 2 considers a complicated moving at a crossroads. These are very common and representative mobilities in practical complex traffic scene.

4.1 Example 1: Simulation scenario at a corner

Fig. 3

JTI at a corner of a road

In this example, we consider a complex turning motion at the corner of a road. Classes differ from each other in turning rates (e.g., a car moving on the inner lane has larger turning rate than a bus moving on the outer lane); therefore, identification is based on this difference. However, the turn rate is unknown in advance and changeable over time, and it is not easy to determine it. Our goal is to track the vehicle’s state and identify its class jointly.

We propose to use the constant turn (CT) model to describe the target motion [36]. Suppose the target state at time k is $x_{k}=[p_{k}^{x},v_{k}^{x},p_{k}^{y},v_{k}^{y}]^{\prime }$, in which $p_{k}^{x},v_{k}^{x},p_{k}^{y},v_{k}^{y}$ denotes position in x-axis, velocity in x-axis, position is y-axis, and velocity in y-axis, respectively. The system model is given by:

$$\begin{aligned} x_{k+1}&= F^{CT}(\omega )x_{k}+w_{k} \\ z_{k+1}&= h_{k}(x_{k+1})+v_{k} \end{aligned}$$

(16)

where the transition function $F^{CT}(\omega )$ is given by

$$\begin{aligned} F^{CT}(\omega )=\left[ \begin{array}{llll} 1 & \frac{\sin \omega T}{\omega } & 0 & -\frac{1-\cos \omega T}{\omega T} \\ 0 & \cos \omega T & 0 & -\sin \omega T \\ 0 & \frac{1-\cos \omega T}{\omega T} & 1 & \frac{\sin \omega T}{\omega } \\ 0 & \sin \omega T & 0 & \cos \omega T \end{array} \right] \end{aligned}$$

(17)

and the covariance of process noise is

$$\begin{aligned} cov(w_{k})= \\ \left[ \begin{array}{llll} \frac{2(\omega T-\sin \omega T)}{\omega ^{3}} & \frac{1-\cos \omega T}{ \omega ^{2}} & 0 & \frac{\omega T-\sin \omega T}{\omega ^{2}} \\ \frac{1-\cos \omega T}{\omega ^{2}} & T & -\frac{\omega T-\sin \omega T}{ \omega ^{2}} & 0 \\ 0 & -\frac{\omega T-\sin \omega T}{\omega ^{2}} & \frac{2(\omega T-\sin \omega T)}{\omega ^{3}} & \frac{1-\cos \omega T}{\omega ^{2}} \\ \frac{\omega T-\sin \omega T}{\omega ^{2}} & 0 & \frac{1-\cos \omega T}{ \omega ^{2}} & T \end{array} \right] \end{aligned}$$

The measurement model is

$$\begin{aligned} h_{k}(x_{k})=\left[ \begin{array}{l} \sqrt{(p_{k}^{x})^{2}+(p_{k}^{y})^{2}} \\ \arctan \frac{p_{k}^{y}}{p_{k}^{x}} \end{array} \right] \end{aligned}$$

(18)

The initial target state $x_{0}=[500,10,500,10]^{\prime };P_{0}=[10^{4},1,10^{4},1]^{\prime }$. The measurement noise $v_{k}$ in one dimension follows ${\mathcal {N}}(0,50^{2}$m$^{2})$. The JPM (15) with $\gamma =1$ is used. $c_{ij}=1,c_{ii}=0,\alpha _{ij}=1,\sum _{i}\beta _{ij}=10^{-4},\beta _{ii}/\beta _{ij}=1.5$. All results were obtained from 10000 MC (Monte Carlo) runs. The true target class is randomly generated with equal probabilities in each MC run, i.e., $P(c_{1})=P(c_{2})=0.5$.

The parameters in EMA estimation are as follows. For class 1, the fixed model set is $\{3\pi /180,4\pi /180\}$; for class 2, the fixed model set is $\{6\pi /180,10\pi /180,12\pi /180,8\pi /180\}$. For each class at each time step, we use one expected model, i.e., $EMA\{2+1\}$ and $EMA\{4+1\}$ for classes 1 and 2, respectively. The transition probability matrix (TPM) for the total model set (containing the expected one) is as follows:

$$\begin{aligned} \text {TPM}1&= \left[ \begin{array}{lll} 0.9 & 0.05 & 0.05 \\ 0.15 & 0.85 & 0 \\ 0.15 & 0 & 0.85 \end{array} \right] \\ \text {TPM}2&= \left[ \begin{array}{lllll} 0.9 & 0.025 & 0.025 & 0.025 & 0.025 \\ 0.15 & 0.85 & 0 & 0 & 0 \\ 0.15 & 0 & 0.85 & 0 & 0 \\ 0.15 & 0 & 0 & 0.85 & 0 \\ 0.15 & 0 & 0 & 0 & 0.85 \end{array} \right] \end{aligned}$$

Simulation results are presented in Fig. 3. They show that for tracking, T-then-I performs best, JTI is in the middle and I-then-T is the worst. Here, T-then-I is best as is desired since its tracking is MMSE estimation, which is optimal in the sense of mse. I-then-T is worst since it does tracking completely based on the decided class without considering possible decision errors. For identification, I-then-T performs best since with $c_{ii}=0,c_{ij}=1(i\ne j)$, identification in I-then-T is the minimal-error-rate decision, which has the highest correct identification rate. T-then-I has the worst decision performance since it does decision based on the one-step predicted estimation.

For the joint performance, JTI outperforms I-then-T and T-then-I. This verifies that JTI can make a good trade-off between optimal decision and optimal estimation and finally performs best in joint performance, which is cared about most in a joint problem. Specifically, within about 18 steps, JTI is significantly superior than I-then-T and is very close to T-then-I. After 18 steps, JTI is close to I-then-T but consistently superior than T-then-I. Quantitatively, the joint performance of JTI is improved by 30% compared with T-then-I at the steady state. In general, the proposed JTI is significantly superior than the traditional two-step methods.

Remark 6

To see more clearly, we also provide the lower bound of the joint performance, as shown in Fig. 4. In Fig. 4, “ideal” means the ideal case in which the true target class is known, and only the joint performance is presented since it is the most desirable performance. Figure 4 verifies that with the accumulation of data, the proposed JTI approach is robust and is near to the lower bound of that in the ideal case.

Fig. 4

JTI at a corner with the ideal case

4.2 Example 2: Simulation scenario at a crossroad

In this simulation, we consider a typical moving at a crossroads, as illustrated in Fig. 1b. The vehicle passing the crossroads goes strait first, and then turns, and then goes strait ahead. This follows the “strait-turn-strait” mode and can be represented by the linear motion and the turn motion. We propose to use the constant acceleration (CA) and CT models to describe this motion [36]. With the target state $x_{k}=[p_{k}^{x},v_{k}^{x},p_{k}^{y},v_{k}^{y}]^{\prime }$, the system model is given by:

$$\begin{aligned} x_{k+1}&= f_{k}(x_{k})+w_{k} \\ z_{k+1}&= h_{k}(x_{k+1})+v_{k} \end{aligned}$$

(19)

When the target moves in a CA model,

$$\begin{aligned} x_{k+1}=\left[ \begin{array}{llll} 1 & T & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & T \\ 0 & 0 & 0 & 1 \end{array} \right] x_{k}+G[\alpha _{CA},\alpha _{CA}]^{\prime }+Gw_{k} \end{aligned},$$

(20)

while when the target moves in a CT model, the dynamic model is the same as in Example 1.

$$\begin{aligned} G=\left[ \begin{array}{ll} T^{2}/2 & 0 \\ T & 0 \\ 0 & T^{2}/2 \\ 0 & T \end{array} \right] ,w_{k}=\left[ \begin{array}{l} w_{k}^{x} \\ w_{k}^{y} \end{array} \right] \end{aligned}$$

(21)

and

$$\begin{aligned} S_{w}=cov(w_{k}^{x})=cov(w_{k}^{y}) \end{aligned}$$

(22)

Here, $w_{k}^{x}$ and $w_{k}^{y}$, which are modeled as process noises, are actually the accelerations along the x- and y-axes, respectively.

When the vehicle moves in a CA model, $\alpha _{CA}^{1}=1g,\alpha _{CA}^{2}=2g,$where $g=9.8m/s;$when the vehicle moves in a CT model, $\omega _{1}=3\pi /180(rad/s),\omega _{2}=-18\pi /180(rad/s)$. The initialization parameters are the same as in Example 1. The total time step is 30s, and the target motion is as follows: 0–10s, CA model; 10–20s, CT model; and 20–30s, CA model. In this example, since the target motion pattern changes over time, single model is adopted to eliminate the interference caused by the model switching and only to verify the proposed JTI algorithm.

Fig. 5

JTI at a crossroads of a road

The simulation results are presented in Fig. 5. It can be seen that for tracking, T-then-I performs best, JTI is in the middle, and I-then-T performs worst. This is consistent with our expectations, and the reason is the same as in Example 1. For identification, I-then-T performs best since it is the minimal-error-rate decision. For joint performance, JTI is the best, which verifies that JTI is robust to complicated mobility and continuously better than the traditional methods. Generally, this fully demonstrates the superiority of the proposed JTI approach in complex motion scenario.

Remark 7

Generally, the JTI problem in complex traffic scene is formulated (by illustration and models) in Sect. 2, the JTI solution with theoretical analyses is provided in Sect. 3, and the simulation verification is presented in Sect. 4.

For problem formulation, there are all kinds of complex traffic scenes, among which complicated mobility and high nonlinearity are common and typical. Therefore, we formulate the JTI problem based on these two scenarios. More importantly, JTI as a joint problem has highly coupled tracking and identification, which is critical in this paper.

For solution, we explore a JTI solution accounting for the practical complex traffic scene and also utilizing the coupling between tracking and identification. Specifically, JTI solution with incorporated UKF-EMA estimation strategy is proposed, which has superior joint performance by utilizing the coupling information and also considers the practical complicated mobilities.

For simulation, two typical examples representing different complex scenes fully demonstrate the superiority of the proposed JTI approach. Simulation results show that the proposed JTI approach with incorporated UKF-EMA estimation can beat the traditional two-step strategies and finally performs best in joint performance.

5 Conclusions

This paper proposes a new joint tracking and identification (JTI) approach for practical JTI problem in complex traffic scene. JTI is essentially a joint decision and estimation (JDE) problem, and better solution requires solving the tracking and identification jointly. The recently proposed JDE framework provides a good framework for solving such problems involving coupled decision and estimation.

First, we formulate the JTI problem in complex traffic scene using a hybrid system model. Then, an applicable JTI approach which considers the complexities of practical traffic scene and also the interdependence between tracking and identification is proposed. Specifically, we propose a CJDE-based JTI risk and then derive a JTI solution by minimizing this risk. A new UKF-EMA-based estimation strategy is proposed. On the one hand, it guarantees the superiority of estimation performance due to UKF in handling nonlinear estimation and EMA in handling complicated motions. On the other hand, it facilitates the decision by providing quantities required in JTI decider. Also presented is a joint performance evaluation metric which can evaluate tracking and identification performance comprehensively.

Simulation results demonstrate the superiority of the proposed JTI approach in complex traffic scene. By considering the characteristics of JTI in practical complex traffic scene and also the highly coupling between tracking and identification, the proposed JTI approach beats the traditional T-then-I and I-then-T methods in joint performance. Note that this paper focuses on the complexity and nonlinearity of the target motion for one single target, multiple targets scenarios will be investigated in the future.