Ensemble hidden Markov models with application to landmine detection

2.1 Hidden Markov models

is generally used to indicate the complete parameter set of the HMM model. In (1), A= [a _ij ] is the state transition probability matrix, where a _ij =P r(q _t =j|q _t−1=i) for i,j=1,…,N; π=[π _i ], where π _i =P r(q ₁=s _i ) are the initial state probabilities; and B=b _i (O _t ),i=1,…,N, where b _i (O _t )=P r(O _t |q _t =i) is the observation probability distribution in state i.

An HMM is called continuous if the observation probability density functions are continuous and discrete otherwise. In the case of the discrete HMM, the observation vectors are commonly quantized into a finite set of symbols, {v ₁,v ₂,…,v _M }, called the codebook. Each state is represented by a discrete probability density function and each symbol has an associated probability of occurring given that the system is in a given state. In other words, B becomes a simple set of fixed probabilities for each state. That is, b _i (O _t )=b _i (k)=P r(v _k |q _t =i), where v _k is the nearest codebook symbol to O _t .

Given the form of the hidden Markov model defined in (1), Rabiner [16] defines three key problems of interest that must be solved for the model to be useful in real-world applications: (i) the classification problem; (ii) the problem of finding an optimal state sequence; and (iii) the problem of estimating the model parameters.

The classification problem involves computing the probability of an observation sequence O={O ₁,O ₂,…,O _T } given a model λ, i.e, P r(O|λ). This probability is computed efficiently using the forward–backward procedure [16].

In most applications, it often turns out that computing an optimal state sequence is more useful than P r(O|λ). There are several possible optimality criteria. One that is particularly useful is to maximize P r(O,Q|λ) over all possible state sequences Q. The Viterbi algorithm [17] is an efficient and formal technique for finding this optimal state sequence and its probability.

The third problem in building an HMM is the training problem: how does one estimate the parameters of the model? The problem is difficult because there are several levels of estimation required in an HMM. First, the states themselves must be estimated. Then, the model parameters λ=(A,B,π) need to be estimated. For the discrete HMM, first the codebook is determined, usually using the K-means algorithm [18], or other vector quantization algorithms. Then, the parameters (A,B,π) are estimated iteratively using the Baum-Welch learning algorithm [19].

2.2 Baseline HMM classifier for landmine detection

The baseline HMM classifier for GPR-based landmine detection was first introduced in [5]. It consists of two HMM models, one for mines and one for the background. Each model has four states and produces a probability value by backtracking through model states using the Viterbi algorithm [17]. The mine model, λ ^m , is designed to capture the spatial distribution of the features. This model assumes that mine signatures have a hyperbolic shape comprised of a succession of rising, horizontal, and falling edges with variable duration in each state. The beginning and the end of the observation vectors correspond typically to a non-edge (or background) state. The background model, λ ^b , is needed to capture the background and clutter characteristics. No prior information or assumptions are used for this model.

The architecture of the baseline HMM classifier is shown in Fig. 2. Full details of the model’s initialization and training can be found in [5]. The probability value produced by the mine (background) model can be thought of as an estimate of the probability of the observation sequence given that there is a mine (background) present.

where #{s _t =1,t=1,⋯,T} corresponds to the number of observations assigned to the background state (state 1). T _max is defined experimentally based on the shortest mine signature. Equation (2) ensures that sequences with a large number of observations assigned to state 1 are considered non-mines.

2.3 Extensions to the baseline HMM for landmine detection

In an effort to improve performance and generalization, several extensions to the baseline HMM have been proposed. For instance, in [12, 13], the authors proposed the multi-stream HMM (MSHMM) that combines multiple sets of features. An optimal weight for each feature was learned in the training phase. In [14], maximum likelihood (ML) and minimization of classification error (MCE) learning methods were derived for the MSHMM. In [15], HMMs with stick-breaking priors (SBHMM) [20] were employed to learn the number of HMM states in the baseline HMM landmine detector. This approach relies on a variational Bayesian learning technique in lieu of standard BW training.

4.1 Similarity between observations in the log-likelihood space

4.2 Pairwise distance-based clustering

The distance matrix, computed using (10), reflects the degree to which pairs of sequences are considered similar. The largest variation is expected to be between sequences from different classes. Other significant variations may exist within the same class, e.g., the groups of signatures shown in Fig. 3. Our goal is to identify the similar groups so that one model can be learned for each group. This task can be achieved using any relational clustering algorithm. In our work, we use the standard agglomerative hierarchical algorithm [18].

where n _k and c _k are the cardinality and the centroid of cluster C _k , respectively. It has been shown in [17] that this approach merges the two clusters that lead to the smallest increase in the overall variance.

4.3 Ensemble HMM initialization and training

The previous clustering step results in K clusters, each comprised of potentially similar sequences. Each cluster is then used to learn an HMM, resulting in an ensemble of K HMMs. Let N _k denote the number of sequences assigned to the same cluster k. Since our clustering step did not use class labels, clusters may include sequences from different classes. Let \(N_{k}^{(c)}\) be the number of sequences in cluster k that belong to class c, such that \(\sum _{c=1}^{C}{N_{k}^{(c)}}=N_{k}\). For instance, for the landmine example, if we let c=1 denote the class of mines and c=0 denote the class of clutter, \(N_{k}^{(1)}\) would be the number of mines assigned to cluster k.

The next step of our approach consists of learning a set of HMMs that reflect the diversity of the training data. Since a cluster contains a set of similar sequences, and each cluster may include observations from different classes, we learn one HMM model \(\left \{\lambda _{k}^{(c)}\right \}\) for each set of sequences assigned to class c within cluster k. Let \(\mathbb {O}_{k}^{(c)}=\left \{O_{r}^{(c)},y_{r}^{(c)}\right \}_{r=1}^{N_{k}^{(c)}}\) be the set of sequences partitioned into cluster k that belong to class c and let \(\left \{\lambda _{r}^{(c)}\right \}_{r=1}^{N_{k}^{(c)}}\) be their corresponding individual HMM models, c∈{1,⋯,C}.

To summarize, for each homogenous cluster i, we define one model \(\lambda _{i}^{(C_{i})}\), i=1,⋯,K ₁, for the dominating class C _i . For mixed clusters, we define C models per cluster: \(\lambda _{j}^{(c)}\), c=1…C, j=1,⋯,K ₂. For each small cluster, we define one model \(\lambda _{k}^{(C_{k})}\) for the dominating class C _k . The ensemble HMM mixture is defined as \(\left \{\lambda _{k}^{(c)}\right \}\), where k∈{1,⋯,K}, and c=C _k if cluster k is dominated by sequences labeled with class C _k , and c∈{1⋯,C} if cluster k is a mixed cluster.

For both the discrete and continuous cases, any clustering algorithm, such as the K-means [27] or the fuzzy c-means [28], could be used to identify the states, codebook, or the multiple components. After initialization, we use one of the training schemes described earlier, to update \(\lambda _{k}^{(c)}\) parameters using the respective observations \(\mathbb {O}_{k}^{(c)}\), k∈{1,⋯,K}, c∈{1,⋯,C}. As mentioned earlier, for homogenous clusters, BW training results in one model λ ^{B W} per cluster; for mixed clusters, MCE training results in C models per cluster, \(\lambda ^{MCE}_{c}, c=1 \ldots C\); and for small clusters, variational Bayesian learning results in one model per cluster, λ ^{V B} . The output of Baum-Welch- and VB-trained cluster models is P r(O|λ _k ) while the output of the MCE-trained cluster models is \(\max _{c}{Pr\left (O|\lambda _{k,c}^{MCE}\right)}\).

4.4 Decision level fusion

The partial confidence values of the different models need to be combined into a single confidence value. Let \(\Lambda =\left \{\lambda ^{BW}_{i},\lambda ^{MCE}_{j},\lambda ^{VB}_{k}\right \}\) be the resulting mixture model composed of a total of K models, K=K ₁+K ₂+K ₃.

Let F(k,r)= logP r(O _r |λ _k ),1≤r≤R,1≤k≤K, be the log-likelihood matrix obtained by testing the R training sequences with the K models. Each column f _r of matrix F represents the feature vector of each sequence in the decision space (recall that f _r is a K-dimensional vector while O _r is a sequence of vector observations of length T). In other words, each column represents the confidences assigned by the K models to each sequence r. Therefore, the set of sequences \(\mathbb {O}=\{O_{r},y_{r}\}_{r=1}^{R}\) is mapped to a confidence space \(\{\textbf {f}_{r}, y_{r}\}_{r=1}^{R}\). Finally, a combination function, \(\mathbb {H}\), takes all the f _r ’s as input and outputs the final decision. The general framework for fusing the K outputs is highlighted in Algorithm ??.

Several decision level fusion techniques such as simple algebraic [29], artificial neural networks (ANN) [30], and hierarchical mixture of experts (HME) [31] can be used. In our work, we use an ANN with a single-layer perceptron and no hidden layers. The ANN weights are learned from the labeled training data using the backpropagation algorithm [32].

The architecture of the proposed eHMM is summarized in Fig. 1. It is composed of four main components: similarity matrix computation, relational clustering, adaptive training scheme, and decision level fusion. To test a new sequence, the outputs of the different models are aggregated into a single confidence value using Algorithm ??.

5.1 Data collections

The proposed eHMM was implemented and tested on GPR data collected with a NIITEK vehicle mounted GPR system [33] (see Fig. 4). This system collects 51 channels of data. Adjacent channels are spaced approximately five centimeters apart in the cross-track direction, and sequences (or scans) are taken at approximately 1-cm down-track intervals. Each A-scan, that is, the measured waveform that is collected in one channel at one down-track position, contains 416 time samples at which the GPR signal return is recorded. We often refer to the time index as depth although, since the radar wave is traveling through different media, this index does not represent a uniform sampling of depth. Thus, we model an entire collection of input data as a three-dimensional matrix of sample values, S(z,x,y),z=1,⋯,416;x=1,⋯,51;y=1,⋯,N _S , where N _S is the total number of collected scans, and the indices z, x, and y represent depth, cross-track position, and down-track positions respectively. A collection of scans, forming a volume of data, is illustrated in Fig. 5.

Figure 6 displays several B-scans (sequences of A-scans) both down-track (formed from a time sequence of A-scans from a single sensor channel) and cross-track (formed from each channel’s response in a single sample). The surveyed object position is highlighted in each figure. The objects scanned are (a) a high-metal content antitank mine, (b) a low-metal antipersonnel mine, and (c) a wood block.

Raw GPR data needs to be preprocessed and prescreened. Preprocessing includes ground-level alignment and signal and noise background removal. Prescreening is needed to focus attention and identify regions with subsurface anomalies. For this step, we use the adaptive least mean squares (LMS) prescreener [34]. The LMS flags locations of interest utilizing a computationally inexpensive algorithm so that more advanced algorithms can be applied only on the small subsets of data flagged by the prescreener.

5.2 Feature extraction

The goal of the feature extraction step is to transform original GPR data into a sequence of observation vectors. We use two types of features that have been proposed and used independently. Each feature represents a different interpretation of the raw data and aims at providing a good discrimination between mine and clutter signatures. These features are outlined in the following subsections.

5.3 Ensemble HMM implementation and results

In all experiments reported in this paper, we use a sixfold cross validation for each data collection \(\mathfrak {D_{l}}\), l∈{1,2,3}. For each fold, a subset of the data (\(\mathfrak {D_{l}}_{\textit {Trn}}\)) is used for training and the remaining data (\(\mathfrak {D_{l}}_{\textit {Tst}}\)) is used for testing. \(\mathfrak {O_{l}}_{\textit {Trn}}^{Feat}\) denotes the set of observation sequences extracted from dataset \(\mathfrak {D_{l}}\), using one of the feature extraction methods, “Feat” (EHD or Gabor).

The first step of the eHMM is the similarity matrix computation. This step requires fitting an individual HMM model for each sequence in the training data \(\mathfrak {O_{l}}_{\textit {Trn}}^{Feat}\). Figure 7 shows the log-likelihood and path mismatch penalty matrices for a training collection that has 521 mines and 1471 clutter signatures (first training fold of \(\mathfrak {D_{1}}\) using EHD features, \(\mathfrak {O_{1}}_{Trn1}^{EHD}\)). In these figures, the indices are rearranged so that the first entries correspond to the mine signatures and the latter ones correspond to non-mine signatures. As it can be seen, the matrices are composed mainly of four blocks. The diagonal blocks correspond to testing mine signatures in mine models and non-mine signatures in non-mine models, and the off-diagonal blocks correspond to testing mine signatures in non-mine models and non-mine signatures in mine models. In these figures, dark pixels correspond to small values of the log-likelihood or path mismatch penalty and bright pixels correspond to larger entries of the corresponding matrices. Note that in the case of the log-likelihood matrix in Fig. 7 a, the diagonal blocks are brighter than the off-diagonal blocks. This means that the signatures from the same class are more similar to each other than to signatures from different classes. Similarly, in the path mismatch penalty matrix of Fig. 7 b, the diagonal blocks are darker than the off-diagonal blocks. This means that when different mines are tested with each other models, the paths are similar. The above observations are trivial as alarms from the same class are expected to be more similar to each other than alarms from different classes. However, they could be used to validate our similarity (and penalty) measures in the log-likelihood space. A more important observation is that within each diagonal block, sub-blocks can be extracted. This is an indication of the existence of different clusters within the mines (and the clutter) themselves.

In the second step, the similarity matrix is transformed into a distance matrix, D, using (10). The hierarchical clustering algorithm [18] is then applied, using D with a fixed number of clusters K=10, to identify sub-categories within the training data. For both the discrete and continuous versions, using any of the features and datasets, the eHMM clustering step successfully assigns groups of similar alarms into clusters. For instance, in Fig. 9, we show the hierarchical clustering results of the first crossvalidation fold of the eCHMM using the EHD features on dataset \(\mathfrak {D_{1}}\). As it can be seen in Fig. 9 a, we have a group of clutter dominated clusters (in brown) and a second group of clusters dominated by mines (in blue). In Fig. 8, we show sample signatures that belong to clusters 1, 6, and 10. As it can be seen from Figs. 8 and 9 a, cluster 1 has only clutter and clusters 6 and 10 are composed exclusively of mine alarms. The mines that belong to cluster 6 have typically strong mine signatures. These mines, as shown in Fig. 9 b, c, are typically mines with high metal content that are buried at shallow depths. The mines that belong to cluster 10 have weak GPR signatures. These mines, as shown in Fig. 9 b, c, are typically mines with weak signatures that are either low metal mines or mines buried at deep depths.

Figure 10 shows the scatter plot of the confidences assigned by \(\lambda _{6}^{(M)}\) and \(\lambda _{10}^{(M)}\) to all the training data. In this figure, we display clutter and mine signatures that belong to each cluster using different symbols and colors. Even though the two models are dominated by mine signatures, we see that not all confidence values are highly correlated. On one hand, some strong mine signatures, particularly those belonging to cluster 6, have high log-likelihoods in model \(\lambda _{6}^{(M)}\) and lower log-likelihoods in model \(\lambda _{10}^{(M)}\) (lower right side of the scatter plot, region R1). This can be attributed to the fact that cluster 6 contains mainly strong mines and is more likely to yield high log-likelihood when testing a strong mine signature. On the other hand, in region R2, the performance of \(\lambda _{10}^{(M)}\) is better as it gives higher likelihood values to the “weak” mines in that region, particularly those belonging to cluster 10. In fact, this result is expected because cluster 10 contains weak mine signatures.

The main conclusion that we can draw from the above example is that \(\lambda _{6}^{(M)}\) and \(\lambda _{10}^{(M)}\) are very different and characterize two distinct subsets of the training data. The standard HMM approach would combine all alarms to learn a single model for mines (weak and strong) and a single model for clutter.

In the final step, the eHMM mixture is combined using a single-layer ANN. The ANN parameters are trained to fit the responses of the eHMM mixture models to the training data labels.

5.4 eHMM vs. baseline HMM results

In this section, we compare the performance of the proposed eHMM to the baseline HMM [5]. For the eHMM, we show the results using the ANN fusion and the hierarchical agglomerative clustering with K=20. In Fig. 11 a, b, we show the ROCs generated by the discrete versions of the eHMM and the baseline HMM on dataset \(\mathfrak {D_{1}}\), using EHD and Gabor features. Similarly, in Fig. 12 a, b, we report the ROCs generated by the continuous versions, i.e., the eCHMM and the baseline CHMM, on dataset \(\mathfrak {D_{1}}\). As it can be seen, in all the ROCs of Figs. 11 and 12, at a given false alarm rate (FAR), the eHMM has a better probability of detecting targets. For instance, in Fig. 11 a, at a FAR of 10 %, the eDHMM using EHD features successfully identifies 94 % of the mines while the baseline DHMM identifies only 87 % of the targets. At the same FAR of 10 %, the ROCs of Fig. 12 a show that the eCHMM successfully identifies 95 % of the targets while the baseline CHMM probability of detection is 85 %.