Two-stage source tracking method using a multiple linear regression model in the expanded phase domain

Time delay estimation (TDE) plays key role in determining the steering capability of microphone array system which produces a direction of the target sound source required for performing spatial processing. Typical applications of microphone array system include teleconferencing, automatic speech recognition, speech enhancement, source separation and automatic auditory system for robots [1–6].

The problem of estimating relative time delay associated with a signal source and a pair of spatially separated microphones has been extensively studied [7–15]. Among TDE methods, the generalized cross-correlation (GCC) method is one of the most widely used because of its simplicity and acceptable performance [7–9]. In the GCC-based method, the time delay is calculated by finding a lag that maximizes the GCC function between acquired signals. The method has been enhanced by introducing a pre-filter or a weighting function such as maximum-likelihood (ML), phase transform (PHAT) and so on. The GCC-ML method derived from the assumption of the ideal single propagation situation is optimal in a statistical point of view in case the observed sample space is large enough. The GCC-PHAT is recognized as reasonably robust to reverberation though it is heuristically designed. Zhang et al. [16] verified that the GCC-PHAT could actually be derived from the ML-based algorithm in reverberant environment if noise level is low. Another technique relied on the identification of the minimum of the average magnitude difference function (AMDF) between two signals, which was recently modified by joint consideration of the AMDF and the average magnitude sum function (AMSF) to improve the performance in reverberant environment [13].

An adaptive filter-based algorithm utilizes the criterion of minimizing the mean-square error between the first channel signal and the filtered second channel signal to estimate relative time delay [17]. In [18], an adaptive eigenvalue decomposition algorithm was proposed to improve TDE performance in reverberant environment. It first identified the room impulse response (RIR) of each channel, and then the delay was determined by finding the direct paths from the two measured RIRs. A systematic overview of the stat-of-the-art of TDE techniques was summarized in the recent literature [14].

The TDE method using the inter-channel phase difference (IPD) has been attracted a lot since 1980s, thanks to its advantage on obtaining the result instantaneously [19–23]. Chan et al. [19] verified that a least-square (LS) estimator to the phase slope of cross power spectrum was equivalent to the ML estimator. They also proved that the distribution of IPD error followed Gaussian probability density function (pdf) if the signal and noise were zero mean Gaussian processes and uncorrelated each other. By raising the coherence issue between dual-microphone noises, Piersol provided the relationship between spatial coherence function and phase bias at specific frequency. Brandstein et al. [20] proposed a generalized cost function of the linear regression model of IPD by adopting a bi-weight function [23]. The method is particularly advantageous in reverberant environment, but there is no benefit in noisy environment. The performance of these approaches commonly degrades when phase wrapping occurs or the phase is corrupted by adverse environmental effects because the phase statistics cannot be modeled by a simple pdf. Since it is hard to find an ideal estimator for a non-Gaussian data set such as wrapped discrete phases, a phase unwrapping process needs to be included in the TDE method [22, 24, 25]. Tribolet [24] proposed an iterative phase unwrapping algorithm that adaptively integrated the derivative of the phase. Brandstein et al. [22] practically implemented a linear regression slope forced unwrapping method which recursively adjusted the estimated wrapping frequency using lower band phase observations. Since these methods commonly include heuristic parts, their performance vary depending on how they are implemented. Recently, recursive unwrapping methods such as maximizing a posteriori probability or adopting the expectation-maximization (EM) using the probability model of the observed phase data set are introduced [26, 27]. In those methods, a reliable phase unwrapping can be achieved at the expense of heavy computational burden.

This article proposes a multiple linear regression model-based instantaneous TDE method that uses the expanded IPD of two channel signals. An estimator designed for operating in the original phase domain, [-π ~ + π), can hardly be optimal because a phase can be wrapped corresponding to the inter-channel distance and the direction of arrival (DOA) angle. To solve the problem, a reasonable statistical model for the distribution of IPD error and its Gaussian approximation are presented. At first, a phase domain expansion method using frequency interpolation and phase shifting methodology is proposed. Conventional linear regression model of IPD can be considered as a multiple linear regression model in the proposed phase expansion framework. By applying the proposed method to TDE, an ambiguous factor due to phase wrapping is dismissed and the LS method results in an optimal estimator. This article also verifies that the proposed estimation method becomes a minimum variance estimator (MVE) in the expanded phase domain. The proposed TDE method is composed of two stages: an LS-based TDE method estimates an initial delay at the first stage, and the estimated delay is applied to the sequential recursive-LS (RLS) estimator. The proposed method is computationally simple since it does not need a minimum or maximum search stage as well as the phase unwrapping process. The proposed algorithm is fairly compared with the optimal GCC methods, a generalized linear regression estimator, and an AMDF method in noisy and reverberant environment. The performance of the candidate estimators is evaluated by detailed assessment items including the percentage of anomalies, the estimation bias for both low and high DOA angles, and the root-mean-squared error (RMSE). Experimental results show that the proposed method can be regarded as the most robust estimator for the outliers and is closer to the unbiased estimator than any other methods. Especially in the RMSE assessment, the proposed RLS-TDE shows the best performance in both noisy and reverberant environment. Finally, the superiority of the tracking performance of the proposed algorithm is verified to a moving source in low SNR conditions.

The contents of the article are divided into four parts. Conventional two-channel TDE is explained in Section 2. Section 3 describes the details of the proposed phase expansion method with a multiple linear regression model. The proposed two-step TDE method for a moving speaker is described in Section 4. Finally, various experimental results are given in Section 5.

The multiple linear regression model-based LS method for IPD estimation is proposed in the expanded phase domain, Ω _d . The proposed method is composed of two stages: the multiple linear regression model-based LS-TDE at the first stage, and the RLS-based source tracking method using the delay information estimated at the first stage. After constructing an LS cost function for the TDE method based on the multiple linear regression model, it is verified that the proposed LS method is an ideal estimator which is unconstrained by phase wrapping. In the second stage, the RLS-TDE method is proposed which works very well for both fixed and moving source tracking. The proposed RLS method can be implemented by a simple equation, and it is also appropriate for conversational speech. Finally, a novel two-channel weighting method for noisy and reverberant environment is described.

4.3. Weighting for LS-TDE in noisy and reverberant condition

In Section 3.1, it is shown that the IPD error distribution can be regarded as Gaussian with variance (2 × SNR)^-1. Actually, this property is implied in the ML TDE explained in the Knapp's method [9] that the ML weighting is derived from MSC. Note that MSC can be regarded as an SNR of the input signal. In practice, MSC must be estimated by the observed data set using a temporal averaging method [33]. However, it is hard to estimate accurate MSC for non-stationary data such as speech signal. The proposed method adopts an approximated-ML weighting which is roughly equivalent to the SNR evaluated from a single frame as follows [12, 22, 23]:

$\psi \left({\omega }_k\right)=\frac{\left|X_1\left({\omega }_k\right)\right|\left|X_2\left({\omega }_k\right)\right|}{\left|N_1\left({\omega }_k\right){|}^2\right|X_2\left({\omega }_k\right){|}^2+\left|N_2\left({\omega }_k\right){|}^2\right|X_1\left({\omega }_k\right){|}^2}.$

(16)

The proposed LS-TDE in the expanded phase domain given in Equation 10 with the weighting function above satisfies all the ML estimation conditions, e.g., the Gaussian assumption of IPD error and weighting of its variance reciprocal. The weighting given in Equation 16 is useful when the coherence between two noises of dual-sensor and the target speech signal are ignor-able. However, it cannot distinguish values of speech from other signals if we assume a reverberant environment. Piersol [20] paid attention to the spatial coherence between two-sensors and proved the effects to the TDE by lots of experimental results, which are consistent with the theoretical analysis. To design a practical two channel system under the reverberant environment, a substitutable method which can suppress the reverberation effect by signal-to-reverberation (SRR)-based weighting is introduced.

To estimate the power of the direct signal and reverberant components, a two-channel generalized side-lobe canceller (GSC) structure is adopted [34]. Figure 5 shows a simplified block diagram to estimate the direct signal power. In this method, the power envelop of the delay-and-sum beamformer (DSB) output, Q(ω, n), and the delay-and-subtract output used for a reference signal, U(ω, n), are obtained by using the first-order recursive equations:

$\begin{array}{ll}\hfill {\lambda }_q\left(w,n\right)& =\eta {\lambda }_q\left(\omega ,n-1\right)+\left(1-\eta \right)|Q\left(\omega ,n\right){|}^2,\\ \hfill {\lambda }_u\left(\omega ,n\right)& =\eta {\lambda }_u\left(\omega ,n-1\right)+\left(1-\eta \right)|U\left(\omega ,n\right){|}^2,\end{array}$

(17)

where n is frame index and η is a forgetting factor set close to, but less than, one. Then, the energy of reverberant residual components, ${\widehat{\lambda }}_r\left(\omega ,n\right)$ is obtained as follows:

${\widehat{\lambda }}_r\left(\omega ,n\right)=W\left(\omega ,n\right){\lambda }_u\left(\omega ,n\right),$

(18)

where W(ω, n) is a frequency dependent gain that is adaptively updated using a quadratic cost function, J _w = {λ _e (ω, n)}², where the error, λ _e (ω, n), is equal to ${\lambda }_q\left(\omega ,n\right)-{\widehat{\lambda }}_r\left(\omega ,n\right)$. Finally, the direct signal power is estimated using a spectral-subtraction method [35]:

$|{Ŝ}_d\left(\omega ,n\right){|}^2=|Q\left(\omega ,n\right){|}^2-{\widehat{\lambda }}_r\left(\omega ,n\right).$

(19)

In Habets's de-reverberation method [34], a post filter is applied to the DSB output, Q(ω, d), however, the spectral subtraction method, given in Equation 19, is good enough in our application because only the power envelop of the direct signal component is needed. Finally, the SRR is represented as follows (omitting frame index similar to Equation 16):

$\psi \left({\omega }_k\right)=\frac{|{Ŝ}_d\left({\omega }_k\right){|}^2}{{\widehat{\lambda }}_r\left({\omega }_k\right)}.$

(20)

The proposed method well suppresses the late reverberation but has no impact on the early reflected component which is the principle reason of bias for the IPD distribution. The bias caused by early reflection entirely depends on the physical conditions including the shape of room, sensor and source position, etc. It is still a challenging research area to deal with the early reflection blindly.

where $G_{X_1{X}_2}\left(\omega \right)$ is the cross spectrum of two channel signal, $X_1\left(\omega \right)X_2^{*}\left(\omega \right)$. The GCC-ML, ψ _ML (ω _k ) given in Equation 16, and the phase transform (PHAT), ${\psi }_{PHAT}\left({\omega }_k\right)=|X_1\left(\omega \right)X_2^{*}\left(\omega \right){|}^{-1}$, are well-known estimators used for noisy and reverberant environments, respectively.

where ε is a fixed positive number to prevent division overflow. The TDE of the modified AMDF, Equation 24, is determined by jointly considering the AMDF and the AMSF. The three reference TDE estimators commonly include a maximum (or minimum) searching process which requires a large amount of computation while the proposed method instantly estimates the time delay with an intra-sample precision.

In the experiment, four conversational speech signals from four different speakers, two-males and two-females are included into the test. An energy ratio-based voice activity detection (VAD) is designed and same voice active intervals are applied to different SNR conditions. The noise PSD of cross spectrum signal gathered in silence intervals is used to calculate the weighting term given in Equation 10. It is also used to GCC-ML to minimize weighting effect. The relative performance of the TDE was evaluated through a number of trials in a simulated rectangular room (12 × 10 × 3 m³). The microphone array is located at (3,3,2) and the distance from the source to the array is maintained 3 m for both fixed and moving source scenarios. We tested eight locations of the fixed source at intervals of 10° from 0° to 70°. The room environment is artificially generated by the modified frequency domain image source model (ISM) with negative reflection coefficients [28, 29]. The reverberation time, T₆₀, is measured by Lehmann's energy decay curve (EDC) [28]. The level of the additive white Gaussian noise (WGN) varies from 5 to 25 dB as the reverberation time is increased from 0 to 500 ms. The sampling frequency is 8000 Hz, 64 ms Hamming window is applied with 50% overlap and the space of microphone is set to 8 cm.

5.1. Fixed source case in noisy and reverberant environments

At first, it is verified whether the actual distribution of the expanded phase follows Gaussian pdf. In Figure 6, the dotted-line depicts a histogram of the expanded phase and the dashed-line shows the IPD of observed signals in the original phase domain at 1500 Hz in 5 dB SNR condition when true IPD is +2π/3. The IPD distribution in the original phase domain (dashed-line) is not symmetric and also a number of phases is concentrated in erroneous IPD near -π region. The solid-line is the approximated Gaussian pdf of Equation 7 with +2π/3 mean and (2 × SNR)^-1 variance. It shows that the IPD of real data is slightly biased to zero direction but we can confirm that the proposed Gaussian assumption in the expanded phase domain is quite reasonable. Figure 7 depicts the IPD distribution in the relatively high reverberant environment. The variance of Gaussian pdf cannot be evaluated from the proposed model in reverberant condition, so we set an appropriate value. In reverberant condition, the actual IPD is more biased to zero than the noisy environment and this phenomenon entirely depends on room environment.

Figures 8, 9, 10 show the delay estimation results in various SNR conditions. The quantitative results by a percentage of outliers, bias and RMSE are evaluated to previously presented five techniques including the proposed LS and RLS methods. The bias and the RMSE are measured using the estimation result except for the outliers. In Figure 8, anomalies percentage is measured when the estimated time delay exceeded 20% of the overall delay range. The GCC-PHAT method shows the worst result, which has a number of severely erroneous estimation outliers in low SNR condition. The GCC-ML and the bi-weight method show similar performance while the bi-weight method shows slightly lower performance in low SNR environment. The AMDF method shows the best result among the comparing methods while it shows certain amount of outliers in low SNR conditions comparing to the proposed method. The proposed LS and RLS have similar anomalies percentage in the proposed algorithm structure, and both have superior performance to others such that the anomalies are suppressed less than 5% even in low SNR condition.

The trend of estimation bias is represented in Figure 9 which shows the results in the low DOA angle and high DOA angle cases separately. The phase of high DOA angle cases are commonly wrapped because the wrapping is occurred when the DOA angle is lager than 32^° in our simulation condition. All of the tested algorithms are hardly biased when the source is located in front direction of dual-sensor as depicted in Figure 9a because the phase wrapping is less likely to occur for a low DOA angle incident case. As shown in Figure 9b, however, the estimation bias for a signal from the high DOA angle generally increases. Since the bias problem becomes more serious when the IPD is getting closer to +π (or -π) as we described in Figures 6 and 7. The proposed algorithm working in the expanded phase domain, however, does not suffer from the bias especially in noisy environment.

The final estimation performance is presented in Figure 10 which depicts RMSE results of averaged whole DOA angles. It is confirmed that the proposed method has superior performance to conventional ones in overall SNR conditions. The proposed LS and the AMDF methods show better performance than the GCC-ML and the bi-weight method while the performance of the bi-weight method and the AMDF method decrease in low SNR condition. The GCC-PHAT shows the worst performance in noisy environment.

Figures 11, 12, 13 show the performance of the test algorithms in reverberant environments. The reverberant signal is synthesized using the ISM model including 20 dB WGN for all conditions. Figure 11 depicts the anomalies percentage in reverberant environments. The number of severe outlier is rapidly increased by increasing T60 for the GCC-ML method while the GCC-PHAT, the AMDF and the bi-weight methods show robust performance even to the long reverberation time. Especially, the AMDF method shows the highest outlier suppression performance that does not affect by the reverberation. From Figures 8, 11, it is clear that the performance of these methods is sensitively affected by noise but robust to reverberation because it is originally designed to targeting reverberant conditions. The outlier suppression performance of the proposed method is similar to the bi-weight method that the anomaly percentage is limited by 10% in reverberant environments.

The estimated bias represented in Figure 12 shows a different trend comparing to the result in noisy environment such that the bias can occur regardless of the DOA angle. The GCC-PHAT method shows the most robust performance irrespective of the reverberation level while its performance also slightly degrades in the high DOA angle case. The other methods except for the GCC-PHAT show that the estimation bias is larger than the result in noisy environment and it is highly affected by the RIR.

Finally, the estimation error except for anomalies is depicted in Figure 13. The GCC-ML method has a relatively small error in low reverberation condition but the error dramatically increases as the reverberation increases. Among the methods immune to reverberation, the AMDF method shows the best performance in overall conditions. As with the previous RMSE results in noisy environment, the proposed two-step method with the SRR-based weighting shows the most accurate TDE results in reverberant environment comparing to the other methods.

Overall, it is verified that the proposed method shows the highest performance especially in the noisy environments, i.e. it has minimum error and the estimation anomalies is less than 5% even in low SNR condition. It is also verified that unlikely to other methods, the proposed multiple linear regression model-based TDE method is not biased by phase wrapping. It also shows the most accurate TDE results in reverberant environments. The proposed method shows similar results to the AMDF method which shows the best performance among the reverberation immune methods in the anomalies percentage and the bias measurements.

5.2. Source tracking scenario for slow and fast moving sources

The tracking performance of the proposed method is verified when the target speech source moves around. A conversational speech of 30 s length is tested in low SNR condition (5 dB). The RLS parameters for slow and fast moving sources are set δ = 0.9, Q = 9 in 64 ms in 50% OLA frame work that the maximum length of observation frames is 352 ms and the minimum weighting for last frame is 0.9⁹. The TDE results of the proposed LS and RLS for slow moving speaker are depicted in Figure 14. From the result, it is also confirmed that the RLS estimation tracks the true TDE (solid-line) better than the first step of the proposed estimation, LS. Note that the proposed RLS can find the true TDE value even there exists a long silence interval. Identical experiments are carried out for fast moving source and the results are depicted in Figure 15. The tracking performance for fast moving source is also good though there are some failed estimations around 11 and 24 s when the source moves from the high DOA angle directions. The proposed RLS, however, adapts the true TDE very quickly and tracks the speaker again even in these cases. The trends of tracking performance of the proposed two-stage method and the conventional methods are presented in Figure 16. For a fair comparison, a smoothing technique is applied to the result of the conventional TDE using a exponentially decaying sliding window that has a same fashion to the second stage of the proposed method. The tracking performance of the conventional methods shows a similar trend to the previous result for a fixed speaker. The proposed two-stage method (RLS) shows the best result in accuracy and the performance does not be affected by the velocity of a moving source.

A LS TDE method based on the multiple linear regression model via the interpolated phase expansion has been proposed. By the proposed phase expansion method, the IPD distribution between two channel signals becomes more advantageous in terms of pdf. It theoretically verified that the approximated Gaussian approaches to the actual IPD distribution for higher SNR and also confirmed it by various experimental results. The proposed TDE method which is composed of two stages shows superior performance especially in the anomalies percentage and RMSE results in both noisy and reverberant environments. It was also demonstrated that the bias to zero problem for high DOA angles could be mitigated in the proposed method. Finally, the superiority of the proposed algorithm in terms of tracking a moving source in low SNR condition was verified. The proposed method provides the explicit TDE solution that can be applied to a real time application. Future work involves improving the method in reverberant environments based on detailed investigation about the IPD statistics for a multi-path effects.