A brief overview of speech enhancement with linear filtering

Abstract

In this paper, we provide an overview of some recently introduced principles and ideas for speech enhancement with linear filtering and explore how these are related and how they can be used in various applications. This is done in a general framework where the speech enhancement problem is stated as a signal vector estimation problem, i.e., with a filter matrix, where the estimate is obtained by means of a matrix-vector product of the filter matrix and the noisy signal vector. In this framework, minimum distortion, minimum variance distortionless response (MVDR), tradeoff, maximum signal-to-noise ratio (SNR), and Wiener filters are derived from the conventional speech enhancement approach and the recently introduced orthogonal decomposition approach. For each of the filters, we derive their properties in terms of output SNR and speech distortion. We then demonstrate how the ideas can be applied to single- and multichannel noise reduction in both the time and frequency domains as well as binaural noise reduction.

1 Review

1.1 Introduction

The problem of speech enhancement, or noise reduction as it is also sometimes called, is a well-known, longstanding problem with important applications in, for example, speech communication systems and hearing aids, where additive noise can, and often does, have a detrimental impact on the speech quality. Although the problem is a classical one and many solutions have been proposed throughout the years, it has arguably not been well-understood, even for the comparably simple case of linear filters. Indeed, it is not until quite recently that steps have been taken to accurately formulate the problem and characterize the desirable properties of possible solutions. Simply put, the performance of speech enhancement methods can be assessed in terms of two quantities, namely noise reduction and speech distortion, and an optimal solution to the speech enhancement would, thus, explicitly take both into account. As an example that this has historically not been done, consider the classical and well-known Wiener filter. It is usually derived from a mean-square error (MSE) criterion, and it is not until recently that its properties in terms of noise reduction and speech distortion have been thoroughly analyzed [1]. Although it was shown in [1] that the Wiener filter indeed improves the signal-to-noise ratio (SNR) (this had not been shown before), it does so without any explicit control over the amount of distortion incurred on the speech signal. As a result, other filters are worth considering. The minimum variance distortionless response (MVDR) filter, in principle, guarantees that no distortion is incurred on the speech signal while the noise is reduced as much as possible. The maximum SNR filter, on the other hand, yields the highest possible output SNR but does so at the cost of a considerable amount of speech distortion. Other competing methods to linear filtering include spectral subtraction methods [2], subspace methods [3, 4], and statistical methods [5–7].

In this paper, we continue the research into methods for speech enhancement based on linear filtering. More specifically, we provide a brief overview of linear filters derived from the conventional approach and the recently introduced orthogonal decomposition approach. We do so in a more general framework than what is typical. More specifically, the speech enhancement problem is stated as the problem of finding a rectangular filter matrix for estimating the speech signal vector from a noisy signal observation vector. Using the two aforementioned approaches, we derive the maximum SNR, Wiener, tradeoff, and MVDR filters and analyze and relate their properties. All of the derived filters are based on second-order statistics of the observed signal as well as the noise. While estimation of these statistics are not considered herein, there exist multiple, well-known methods for conducting this estimation in practice both in single-channel [8, 9] and multichannel [10–12] scenarios. Finally, we then proceed to demonstrate and discuss their application in various settings, including time and frequency domain enhancement and single- and multichannel enhancement.

The rest of the paper is organized as follows. In Section 1.2, we introduce the signal model and basic assumptions and state the problem at hand. We then, in Section 1.3, address the problem using the conventional approach, define various useful performance measures, and derive and compare some optimal filters. We then proceed to present an alternative approach based on the orthogonal decomposition in Section 1.4, and we use this to derive optimal filters. These are then also compared in terms of their noise reduction and speech distortion properties. In Section 1.5, we show how the two approaches can be applied in various speech enhancement contexts before concluding on the work in Section 2.

1.2 Signal model and problem formulation

We consider the very general signal model of an observation signal vector of length M:

$$\begin{array}{lcr}\mathbf{y}& =& {\left[\begin{array}{cccc}{y}_1& y_2& \cdots & y_M\end{array}\right]}^T\\ =& \mathbf{x}+\mathbf{v},\end{array}$$

(1)

where the superscript ^T is the transpose operator, and x and v are the speech and noise signal vectors, respectively, which are defined similarly to the noisy signal vector, y. We assume that the components of the two vectors x and v are zero mean, stationary, and circular. We further assume that these two vectors are uncorrelated, i.e., E(x v^H)=E(v x^H)=0_M×M, where E(·) denotes mathematical expectation, the superscript ^H is the conjugate-transpose operator, and 0_M×M is a matrix of size M×M with all its elements equal to 0. In this context, the correlation matrix (of size M×M) of the observations is:

$$\begin{array}{lcr}{\mathbf{\Phi }}_{\mathbf{y}}& =& E\left(\mathbf{y}{\mathbf{y}}^H\right)\hfill \\ =& {\mathbf{\Phi }}_{\mathbf{x}}+{\mathbf{\Phi }}_{\mathbf{v}},\end{array}$$

(2)

where Φ _x =E(x x^H) and Φ _v =E(v v^H) are the correlation matrices of x and v, respectively. In the rest of this paper, we assume that the rank of the speech correlation matrix, Φ _x , is equal to P≤M and the rank of the noise correlation matrix, Φ _v , is equal to M.

In order to be able to derive appropriate performance measures and optimal linear filters that can achieve a clear objective according to these measures, it is of great importance to define, without any ambiguity, the desired signal that we want to estimate or extract from the observations. Also, in general, y should be written explicitly as a function of this desired signal. In some context, x₁, the first element of x, is the desired signal; in some other situations, the whole vector x or part of it is the desired signal vector. Therefore, in a general manner, our desired signal vector is defined as:

$$\begin{array}{l}{\mathbf{x}}_Q={\left[\begin{array}{cccc}{x}_1& x_2& \cdots & x_Q\end{array}\right]}^T,\end{array}$$

(3)

where 1≤Q≤M. In the same way, we define the vector v _Q as the first Q components of v. Then, the objective of speech enhancement (or noise reduction) is to estimate x _Q from y. This should be done in such a way that the noise is reduced as much as possible with no or little distortion of the desired signal vector [1, 13–15]. In the rest of this study, we consider two important cases: without (conventional approach) and with the orthogonal decomposition of the speech signal vector.

1.3 Speech enhancement with the conventional approach

1.3.1 Principle

Our objective is to estimate x _Q from y, even though y is not an explicit function of x _Q . With linear filtering techniques [3, 4, 16–20], the desired signal vector is estimated as:

$$\begin{array}{lcr}\mathbf{z}& =& \mathbf{H}\mathbf{y}\hfill \\ =& \mathbf{H}\left(\mathbf{x}+\mathbf{v}\right)\hfill \\ =& {\mathbf{x}}_{\text{fd}}+{\mathbf{v}}_{\text{rn}},\end{array}$$

(4)

where z is supposed to be the estimate of x _Q ,

$$\begin{array}{l}\mathbf{H}=\left[\begin{array}{c}{\mathbf{h}}_1^H\\ {\mathbf{h}}_2^H\\ ⋮\\ {\mathbf{h}}_Q^H\end{array}\right]\end{array}$$

(5)

is a rectangular filtering matrix of size Q×M, h _q , q=1,2,…,Q are complex-valued filters of length M, x_fd=H x is the filtered desired signal, and v_rn=H v is the residual noise. We deduce that the correlation matrix of z is:

$$\begin{array}{l}{\mathbf{\Phi }}_{\mathbf{z}}={\mathbf{\Phi }}_{{\mathbf{x}}_{\text{fd}}}+{\mathbf{\Phi }}_{{\mathbf{v}}_{\text{rn}}},\end{array}$$

(6)

where ${\mathbf{\Phi }}_{{\mathbf{x}}_{\text{fd}}}=\mathbf{H}{\mathbf{\Phi }}_{\mathbf{x}}{\mathbf{H}}^H$ and ${\mathbf{\Phi }}_{{\mathbf{v}}_{\text{rn}}}=\mathbf{H}{\mathbf{\Phi }}_{\mathbf{v}}{\mathbf{H}}^H$.

An interesting particular case is Q=P=1. In this scenario, Equation 4 simplifies to:

$$\begin{array}{l}z={\mathbf{h}}^H\mathbf{y},\end{array}$$

(7)

where h is a complex-valued filter of length M. Since Φ _x is a rank 1 matrix, it can be written as:

$$\begin{array}{l}{\mathbf{\Phi }}_{\mathbf{x}}={\varphi }_{x_1}\mathbf{d}{\mathbf{d}}^H,\end{array}$$

(8)

where ${\varphi }_{x_1}=E\left({\left|x_1\right|}^2\right)$ is the variance of x₁ and d is a vector of length M, whose first element is equal to 1.

1.3.2 Performance measures

We are now ready to define the most important performance measures in the context of linear filtering.

The input SNR is defined as:

$$\begin{array}{l}\text{iSNR}=\frac{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_Q}\right)}{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{v}}_Q}\right)},\end{array}$$

(9)

where tr(·) denotes the trace of a square matrix, and ${\mathbf{\Phi }}_{{\mathbf{x}}_Q}$ and ${\mathbf{\Phi }}_{{\mathbf{v}}_Q}$ are the correlation matrices (of size Q×Q) of x _Q and v _Q , respectively.

The output SNR, obtained from Equation 6, helps quantify the SNR after filtering. It is given by:

$$\begin{array}{lcr}\text{oSNR}\left(\mathbf{H}\right)& =& \frac{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_{\text{fd}}}\right)}{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{v}}_{\text{rn}}}\right)}\hfill \\ =& \frac{\text{tr}\left(\mathbf{H}{\mathbf{\Phi }}_{\mathbf{x}}{\mathbf{H}}^H\right)}{\text{tr}\left(\mathbf{H}{\mathbf{\Phi }}_{\mathbf{v}}{\mathbf{H}}^H\right)}.\end{array}$$

(10)

Then, the main objective of speech enhancement is to find an appropriate H that makes the output SNR greater than the input SNR. Consequently, the quality of the noisy signal may be enhanced.

The noise reduction factor quantifies the amount of noise being rejected by H. This quantity is defined as the ratio of the power of the original noise over the power of the noise remaining after filtering, i.e.,

$$\begin{array}{l}{\xi }_{\text{nr}}\left(\mathbf{H}\right)=\frac{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{v}}_Q}\right)}{\text{tr}\left(\mathbf{H}{\mathbf{\Phi }}_{\mathbf{v}}{\mathbf{H}}^H\right)}.\end{array}$$

(11)

Any good choice of H should lead to ξ_nr(H)≥1, in which case the noise has been attenuated.

The desired speech signal can be distorted by the rectangular filtering matrix. Therefore, the speech reduction factor is defined as:

$$\begin{array}{l}{\xi }_{\text{sr}}\left(\mathbf{H}\right)=\frac{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_Q}\right)}{\text{tr}\left(\mathbf{H}{\mathbf{\Phi }}_{\mathbf{x}}{\mathbf{H}}^H\right)}.\end{array}$$

(12)

For optimal filters, we should have ξ_sr(H)≥1 as the optimal filter would otherwise amplify the desired signal.

By making the appropriate substitutions, one can derive the relationship among the measures defined so far, i.e.,

$$\begin{array}{l}\frac{\text{oSNR}\left(\mathbf{H}\right)}{\text{iSNR}}=\frac{{\xi }_{\text{nr}}\left(\mathbf{H}\right)}{{\xi }_{\text{sr}}\left(\mathbf{H}\right)}.\end{array}$$

(13)

This fundamental expression indicates the equivalence between gain/loss in SNR and distortion (for both speech and noise).

Another way to measure the distortion of the desired signal vector due to the filtering operation is via the speech distortion index defined as:

$$\begin{array}{l}{\upsilon }_{\text{sd}}\left(\mathbf{H}\right)=\frac{E\left[{\left({\mathbf{x}}_{\text{fd}}-{\mathbf{x}}_Q\right)}^H\left({\mathbf{x}}_{\text{fd}}-{\mathbf{x}}_Q\right)\right]}{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_Q}\right)}.\end{array}$$

(14)

The speech distortion index is always greater than or equal to 0 and should be upper bounded by 1 for optimal rectangular filtering matrices^a; so the higher the value of υ_sd(H) is, the more the desired signal is distorted.

We define the error signal vector between the estimated and desired signals as:

$$\begin{array}{lcr}\mathbf{e}& =& \mathbf{z}-{\mathbf{x}}_Q\hfill \\ =& \mathbf{H}\mathbf{y}-{\mathbf{x}}_Q,\end{array}$$

(15)

which can also be expressed as the sum of two uncorrelated error signal vectors:

$$\begin{array}{l}\mathbf{e}={\mathbf{e}}_{\text{ds}}+{\mathbf{e}}_{\text{rs}},\end{array}$$

(16)

where

$$\begin{array}{l}{\mathbf{e}}_{\text{ds}}={\mathbf{x}}_{\text{fd}}-{\mathbf{x}}_Q\end{array}$$

(17)

is the signal distortion due to the rectangular filtering matrix and

$$\begin{array}{l}{\mathbf{e}}_{\text{rs}}={\mathbf{v}}_{\text{rn}}\end{array}$$

(18)

represents the residual noise. Therefore, the MSE criterion is:

$$\begin{array}{ll}J\left(\mathbf{H}\right)& =\text{tr}\left[E\left(\mathbf{e}{\mathbf{e}}^H\right)\right]=\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_Q}\right)+\text{tr}\left(\mathbf{H}{\mathbf{\Phi }}_{\mathbf{y}}{\mathbf{H}}^H\right)\\ -\text{tr}\left(\mathbf{H}{\mathbf{\Phi }}_{\mathbf{x}}{\mathbf{I}}_{\mathrm{i}}^T\right)-\text{tr}\left({\mathbf{I}}_{\mathrm{i}}{\mathbf{\Phi }}_{\mathbf{x}}{\mathbf{H}}^H\right),\end{array}$$

(19)

where

$$\begin{array}{l}{\mathbf{I}}_{\mathrm{i}}=\left[\begin{array}{cc}{\mathbf{I}}_Q& {\mathbf{0}}_{Q\times (M-Q)}\end{array}\right]\end{array}$$

(20)

is the identity filtering matrix, with I _Q being the Q×Q identity matrix. Using the fact that $E\left({\mathbf{e}}_{\text{ds}}{\mathbf{e}}_{\text{rs}}^H\right)={\mathbf{0}}_{Q\times Q}$, J(H) can be expressed as the sum of two other MSEs, i.e.,

$$\begin{array}{lcr}J\left(\mathbf{H}\right)& =& \text{tr}\left[E\left({\mathbf{e}}_{\text{ds}}{\mathbf{e}}_{\text{ds}}^H\right)\right]+\text{tr}\left[E\left({\mathbf{e}}_{\text{rs}}{\mathbf{e}}_{\text{rs}}^H\right)\right]\hfill \\ =& J_{\text{ds}}\left(\mathbf{H}\right)+J_{\text{rs}}\left(\mathbf{H}\right),\hfill \end{array}$$

(21)

where

$$\begin{array}{l}{J}_{\text{ds}}\left(\mathbf{H}\right)=\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_Q}\right){\upsilon }_{\text{sd}}\left(\mathbf{H}\right)\end{array}$$

(22)

and

$$\begin{array}{l}{J}_{\text{rs}}\left(\mathbf{H}\right)=\frac{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{v}}_Q}\right)}{{\xi }_{\text{nr}}\left(\mathbf{H}\right)}.\end{array}$$

(23)

We deduce that

$$\begin{array}{lcr}\frac{J_{\text{ds}}\left(\mathbf{H}\right)}{J_{\text{rs}}\left(\mathbf{H}\right)}& =& \text{iSNR}\times {\xi }_{\text{nr}}\left(\mathbf{H}\right)\times {\upsilon }_{\text{sd}}\left(\mathbf{H}\right)\hfill \\ =& \text{oSNR}\left(\mathbf{H}\right)\times {\xi }_{\text{sr}}\left(\mathbf{H}\right)\times {\upsilon }_{\text{sd}}\left(\mathbf{H}\right).\hfill \end{array}$$

(24)

We observe how the MSEs are related to the different performance measures.

1.3.3 Optimal filters

Let λ_max be the maximum eigenvalue of the matrix ${\mathbf{\Phi }}_{\mathbf{v}}^{-1}{\mathbf{\Phi }}_{\mathbf{x}}$ with corresponding eigenvector b_max. It can be shown that the maximum SNR filtering matrix is given by [20]:

$$\begin{array}{l}{\mathbf{H}}_{\text{max}}=\left[\begin{array}{c}{\beta }_1{\mathbf{b}}_{\text{max}}^T\\ {\beta }_2{\mathbf{b}}_{\text{max}}^T\\ ⋮\\ {\beta }_Q{\mathbf{b}}_{\text{max}}^T\end{array}\right],\end{array}$$

(25)

where β _q , q=1,2,…,Q are arbitrary complex numbers with at least one of them different from 0. The corresponding output SNR is:

$$\begin{array}{l}\text{oSNR}\left({\mathbf{H}}_{\text{max}}\right)={\lambda }_{\text{max}}.\end{array}$$

(26)

The output SNR with the maximum SNR filtering matrix is always greater than or equal to the input SNR, i.e., oSNR(H_max)≥iSNR. We also have oSNR(H)≤λ_max,∀H. The best way to find the β _q s is by minimizing distortion. By substituting H_max into the distortion-based MSE and minimizing with respect to the β _q s, we get

$$\begin{array}{lcr}{\mathbf{H}}_{\text{max}}& =& {\mathbf{I}}_{\mathrm{i}}{\mathbf{\Phi }}_{\mathbf{x}}\frac{{\mathbf{b}}_{\text{max}}{\mathbf{b}}_{\text{max}}^H}{{\lambda }_{\text{max}}}\\ =& {\mathbf{I}}_{\mathrm{i}}{\mathbf{\Phi }}_{\mathbf{v}}{\mathbf{b}}_{\text{max}}{\mathbf{b}}_{\text{max}}^H.\end{array}$$

(27)

If we differentiate the MSE criterion, J(H), with respect to H and equate the result to zero, we find the Wiener filtering matrix:

$$\begin{array}{lcr}{\mathbf{H}}_{\mathrm{W}}& =& {\mathbf{I}}_{\mathrm{i}}{\mathbf{\Phi }}_{\mathbf{x}}{\mathbf{\Phi }}_{\mathbf{y}}^{-1}\hfill \\ =& {\mathbf{I}}_{\mathrm{i}}\left({\mathbf{I}}_M-{\mathbf{\Phi }}_{\mathbf{v}}{\mathbf{\Phi }}_{\mathbf{y}}^{-1}\right),\end{array}$$

(28)

where I _M is the M×M identity matrix. The output SNR with the Wiener filtering matrix is always greater than or equal to the input SNR, i.e., oSNR(H_W)≥iSNR. Obviously, we have

$$\begin{array}{l}\text{oSNR}\left({\mathbf{H}}_{\mathrm{W}}\right)\le \text{oSNR}\left({\mathbf{H}}_{\text{max}}\right)\end{array}$$

(29)

and, in general,

$$\begin{array}{l}{\upsilon }_{\text{sd}}\left({\mathbf{H}}_{\mathrm{W}}\right)\le {\upsilon }_{\text{sd}}\left({\mathbf{H}}_{\text{max}}\right).\end{array}$$

(30)

To better compromise between noise reduction and speech distortion, we can minimize the speech distortion index with the constraint that the noise reduction factor is equal to a positive value that is greater than 1, i.e.,

$$\begin{array}{l}\underset{\mathbf{H}}{min}{J}_{\text{ds}}\left(\mathbf{H}\right)\mathrm{subject}\mathit{\text{to}}{J}_{\text{rs}}\left(\mathbf{H}\right)=\beta \text{tr}\left({\mathbf{\Phi }}_{{\mathbf{v}}_Q}\right),\end{array}$$

(31)

where 0<β<1 to insure that we get some noise reduction. The previous optimization leads to the tradeoff filter:

$$\begin{array}{l}{\mathbf{H}}_{\mathrm{T},\mu }={\mathbf{I}}_{\mathrm{i}}{\mathbf{\Phi }}_{\mathbf{x}}{\left({\mathbf{\Phi }}_{\mathbf{x}}+\mu {\mathbf{\Phi }}_{\mathbf{v}}\right)}^{-1},\end{array}$$

(32)

where μ>0 is a Lagrange multiplier. The output SNR with the tradeoff filtering matrix is always greater than or equal to the input SNR, i.e., oSNR(H_T,μ)≥iSNR, ∀μ>0. Usually, μ is chosen in a heuristic way, so that for

• μ=1, H_T,1=H_W, which is the Wiener filtering matrix;

• μ>1, results in a filtering matrix with low residual noise at the expense of high speech distortion (as compared to Wiener); and

• μ<1, results in a filtering matrix with high residual noise and low speech distortion (as compared to Wiener).

We should have for μ≥1,

$$\begin{array}{l}\text{oSNR}\left({\mathbf{H}}_{\mathrm{W}}\right)\le \text{oSNR}\left({\mathbf{H}}_{\mathrm{T},\mu }\right)\le \text{oSNR}\left({\mathbf{H}}_{\text{max}}\right),\end{array}$$

(33)

$$\begin{array}{l}{\upsilon }_{\text{sd}}\left({\mathbf{H}}_{\mathrm{W}}\right)\le {\upsilon }_{\text{sd}}\left({\mathbf{H}}_{\mathrm{T},\mu }\right),\end{array}$$

(34)

and for μ≤1,

$$\begin{array}{l}\text{oSNR}\left({\mathbf{H}}_{\mathrm{T},\mu }\right)\le \text{oSNR}\left({\mathbf{H}}_{\mathrm{W}}\right)\le \text{oSNR}\left({\mathbf{H}}_{\text{max}}\right),\end{array}$$

(35)

$$\begin{array}{l}{\upsilon }_{\text{sd}}\left({\mathbf{H}}_{\mathrm{T},\mu }\right)\le {\upsilon }_{\text{sd}}\left({\mathbf{H}}_{\mathrm{W}}\right).\end{array}$$

(36)

Another filter can be derived by just minimizing J_ds(H). We obtain the minimum distortion (MD) rectangular filtering matrix:

$$\begin{array}{l}{\mathbf{H}}_{\text{MD}}={\mathbf{I}}_{\mathrm{i}}{\mathbf{\Phi }}_{\mathbf{x}}{\mathbf{\Phi }}_{\mathbf{x}}^{†},\end{array}$$

(37)

where ${\mathbf{\Phi }}_{\mathbf{x}}^{†}$ is the pseudoinverse of Φ _x . If Φ _x is a full-rank matrix, H_MD becomes the identity filter, I_i, which does not affect the observations. The MD filter is very close to the well-known MVDR filter.

For Q=P=1, it is possible to derive the MVDR filter. Indeed, by minimizing the variance of the filter’s output, ϕ _z =h^HΦ _y h, or the variance of the residual noise, ${\varphi }_{v_{\text{rn}}}={\mathbf{h}}^H{\mathbf{\Phi }}_{\mathbf{v}}\mathbf{h}$, subject to the distortionless constraint, h^Hd=1, we easily get

$$\begin{array}{lcr}{\mathbf{h}}_{\text{MVDR}}& =& \frac{{\mathbf{\Phi }}_{\mathbf{y}}^{-1}\mathbf{d}}{{\mathbf{d}}^H{\mathbf{\Phi }}_{\mathbf{y}}^{-1}\mathbf{d}}\\ =& \frac{{\mathbf{\Phi }}_{\mathbf{v}}^{-1}\mathbf{d}}{{\mathbf{d}}^H{\mathbf{\Phi }}_{\mathbf{v}}^{-1}\mathbf{d}}.\end{array}$$

(38)

It can be checked that J_ds(h_MVDR)=0, proving that h_MVDR is distortionless.

It is also possible to derive the MVDR (square) filtering matrix for Q=M. Using the well-known eigenvalue decomposition [21], the speech correlation matrix can be diagonalized as:

$$\begin{array}{l}{\mathbf{Q}}_{\mathbf{x}}^H{\mathbf{\Phi }}_{\mathbf{x}}{\mathbf{Q}}_{\mathbf{x}}={\mathbf{\Lambda }}_{\mathbf{x}},\end{array}$$

(39)

where

$$\begin{array}{l}{\mathbf{Q}}_{\mathbf{x}}=\left[\begin{array}{cccc}{\mathbf{q}}_{\mathbf{x},1}& {\mathbf{q}}_{\mathbf{x},2}& \cdots & {\mathbf{q}}_{\mathbf{x},M}\end{array}\right]\end{array}$$

(40)

is a unitary matrix, i.e., ${\mathbf{Q}}_{\mathbf{x}}^H{\mathbf{Q}}_{\mathbf{x}}={\mathbf{Q}}_{\mathbf{x}}{\mathbf{Q}}_{\mathbf{x}}^H={\mathbf{I}}_M$ and

$$\begin{array}{l}{\mathbf{\Lambda }}_{\mathbf{x}}=\text{diag}\left({\lambda }_{\mathbf{x},1},{\lambda }_{\mathbf{x},2},\dots ,{\lambda }_{\mathbf{x},M}\right)\end{array}$$

(41)

is a diagonal matrix. The orthonormal vectors q_x,1,q_x,2,…,q_x,M are the eigenvectors corresponding, respectively, to the eigenvalues λ_x,1,λ_x,2,…,λ_x,M of the matrix Φ _x , where λ_x,1≥λ_x,2≥⋯≥λ_x,P>0 and λ_x,P+1=λ_x,P+2=⋯=λ_x,M=0. Let

$$\begin{array}{l}{\mathbf{Q}}_{\mathbf{x}}=\left[\begin{array}{cc}{\mathbf{T}}_{\mathbf{x}}& {\mathbf{\Xi }}_{\mathbf{x}}\end{array}\right],\end{array}$$

(42)

where the M×P matrix T _x contains the eigenvectors corresponding to the nonzero eigenvalues of Φ _x and the M×(M−P) matrix Ξ _x contains the eigenvectors corresponding to the null eigenvalues of Φ _x . It can be verified that

$$\begin{array}{l}{\mathbf{I}}_M={\mathbf{T}}_{\mathbf{x}}{\mathbf{T}}_{\mathbf{x}}^H+{\mathbf{\Xi }}_{\mathbf{x}}{\mathbf{\Xi }}_{\mathbf{x}}^H.\end{array}$$

(43)

Notice that ${\mathbf{T}}_{\mathbf{x}}{\mathbf{T}}_{\mathbf{x}}^H$ and ${\mathbf{\Xi }}_{\mathbf{x}}{\mathbf{\Xi }}_{\mathbf{x}}^H$ are two orthogonal projection matrices of rank P and M−P, respectively. Hence, ${\mathbf{T}}_{\mathbf{x}}{\mathbf{T}}_{\mathbf{x}}^H$ is the orthogonal projector onto the speech subspace (where all the energy of the speech signal is concentrated), or the range of Φ _x and ${\mathbf{\Xi }}_{\mathbf{x}}{\mathbf{\Xi }}_{\mathbf{x}}^H$ is the orthogonal projector onto the null subspace of Φ _x . Using Equation 43, we can write the speech vector as:

$$\begin{array}{lcr}\mathbf{x}& =& {\mathbf{Q}}_{\mathbf{x}}{\mathbf{Q}}_{\mathbf{x}}^H\mathbf{x}\\ =& {\mathbf{T}}_{\mathbf{x}}{\mathbf{T}}_{\mathbf{x}}^H\mathbf{x}.\end{array}$$

(44)

We deduce from Equation 44 that the distortionless constraint is:

$$\begin{array}{l}\mathbf{H}{\mathbf{T}}_{\mathbf{x}}={\mathbf{T}}_{\mathbf{x}},\end{array}$$

(45)

since, in this case, $\mathbf{H}\mathbf{x}=\mathbf{H}{\mathbf{T}}_{\mathbf{x}}{\mathbf{T}}_{\mathbf{x}}^H\mathbf{x}={\mathbf{T}}_{\mathbf{x}}{\mathbf{T}}_{\mathbf{x}}^H\mathbf{x}=\mathbf{x}$. Now, from the criterion:

$$\begin{array}{l}\underset{\mathbf{H}}{min}\text{tr}\left(\mathbf{H}{\mathbf{\Phi }}_{\mathbf{v}}{\mathbf{H}}^H\right)\mathrm{subject}\mathit{\text{to}}\mathbf{H}{\mathbf{T}}_{\mathbf{x}}={\mathbf{T}}_{\mathbf{x}},\end{array}$$

(46)

we find the MVDR:

$$\begin{array}{l}{\mathbf{H}}_{\text{MVDR}}={\mathbf{T}}_{\mathbf{x}}{\left({\mathbf{T}}_{\mathbf{x}}^H{\mathbf{\Phi }}_{\mathbf{v}}^{-1}{\mathbf{T}}_{\mathbf{x}}\right)}^{-1}{\mathbf{T}}_{\mathbf{x}}^H{\mathbf{\Phi }}_{\mathbf{v}}^{-1}.\end{array}$$

(47)

Equation 47 can also be expressed as:

$$\begin{array}{l}{\mathbf{H}}_{\text{MVDR}}={\mathbf{T}}_{\mathbf{x}}{\left({\mathbf{T}}_{\mathbf{x}}^H{\mathbf{\Phi }}_{\mathbf{y}}^{-1}{\mathbf{T}}_{\mathbf{x}}\right)}^{-1}{\mathbf{T}}_{\mathbf{x}}^H{\mathbf{\Phi }}_{\mathbf{y}}^{-1}.\end{array}$$

(48)

Of course, for P=M, the MVDR filtering matrix simplifies to the identity matrix, i.e., H_MVDR=I _M . As a consequence, we can state that the higher the dimension of the null space of Φ _x is, the more the MVDR is efficient in terms of noise reduction. The best scenario corresponds to P=1. We can verify that J_ds(H_MVDR)=0.

1.4 Speech enhancement with the orthogonal decomposition of the speech signal vector

1.4.1 Principle

Another perspective for speech enhancement is to extract the desired signal vector, x _Q , from x. This way, the observation signal vector, y, will be an explicit function of x _Q . As a consequence, the objectives that we wish to achieve are much easier to handle.

In this section, we assume that the elements x _q , q=1,2,…,Q are not fully coherent, so that ${\mathbf{\Phi }}_{{\mathbf{x}}_Q}$ is a full-rank matrix. To extract x _Q from x, we need to decompose x into two orthogonal components: one correlated with (or is a linear transformation of) the desired signal vector and the other one orthogonal to x _Q and, hence, will be considered as an interference signal vector. Specifically, the vector x is decomposed into the following form [22, 23]:

$$\begin{array}{lcr}\mathbf{x}& =& {\mathbf{\Phi }}_{\mathbf{x}{\mathbf{x}}_Q}{\mathbf{\Phi }}_{{\mathbf{x}}_Q}^{-1}{\mathbf{x}}_Q+{\mathbf{x}}_{\mathrm{i}}\\ =& {\mathbf{x}}_{\mathrm{d}}+{\mathbf{x}}_{\mathrm{i}},\hfill \end{array}$$

(49)

where

$$\begin{array}{lcr}{\mathbf{x}}_{\mathrm{d}}& =& {\mathbf{\Phi }}_{\mathbf{x}{\mathbf{x}}_Q}{\mathbf{\Phi }}_{{\mathbf{x}}_Q}^{-1}{\mathbf{x}}_Q\\ =& {\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}{\mathbf{x}}_Q\hfill \end{array}$$

(50)

is a linear transformation of the desired signal vector, ${\mathbf{\Phi }}_{\mathbf{x}{\mathbf{x}}_Q}=E\left(\mathbf{x}{\mathbf{x}}_Q^H\right)$ is the cross-correlation matrix (of size M×Q) between x and x _Q , ${\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}={\mathbf{\Phi }}_{\mathbf{x}{\mathbf{x}}_Q}{\mathbf{\Phi }}_{{\mathbf{x}}_Q}^{-1}$, and

$$\begin{array}{l}{\mathbf{x}}_{\mathrm{i}}=\mathbf{x}-{\mathbf{x}}_{\mathrm{d}}\end{array}$$

(51)

is the interference signal vector. It is easy to see that x_d and x_i are orthogonal^b, i.e.,

$$\begin{array}{l}E\left({\mathbf{x}}_{\mathrm{d}}{\mathbf{x}}_{\mathrm{i}}^H\right)={\mathbf{0}}_{M\times M}.\end{array}$$

(52)

We observe that the first Q elements of x_d and x_i are equal to x _Q and 0_Q×1, respectively. Now, we can express the observation signal vector as an explicit function of x _Q , i.e.,

$$\begin{array}{l}\mathbf{y}={\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}{\mathbf{x}}_Q+{\mathbf{x}}_{\mathrm{i}}+\mathbf{v}.\end{array}$$

(53)

With this approach, the estimator is:

$$\begin{array}{lcr}{\mathbf{z}}^{\prime }& =& {\mathbf{H}}^{\prime }\left[{\mathbf{x}}_{\mathrm{d}}+{\mathbf{x}}_{\mathrm{i}}+\mathbf{v}\right]\\ =& {\mathbf{x}}_{\text{fd}}^{\prime }+{\mathbf{x}}_{\text{ri}}^{\prime }+{\mathbf{v}}_{\text{rn}}^{\prime },\end{array}$$

(54)

where

$$\begin{array}{l}{\mathbf{H}}^{\prime }=\left[\begin{array}{c}{\mathbf{h}}_1^{\mathrm{\prime H}}\\ {\mathbf{h}}_2^{\mathrm{\prime H}}\\ ⋮\\ {\mathbf{h}}_Q^{\mathrm{\prime H}}\end{array}\right]\end{array}$$

(55)

is a rectangular filtering matrix of size Q×M, ${\mathbf{h}}_q^{\prime },q=1,2,\dots ,Q$ are complex-valued filters of length M, ${\mathbf{x}}_{\text{fd}}^{\prime }={\mathbf{H}}^{\prime }{\mathbf{x}}_{\mathrm{d}}$ is the filtered desired signal, ${\mathbf{x}}_{\text{ri}}^{\prime }={\mathbf{H}}^{\prime }{\mathbf{x}}_{\mathrm{i}}$ is the residual interference, and ${\mathbf{v}}_{\text{rn}}^{\prime }={\mathbf{H}}^{\prime }\mathbf{v}$ is the residual noise. The correlation matrix of z^′ is then:

$$\begin{array}{l}{\mathbf{\Phi }}_{{\mathbf{z}}^{\prime }}={\mathbf{\Phi }}_{{\mathbf{x}}_{\text{fd}}^{\prime }}+{\mathbf{\Phi }}_{{\mathbf{x}}_{\text{ri}}^{\prime }}+{\mathbf{\Phi }}_{{\mathbf{v}}_{\text{rn}}^{\prime }},\end{array}$$

(56)

where ${\mathbf{\Phi }}_{{\mathbf{x}}_{\text{fd}}^{\prime }}={\mathbf{H}}^{\prime }{\mathbf{\Phi }}_{{\mathbf{x}}_{\mathrm{d}}}{\mathbf{H}}^{\mathrm{\prime H}}$, with ${\mathbf{\Phi }}_{{\mathbf{x}}_{\mathrm{d}}}={\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}{\mathbf{\Phi }}_{{\mathbf{x}}_Q}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H$ being the correlation matrix (whose rank is equal to Q) of x_d, ${\mathbf{\Phi }}_{{\mathbf{x}}_{\text{ri}}^{\prime }}={\mathbf{H}}^{\prime }{\mathbf{\Phi }}_{{\mathbf{x}}_{\mathrm{i}}}{\mathbf{H}}^{\mathrm{\prime H}}$, with ${\mathbf{\Phi }}_{{\mathbf{x}}_{\mathrm{i}}}=E\left({\mathbf{x}}_{\mathrm{i}}{\mathbf{x}}_{\mathrm{i}}^H\right)$ being the correlation matrix of x_i, and ${\mathbf{\Phi }}_{{\mathbf{v}}_{\text{rn}}^{\prime }}={\mathbf{H}}^{\prime }{\mathbf{\Phi }}_{\mathbf{v}}{\mathbf{H}}^{\mathrm{\prime H}}$.

1.4.2 Performance measures

The input SNR is identical to the definition given in Equation 9.

From Equation 56, we deduce the output SNR:

$$\begin{array}{lcr}\text{oSNR}\left({\mathbf{H}}^{\prime }\right)& =& \frac{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_{\text{fd}}^{\prime }}\right)}{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_{\text{ri}}^{\prime }}+{\mathbf{\Phi }}_{{\mathbf{v}}_{\text{rn}}^{\prime }}\right)}\hfill \\ =& \frac{\text{tr}\left({\mathbf{H}}^{\prime }{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}{\mathbf{\Phi }}_{{\mathbf{x}}_Q}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{H}}^{\mathrm{\prime H}}\right)}{\text{tr}\left({\mathbf{H}}^{\prime }{\mathbf{\Phi }}_{\text{in}}{\mathbf{H}}^{\mathrm{\prime H}}\right)},\hfill \end{array}$$

(57)

where

$$\begin{array}{l}{\mathbf{\Phi }}_{\text{in}}={\mathbf{\Phi }}_{{\mathbf{x}}_{\mathrm{i}}}+{\mathbf{\Phi }}_{\mathbf{v}}\end{array}$$

(58)

is the correlation matrix of the interference-plus-noise. The obvious objective is to find an appropriate H^′ in such a way that oSNR(H^′)≥iSNR.

The noise reduction factor is:

$$\begin{array}{l}{\xi }_{\text{nr}}\left({\mathbf{H}}^{\prime }\right)=\frac{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{v}}_Q}\right)}{\text{tr}\left({\mathbf{H}}^{\prime }{\mathbf{\Phi }}_{\text{in}}{\mathbf{H}}^{\mathrm{\prime H}}\right)}.\end{array}$$

(59)

A reasonable choice of H^′ should give a value of the noise reduction factor greater than 1, meaning that the noise and interference have been attenuated by the filter.

The speech reduction factor is defined as:

$$\begin{array}{l}{\xi }_{\text{sr}}\left({\mathbf{H}}^{\prime }\right)=\frac{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_Q}\right)}{\text{tr}\left({\mathbf{H}}^{\prime }{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}{\mathbf{\Phi }}_{{\mathbf{x}}_Q}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{H}}^{\mathrm{\prime H}}\right)}.\end{array}$$

(60)

A rectangular filtering matrix that does not affect the desired signal requires the constraint^c:

$$\begin{array}{l}{\mathbf{H}}^{\prime }{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}={\mathbf{I}}_Q.\end{array}$$

(61)

Hence, ξ_sr(H^′)=1 in the absence of (correlated) distortion and ξ_sr(H^′)>1 in the presence of distortion.

Again, we have the fundamental relationship:

$$\begin{array}{l}\frac{\text{oSNR}\left({\mathbf{H}}^{\prime }\right)}{\text{iSNR}}=\frac{{\xi }_{\text{nr}}\left({\mathbf{H}}^{\prime }\right)}{{\xi }_{\text{sr}}\left({\mathbf{H}}^{\prime }\right)}.\end{array}$$

(62)

When no distortion occurs, the gain in SNR coincides with the noise reduction factor.

We can also quantify the distortion with the speech distortion index:

$$\begin{array}{l}{\upsilon }_{\text{sd}}\left({\mathbf{H}}^{\prime }\right)=\frac{\text{tr}\left[\left({\mathbf{H}}^{\prime }{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}-{\mathbf{I}}_Q\right){\mathbf{\Phi }}_{{\mathbf{x}}_Q}{\left({\mathbf{H}}^{\prime }{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}-{\mathbf{I}}_Q\right)}^H\right]}{\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_Q}\right)}.\end{array}$$

(63)

The speech distortion index is always greater than or equal to 0 and should be upper bounded by 1 for optimal filtering matrices, which corresponds to the case where the filtering matrix is just a matrix of zeros; so the higher the value of υ_sd(H^′) is, the more the desired signal is distorted.

The error signal is:

$$\begin{array}{lcr}{\mathbf{e}}^{\prime }& =& {\mathbf{z}}^{\prime }-{\mathbf{x}}_Q\hfill \\ =& {\mathbf{H}}^{\prime }\mathbf{y}-{\mathbf{x}}_Q.\end{array}$$

(64)

It can be written as the sum of two orthogonal error signal vectors:

$$\begin{array}{l}{\mathbf{e}}^{\prime }={\mathbf{e}}_{\text{ds}}^{\prime }+{\mathbf{e}}_{\text{rs}}^{\prime },\end{array}$$

(65)

where

$$\begin{array}{lcr}{\mathbf{e}}_{\text{ds}}^{\prime }& =& {\mathbf{x}}_{\text{fd}}^{\prime }-{\mathbf{x}}_Q\hfill \\ =& \left({\mathbf{H}}^{\prime }{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}-{\mathbf{I}}_Q\right){\mathbf{x}}_Q\hfill \end{array}$$

(66)

is the signal distortion due to the rectangular filtering matrix and

$$\begin{array}{lcr}{\mathbf{e}}_{\text{rs}}^{\prime }& =& {\mathbf{x}}_{\text{ri}}^{\prime }+{\mathbf{v}}_{\text{rn}}^{\prime }\\ =& {\mathbf{H}}^{\prime }{\mathbf{x}}_{\mathrm{i}}+{\mathbf{H}}^{\prime }\mathbf{v}\end{array}$$

(67)

represents the residual interference-plus-noise. Having defined the error signal, we can now write the MSE criterion:

$$\begin{array}{ll}J\left({\mathbf{H}}^{\prime }\right)& =\text{tr}\left[E\left({\mathbf{e}}^{\prime }{\mathbf{e}}^{\mathrm{\prime H}}\right)\right]\\ =\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_Q}\right)+\text{tr}\left({\mathbf{H}}^{\prime }{\mathbf{\Phi }}_{\mathbf{y}}{\mathbf{H}}^{\mathrm{\prime H}}\right)-\text{tr}\left({\mathbf{H}}^{\prime }{\mathbf{\Phi }}_{\mathbf{x}}{\mathbf{I}}_{\mathrm{i}}^T\right)\\ -\text{tr}\left({\mathbf{I}}_{\mathrm{i}}{\mathbf{\Phi }}_{\mathbf{x}}{\mathbf{H}}^{\mathrm{\prime H}}\right)\\ =J_{\text{ds}}\left({\mathbf{H}}^{\prime }\right)+J_{\text{rs}}\left({\mathbf{H}}^{\prime }\right),\end{array}$$

(68)

where

$$\begin{array}{ll}{J}_{\text{ds}}\left({\mathbf{H}}^{\prime }\right)& =\text{tr}\left({\mathbf{\Phi }}_{{\mathbf{x}}_Q}\right)+\text{tr}\left({\mathbf{H}}^{\prime }{\mathbf{\Phi }}_{{\mathbf{x}}_{\mathrm{d}}}{\mathbf{H}}^{\mathrm{\prime H}}\right)-\text{tr}\left({\mathbf{H}}^{\prime }{\mathbf{\Phi }}_{{\mathbf{x}}_{\mathrm{d}}}{\mathbf{I}}_{\mathrm{i}}^T\right)\\ -\text{tr}\left({\mathbf{I}}_{\mathrm{i}}{\mathbf{\Phi }}_{{\mathbf{x}}_{\mathrm{d}}}{\mathbf{H}}^{\mathrm{\prime H}}\right)\end{array}$$

(69)

and

$$\begin{array}{l}{J}_{\text{rs}}\left({\mathbf{H}}^{\prime }\right)={\mathbf{H}}^{\prime }{\mathbf{\Phi }}_{\text{in}}{\mathbf{H}}^{\mathrm{\prime H}}.\end{array}$$

(70)

We deduce that

$$\begin{array}{lcr}\frac{J_{\text{ds}}\left({\mathbf{H}}^{\prime }\right)}{J_{\text{rs}}\left({\mathbf{H}}^{\prime }\right)}& =& \text{iSNR}\times {\xi }_{\text{nr}}\left({\mathbf{H}}^{\prime }\right)\times {\upsilon }_{\text{sd}}\left({\mathbf{H}}^{\prime }\right)\hfill \\ =& \text{oSNR}\left({\mathbf{H}}^{\prime }\right)\times {\xi }_{\text{sr}}\left({\mathbf{H}}^{\prime }\right)\times {\upsilon }_{\text{sd}}\left({\mathbf{H}}^{\prime }\right),\end{array}$$

(71)

showing how the MSEs are related to the different performance measures.

1.4.3 Optimal filters

Let ${\lambda }_{\text{max}}^{\prime }$ be the maximum eigenvalue of the matrix ${\mathbf{\Phi }}_{\text{in}}^{-1}{\mathbf{\Phi }}_{{\mathbf{x}}_{\mathrm{d}}}$ with corresponding eigenvector ${\mathbf{b}}_{\text{max}}^{\prime }$. We easily find that the maximum SNR filtering matrix with minimum distortion is:

$$\begin{array}{lcr}{\mathbf{H}}_{\text{max}}^{\prime }& =& {\mathbf{I}}_{\mathrm{i}}{\mathbf{\Phi }}_{{\mathbf{x}}_{\mathrm{d}}}\frac{{\mathbf{b}}_{\text{max}}^{\prime }{\mathbf{b}}_{\text{max}}^{\mathrm{\prime H}}}{{\lambda }_{\text{max}}^{\prime }}\\ =& {\mathbf{I}}_{\mathrm{i}}{\mathbf{\Phi }}_{\text{in}}{\mathbf{b}}_{\text{max}}^{\prime }{\mathbf{b}}_{\text{max}}^{\mathrm{\prime H}}\end{array}$$

(72)

and

$$\begin{array}{l}\text{oSNR}\left({\mathbf{H}}_{\text{max}}^{\prime }\right)={\lambda }_{\text{max}}^{\prime }.\end{array}$$

(73)

The output SNR with the maximum SNR filtering matrix is always greater than or equal to the input SNR, i.e., $\text{oSNR}\left({\mathbf{H}}_{\text{max}}^{\prime }\right)\ge \text{iSNR}$. We also have $\text{oSNR}\left({\mathbf{H}}^{\prime }\right)\le {\lambda }_{\text{max}}^{\prime },\forall {\mathbf{H}}^{\prime }$.

The minimization of the MSE criterion leads to the Wiener filtering matrix:

$$\begin{array}{lcr}{\mathbf{H}}_{\mathrm{W}}^{\prime }& =& {\mathbf{I}}_{\mathrm{i}}{\mathbf{\Phi }}_{\mathbf{x}}{\mathbf{\Phi }}_{\mathbf{y}}^{-1}\\ =& {\mathbf{H}}_{\mathrm{W}},\hfill \end{array}$$

(74)

which is identical to the Wiener filter obtained with the classical approach. Even though the Wiener filter obtained with the two different approaches is the same, its evaluation with the performance measures is slightly different due the conceptual difference between the two methods. We always have $\text{oSNR}\left({\mathbf{H}}_{\mathrm{W}}^{\prime }\right)\ge \text{iSNR}$.

We can rewrite the Wiener filtering matrix as:

$$\begin{array}{lcr}{\mathbf{H}}_{\mathrm{W}}^{\prime }& =& {\left({\mathbf{I}}_Q+{\mathbf{\Phi }}_{{\mathbf{x}}_Q}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{\Phi }}_{\text{in}}^{-1}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}\right)}^{-1}{\mathbf{\Phi }}_{{\mathbf{x}}_Q}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{\Phi }}_{\text{in}}^{-1}\\ =& {\left({\mathbf{\Phi }}_{{\mathbf{x}}_Q}^{-1}+{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{\Phi }}_{\text{in}}^{-1}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}\right)}^{-1}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{\Phi }}_{\text{in}}^{-1}.\hfill \end{array}$$

(75)

This form is interesting because it shows an obvious link with some other optimal filtering matrices as it will be verified later.

Another way to express the Wiener filter is:

$$\begin{array}{lcr}{\mathbf{H}}_{\mathrm{W}}^{\prime }& =& {\mathbf{I}}_{\mathrm{i}}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}{\mathbf{\Phi }}_{{\mathbf{x}}_Q}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{\Phi }}_{\mathbf{y}}^{-1}\\ =& {\mathbf{I}}_{\mathrm{i}}\left({\mathbf{I}}_M-{\mathbf{\Phi }}_{\text{in}}{\mathbf{\Phi }}_{\mathbf{y}}^{-1}\right).\hfill \end{array}$$

(76)

The MVDR^d rectangular filtering matrix is obtained by minimizing the MSE of the residual interference-plus-noise, J_rs(H^′), subject to the constraint that the desired signal vector is not distorted. Mathematically, this is equivalent to:

$$\begin{array}{l}\underset{{\mathbf{H}}^{\prime }}{min}\text{tr}\left({\mathbf{H}}^{\prime }{\mathbf{\Phi }}_{\text{in}}{\mathbf{H}}^{\mathrm{\prime H}}\right)\mathrm{subject}\mathit{\text{to}}\mathbf{H}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}={\mathbf{I}}_Q.\end{array}$$

(77)

The solution to the above optimization problem is:

$$\begin{array}{l}{\mathbf{H}}_{\text{MVDR}}^{\prime }={\left({\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{\Phi }}_{\text{in}}^{-1}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}\right)}^{-1}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{\Phi }}_{\text{in}}^{-1},\end{array}$$

(78)

which is interesting to compare to ${\mathbf{H}}_{\mathrm{W}}^{\prime }$ [Equation 75]. We can rewrite the MVDR as:

$$\begin{array}{l}{\mathbf{H}}_{\text{MVDR}}^{\prime }={\left({\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{\Phi }}_{\mathbf{y}}^{-1}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}\right)}^{-1}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{\Phi }}_{\mathbf{y}}^{-1}.\end{array}$$

(79)

We should always have

$$\begin{array}{l}\text{oSNR}\left({\mathbf{H}}_{\text{MVDR}}^{\prime }\right)\le \text{oSNR}\left({\mathbf{H}}_{\mathrm{W}}^{\prime }\right)\le \text{oSNR}\left({\mathbf{H}}_{\text{max}}^{\prime }\right).\end{array}$$

(80)

By minimizing the speech distortion index with the constraint that the noise reduction factor is equal to a positive value that is greater than 1, we find the tradeoff filtering matrix:

$$\begin{array}{l}{\mathbf{H}}_{\mathrm{T},{\mu }^{\prime }}^{\prime }={\mathbf{\Phi }}_{{\mathbf{x}}_Q}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\left({\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}{\mathbf{\Phi }}_{{\mathbf{x}}_Q}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H+{\mu }^{\prime }{\mathbf{\Phi }}_{\text{in}}\right)}^{-1},\end{array}$$

(81)

which can be rewritten as:

$$\begin{array}{l}{\mathbf{H}}_{\mathrm{T},{\mu }^{\prime }}^{\prime }={\left({\mu }^{\prime }{\mathbf{\Phi }}_{{\mathbf{x}}_Q}^{-1}+{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{\Phi }}_{\text{in}}^{-1}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}\right)}^{-1}{\mathbf{\Gamma }}_{\mathbf{x}{\mathbf{x}}_Q}^H{\mathbf{\Phi }}_{\text{in}}^{-1},\end{array}$$

(82)

where μ^′≥0 is a Lagrange multiplier. Usually, μ^′ is chosen in an ad hoc way, so that for

• μ^′=1, ${\mathbf{H}}_{\mathrm{T},1}^{\prime }={\mathbf{H}}_{\mathrm{W}}^{\prime }$, which is the Wiener filtering matrix;

• μ^′=0 [from Equation 82], ${\mathbf{H}}_{\mathrm{T},0}^{\prime }={\mathbf{H}}_{\text{MVDR}}^{\prime }$, which is the MVDR filtering matrix;

• μ^′>1, results in a filtering matrix with low residual noise (as compared to Wiener) at the expense of high speech distortion; and

• μ^′<1, results in a filtering matrix with high residual noise and low speech distortion (as compared to Wiener).

We always have $\text{oSNR}\left({\mathbf{H}}_{\mathrm{T},{\mu }^{\prime }}^{\prime }\right)\ge \text{iSNR},\forall {\mu }^{\prime }\ge 0$. We should also have for μ^′≥1,

$$\begin{array}{ll}\text{oSNR}\left({\mathbf{H}}_{\text{MVDR}}^{\prime }\right)& \le \text{oSNR}\left({\mathbf{H}}_{\mathrm{W}}^{\prime }\right)\le \text{oSNR}\left({\mathbf{H}}_{\mathrm{T},{\mu }^{\prime }}^{\prime }\right)\\ \le \text{oSNR}\left({\mathbf{H}}_{\text{max}}^{\prime }\right)\end{array}$$

(83)

and for μ^′≤1,

$$\begin{array}{ll}\text{oSNR}\left({\mathbf{H}}_{\text{MVDR}}^{\prime }\right)& \le \text{oSNR}\left({\mathbf{H}}_{\mathrm{T},{\mu }^{\prime }}^{\prime }\right)\le \text{oSNR}\left({\mathbf{H}}_{\mathrm{W}}^{\prime }\right)\\ \le \text{oSNR}\left({\mathbf{H}}_{\text{max}}^{\prime }\right).\end{array}$$

(84)

The case Q=M is interesting because for both approaches, performance measures and optimal square filtering matrices are identical. We can draw the same conclusions for the case Q=P=1.

1.5 Application examples

In this section, we show how the two approaches can be applied to different applications of speech enhancement.

1.5.1 Single-channel noise reduction in the time domain

The single-channel noise reduction problem in the time domain consists of recovering the desired signal (or clean speech) x(t), t being the discrete-time index, of zero mean from the noisy observation (microphone signal) [1]:

$$\begin{array}{l}y\left(t\right)=x\left(t\right)+v\left(t\right),\end{array}$$

(85)

where v(t), assumed to be a zero-mean random process, is the unwanted additive noise that can be either white or colored but is uncorrelated with x(t).

The signal model given in Equation 85 can be put into a vector form by considering the L most recent successive time samples, i.e.,

$$\begin{array}{lcr}\mathbf{y}\left(t\right)& =& {\left[\begin{array}{cccc}y\left(t\right)& y(t-1)& \cdots & y(t-L+1)\end{array}\right]}^T\\ =& \mathbf{x}\left(t\right)+\mathbf{v}\left(t\right),\hfill \end{array}$$

(86)

where x(t) and v(t) are defined in a similar way to y(t). We define the desired signal vector as:

$$\begin{array}{l}{\mathbf{x}}_Q\left(t\right)={\left[\begin{array}{cccc}x\left(t\right)& x(t-1)& \cdots & x(t-Q+1)\end{array}\right]}^T,\end{array}$$

(87)

that we can estimate from y(t) with either of the two methods. Estimating the desired signal using conventional, rectangular filters was considered in [24], while [22] considers the orthogonal decomposition approach. Simulation results showing the performance of the two filtering methods for single-channel noise reduction are also found in [22, 24].

1.5.2 Single-channel noise reduction in the time-frequency domain

Using the short-time Fourier transform (STFT), Equation 85 can be rewritten in the time-frequency domain as [13, 25]:

$$\begin{array}{l}Y(k,n)=X(k,n)+V(k,n),\end{array}$$

(88)

where the zero-mean complex random variables Y(k,n), X(k,n), and V(k,n) are the STFTs of y(t), x(t), and v(t), respectively, at frequency bin k∈{0,1,…,K−1} and time frame n. Here, the sample X(k,n) is the desired signal.

The simplest way to estimate X(k,n) is by applying a positive gain to Y(k,n) with the conventional approach. However, the noise reduction performance may be limited.

A more general approach to estimate the desired signal is by filtering the observation signal vector of length P[25]:

$$\mathbf{y}(k,n)={\left[Y(k,n)Y(k,n-1)\cdots Y(k,n-P+1)\right]}^T,$$

(89)

and using the orthogonal decomposition to extract X(k,n) from x(k,n), which is defined in a similar way to y(k,n). Thanks to this approach, the non-negligible interframe correlation is taken into account, which is not the case when just a gain is used. As a consequence, we can better compromise between noise reduction and speech distortion.

The STFT-based filtering methods for single-channel noise reduction was considered in, e.g., [26–28], where experimental results can also be found.

1.5.3 Multichannel noise reduction in the time domain

In the multichannel scenario, we have a microphone array with M sensors that captures a convolved source signal in some noise field. In the time domain, the received signals are expressed as [29, 30]:

$$\begin{array}{lcr}{y}_m\left(t\right)& =& g_m\left(t\right)\ast s\left(t\right)+v_m\left(t\right)\hfill \\ =& x_m\left(t\right)+v_m\left(t\right),m=1,2,\dots ,M,\end{array}$$

(90)

where g _m (t) is the acoustic impulse response from the unknown speech source, s(t), location to the m th microphone, ∗ stands for linear convolution, and x _m (t) and v _m (t) are, respectively, the convolved speech and additive noise at microphone m. We assume that the signals x _m (t)=g _m (t)∗s(t) and v _m (t) are uncorrelated, zero mean, real, and broadband. By definition, x _m (t) is coherent across the array. The noise signals, v _m (t), are typically only partially coherent across the array.

By processing the data by blocks of L time samples, the signal model given in Equation 90 can be put into a vector form as:

$$\begin{array}{l}{\mathbf{y}}_m\left(t\right)={\mathbf{x}}_m\left(t\right)+{\mathbf{v}}_m\left(t\right),m=1,2,\dots ,M,\end{array}$$

(91)

where

$$\begin{array}{l}{\mathbf{y}}_m\left(t\right)={\left[\begin{array}{cccc}{y}_m\left(t\right)& y_m(t-1)& \cdots & y_m(t-L+1)\end{array}\right]}^T\end{array}$$

(92)

is a vector of length L, and x _m (t) and v _m (t) are defined similarly to y _m (t). It is more convenient to concatenate the M vectors y _m (t), m=1,2,…,M together as:

$$\begin{array}{lcr}\underset{-}{\mathbf{y}}\left(t\right)& =& {\left[\begin{array}{cccc}{\mathbf{y}}_1^T\left(t\right)& {\mathbf{y}}_2^T\left(t\right)& \cdots & {\mathbf{y}}_M^T\left(t\right)\end{array}\right]}^T\\ =& \underset{-}{\mathbf{x}}\left(t\right)+\underset{-}{\mathbf{v}}\left(t\right),\hfill \end{array}$$

(93)

where vectors $\underset{-}{\mathbf{x}}\left(t\right)$ and $\underset{-}{\mathbf{v}}\left(t\right)$ of length ML are defined in a similar way to $\underset{-}{\mathbf{y}}\left(t\right)$.

We consider x₁(t) as the desired signal vector. Our problem then may be stated as follows: given $\underset{-}{\mathbf{y}}\left(t\right)$, our aim is to preserve x₁(t) while minimizing the contribution of $\underset{-}{\mathbf{v}}\left(t\right)$. Both approaches can be used but the one based on the orthogonal decomposition is more appropriate since it will better exploit the correlation among the convolved speech signals at the microphones for noise reduction. The orthogonal decomposition approach for multichannel noise reduction was considered in, e.g., [31, 32], where experimental results can also be found.

1.5.4 Multichannel noise reduction in the frequency domain

In the frequency domain, at the frequency index f, Equation 90 can be expressed as:

$$\begin{array}{lcr}{Y}_m\left(f\right)& =& G_m\left(f\right)S\left(f\right)+V_m\left(f\right)\hfill \\ =& X_m\left(f\right)+V_m\left(f\right),m=1,2,\dots ,M,\end{array}$$

(94)

where Y _m (f), G _m (f), S(f), V _m (f), and X _m (f) are the frequency-domain representations of y _m (t), g _m (t), s(t), v _m (t), and x _m (t), respectively.

It is more convenient to write the M frequency-domain microphone signals in a vector notation:

$$\begin{array}{lcr}\mathbf{y}\left(f\right)& =& \mathbf{g}\left(f\right)S\left(f\right)+\mathbf{v}\left(f\right)\hfill \\ =& \mathbf{x}\left(f\right)+\mathbf{v}\left(f\right)\hfill \\ =& \mathbf{d}\left(f\right)X_1\left(f\right)+\mathbf{v}\left(f\right),\hfill \end{array}$$

(95)

where

$$\begin{array}{lcr}\mathbf{y}\left(f\right)& =& {\left[\begin{array}{cccc}{Y}_1\left(f\right)& Y_2\left(f\right)& \cdots & Y_M\left(f\right)\end{array}\right]}^T,\hfill \\ \mathbf{x}\left(f\right)& =& {\left[\begin{array}{cccc}{X}_1\left(f\right)& X_2\left(f\right)& \cdots & X_M\left(f\right)\end{array}\right]}^T\hfill \\ =& S\left(f\right)\mathbf{g}\left(f\right),\hfill \\ \mathbf{g}\left(f\right)& =& {\left[\begin{array}{cccc}{G}_1\left(f\right)& G_2\left(f\right)& \cdots & G_M\left(f\right)\end{array}\right]}^T,\hfill \\ \mathbf{v}\left(f\right)& =& {\left[\begin{array}{cccc}{V}_1\left(f\right)& V_2\left(f\right)& \cdots & V_M\left(f\right)\end{array}\right]}^T,\hfill \end{array}$$

and

$$\begin{array}{lcr}\mathbf{d}\left(f\right)& =& {\left[\begin{array}{cccc}1& \frac{G_2\left(f\right)}{G_1\left(f\right)}& \cdots & \frac{G_M\left(f\right)}{G_1\left(f\right)}\end{array}\right]}^T\\ =& \frac{\mathbf{g}\left(f\right)}{G_1\left(f\right)}.\hfill \end{array}$$

(96)

Expression in Equation 95 depends explicitly on the desired signal, X₁(f), that we want to estimate from y(f).

There is another interesting way to write Equation 95. First, it is easy to see that

$$\begin{array}{l}{X}_m\left(f\right)={\gamma }_{X_m{X}_1}\left(f\right)X_1\left(f\right),m=1,2,\dots ,M,\end{array}$$

(97)

where

$$\begin{array}{lcr}{\gamma }_{X_m{X}_1}\left(f\right)& =& \frac{E\left[X_m\left(f\right)X_1^{\ast }\left(f\right)\right]}{E\left[{\left|X_1\left(f\right)\right|}^2\right]}\hfill \\ =& \frac{G_m\left(f\right)}{G_1\left(f\right)},m=1,2,\dots ,M\end{array}$$

(98)

is the partially normalized [with respect to X₁(f)] coherence function between X _m (f) and X₁(f). Using Equation 97, we can rewrite Equation 95 as:

$$\begin{array}{l}\mathbf{y}\left(f\right)={\gamma }_{\mathbf{x}{X}_1}\left(f\right)X_1\left(f\right)+\mathbf{v}\left(f\right),\end{array}$$

(99)

where

$$\begin{array}{lcr}{\gamma }_{\mathbf{x}{X}_1}\left(f\right)& =& {\left[\begin{array}{cccc}1& {\gamma }_{X_2{X}_1}\left(f\right)& \cdots & {\gamma }_{X_M{X}_1}\left(f\right)\end{array}\right]}^T\\ =& \frac{E\left[\mathbf{x}\left(f\right)X_1^{\ast }\left(f\right)\right]}{E\left[{\left|X_1\left(f\right)\right|}^2\right]}\hfill \\ =& \mathbf{d}\left(f\right)\hfill \end{array}$$

(100)

is the partially normalized [with respect to X₁(f)] coherence vector (of length M) between x(f) and X₁(f). This shows that the two approaches for noise reduction are identical. More details on multichannel noise reduction in the frequency domain as well as experimental results can be found in [25, 33].

1.5.5 Binaural noise reduction

Binaural noise reduction [34] consists of the estimation of the received speech signal at two different microphones with a sensor array of M microphones. One estimate is for the left ear and the other for the right ear. This way and thanks to our binaural hearing system, we will be able to localize the speech source in the space. In the frequency domain, we can estimate, for example, X₁(f) and X₂(f). As explained above, the two methods are the same. In the time domain, we can estimate, for example, x₁(t) and x₂(t). The method based on the orthogonal decomposition is more appropriate since it may distort less the signals. Distortion in binaural noise reduction is problematic since it may affect the cues for localization and separation. Experimental results and further theoretical details on binaural noise reduction using the approaches mentioned herein are found in [35, 36].

2 Conclusions

In this paper, we have given a brief overview of linear filtering methods for speech enhancement based on two approaches: a so-called conventional approach and an approach based on the orthogonal decomposition. In the context of these two different approaches, various optimal filters (e.g., MVDR, maximum SNR, and Wiener filters) have been derived and their properties in terms of different performance measures have been assessed and compared. These performance measures, simply put, quantify the properties of the filters and approaches in terms of noise reduction and speech distortion and show how they offer different tradeoffs between the two. We have also demonstrated how the approaches can be applied in various speech enhancement contexts, including single- and multichannel enhancement in both the time and frequency domains and in binaural noise reduction.

Endnotes

^a The upper bound comes from the fact that this distortion is obtained when the filtering matrix only contains zeros, which should be the maximum expected distortion.

^b It is legitimate to consider x_i as an interference, since the desired signal is entirely in x_d, and x_i and x_d are uncorrelated.

^c Here, the distortionless constraint is in the sense that we can perfectly recover the desired signal vector, even though the residual interference can add some uncorrelated distortion to the desired signal.

^d We use the terminology MVDR because we can completely extract the desired signal with this filter.