Elaborate analysis and design of filter-bank-based sensing for wideband cognitive radios

Abstract

The successful operation of a cognitive radio system strongly depends on its ability to sense the radio environment. With the use of spectrum sensing algorithms, the cognitive radio is required to detect co-existing licensed primary transmissions and to protect them from interference. This paper focuses on filter-bank-based sensing and provides a solid theoretical background for the design of these detectors. Optimum detectors based on the Neyman-Pearson theorem are developed for uniform discrete Fourier transform (DFT) and modified DFT filter banks with root-Nyquist filters. The proposed sensing framework does not require frequency alignment between the filter bank of the sensor and the primary signal. Each wideband primary channel is spanned and monitored by several sensor subchannels that analyse it in narrowband signals. Filter-bank-based sensing is proved to be robust and efficient under coloured noise. Moreover, the performance of the weighted energy detector as a sensing technique is evaluated. Finally, based on the Locally Most Powerful and the Generalized Likelihood Ratio test, real-world sensing algorithms that do not require a priori knowledge are proposed and tested.

1 Introduction

Spectrum sensing has been brought into the center of research activities due to its application in the context of cognitive radio (CR) [1]. Cognitive radio and dynamic spectrum access have been identified as the means to maximize spectrum exploitation and efficiency. The cognitive radios share the available spectrum with a licensed primary system (PS) and have the responsibility not to adversely affect the PS user operation by causing interference. Spectrum sensing is used to identify and consequently avoid co-existing primary signals. Several spectrum sensing techniques have been derived and studied [2, 3]. These algorithms present pros and cons concerning the need for a priori knowledge of PS signals features, the computational complexity, the robustness in channel variations and coloured noise etc.

In [4], the introduction of filter-bank-based sensing was made and further analysis was provided in [5]. The main advantage of this technique lies in the fact that since the CR networks will use multicarrier modulations for transmission, the analysis filters that are intended for receiver operation, e.g. in OFDM-OQAM [6, 7], in discrete wavelet multiTone [8] and in filtered multiTone [9], could also be used for sensing without extra computational workload. In [10], filter-bank-based physical layer design for CR systems was introduced, where simultaneous spectrum sensing and transmission may be possible using the filter bank. In [4] and [5], Fahrang-Boroujeny examined the filter bank operation as an estimator of the power Spectral density (PSD) incorporating a spectrum analyser in the receiver structure. A comparison in the performance of the estimation with the non-parametric multitaper method [11] was also made. It was confirmed that the use of a filter bank can similarly achieve remarkable spectral analysis with the use of a larger set of samples but much less computational complexity.

However, in [4, 5], no detectors were presented that could be practically used to identify primary emissions in a given band of interest. Since then, several studies were made regarding filter-bank-based sensing. Most studies consider simplified models for the operation of the filter bank and do not attain optimality [12, 13]. In [14], filter bank sensing is performed using data-aided feature detection achieving results in extremely low signal-to-noise ratio (SNR); however, the algorithm required knowledge for specific features of the PS signal. Some studies focus on the reduction of the computational workload [15–17], while others propose new filters suitable for sensing [18, 19]. In addition, special issues have been addressed that concern the application of filter banks in spectrum sensing [20, 21]. Moreover, in [22, 23], efficient implementations of filter bank sensors are presented.

This study is motivated by the absence of strong theoretical description for filter-bank-based sensors. A variety of detectors are presented for uniform and modified discrete Fourier transform (uniform DFT-MDFT) filter banks and especially root-Nyquist filters. As in [14], filter banks are implemented using the polyphase structure in order to perform simultaneous parallel sensing on all subchannels; however, the proposed algorithms are not based on specific signal feature extraction. Optimality based on the Neyman-Pearson theorem [24] is achieved. Moreover, the designed sensors are based on the approach that there is no ‘1-1’ matching of the receiver filters and the PS signal bandwidth. This is a common flaw among the majority of filter-bank-based sensing studies. Primary signals are in their vast majority wideband, while the CR filter bank has the objective to divide the monitored spectrum into narrowband channels. Therefore, in a common configuration, a primary channel is spanned and analysed by a number of CR filters. The use of multiple CR filters on a single PS transmission also eliminates the need for frequency alignment between the primary signal and the CR detector. An additional advantage is that the filter bank can partially exploit the radio channel frequency selectivity without the need for complicated equalization procedures on the primary signal. In [25] and [26], multiple CR subchannels are also used to span the primary signal. In the first study, a technique for SNR estimation of the PS signal is proposed using simple approximations, while in [26] the authors introduce a weighted energy detector scheme that is able to efficiently scan Bluetooth channels.

In Section 2, the used system model is presented. In Section 3, the Neyman-Pearson optimal detectors for uniform DFT banks are extracted, while in Section 4, the optimal detectors in the output of an OFDM-OQAM receiver (using an MDFT bank) are studied. In Section 5, the weighted energy detector for uniform DFT banks is analysed, and in Section 6, the extension of the filter bank sensing application in coloured noise is presented. It is noted that coloured noise has been identified [5] as a significant challenge for spectrum sensing applications. Finally in Section 7, practical implementations of the detectors - the Locally Most Powerful (LMP) test and the Generalized Likelihood Ratio Test (GLRT) - are proposed. Simulation results are presented in Section 8.

2 System model

A CR system is assumed that digitizes a large portion of spectrum containing K channels of the PS. The wideband received signal is analysed by a bank of M>K filters that span the whole digitized bandwidth. Consequently, each PS channel is analysed by ⌈L=M / K⌉ filters. A graphical representation of a 32-channel filter bank that spans the bandwidth that contains four PS channels is presented in Figure 1. In the specific example, the output from eight subchannels can be used to extract the decision on whether a PS channel is occupied or not.

Figure 1

Filter bank configuration example for M = 32 in a bandwidth containing four PS channels.

Given the fact that the wideband CR receiver should optimize the use of the limited computational resources, the analysis focuses on uniform DFT [27] filter banks with downsampling at the Nyquist rate. The well-known computationally efficient polyphase structure can be assumed. The type-1 polyphase representation of an finite impulse response (FIR) filter is the following [28]:

$$h_i^{\left(1\right)}\left(n\right)=h\left(\mathit{\text{Mn}}+i\right),H\left(z\right)=\sum _{i=0}^{M-1}{z}^{-i}{H}_i^{\left(1\right)}\left(z^M\right)$$

(1)

where the subscript indicates the i th polyphase filter path, while the superscript defines the type of polyphase representation. Analysis is also expanded in filter banks used by OFDM-OQAM systems. The specific filters present similarities with the cosine-modulated filter banks and their digitally equivalent representation is the MDFT [29] filter banks [29]. Ideally, a digital implementation of an OFDM-OQAM system will use MDFT filter banks with perfect reconstruction (PR) properties. In many cases, the PR property is sacrificed for better time-frequency localization; however, these filters can also be expressed with the MDFT structure and it is highly desired to approximate the behaviour of MDFT-PR. Two equivalent structures of an OFDM-OQAM receiver are presented in Figure 2 (where normalization and phase correction coefficients are arbitrarily set since they do not affect the results of the specific study). The two filter structures are equivalent for D filter coefficients where $\alpha =⌊\frac{D-1}{M/2}⌋=2{\alpha }^{\prime }$, i.e., α is even. Nevertheless, the selection of odd α does not affect the MDFT-filter bank-based spectrum sensing results.

Figure 2

Two OFDM-OQAM receiver filter structures. (a) Conventional implementation and (b) parallel uniform DFT bank implementation.

Moreover, the assumed prototype filter is real and symmetric ensuring linear phase response. Deeper analysis is performed for filter banks using a root-Nyquist prototype filter [30]. The root-Nyquist filters are typically used for pulse shaping in radio communication systems due to the InterSymbol Interference (ISI)-free transmission properties. It is noted that the necessary and sufficient condition for MDFT-PR filters suitable for OFDM-OQAM also leads to root-Nyquist filters [29].

The n th time sample of the received signal from the output of the A/D converter is represented by r _n , while $y_n^{\left(i\right)}$ represents the output of the i th filter. It is assumed that the filter impulse response is normalized so that the mean input and output signal powers remain constant, if the incoming signal is contained in the specific filter passband. Thus, $h_u=p_u/\sqrt{\sum _{u=-\infty }^{\infty }{\left|p_u\right|}^2}$, where p _u is the FIR filter prototype. Using a non-casual filter representation, the Nyquist ISI-free condition is expressed by

$$g_n=\sum _{u=-\infty }^{\infty }{h}_u{h}_{n-u}=\left\{\begin{array}{c}1,n=0\\ 0,n=\mathit{\text{lM}},l\ne 0\end{array}\right.$$

(2)

The presented theoretical analysis assumes that the transmitted PS signal s _n is circular white Gaussian random variable with mean power at the receiver (over a flat channel) ${\sigma }_s^2$. This assumption is accurate for orthogonal frequency division multiplexing (OFDM) signals, but it is questionable for single carrier signals. However, since the signal passes through a cascade of filters and transceiver impairments, each signal sample is the result of linear combinations and shifts that includes a large number of random variables. Thus, a generalized version of the Central Limit Theorem can be invoked to justify the Gaussian assumption for the PS signal. The incoming signal at the CR receiver for given PS transmissions under frequency selective channels is provided by

$$r_n=\sum _{k=0}^{K-1}{\zeta }_k\sum _{u=0}^{\infty }{c}_u^{\left(k\right)}{s}_{n-u}^{\left(k\right)}+w_n$$

(3)

where the superscripts indicate the transmitted signal at the k th primary channel, and the ζ _k coefficient is equal to 0 if the k th channel is free, or 1 if the k th channel is occupied. Filter $c_u^{\left(k\right)}$ represents the radio channel impulse response for the k th PS channel. The formulated detection problem concerns the use of the available information from multiple filter bank outputs in the decision of whether a primary channel is occupied or not. Therefore, the binary decision problem for the k th primary channel is expressed as

$$\begin{array}{l}{\mathcal{\mathscr{H}}}_0:{\zeta }_k=0\to r_n^{\left(k\right)}=w_n\\ {\mathcal{\mathscr{H}}}_1:{\zeta }_k=1\to r_n^{\left(k\right)}=\sum _{u=0}^{\infty }{c}_u^{\left(k\right)}{s}_{n-u}^{\left(k\right)}+w_n\end{array}$$

(4)

The observation vector that contains the information used for the decision is a set of N samples (depending on the sensing duration) for each of the L filters that span the primary channel under investigation. The following vectors are defined:

$${\mathbf{y}}_l={\left[\begin{array}{cccc}{y}_n^{\left(l\right)}& y_{n-1}^{\left(l\right)}& \dots & y_{n-N+1}^{\left(l\right)}\end{array}\right]}^{\text{T}}$$

(5)

with

$$y_n^{\left(l\right)}=\sum _{u=-\infty }^{\infty }{h}_u^{\left(l\right)}{r}_{\mathit{\text{Mn}}-u}$$

(6)

where $h_u^{\left(l\right)}$ is the impulse response for the l th subchannel of the filter bank. In (6), it is assumed that the signal is maximally decimated at the output. Without loss of generality, L is assumed even for presentation purposes. No additional information on primary signal features and specifications is assumed. The next step is the definition of the detectors.

3 Neyman-Pearson optimal detectors for maximally decimated signals

The energy detector is the simplest and most common spectrum sensing technique with low complexity and minimum knowledge regarding the primary system. The energy detector can be seen as a special case of filter-bank-based sensing with M=1 and K=1. It is proved that for zero-mean white Gaussian input, the energy detector is optimal according to the Neyman-Pearson theorem [24] for known PS signal power ${\sigma }_s^2$ and additive white Gaussian noise (AWGN) with ${\sigma }_w^2$.

$$T=\sum _{n=0}^{N-1}{\left|y\left[n\right]\right|}^2\underset{{\mathcal{\mathscr{H}}}_0}{\overset{{\mathcal{\mathscr{H}}}_1}{\gtrless }}\gamma$$

(7)

The metric T follows a scaled χ² distribution as the sum of squares of white Gaussian zero-mean random variables [31] with 2N degrees of freedom. It is proved that T follows the distributions below:

$$T=\left\{\begin{array}{c}\frac{{\sigma }_w^2}{2}{\chi }_{2N}^2\text{If}{\mathcal{\mathscr{H}}}_0,\\ \frac{{\sigma }_w^2+{\sigma }_s^2}{2}{\chi }_{2N}^2\text{If}{\mathcal{\mathscr{H}}}_1\end{array}\right.$$

(8)

In this paper, the energy detector is used as a reference for the evaluation of the extracted algorithms.

When referring to maximally decimated signals, it is considered that the rate of the signals at the filter output is reduced to the minimum as it is defined by the Nyquist sampling theorem. This means that the specific detector operates at the output of a uniform DFT filter bank (or the output of one of the parallel banks in an OFDM-OQAM system before the real-imaginary separation - Figure 2b). The sensing mechanism can be considered as a vector detection problem with vector observations from L different sensors. Assuming that the k₀th primary channel is observed, the output from the filter paths with indexes k₀L+l, l=0,…,L-1 is collected.

The first step is to define a single observation vector with the use of a reordering technique for the selected vectors. For reasons that will be cleared out at the next steps, the following reordering is selected:

• The observation vectors are divided in two groups:

  • Group 0 contains the even subchannels.

  • Group 1 contains the odd subchannels.

• Spatial reordering (or row rollout [24]) is performed (choosing first the group 0 vector). Therefore,

  $$\begin{array}{l}\mathbf{y}={\left[\begin{array}{ccc}{\left({\mathbf{y}}^{\left(0\right)}\right)}^{\text{H}}& |& {\left({\mathbf{y}}^{\left(1\right)}\right)}^{\text{H}}\end{array}\right]}^{\text{H}}\\ ={\left[\begin{array}{ccccccccc}{\mathbf{y}}_0^{\text{H}}& {\mathbf{y}}_2^{\text{H}}& \dots & {\mathbf{y}}_{L-2}^{\text{H}}& |& {\mathbf{y}}_1^{\text{H}}& {\mathbf{y}}_3^{\text{H}}& \dots & {\mathbf{y}}_{L-1}^{\text{H}}\end{array}\right]}^{\text{H}}\end{array}$$

  (9)

  where y _l is given in (5). For simplicity reasons, the common term k₀L of the vector subscripts was omitted. Subvectors y _i represent the observation vector per filter path, i.e. per subchannel.

The signal PSD under ${\mathcal{\mathscr{H}}}_1$ after filtering and before decimation is given by

$$\begin{array}{l}{S}_{y_i{y}_i}\left(f\right)=S_{\mathit{\text{rr}}}\left(f\right){\left|H\left(f-f_i\right)\right|}^2\\ =\left(\sum _{k=0}^{K-1}{\left|C^{\left(k\right)}\left(f\right)\right|}^2{S}_{\mathit{\text{ss}}}^{\left(k\right)}\left(f\right)+S_{\mathit{\text{ww}}}\left(f\right)\right){\left|H\left(f-f_i\right)\right|}^2\end{array}$$

(10)

where C^(k)(f) is the radio channel transfer function for the PS signal that occupies the k th subchannel and $S_{\mathit{\text{ss}}}^{\left(k\right)}\left(f\right),S_{\mathit{\text{rr}}}\left(f\right)$ the respective PSDs for s _n and r _n respectively. As in [4], it is considered that the receiver filters are adequately narrowband. This assumption is valid for OFDM-OQAM systems since by design the filters should divide the whole bandwidth in subchannels where the radio channel frequency response can be considered approximately flat (since otherwise computationally cumbersome equalization methods would be required). Therefore, the following approximation can be considered:

$$\begin{array}{l}{S}_{y_i{y}_i}\left(f\right)\simeq \left({\left|C\left(f_i\right)\right|}^2{}_S^{\mathit{\text{ss}}}\left(f\right)+S_{\mathit{\text{ww}}}\left(f\right)\right){\left|H\left(f-f_i\right)\right|}^2\\ =\left({\left|c_i\right|}^2{\sigma }_s^2+{\sigma }_w^2\right){\left|H\left(f-f_i\right)\right|}^2\end{array}$$

(11)

where the PSDs for the white Gaussian signal and noise were used. The c _i coefficient is a measure of the radio channel effect for the specific subchannel obtained by the radio channel Fourier transform. Although this study assumes flat radio channel per filter subchannel, this requirement is not very strict. During simulation and detector evaluation, the radio channel models did not provide strictly flat radio channels per filter. Nevertheless, under realistic channel conditions, no significant fluctuations from the assumed optimum performance under the flat channel assumption was observed. Moreover, a better approximation of the c _i coefficients can be made with the use of a mean value of the frequency selective channel transfer function for each subchannel filter. More specifically,

$${\left|c_i\right|}^2=\frac{1}{\text{BW}}\underset{-\infty }{\overset{\infty }{\int }}{\left|C\left(f\right)H\left(f-f_i\right)\right|}^2\mathit{\text{df}}$$

(12)

where BW is the total bandwidth. From (11), it is concluded that with the use of the narrowband filter, the input signal for each subchannel can be considered an AWGN random variable with ${\mathbf{r}}_i\sim \mathcal{N}\left(0,\left({\left|c_i\right|}^2{\sigma }_s^2+{\sigma }_w^2\right){\mathbf{I}}_N\right)$, despite the radio channel effects. Under ${\mathcal{\mathscr{H}}}_0$, the input signal is also AWGN with ${\mathbf{r}}_i\sim \mathcal{N}\left(0,{\sigma }_w^2{\mathbf{I}}_N\right)$. In Figure 3, two examples of Rayleigh channels with exponential power delay profiles (PDP) are presented. The first example (maximum delay spread 12 samples and exponent λ=1) presents a quite accurate flat narrowband channel approximation. In the second case (maximum delay spread 24 samples with exponent 0.5), frequency selectivity is apparent in the passband. However, the flat approximation is used in both cases with higher expected deviation from the ideal for the second example.

Figure 3

Examples of radio channel transfer functions per subchannel for M = 32 and Rayleigh channels with exponential power delay profiles.

Generally, for large M and adequately small number of filter coefficients, the filter transition bandwidth is not quite sharp. However, it can be assumed that the subchannels of the filter bank provide sufficient attenuation at the passband so that the following approximation stands:

$$H\left(f-f_i\right)H^{\ast }\left(f+f_{i+l}\right)\simeq 0,\begin{array}{cc}i& =0,\dots ,M-1\\ i+l& =0,\dots ,M-1\\ l& \ne -1,0,1\end{array}$$

(13)

In (13) it is claimed that the outputs from two filter paths that are not directly adjacent are uncorrelated and under the Gaussian assumption independent. According to the Neyman-Pearson theorem, the detector that optimizes the probability of detection P_D for given probability of false alarm P_FA is provided by the likelihood ratio:

$$\frac{f\left(\mathbf{y}/{\mathcal{\mathscr{H}}}_1\right)}{f\left(\mathbf{y}/{\mathcal{\mathscr{H}}}_0\right)}\underset{{\mathcal{\mathscr{H}}}_0}{\overset{{\mathcal{\mathscr{H}}}_1}{\gtrless }}\theta$$

(14)

where f is the probability density function (PDF) for the random variable (operand ‘ /’ indicates a conditional probability). Threshold θ is selected in order to achieve the desired P_FA. In order to define the threshold, the distributions of the vector random variables, as well as the distributions of the final detection metric must be defined. Based on the initial assumptions, y is a multidimensional Gaussian variable. The first task is to determine its covariance matrix. Similarly to the energy detector, the SNR, or equivalently the signal and noise powers, should be known. More specifically, the investigated detector should be aware of the SNR per subchannel that is given by ${\gamma }_i=\frac{{\left|c_i\right|}^2{\sigma }_s^2}{{\sigma }_w^2}$. Initially, this information is assumed known. In a real-world design of the detector, an estimation procedure is performed before or during the detection (Section 7).

3.1 Distribution under ${\mathcal{\mathscr{H}}}_{\mathbf{0}}$

The determination of the covariance matrix for the specified block vector with spatial reordering is made using the following relationship:

$$\mathrm{E}\left(\mathbf{y}{\mathbf{y}}^{\text{H}}\right)=\left[\begin{array}{ccc}{\mathbf{E}}_{00}& |& {\mathbf{E}}_{01}\\ ---& |& ---\\ {\mathbf{E}}_{10}& |& {\mathbf{E}}_{11}\end{array}\right]$$

(15)

where,

$$\begin{array}{c}{\mathbf{E}}_{\mathit{\text{ij}}}=\left[\begin{array}{cccc}\mathrm{E}\left({\mathbf{y}}_i{\mathbf{y}}_j^{\text{H}}\right)& \mathrm{E}\left({\mathbf{y}}_i{\mathbf{y}}_{j+2}^{\text{H}}\right)& \dots & \mathrm{E}\left({\mathbf{y}}_i{\mathbf{y}}_{j+L-2}^{\text{H}}\right)\\ \mathrm{E}\left({\mathbf{y}}_{i+2}{\mathbf{y}}_j^{\text{H}}\right)& \mathrm{E}\left({\mathbf{y}}_{i+2}{\mathbf{y}}_{j+2}^{\text{H}}\right)& \mathrm{E}\left({\mathbf{y}}_{i+2}{\mathbf{y}}_{j+L-2}^{\text{H}}\right)\\ ⋮& \ddots & \ddots & ⋮\\ \mathrm{E}\left({\mathbf{y}}_{i+L-2}{\mathbf{y}}_j^{\text{H}}\right)& \mathrm{E}\left({\mathbf{y}}_{i+L-2}{\mathbf{y}}_{j+2}^{\text{H}}\right)& \dots & \mathrm{E}\left({\mathbf{y}}_{i+L-2}{\mathbf{y}}_{j+L-2}^{\text{H}}\right)\end{array}\right]\end{array}$$

(16)

Initially, in order to quantify the effect of the filter bank on the variables, the input signal variance for all subchannels is assumed to be σ²=1. The covariance of the maximally decimated signal for each subchannel separately is given by [4]

$$\begin{array}{l}{\rho }_{y_i{y}_i}\left(n\right)=\sum _{u=-\infty }^{\infty }{h}_u{e}^{2\mathrm{\pi j}\frac{\mathit{\text{ui}}}{M}}{h}_{\mathit{\text{Mn}}+u}{e}^{-2\mathrm{\pi j}\frac{\mathit{\text{ui}}+\mathit{\text{Mni}}}{M}}\\ =\sum _{u=-\infty }^{\infty }{h}_u{h}_{\mathit{\text{Mn}}+u}=g_{\mathit{\text{Mn}}}\end{array}$$

(17)

For a root-Nyquist filter and according to (2), the result is equal to unity for n=0 and zero otherwise. Therefore, the output samples of each subchannel are uncorrelated despite the filter bank.

$$\mathrm{E}\left({\mathbf{y}}_i{\mathbf{y}}_i^{\text{H}}\right)=\left[\begin{array}{ccc}{\rho }_{y_i{y}_i}\left(0\right)& \dots & {\rho }_{y_i{y}_i}\left(N-1\right)\\ ⋮& \ddots \\ {\rho }_{y_i{y}_i}\left(-N+1\right)& {\rho }_{y_i{y}_i}\left(0\right)\end{array}\right]={\mathbf{I}}_N$$

(18)

Based on the aforementioned assumption (13), the output vectors from not direct-adjacent filter paths are uncorrelated and therefore,

$$\mathrm{E}\left({\mathbf{y}}_i{\mathbf{y}}_{i+l}^{\text{H}}\right)={\mathbf{0}}_N,l\ne -1,0,1$$

(19)

It remains to calculate the cross-correlation ${\left[\mathbf{R}\right]}_{m,n}={\rho }_{y_i{y}_{i+l}}\left(m-n\right)(m,n=1\dots N)$ between the output samples of adjacent filters (l=-1,1).

The elements of R are calculated with the following formula:

$$\begin{array}{l}{\left[\mathbf{R}\right]}_{k,m}=\text{E}\left(y_n^{\left(i\right)}{y}_{n+m-k}^{(i+l)\ast }\right)=\text{E}\left(y_n^{\left(i\right)}{y}_{n+\epsilon }^{(i+l)\ast }\right)=\\ \sum _{u=-\infty }^{\infty }\sum _{p=-\infty }^{\infty }{h}_u{h}_p{e}^{2\mathrm{\pi j}\frac{\mathit{\text{iu}}}{M}}{e}^{-2\mathrm{\pi j}\frac{(i+l)p}{M}}\text{E}\left(r_{\mathit{\text{Mn}}-u}{r}_{\mathit{\text{Mn}}-p+\mathrm{M\epsilon }}^{\ast }\right)\end{array}$$

(20)

where m-k=ε. For zero-mean white Gaussian input with σ²=1, the following result is extracted:

$$\begin{array}{l}{\left[\mathbf{R}\right]}_{k,m}={\delta }_{p+\mathrm{M\epsilon }-u}\sum _{u=-\infty }^{\infty }\sum _{p=-\infty }^{\infty }{h}_u{h}_p{e}^{2\mathrm{\pi j}\frac{\mathit{\text{iu}}}{M}}{e}^{-2\mathrm{\pi j}\frac{(i+l)p}{M}}\\ =\sum _{u=-\infty }^{\infty }{h}_u{h}_{u-\mathrm{M\epsilon }}{e}^{2\mathrm{\pi j}\frac{\mathit{\text{iu}}}{M}}{e}^{-2\mathrm{\pi j}\frac{(i+l)\left(u-\mathrm{M\epsilon }\right)}{M}}\\ =\sum _{u=-\infty }^{\infty }{h}_u{e}^{-2\mathrm{\pi j}\frac{\mathit{\text{lu}}}{M}}{h}_{u-\mathrm{M\epsilon }}\end{array}$$

(21)

which can be proved to be equal to the following:

$$\begin{array}{l}{\left[\mathbf{R}\right]}_{k,m}=g_{M\left(m-k\right)}^{cos}\triangleq \sum _{u=-\infty }^{\infty }{h}_{u}cos\left(\frac{2\mathrm{\pi u}}{M}\right)h_{u+M\left(k-m\right)}\\ \mathbf{R}=\left[\begin{array}{cccc}{g}_0^{cos}& g_M^{cos}& \dots & g_{M\left(N-1\right)}^{cos}\\ g_{-M}^{cos}& g_0^{cos}\\ ⋮& \ddots & ⋮\\ g_{-M\left(N-1\right)}^{cos}& \dots & g_0^{cos}\end{array}\right]\hfill \end{array}$$

(22)

The matrix R is real. It is noted that filters are expressed with a non-casual representation in order to exploit symmetries in a simple way. As a next step, the following $\frac{\mathit{\text{LN}}}{2}\times \frac{\mathit{\text{LN}}}{2}$ low triangular block matrix is defined:

$$\mathbf{P}\triangleq \underset{L/2N\times N\text{matrices}}{\underbrace{\left[\begin{array}{cccc}\mathbf{R}& \mathbf{0}& \dots & \mathbf{0}\\ \mathbf{R}& \mathbf{R}& \mathbf{0}\\ \mathbf{0}& \mathbf{R}& \mathbf{R}& \ddots \\ ⋮& \ddots & \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{R}& \mathbf{R}\end{array}\right]}}$$

(23)

Based on the definition of (23) and using (18), (19) and (22), the produced result is given by

$$\mathrm{E}\left(\mathbf{y}{\mathbf{y}}^{\text{H}}/r\sim \mathcal{N}\left(0,1\right)\right)\Delta =\mathbf{C}=\left[\begin{array}{ccc}{\mathbf{I}}_{\mathit{\text{LN}}/2}& |& \mathbf{P}\\ -& -& -\\ {\mathbf{P}}^{\text{T}}& |& {\mathbf{I}}_{\mathit{\text{LN}}/2}\end{array}\right]$$

(24)

The definition of the L N×L N matrix leads directly to the extraction of the covariance matrix for variable y given that the primary channel is free $\left({\mathcal{\mathscr{H}}}_0\right)$. It is straightforward to claim that

$${\mathcal{\mathscr{H}}}_0:{\mathbf{C}}_y={\mathbf{C}}_w={\sigma }_w^2\mathbf{C}\Rightarrow \mathbf{y}/{\mathcal{\mathscr{H}}}_0\sim \mathcal{N}\left(\mathbf{0},{\sigma }_w^2\mathbf{C}\right)$$

(25)

The fact that C is expressed as a 2×2 block matrix with identity matrices in the block diagonal justifies the selection of the specific reordering since the computations are now quite simplified. More specifically, the inverse of C is given by ([32])

$$\begin{array}{c}{\mathbf{C}}^{-1}=\left[\begin{array}{ccc}{\left({\mathbf{I}}_{\mathit{\text{LN}}/2}-\mathbf{P}{\mathbf{P}}^{\text{T}}\right)}^{-1}& |& -\mathbf{P}{\left({\mathbf{I}}_{\mathit{\text{LN}}/2}-{\mathbf{P}}^{\text{T}}\mathbf{P}\right)}^{-1}\\ -& -& -\\ -{\mathbf{P}}^{\text{T}}{\left({\mathbf{I}}_{\mathit{\text{LN}}/2}-\mathbf{P}{\mathbf{P}}^{\text{T}}\right)}^{-1}& |& {\left({\mathbf{I}}_{\mathit{\text{LN}}/2}-{\mathbf{P}}^{\mathrm{T}}\mathbf{P}\right)}^{-1}\end{array}\right]\end{array}$$

(26)

An important feature is that the determinant of C can be easily calculated using $\left|\mathbf{C}\right|=\left|{\mathbf{I}}_{\mathit{\text{LN}}/2}-\mathbf{P}{\mathbf{P}}^{\text{T}}\right|$[33], since the matrix is real and symmetric. In addition, the conditional probabilities between groups 0 (even filter paths) and 1 (odd filter paths) are extracted:

$$\begin{array}{c}\mathbf{y}\sim \mathcal{N}\left(\mathbf{0},{\mathbf{C}}_w\right)\Rightarrow \left\{\begin{array}{c}{\mathbf{y}}^{\left(0\right)}/{\mathbf{y}}^{\left(1\right)}\sim \mathcal{N}\left(\mathbf{P}{\mathbf{y}}^{\left(1\right)},{\sigma }_w^2\left(\mathbf{I}-\mathbf{P}{\mathbf{P}}^{\text{T}}\right)\right)\\ {\mathbf{y}}^{\left(1\right)}/{\mathbf{y}}^{\left(0\right)}\sim \mathcal{N}\left({\mathbf{P}}^{\text{T}}{\mathbf{y}}^{\left(0\right)},{\sigma }_w^2\left(\mathbf{I}-{\mathbf{P}}^{\text{T}}\mathbf{P}\right)\right)\end{array}\right.\end{array}$$

(27)

3.2 Distribution under ${\mathcal{\mathscr{H}}}_{\mathbf{1}}$

According to the initial assumptions, the primary signal s _n follows the normal distribution with mean power ${\sigma }_s^2$. Initially, a noiseless channel is assumed. Due to the radio channel and based on the approximation in (11), the input signal power per subchannel is given by the multiplication of the mean signal power with a radio channel coefficient, i.e. the assumed input per subchannel is given by $r_n^{\left(i\right)}=c_i{s}_n$ and the mean input signal power per subchannel is $\text{E}\left({\left|r_n^{\left(i\right)}\right|}^2\right)=\text{E}\left({\left|c_i{s}_n\right|}^2\right)={\left|c_i\right|}^2{\sigma }_s^2$. The calculation of the covariance matrix of the observation vector is made using the procedure of the previous paragraph with the introduction of the radio channel coefficient effects. Since the result presented in (18) has been computed for standard zero-mean Gaussian input (σ²=1), the autocovariance matrix for the output of the i th filter path, when the input is given by $r_n^{\left(i\right)}=c_i{s}_n$, is provided by

$$\mathrm{E}\left({\mathbf{y}}_i{\mathbf{y}}_i^{\text{H}}\right)={\left|c_i\right|}^2{\sigma }_s^2{\mathbf{I}}_N$$

(28)

The cross-covariance submatrix between filter outputs of non-directly adjacent paths is once again zero. For directly adjacent filter paths, computation is performed as in (20) with the inclusion of the channel coefficients.

$$\begin{array}{l}{\left[\mathbf{R}\right]}_{k,m}=\mathrm{E}\left(y_n^{\left(i\right)}{y}_{n+m-k}^{(i+l)\ast }\right)=\mathrm{E}\left(y_n^{\left(i\right)}{y}_{n+\epsilon }^{(i+l)\ast }\right)=\\ \sum _{u=-\infty }^{\infty }\sum _{p=-\infty }^{\infty }{h}_u{h}_p{e}^{2\mathrm{\pi j}\frac{\mathit{\text{iu}}}{M}}{e}^{-2\mathrm{\pi j}\frac{(i+l)p}{M}}\mathrm{E}\left(c_i{s}_{\mathit{\text{Mn}}-u}{c}_{i+l}^{\ast }{s}_{\mathit{\text{Mn}}-p+\mathrm{M\epsilon }}^{\ast }\right)\end{array}$$

(29)

In order to avoid complex coefficients, since there is no simple method to estimate their phase, the following rationale is considered: A complex zero-mean Gaussian variable can also be seen as a pair of two random variables: (a) the Rayleigh amplitude $\left|s_n\right|\sim \text{Rayleigh}\left({\sigma }_s^2\right)$ and (b) the uniform in [0,2π) phase ${\phi }_{s_n}\sim \mathcal{U}\left(0,2\pi \right)$. After the application of the channel coefficient on the signal, the considered input of the i th filter is given by

$$r_n^{\left(i\right)}=c_i\left|s_n\right|e^{j{\phi }_{s_n}}=\left|c_i\right|\left|s_n\right|e^{j{\phi }_{s_n}+{\phi }_{c_i}}$$

(30)

However, the radio-channel-induced phase shifts due to the uniform phase distribution and periodicity will not cause any variation in the signal phase distribution and therefore:

$${\phi }_{s_n}+{\phi }_{c_i}\sim \mathcal{U}\left(0,2\pi \right)\Rightarrow s_n^{\prime }=\left|s_n\right|e^{j({\phi }_{s_n}+{\phi }_{c_i})}\sim \mathcal{N}\left(0,{\sigma }_s^2\right)$$

(31)

From (31) and (30) it is extracted that

$$r_n^{\left(i\right)}\sim \mathcal{N}\left(0,{\left|c_i\right|}^2{\sigma }_s^2\right)$$

(32)

Using (32) and (29) similarly to (21), it is concluded that

$$\text{E}\left({\mathbf{y}}_i{\mathbf{y}}_{i+l}^{\text{H}}\right)={\sigma }_s^2\left|c_i\right|\left|c_{i+l}\right|\mathbf{R}$$

(33)

The diagonal matrix Σ is defined:

$$\begin{array}{c}\mathit{\Sigma }=\left[\begin{array}{ccccc}\left|c_0\right|{\mathbf{I}}_N& \mathbf{0}& \mathbf{0}& \dots & \mathbf{0}\\ \mathbf{0}& \left|c_2\right|{\mathbf{I}}_N& \mathbf{0}& \mathbf{0}\\ \mathbf{0}& \ddots & \ddots & \ddots & \ddots \\ \ddots & \mathbf{0}& \left|c_{L-2}\right|{\mathbf{I}}_N& \mathbf{0}& \mathbf{0}\\ ⋮& \mathbf{0}& \mathbf{0}& \left|c_1\right|{\mathbf{I}}_N& \mathbf{0}& \ddots \\ \mathbf{0}& \mathbf{0}& \left|c_3\right|{\mathbf{I}}_N& \mathbf{0}& \mathbf{0}\\ \ddots & \ddots & \ddots & \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{0}& \left|c_{L-1}\right|{\mathbf{I}}_N\end{array}\right]\end{array}$$

(34)

where the diagonal elements contain the channel coefficients according to the adopted reordering. Based on the definition of Σ, it is concluded that under ${\mathcal{\mathscr{H}}}_1$ with no noise, the received signal covariance matrix is given by

$${\mathbf{C}}_s={\sigma }_s^2\mathit{\Sigma }\mathbf{C}\mathit{\Sigma }$$

(35)

Let us now assume that AWGN with mean power ${\sigma }_w^2$ is also present during reception. In this case, the input for the i th subchannel assuming that the primary signal s _n follows the zero-mean Gaussian distribution with mean power ${\sigma }_s^2$ is given by

$$r_n^{\left(i\right)}=c_i{s}_n+w_n$$

(36)

where w _n is the noise component with $w_n\sim \mathcal{N}\left(0,{\sigma }_w^2\right)$. It was proved for a noiseless signal in (32) that $c_i{s}_n\sim \mathcal{N}\left(\mathbf{0},{\left|c_i\right|}^2{\sigma }_s^2\right)$. The input signal for the i th subchannel $r_n^{\left(i\right)}$ is given by the sum of two zero-mean Gaussian variables with variances ${\left|c_i\right|}^2{\sigma }_s^2$ and ${\sigma }_w^2$. Based on the properties of the Gaussian distribution [24] and given that signal and noise are independent and uncorrelated random variables, $r_n^{\left(i\right)}$ will also follow the zero-mean Gaussian distribution and its variance will be provided by the sum of the variances of the two random variables (signal and noise). Therefore,

$$r_n^{\left(i\right)}\sim \mathcal{N}\left(0,{\left|c_i\right|}^2{\sigma }_s^2+{\sigma }_w^2\right)$$

(37)

If z _n is a zero-mean Gaussian variable with σ²=1, then the variable $\left(\sqrt{{\left|c_i\right|}^2{\sigma }_s^2+{\sigma }_w^2}\right)z_n$ follows the same distribution with $r_n^{\left(i\right)}$ and it can be used for the calculation of the autocovariance and cross-covariance matrices. Similarly to (28) and (29), it is proved that

$$\text{E}\left({\mathbf{y}}_i{\mathbf{y}}_i^{\text{H}}\right)=\left({\left|c_i\right|}^2{\sigma }_s^2+{\sigma }_w^2\right){\mathbf{I}}_N$$

(38)

and for l=±1,

$$\begin{array}{c}\text{E}\left(y_n^{\left(i\right)}{y}_{n+m-k}^{(i+l)\ast }\right)=\sum _{u=-\infty }^{\infty }\sum _{p=-\infty }^{\infty }{h}_u{h}_p{e}^{2\mathrm{\pi j}\frac{\mathit{\text{iu}}}{M}}{e}^{-2\mathrm{\pi j}\frac{(i+l)p}{M}}\\ \times \text{E}\left(r_{\mathit{\text{Mn}}-u}{r}_{\mathit{\text{Mn}}-p+\mathrm{M\epsilon }}^{\ast }\right)\\ =\sum _{u=-\infty }^{\infty }\sum _{p=-\infty }^{\infty }{h}_u{h}_p{e}^{2\mathrm{\pi j}\frac{\mathit{\text{iu}}}{M}}{e}^{-2\mathrm{\pi j}\frac{(i+l)p}{M}}\times \\ \times \text{E}\left(\left(\sqrt{{\left|c_i\right|}^2{\sigma }_s^2+{\sigma }_w^2}\right)z_{\mathit{\text{Mn}}-u}\right.\\ \times \left.\left(\sqrt{{\left|c_{i+l}\right|}^2{\sigma }_s^2+{\sigma }_w^2}\right)z_{\mathit{\text{Mn}}-p+\mathrm{M\epsilon }}^{\ast }\right)\\ =\left(\sqrt{{\left|c_i\right|}^2{\sigma }_s^2+{\sigma }_w^2}\right)\left(\sqrt{{\left|c_{i+l}\right|}^2{\sigma }_s^2+{\sigma }_w^2}\right){\left[\mathbf{R}\right]}_{k,m}\end{array}$$

(39)

which concludes that $\text{E}\left({\mathbf{y}}_i{\mathbf{y}}_{i+l}^{\text{H}}\right)=\left(\sqrt{{\left|c_i\right|}^2{\sigma }_s^2+{\sigma }_w^2}\right)\left(\sqrt{{\left|c_{i+l}\right|}^2{\sigma }_s^2+{\sigma }_w^2}\right)\mathbf{R}$. The combination of the results from (38) and (39) are now used to extract the correlation matrix for vector y defined in (9). It is proved that

$${\mathbf{C}}_y={\mathit{\Sigma }}_y\mathbf{C}{\mathit{\Sigma }}_y,\iff {\mathbf{C}}_y^{-1}={\mathit{\Sigma }}_y^{-1}{\mathbf{C}}^{-1}{\mathit{\Sigma }}_y^{-1}$$

(40)

where Σ _y is the diagonal matrix containing the factors $\left(\sqrt{{\left|c_i\right|}^2{\sigma }_s^2+{\sigma }_w^2}\right)$.

$${\mathit{\Sigma }}_y={\left({\sigma }_s^2{\mathit{\Sigma }}^2+{\sigma }_w^2{\mathbf{I}}_{\mathit{\text{LN}}}\right)}^{1/2}$$

(41)

Thus, based on the Gaussian distribution properties, since the observation vector under ${\mathcal{\mathscr{H}}}_1$ is a linear transformation of the zero-mean Gaussian input and its covariance matrix is given by (40), it follows the distribution of (42):

$$\begin{array}{c}\mathbf{y}/{\mathcal{\mathscr{H}}}_1\sim \mathcal{N}\left(\mathbf{0},{\left({\sigma }_s^2{\mathit{\Sigma }}^2+{\sigma }_w^2{\mathbf{I}}_{\mathit{\text{LN}}}\right)}^{1/2}\mathbf{C}{\left({\sigma }_s^2{\mathit{\Sigma }}^2+{\sigma }_w^2{\mathbf{I}}_{\mathit{\text{LN}}}\right)}^{1/2}\right)\end{array}$$

(42)

If the known (or estimated) quantity is the SNR γ _i , the channel coefficients can be calculated from ${\left|c_i\right|}^2={{\gamma }_i}^{{\sigma }_w^2}/{}_{{\sigma }_s^2}$.

3.3 The detector metric

In order to define the detector, starting from (14), the log-likelihood relationship is formed:

$$\begin{array}{c}ln\left(\left|{\mathbf{C}}_w\right|\right)-ln\left(\left|{\mathbf{C}}_y\right|\right)+{\mathbf{y}}^{\text{H}}\underset{w}{\overset{-1}{\mathbf{C}}}\mathbf{y}-{\mathbf{y}}^{\text{H}}\underset{y}{\overset{-1}{\mathbf{C}}}\mathbf{y}\underset{{\mathcal{\mathscr{H}}}_0}{\overset{{\mathcal{\mathscr{H}}}_1}{\gtrless }}ln\left(\theta \right)\Rightarrow \\ {\mathbf{y}}^{\text{H}}\left({\mathbf{C}}_w^{-1}-{\mathbf{C}}_y^{-1}\right)\mathbf{y}\underset{{\mathcal{\mathscr{H}}}_0}{\overset{{\mathcal{\mathscr{H}}}_1}{\gtrless }}{\theta }^{\prime }=ln\left(\theta \right)+ln\left(\left|{\mathbf{C}}_y\right|\right)-ln\left(\left|{\mathbf{C}}_w\right|\right)\end{array}$$

(43)

Therefore, the metric that will be used for the decision is provided by

$$T\left(\mathbf{y}\right)={\mathbf{y}}^{\text{H}}\left({\mathbf{C}}_w^{-1}-{\mathbf{C}}_y^{-1}\right)\mathbf{y}$$

(44)

3.4 Metric distribution under ${\mathcal{\mathscr{H}}}_{\mathbf{0}}$

Initially, the eigenvalue decomposition of matrix C=U Λ U^H is performed and since C is hermitian U U^H=U^HU=I. With the use of the eigenvalues and eigenvectors, an interim variable can be defined:

$$\mathbf{z}=\frac{1}{{}_{\sigma }^w}{\mathit{\Lambda }}^{-1/2}{\mathbf{U}}^{\text{H}}\mathbf{y}$$

(45)

It is simple to prove that this variable decorrelates the samples of vector y and that $\mathbf{z}\sim \mathcal{N}\left(\mathbf{0},{\mathbf{I}}_{\mathit{\text{LN}}}\right)$. Thus, the metric can be written as

$$\begin{array}{l}{\mathbf{y}}^{\text{H}}\left({\mathbf{C}}_w^{-1}-{\mathbf{C}}_y^{-1}\right)\mathbf{y}\Rightarrow {\mathbf{z}}^{\text{H}}{\sigma }_w{\mathit{\Lambda }}^{1/2}{\mathbf{U}}^{\text{H}}\left({}^1/{}_{{\sigma }_w^2}{\mathbf{C}}^{-1}-{{\mathit{\Sigma }}_y}^{-1}{\mathbf{C}}^{-1}{{\mathit{\Sigma }}_y}^{-1}\right)\\ \times \mathbf{U}{\mathit{\Lambda }}^{1/2}{\sigma }_w\mathbf{z}\\ ={\mathbf{z}}^{\text{H}}\left(\mathbf{I}-{\sigma }_w^2{\mathit{\Lambda }}^{1/2}{\mathbf{U}}^{\text{H}}{{\mathit{\Sigma }}_y}^{-1}{\mathbf{C}}^{-1}{{\mathit{\Sigma }}_y}^{-1}\mathbf{U}{\mathit{\Lambda }}^{1/2}\right)\mathbf{z}\end{array}$$

(46)

In order to calculate the eigenvalues of the matrix that appears at the quadratic form of the metric, the definition of the characteristic polynomial is used:

$$\begin{array}{c}\text{det}\left(\mathbf{I}-\underset{w}{\overset{2}{\sigma }}{\mathit{\Lambda }}^{1/2}{\mathbf{U}}^{\text{H}}{{\mathit{\Sigma }}_y}^{-1}{\mathbf{C}}^{-1}{{\mathit{\Sigma }}_y}^{-1}\mathbf{U}{\mathit{\Lambda }}^{1/2}-\chi \mathbf{I}\right)\\ =\text{det}\left(\underset{w}{\overset{2}{\sigma }}{\mathit{\Lambda }}^{1/2}{\mathbf{U}}^{\text{H}}{{\mathit{\Sigma }}_y}^{-1}{\mathbf{C}}^{-1}{{\mathit{\Sigma }}_y}^{-1}\mathbf{U}{\mathit{\Lambda }}^{1/2}+\left(\chi -1\right)\mathbf{I}\right)\\ =\text{det}\left(\mathit{\Lambda }\right)\text{det}\left(\underset{w}{\overset{2}{\sigma }}{{\mathit{\Sigma }}_y}^{-1}{\mathbf{C}}^{-1}{{\mathit{\Sigma }}_y}^{-1}+\left(\chi -1\right){\mathbf{C}}^{-1}\right)=0\end{array}$$

(47)

The determinant of Λ can be omitted. In order to move forward, the approach described below is followed:

• The inverse of C^-1 is also an hermitian matrix.

• Thus, the matrix $\mathbf{V}={\sigma }_w^2{{\mathit{\Sigma }}_y}^{-1}{\mathbf{C}}^{-1}{{\mathit{\Sigma }}_y}^{-1}+\left(\chi -1\right){\mathbf{C}}^{-1}$ can be seen as the sum of two covariance matrices of multidimensional Gaussian variables. It is known from probability theory [34] that if two random variables are independent, the covariance matrix of their sum is equal to the sum of the covariance matrices. Let us assume a random variable that follows the distribution $\mathcal{N}(0,{\sigma }_w^2{{\mathit{\Sigma }}_y}^{-2})$ and a second random variable that follows the distribution $\mathcal{N}(0,(\chi -1\left)\mathbf{I}\right)$. In addition, the two variables are considered independent. It is assumed that the variables are added and applied to the selected filter bank. Then, the covariance matrix of their sum at the output of the bank will be provided by matrix V.

• However, for independent Gaussian variables a different approach can be used that produces equivalent results. The sum of two independent zero-mean Gaussian random variables is also a zero-mean Gaussian random variable. The variance of the sum is equal with the sum of variances [34]. Therefore, instead of considering the sum of two independent random variables applied to the filter bank, it is equivalent to assume a single random variable that follows the distribution $\mathcal{N}(0,{\sigma }_w^2{{\mathit{\Sigma }}_y}^{-2}+(\chi -1\left)\mathbf{I}\right)$.

• Based on the second approach, the covariance matrix at the output of the filter will be given by

  $$\mathbf{W}={\left({\sigma }_w^2{{\mathit{\Sigma }}_y}^{-2}+\left(\chi -1\right)\mathbf{I}\right)}^{\frac{1}{2}}{\mathbf{C}}^{-1}{\left({\sigma }_w^2{{\mathit{\Sigma }}_y}^{-2}+\left(\chi -1\right)\mathbf{I}\right)}^{\frac{1}{2}}$$

  (48)

• The extraction of the covariance matrix in this case is performed using the same procedure that was followed in (39), (40) and (41).

• Since the two procedures are equivalent for Gaussian random variables, it is expected that the equivalence will also stand between the covariance matrices V and W and therefore they can be used interchangeably. The followed approach is valid, since the eigenvalues extracted by V are applied on the Gaussian random variable $\mathbf{z}\sim \mathcal{N}\left(\mathbf{0},{\mathbf{I}}_{\mathit{\text{LN}}}\right)$. In addition, the validity of the procedure was verified with extended tests.

• Thus, instead of calculating the determinant of V, we proceed using matrix W. From (48) and given the fact that the determinant of C is non-zero (since C is invertible), it is extracted that

  $$\text{det}\left(\underset{w}{\overset{2}{\sigma }}{{\mathit{\Sigma }}_y}^{-2}+\left(\chi -1\right)\mathbf{I}\right)=0\Rightarrow {\chi }_i=1-\frac{\underset{w}{\overset{2}{\sigma }}}{\underset{s}{\overset{2}{\sigma }}{\left|c_i\right|}^2+\underset{w}{\overset{2}{\sigma }}}$$

  (49)

• With the use of V- W equivalence, the metric matrix is considered to have L discrete eigenvalues with N degrees of multiplicity each. The eigenvector matrix does not play any role in the procedure and it does not need to be calculated. Since U is unitary, the random variable $\tilde{\mathbf{z}}={\mathbf{U}}^{\text{H}}\mathbf{z}$ will follow the same distribution as z. Therefore, the metric under ${\mathcal{\mathscr{H}}}_0$ is given by

  $$T\left(\mathbf{y}/{\mathcal{\mathscr{H}}}_0\right)=\sum _{l=0}^{L-1}\sum _{n=0}^{N-1}\frac{{\sigma }_s^2{\left|c_l\right|}^2}{{\sigma }_s^2{\left|c_l\right|}^2+{\sigma }_w^2}{\left|{\tilde{z}}_{n+\mathit{\text{lN}}}\right|}^2$$

  (50)

• In order to verify the validity of the predescribed equivalence, extended tests were performed for various filters and radio channels. The eigenvalues of matrix V were numerically extracted and used directly for metric calculation. It was observed that in all cases, the eigenvalues of V and W were very similar. However, the most important observation was that they produced, as expected, identical distributions when applied to z for metric calculation. Therefore, the equivalence was also verified via simulations.

In (50), each ${\tilde{z}}_i$ element follows the standard complex Gaussian distribution (unit variance). Consequently,

$$2{\left|{\tilde{z}}_{n+\mathit{\text{lN}}}\right|}^2\sim {\chi }_2^2$$

(51)

The chi-square distribution can be considered a special case of Gamma distribution. In fact, if a random variable ω follows the ${\chi }_2^2$ distribution, for a positive constant β, the variable β ω follows the gamma distribution with parameters (1,2β) [35]. Therefore,

$$\frac{{\sigma }_s^2{\left|c_l\right|}^2}{{\sigma }_s^2{\left|c_l\right|}^2+{\sigma }_w^2}{\left|{\tilde{z}}_{n+\mathit{\text{lN}}}\right|}^2\sim \mathcal{G}\left(1,\frac{{\sigma }_s^2{\left|c_l\right|}^2}{{\sigma }_s^2{\left|c_l\right|}^2+{\sigma }_w^2}\right)$$

(52)

where denotes the gamma distribution. In order to calculate the metric distribution, the characteristic function for is used. Moreover, since the addends in (50) are independent, the characteristic function of the total metric will be given by the product of the functions of the individual variables. Thus,

$${\phi }_{T(\mathbf{y}/{\mathcal{\mathscr{H}}}_0)}\left(\tau \right)=\prod _{n=0}^{L-1}{\left(\frac{1}{1-j\frac{{\sigma }_s^2{\left|c_l\right|}^2}{{\sigma }_s^2{\left|c_l\right|}^2+{\sigma }_w^2}\tau }\right)}^N$$

(53)

The PDF is given by the inverse Fourier of the characteristic function at point -t.

$$f_{T\left(\mathbf{y}/{\mathcal{\mathscr{H}}}_0\right)}\left(t\right)={\mathcal{F}}_{-t}^{-1}\left\{\prod _{n=0}^{L-1}{\left(\frac{1}{1-j\frac{{\sigma }_s^2{\left|c_l\right|}^2}{{\sigma }_s^2{\left|c_l\right|}^2+{\sigma }_w^2}\tau }\right)}^N\right\}$$

(54)

The calculation of the analytical solution is not a straightforward task because of the fact that each eigenvalue is repeated N times. An analytical solution can be achieved using the distribution provided in [36] where the positive definite matrix of [36] is the identity matrix. The metric distribution is given by

$$\begin{array}{l}{f}_{T\left(\mathbf{y}\right)}\left(t>0/{\mathcal{\mathscr{H}}}_0\right)=\prod _{i=1}^{L-1}{\left(\frac{{\lambda }_{min}^{\text{udft}/{\mathcal{\mathscr{H}}}_0}}{{\lambda }_l^{\text{udft}/{\mathcal{\mathscr{H}}}_0}}\right)}^N\\ \times \sum _{k=0}^{\infty }\frac{d_k{t}^{\mathit{\text{LN}}+k-1}{e}^{-\left(\frac{t}{{\lambda }_{min}^{\text{udft}/{\mathcal{\mathscr{H}}}_0}}\right)}}{{\left({\lambda }_{min}^{\text{udft}/{\mathcal{\mathscr{H}}}_0}\right)}^{\mathit{\text{LN}}+k}\Gamma \left(\mathit{\text{LN}}+k\right)}\end{array}$$

(55)

where coefficients d _k are calculated iteratively using

$$\begin{array}{c}\begin{array}{c}{d}_0=1,\\ d_{k+1}=\frac{N}{k+1}\sum _{i=1}^{k+1}\left[\sum _{l=1}^{L-1}{\left(1-\frac{{\lambda }_{min}^{\text{udft}/H_0}}{{\lambda }_l^{\text{udft}/H_0}}\right)}^i\right]d_{k+1-i},k=0,1,2\dots \end{array}\end{array}$$

(56)

and the λ _i s are the L discrete eigenvalues, thus,

$${\lambda }_i^{\text{udft}/{\mathcal{\mathscr{H}}}_0}=\frac{{\sigma }_s^2{\left|c_i\right|}^2}{{\sigma }_w^2+{\sigma }_s^2{\left|c_i\right|}^2}$$

(57)

According to the Neyman-Pearson theorem, the threshold is calculated from the probability of false alarm P_FA under ${\mathcal{\mathscr{H}}}_0$ through the complimentary cumulative distribution function (CDF). The following relationship is used [36]:

$$\begin{array}{c}{P}_{\text{FA}}=1-\prod _{i=1}^{L-1}{\left(\frac{{\lambda }_{min}^{\text{udft}/{\mathcal{\mathscr{H}}}_0}}{{\lambda }_l^{\text{udft}/{\mathcal{\mathscr{H}}}_0}}\right)}^N\sum _{k=0}^{\infty }{d}_k\left(\frac{\Gamma \left(\mathit{\text{LN}}+k,\frac{\theta }{{\lambda }_l^{\text{udft}/{\mathcal{\mathscr{H}}}_0}}\right)}{\Gamma \left(\mathit{\text{LN}}+k\right)}\right)\end{array}$$

(58)

Threshold θ is calculated for a given P_FA with numerical inversion of the function.

The main problem regarding the use of (55), (56) and (58) in a sensing algorithm is the infinite sum. In a practical implementation, the truncation of the infinite sum is unavoidable. In order to calculate the number of needed addends (or equivalently d _k coefficients) that will provide a sufficient approximation of the distribution, the following simulation procedure was followed. A large set of 50,000 channels with exponential PDP was produced. The exponential PDP is given by

$$\text{PDP}=exp\left(-\mathrm{\lambda n}\right),\text{for}n=0,1\dots ,C_{max}$$

(59)

During the simulation, the used PDP parameters were λ=0.7, C_max=48. The c _i coefficients were calculated using (12) and the eigenvalues ${\lambda }_i^{\text{udft}/{\mathcal{\mathscr{H}}}_0}$ were determined using (57). It was concluded that the number of the necessary d _k coefficients depends on the spread of the extracted eigenvalues. This means that if the standard deviation of the L eigenvalues is small, then the approximation converges fast to the distribution of (55); otherwise, a very large number of d _k coefficients should be calculated. In order to extract an empirical rule for the number of needed coefficients, the ratio of the maximum eigenvalue ${\lambda }_{max}^{\text{udft}/{\mathcal{\mathscr{H}}}_0}$ over the minimum eigenvalue ${\lambda }_{min}^{\text{udft}/{\mathcal{\mathscr{H}}}_0}$ was used as a measure of the eigenvalue spread. For each channel, the values in (55) and (56) were calculated iteratively as k was continuously increasing. In each step, numerical integration of the estimated PDF was performed. When the integration result becomes larger than 0.9999, it is assumed that the approximation has converged to the desired distribution.In Figure 4, the simulation results are presented. It can be seen that in the vast majority of cases, less than 1,000 coefficients are needed for successful convergence. The prediction bounds of the linear fit (using the minimum mean square error (MMSE) technique) for 99.99% confidence level are also provided in Figure 4. It was estimated that based on the simulation, the upper prediction bound is described by the following straight line:

$$y=25.26x+300.3$$

(60)

Figure 4

Simulation results of the number of coefficients for distribution approximation.

Therefore, if the number of coefficients is estimated using (60), then there will be 99.99% probability that the approximated distribution has successfully converged. It is noted that with the use of the described procedure, a very accurate estimate of the P_FA from (58) is achieved with absolute error less than 10^-5 for P_FA>0.25.

In order to avoid the need for an approximation of an infinite sum, a method of numerical integration of the characteristic function can be used. During this study, a simple fast Fourier transform (FFT)-based algorithm that performs the integration was developed. Let us assume that the energy of the characteristic function is concentrated in the space τ∈[ -δ δ ]. Then, it is proved that (54) can be accurately approximated by

$$\begin{array}{c}{\widehat{f}}_{T\left(\mathbf{y}\right)}\left(t_k=\frac{\mathrm{\pi k}}{\delta }/{\mathcal{\mathscr{H}}}_0\right)=\frac{\delta e^{\mathrm{j\pi k}}}{\pi N^{\prime }}\text{FFT}{\left[{\phi }_{T(\mathbf{y}/{\mathcal{\mathscr{H}}}_0)}\left(\mathrm{n\Delta \tau }-\delta \right)\right]}_{N^{\prime }}\end{array}$$

(61)

The FFT-length N^′ can be found as an integer with N^′Δ τ=τ_max with τ_max a value where the PDF is expected to be practically zero and $\mathrm{\Delta \tau }=\frac{1}{2\delta }$. Therefore, fast Fourier transforms can be used to approximate with high accuracy the distribution under investigation. The complimentary CDF can be approximated with the trapezoidal numerical integration technique. Thus, for a threshold ${\theta }^{\prime }=\frac{\pi k_0}{\delta }$, the P_FA is given by

$$P_{\text{FA}}\simeq \sum _{n=0}^{k_0}\frac{\pi }{\delta }\left(\frac{{\widehat{f}}_{T\left(\mathbf{y}\right)}\left(\frac{\mathrm{\pi k}}{\delta }/{\mathcal{\mathscr{H}}}_0\right)+{\widehat{f}}_{T\left(\mathbf{y}\right)}\left(\frac{\pi \left(k+1\right)}{\delta }/{\mathcal{\mathscr{H}}}_0\right)}{2}\right)$$

(62)

Given the fact that based on the aforementioned analysis, the selected threshold θ^′ is also discrete, an approximation of the P_FA for any threshold can be calculated using linear interpolation.

3.5 Metric distribution under ${\mathcal{\mathscr{H}}}_{\mathbf{1}}$

Under ${\mathcal{\mathscr{H}}}_1$, the interim variable can be defined as

$$\mathbf{z}={\mathit{\Lambda }}^{-1/2}{\mathbf{U}}^{\text{H}}{{\mathit{\Sigma }}_y}^{-1}\mathbf{y}$$

(63)

In this case, the determinant that provides the eigenvalues is given by

$$\begin{array}{l}\text{det}\left(\frac{{\mathit{\Sigma }}_y{\mathbf{C}}^{-1}{\mathit{\Sigma }}_y}{\underset{w}{\overset{2}{\sigma }}}-\left(\chi +1\right){\mathbf{C}}^{-1}\right)\\ =\text{det}\left({\left(\frac{{{\mathit{\Sigma }}_y}^2}{\underset{w}{\overset{2}{\sigma }}}-\left(\chi +1\right)\mathbf{I}\right)}^{\frac{1}{2}}{\mathbf{C}}^{-1}{\left(\frac{{{\mathit{\Sigma }}_y}^2}{\underset{w}{\overset{2}{\sigma }}}-\left(\chi +1\right)\mathbf{I}\right)}^{\frac{1}{2}}\right)=0\end{array}$$

(64)

which leads to ${\chi }_i=\frac{{\sigma }_s^2{\left|c_i\right|}^2}{{\sigma }_w^2}$.

The calculation of the metric distribution under ${\mathcal{\mathscr{H}}}_1$ can be made with the same techniques as before:

• Using the analytical solution (55) and (56), where the used eigenvalues are ${\lambda }_i^{\text{udft}/{\mathcal{\mathscr{H}}}_1}=\frac{{\sigma }_s^2{\left|c_i\right|}^2}{{\sigma }_w^2}$

• With FFT-based approximation of the PDF from the characteristic function given by

  $${\phi }_{T\left(\mathbf{y}/{\mathcal{\mathscr{H}}}_1\right)}\left(\tau \right)=\prod _{n=0}^{L-1}{\left(1-j\frac{{\sigma }_s^2{\left|c_i\right|}^2}{{\sigma }_w^2}\tau \right)}^{-N}$$

  (65)

After the extraction of the metric distribution under ${\mathcal{\mathscr{H}}}_1$ and for a given threshold, the probability of correct detection P_D can be calculated. Using the analytical solution, the following formula is derived [36]:

$$\begin{array}{c}{P}_{\text{D}}=1-\prod _{i=1}^{L-1}{\left(\frac{{\lambda }_{min}^{\text{udft}/{\mathcal{\mathscr{H}}}_1}}{{\lambda }_l^{\text{udft}/{\mathcal{\mathscr{H}}}_1}}\right)}^N\sum _{k=0}^{\infty }{d}_k\left(\frac{\Gamma \left(\mathit{\text{LN}}+k,\frac{{\theta }^{\prime }}{{\lambda }_l^{\text{udft}/{\mathcal{\mathscr{H}}}_1}}\right)}{\Gamma \left(\mathit{\text{LN}}+k\right)}\right)\end{array}$$

(66)

When the truncation of the infinite sum is done using the ratio of the maximum over the minimum eigenvalue and the empirical formula of (60), then the approximation of the P_D value is extremely accurate (absolute error less than 10^-4).

4 Detector at the OFDM-OQAM receiver output

The candidacy of OFDM-OQAM as a modulation technique for CR system leads inevitably to the need for a detector design for such systems. As mentioned in Section 2, the OFDM-OQAM systems can be assumed to use an MDFT filter bank structured as the block diagrams in Figure 2. The two structures are equivalent and can be used to implement the filtering unit of an OFDM-OQAM demodulator. In order to design the detector, the samples after the real/imaginary separation blocks are collected. At this point, the signal has practically returned to the Nyquist sampling rate. The second structure is selected, which contains two similar parallel uniform DFT banks with maximally decimated output in order to simplify the mathematical formulation of the problem. The two banks are operating with relevant time offset of M/2 symbols. The signal that is led to the detector can be defined as

$$\begin{array}{cc}{\mathbf{y}}_{2i}^{\text{oqam}}& =\Re \left({\mathbf{y}}_{2i}^{\left(\text{A}\right)}\right)+\mathit{\text{jI}}\left({\mathbf{y}}_{2i}^{\left(\text{B}\right)}\right)\\ {\mathbf{y}}_{2i+1}^{\text{oqam}}& =\mathit{\text{jI}}\left({\mathbf{y}}_{2i+1}^{\left(\text{A}\right)}\right)+\Re \left({\mathbf{y}}_{2i+1}^{\left(\text{B}\right)}\right)\\ i& =0,1\mathrm{..M}/2\end{array}$$

(67)

The superscript indicates whether the specific signal is extracted by the first upper bank (A) or the second lower bank (B). Initially, an examination is performed to verify that there is no correlation between the real and imaginary part of the signal. The real/imaginary separation blocks can be expressed as simple linear relationships:

$$\begin{array}{l}\Re \left({\mathbf{y}}_i^{\left(\text{X}\right)}\right)=\left[\frac{{\mathbf{y}}_i^{\left(\text{X}\right)}+{\left({\mathbf{y}}_i^{\left(\text{X}\right)}\right)}^{\ast }}{2}\right],I\left({\mathbf{y}}_i^{\left(\text{X}\right)}\right)\\ =-j\left[\frac{{\mathbf{y}}_i^{\left(\text{X}\right)}-{\left({\mathbf{y}}_i^{\left(\text{X}\right)}\right)}^{\ast }}{2}\right]\\ \text{X}=\left[\text{A},\text{B}\right],i=0,1\dots M-1\end{array}$$

(68)

First, the autocorrelation matrix for each subchannel is calculated. It is noted that the presented analysis is provided for the even channels, but the exact same results are extracted for the odd subchannels with permutation of the ℜ(∙) and I(∙) operators:

$$\begin{array}{l}{\left[\mathrm{E}\left({\mathbf{y}}_i^{\text{oqam}}{\left({\mathbf{y}}_i^{\text{oqam}}\right)}^{\text{H}}\right)\right]}_{k,m}=\mathrm{E}\left(y_n^{\text{oqam,}i}{\left(y_{n+m-k}^{\text{oqam,}i}\right)}^{\ast }\right)\\ =\mathrm{E}\left(y_n^{\text{oqam,}i}{\left(y_{n+\epsilon }^{\text{oqam,}i}\right)}^{\ast }\right)\\ =\mathrm{E}\left(\Re \left(y_n^{\left(\text{A,}i\right)}\right)\Re \left(y_{n+\epsilon }^{\left(\text{A,}i\right)}\right)\right)\\ +\mathrm{E}\left(I\left(y_n^{\left(\text{B,}i\right)}\right)I\left(y_{n+\epsilon }^{\left(\text{B,}i\right)}\right)\right)\\ +j\mathrm{E}\left(\Re \left(y_{n+\epsilon }^{\left(\text{A,}i\right)}\right)I\left(y_n^{\left(\text{B,}i\right)}\right)\right)\\ -j\mathrm{E}\left(\Re \left(y_n^{\left(\text{A,}i\right)}\right)I\left(y_{n+\epsilon }^{\left(\text{B,}i\right)}\right)\right)\\ \text{where}(m-k=\epsilon )\end{array}$$

(69)

Assuming a circular white Gaussian input, the following result is extracted from (18) and (19):

$$\begin{array}{l}\mathrm{E}\left(\Re \left(y_n^{\left(\text{A}\right)}\right)\Re \left(y_{n+\epsilon }^{\left(\text{A}\right)}\right)\right)=\frac{1}{2}{\sigma }_{x,i}^2\delta \left(\epsilon \right)\\ \mathrm{E}\left(I\left(y_n^{\left(\text{B}\right)}\right)I\left(y_{n+\epsilon }^{\left(\text{B}\right)}\right)\right)=\frac{1}{2}{\sigma }_{x,i}^2\delta \left(\epsilon \right)\\ \text{with}{\sigma }_{x,i}^2=\left\{\begin{array}{c}{\sigma }_w^2,\text{for}{\mathcal{\mathscr{H}}}_0\\ {\left|c_i\right|}^2{\sigma }_s^2+{\sigma }_w^2,\text{for}{\mathcal{\mathscr{H}}}_1\end{array}\right.\end{array}$$

(70)

The investigation of the correlation of the imaginary parts in (69) leads to

$$\begin{array}{l}\text{E}\left(\Re \left(y_{n+\epsilon }^{\left(\text{A,}i\right)}\right)I\left(y_n^{\left(\text{B,}i\right)}\right)\right)=\text{E}\left(-j\left(\frac{y_{n+\epsilon }^{\left(\text{A,}i\right)}+{\left(y_{n+\epsilon }^{\left(\text{A,}i\right)}\right)}^{\ast }}{2}\right)\left(\frac{y_n^{\left(\text{B,}i\right)}-{\left(y_n^{\left(\text{B,}i\right)}\right)}^{\ast }}{2}\right)\right)\\ =\frac{j}{4}\left(\text{E}\left(y_{n+\epsilon }^{\left(\text{A,}i\right)}\right)\text{E}\left({\left(y_n^{\left(\text{B,}i\right)}\right)}^{\ast }\right)\right)-\frac{j}{4}\left(\text{E}\left({\left(y_{n+\epsilon }^{\left(\text{A,}i\right)}\right)}^{\ast }\right)\text{E}\left(y_n^{\left(\text{B,}i\right)}\right)\right)\\ =\frac{j}{4}\left(\sum _{u=-\infty }^{\infty }\sum _{p=-\infty }^{\infty }{h}_u^{\ast }{h}_p{e}^{2\mathrm{\pi j}\frac{i\left(p-u\right)}{M}}\text{E}\left(x_{\mathit{\text{Mn}}-u-\frac{M}{2}}^{\ast }{x}_{\mathit{\text{Mn}}+\mathrm{M\epsilon }-p}\right)\right.\\ \left.-\sum _{u=-\infty }^{\infty }\sum _{p=-\infty }^{\infty }{h}_u^{\ast }{h}_p{e}^{2\mathrm{\pi j}\frac{i\left(p-u\right)}{M}}\text{E}\left(x_{\mathit{\text{Mn}}+\mathrm{M\epsilon }-u}^{\ast }{x}_{\mathit{\text{Mn}}-p-\frac{M}{2}}\right)\right)\\ =\frac{j}{4}\left(\sum _{p=-\infty }^{\infty }{h}_p{h}_{p-\mathrm{M\epsilon }-\frac{M}{2}}{e}^{\mathrm{j\pi i}}-\sum _{u=-\infty }^{\infty }{h}_u{h}_{u-\mathrm{M\epsilon }-\frac{M}{2}}{e}^{-\mathrm{j\pi i}}\right)=0\end{array}$$

(71)

In (71), all the terms containing E(x _n x_n+ε) and E((x _n x_n+ε)^∗) were omitted since in complex circular normal variables, these terms are zeroed, if the imaginary and real parts have the same variances. The same result is extracted for the remaining term of (69). The combined result is given by

$$\mathrm{E}\left({\mathbf{y}}_i^{\text{oqam}}{\left({\mathbf{y}}_i^{\text{oqam}}\right)}^{\text{H}}\right)={\sigma }_{x,i}^2{\mathbf{I}}_N$$

(72)

Similarly to the previous analysis, the correlation of the output from non-directly adjacent filter paths is considered zero. The next step is to calculate the cross-correlation between outputs of directly adjacent filters. Assuming that filter path i is even, then i+1 is odd. The exact same procedure can be used for an odd i th filter. Thus,

$$\begin{array}{l}\begin{array}{c}\mathrm{E}\left({\mathbf{y}}_i^{\text{oqam}}{\left({\mathbf{y}}_{i+1}^{\text{oqam}}\right)}^{\text{H}}\right)=\mathrm{E}\left(\left(\frac{{\mathbf{y}}_i^{\text{(A)}}+{\left({\mathbf{y}}_i^{\text{(A)}}\right)}^{\ast }}{2}+\frac{{\mathbf{y}}_i^{\text{(B)}}-{\left({\mathbf{y}}_i^{\text{(B)}}\right)}^{\ast }}{2}\right)\left({\left(\frac{{\mathbf{y}}_{i+1}^{\text{(A)}}-{\left({\mathbf{y}}_{i+1}^{\text{(A)}}\right)}^{\ast }}{2}\right)}^{\text{H}}+{\left(\frac{{\mathbf{y}}_{i+1}^{\text{(B)}}+{\left({\mathbf{y}}_{i+1}^{\text{(B)}}\right)}^{\ast }}{2}\right)}^{\text{H}}\right)\right)\\ =\frac{1}{4}\left(\begin{array}{c}\mathrm{E}\left({\mathbf{y}}_i^{\text{(A)}}{\left({\mathbf{y}}_{i+1}^{\text{(A)}}\right)}^{\text{H}}\right)-\mathrm{E}\left({\left({\mathbf{y}}_i^{\text{(A)}}\right)}^{\ast }{\left({\mathbf{y}}_{i+1}^{\text{(A)}}\right)}^{\text{T}}\right)+\mathrm{E}\left({\mathbf{y}}_i^{\text{(B)}}{\left({\mathbf{y}}_{i+1}^{\text{(B)}}\right)}^{\text{H}}\right)-\mathrm{E}\left({\left({\mathbf{y}}_i^{\text{(B)}}\right)}^{\ast }{\left({\mathbf{y}}_{i+1}^{\text{(B)}}\right)}^{\text{T}}\right)\\ +\mathrm{E}\left({\mathbf{y}}_i^{\text{(B)}}{\left({\mathbf{y}}_{i+1}^{\text{(A)}}\right)}^{\text{H}}\right)+\mathrm{E}\left({\mathbf{y}}_i^{\text{(A)}}{\left({\mathbf{y}}_{i+1}^{\text{(B)}}\right)}^{\text{H}}\right)+\mathrm{E}\left({\left({\mathbf{y}}_i^{\text{(B)}}\right)}^{\ast }{\left({\mathbf{y}}_{i+1}^{\text{(A)}}\right)}^{\text{T}}\right)+\mathrm{E}\left({\left({\mathbf{y}}_i^{\text{(A)}}\right)}^{\ast }{\left({\mathbf{y}}_{i+1}^{\text{(B)}}\right)}^{\text{T}}\right)\end{array}\right)\end{array}\end{array}$$

(73)

where once more, all the terms that did not contain products between conjugates were ignored. The first finding concerns the first four terms. Since the output from each separate filter bank is the same with the output of a uniform DFT filter bank at Nyquist rate, then the cross-correlation matrix between the outputs of adjacent paths coming from the same filter bank has already been calculated in (22) for white Gaussian input with unitary variance ($\mathcal{N}\left(0,1\right)$). Given that the only difference between filter banks (A) and (B) is a delay offset of the input and that filter paths ${\mathbf{y}}_i^{\text{(A)}}$, ${\mathbf{y}}_i^{\text{(B)}}$ monitor the same spectral band (with the same radio channel coefficient c _i ), the output cross-correlations for adjacent subchannels for the two filter banks are equal. Therefore,

$$\begin{array}{c}\mathrm{E}\left({\mathbf{y}}_i^{\text{(A)}}{\left({\mathbf{y}}_{i+1}^{\text{(A)}}\right)}^{\text{H}}\right)=\mathrm{E}\left({\mathbf{y}}_i^{\text{(B)}}{\left({\mathbf{y}}_{i+1}^{\text{(B)}}\right)}^{\text{H}}\right)={\sigma }_{x,i}{\sigma }_{x,i+1}\mathbf{R},\\ \text{and given that}\mathbf{R}\text{is real and symmetric:}\hfill \\ \mathrm{E}\left({\left({\mathbf{y}}_i^{\text{(A)}}\right)}^{\ast }{\left({\mathbf{y}}_{i+1}^{\text{(A)}}\right)}^{\text{T}}\right)=\mathrm{E}\left({\left({\mathbf{y}}_i^{\text{(B)}}\right)}^{\ast }{\left({\mathbf{y}}_{i+1}^{\text{(B)}}\right)}^{\text{T}}\right)\hfill \\ ={\sigma }_{x,i}{\sigma }_{x,i+1}{\mathbf{R}}^{\text{H}}={\sigma }_{x,i}{\sigma }_{x,i+1}\mathbf{R}\end{array}$$

(74)

where σ_x,i=σ _s |c _i |. Therefore, the sum of the first four terms in (73) is zero. Regarding the following four terms, it can be proved after some mathematical manipulation that

$$\begin{array}{l}{\left[\mathrm{E}\left({\mathbf{y}}_i^{\left(\text{A}\right)}{\left({\mathbf{y}}_{i+l}^{\left(\text{B}\right)}\right)}^{\text{H}}\right)\right]}_{k,m}={\sigma }_{x,i}{\sigma }_{x,i+l}{\left(-1\right)}^{i+l}\\ \times \sum _{u=-\infty }^{\infty }{h}_u{h}_{u-\frac{M}{2}+\mathrm{M\epsilon }}{e}^{-2\mathrm{\pi j}\frac{\mathit{\text{lu}}}{M}}\\ ={\sigma }_{x,i}{\sigma }_{x,i+l}{\left(-1\right)}^{i+l}{\left(-1\right)}^{⌊1+\frac{l}{2}⌋}j\\ \times \sum _{u=-\infty }^{\infty }{h}_u{h}_{u-\frac{M}{2}+\mathrm{M\epsilon }}\text{sin}\left(\frac{2\mathrm{\pi u}}{M}\right)\\ \triangleq {\sigma }_{x,i}{\sigma }_{x,i+l}{\left(-1\right)}^{i+l}{\left(-1\right)}^{⌊1+\frac{l}{2}⌋}{\mathit{\text{jg}}}_{\mathrm{M\epsilon }-\frac{M}{2}}^{\text{sin}}\end{array}$$

(75)

This result is achieved using the symmetry of the prototype FIR filter. Matrix T is defined in order to store the outcome for σ_x,i=1, σ_x,i+l=1. With unification of the sign manipulation factors, the matrix elements are given by

$${\left[{\mathbf{T}}_{i,l}\right]}_{k,m}=\left\{\begin{array}{c}j{\left(-1\right)}^{i+1}{g}_{M\left(m-k\right)-\frac{M}{2}}^{\text{sin}},l=1\\ j{\left(-1\right)}^i{g}_{M\left(m-k\right)-\frac{M}{2}}^{\text{sin}},l=-1\end{array}\right.$$

(76)

Based on (75) and given the fact that T is purely imaginary, it is also concluded that

$$\begin{array}{l}{\left[\mathrm{E}\left({\left({\mathbf{y}}_i^{\text{(A)}}\right)}^{\ast }{\left({\mathbf{y}}_{i+l}^{\text{(B)}}\right)}^{\text{T}}\right)\right]}_{k,m}={\left({\sigma }_{x,i}{\sigma }_{x,i+l}{\left[{\mathbf{T}}_{i,l}\right]}_{k,m}\right)}^{\ast }\\ =-{\sigma }_{x,i}{\sigma }_{x,i+l}{\left[{\mathbf{T}}_{i,l}\right]}_{k,m}\\ \mathrm{E}\left({\mathbf{y}}_{i+l}^{\text{(B)}}{\left({\mathbf{y}}_i^{\text{(A)}}\right)}^{\text{H}}\right)={\sigma }_{x,i}{\sigma }_{x,i+l}{\mathbf{T}}_{i,l}^{\text{H}},\mathrm{E}\left({\left({\mathbf{y}}_{i+l}^{\text{(B)}}\right)}^{\ast }{\left({\mathbf{y}}_i^{\text{(A)}}\right)}^{\text{T}}\right)\\ =-{\sigma }_{x,i}{\sigma }_{x,i+l}{\mathbf{T}}_{i,l}^{\text{H}}\end{array}$$

(77)

Combining (76) and (??) in (73), the result is given by

$$\mathrm{E}\left({\mathbf{y}}_i^{\text{oqam}}{\left({\mathbf{y}}_{i+1}^{\text{oqam}}\right)}^{\text{H}}\right)={\mathbf{0}}_N$$

(78)

Thus, if the MDFT filter bank output after the operators ℜ(∙) and I(∙) is used, then all the signal vectors that participate in the detection procedure are uncorrelated. The observation vector is defined as

$${\mathbf{y}}^{\text{oqam}}={\left[\begin{array}{cccc}{\left({\mathbf{y}}_0^{\text{oqam}}\right)}^{\text{T}}& {\left({\mathbf{y}}_1^{\text{oqam}}\right)}^{\text{T}}& \dots & {\left({\mathbf{y}}_1^{\text{oqam}}\right)}^{\text{T}}\end{array}\right]}^{\text{T}}$$

(79)

where due to the independence of signals, reordering is not needed. The covariance matrix of y^oqam is the following L N×L N matrix:

$$\mathrm{E}\left({\mathbf{y}}^{\text{oqam}}{\left({\mathbf{y}}^{\text{oqam}}\right)}^{\text{H}}\right)=\left[\begin{array}{cccc}{\sigma }_{x,0}^2{\mathbf{I}}_N& {\mathbf{0}}_N& \dots & {\mathbf{0}}_N\\ {\mathbf{0}}_N& {\sigma }_{x,1}^2{\mathbf{I}}_N& ⋮\\ ⋮& \ddots & {\mathbf{0}}_N\\ {\mathbf{0}}_N& \dots & {\mathbf{0}}_N& {\sigma }_{x,L-1}^2{\mathbf{I}}_N\end{array}\right]$$

(80)

The form of this matrix leads to the pleasant conclusion that the weighted energy detector (with proper scaling for each filter path) is the Neyman-Pearson optimum detector. The metric of the detector is given by

$$T_{\text{oqam}}\left(\mathbf{y}\right)=\frac{{\sigma }_s^2}{{\sigma }_w^2}\sum _{i=0}^{L-1}\frac{{\left|c_i\right|}^2}{{\sigma }_s^2{\left|c_i\right|}^2+{\sigma }_w^2}{\left({\mathbf{y}}_i^{\left(\text{oqam}\right)}\right)}^{\text{H}}{\mathbf{y}}_i^{\left(\text{oqam}\right)}$$

(81)

The metric distribution can be extracted by

• With the use of the analytical method of (55), (56) and (58) using as eigenvalues the detector weights, i.e.

  $$\begin{array}{cc}{\lambda }_i^{\text{oqam}/{\mathcal{\mathscr{H}}}_0}& =\frac{{\sigma }_s^2{\left|c_i\right|}^2}{\left({\sigma }_w^2+{\sigma }_s^2{\left|c_i\right|}^2\right)}\\ {\lambda }_i^{\text{oqam}/{\mathcal{\mathscr{H}}}_1}& =\frac{{\sigma }_s^2{\left|c_i\right|}^2}{{\sigma }_w^2}\end{array}$$

  (82)

• Or using the FFT-based numerical integration with ${\phi }_{T({\mathbf{y}}_{\text{oqam}}/H_m)}\left(\omega \right)=\prod _{i=0}^{L-1}{\left(1-j{\lambda }_i^{\text{oqam}/{\mathcal{\mathscr{H}}}_m}\omega \right)}^{-N}$ for the outcomes m=0,1.

In Figure 5, the histogram of the simulated OFDM-OQAM detector metric is compared with the theoretically calculated distribution. Both methods of distribution calculation return practically identical results. A very similar figure can be extracted for the Neyman-Pearson detector for uniform DFT filter banks, since both optimal detectors have identical metric distributions. The results of the two optimum Neyman-Pearson detectors for the uniform DFT and the MDFT filter bank present similar performance since in both cases the maximum possible amount of information is used. The MDFT filter bank uses double computational workload for filtering. However, in most cases for relatively large sample sets and L>3, the matrix multiplications for the computation of the uniform DFT metric lead to heavier computational burden. Therefore, in general, the OFDM-OQAM optimum detection is favourable.

Figure 5

Distribution fit example for the Neyman-Pearson detection metric for an MDFT filter bank.

5 Energy detection in the uniform DFT filter bank

The optimum detector for an MDFT filter bank in Section 4 is proven to be a weighted (per subchannel) energy detector. Although the performance is identical, the uniform DFT optimum detector is much more complicated since it involves multiplications with the matrix C^-1. In this section, the use of the weighted energy detector in a uniform DFT is investigated. It can be proved that the weighted energy detector is optimal if R=0; however, this is not possible in a DFT filter bank with a root-Nyquist prototype filter.

The weighted energy detector as a filter bank-based algorithm was presented in [26], where it was assumed that the subchannel filter outputs are uncorrelated and that the metric performance is determined using Gaussian distributions. It is also claimed that the weighted energy detector operation with optimal weights is similar to the maximum ratio combining (MRC) for independent variables; however, the optimal weights are not determined. The MRC consideration is proved to be the correct and optimum approach for the OFDM-OQAM detector in Section 4, where filter outputs are uncorrelated. In this section, an extension and generalization of the weighted energy detector is performed. The subchannel filter outputs are considered correlated, a fact that is inevitable for uniform DFT filter banks. Moreover, accurate and analytic distributions for the metric under ${\mathcal{\mathscr{H}}}_0$ and ${\mathcal{\mathscr{H}}}_1$ are extracted based on probability theory.

The metric of the weighted energy detector is given by

$$T_{\mathit{\text{ed}}}\left(\mathbf{y}\right)=\frac{{\sigma }_s^2}{{\sigma }_w^2}\sum _{i=0}^{L-1}\frac{{\left|c_i\right|}^2}{{\sigma }_s^2{\left|c_i\right|}^2+{\sigma }_w^2}{\mathbf{y}}_i^{\text{H}}{\mathbf{y}}_i=\sum _{i=0}^{L-1}{T}_i\left({\mathbf{y}}_i\right)$$

(83)

For zero-mean complex Gaussian input, additive white noise and root-Nyquist prototype filter, the output for each subchannel is proved to follow the distributions below:

$$\begin{array}{l}{\mathcal{\mathscr{H}}}_0:T_i\left({\mathbf{y}}_i\right)\sim \mathcal{G}\left(N,\frac{{\sigma }_s^2{\left|c_i\right|}^2}{{\sigma }_s^2{\left|c_i\right|}^2+{\sigma }_w^2}\right)\\ {\mathcal{\mathscr{H}}}_1:T_i\left({\mathbf{y}}_i\right)\sim \mathcal{G}\left(N,\frac{{\sigma }_s^2{\left|c_i\right|}^2}{{\sigma }_w^2}\right)\end{array}$$

(84)

Since independence between samples from the adjacent filters y_i+1 and y_i-1 is not possible, the performance of the detector will be inferior to the detector in Section 3. The hysteresis of the detector depends on the transition bandwidth and the roll-off of the prototype filter.

Therefore, T_ed(y) is a sum of weighted, correlated gamma variables. In order to extract the metric distribution, it is necessary to calculate the correlation coefficient for the energy between the outputs of two adjacent filter paths. According to [37] (Jensen approximation), it is claimed that for a stream of N pairs of samples of Gaussian variables z _k =(z_{k 1},z_{k 2}) with (μ=0,σ=1) that are mutually correlated (but each one has uncorrelated time samples), then the correlation of the energies $u_1=\sum _{k=0}^{N-1}\left|z_{k1}^2\right|,u_2=\sum _{k=0}^{N-1}\left|z_{k2}^2\right|$ can be calculated by

$$p_{u_1{u}_2}=\frac{1}{N}\sum _{n=0}^{N-1}{q}_n^2$$

(85)

where the q _n s are the normal correlations for variables (z_{k 1},z_{k 2}). Jensen’s result is valid for real random variables with standard normal distribution. However, it can easily be proved through the correlation coefficient that the same formula is valid for 2N-length real normal variables with μ=0,σ²=1/2. Since a N-complex normal vector is equivalent with a 2N-real vector, the Jensen’s corollary can be used for the current analysis.

Normal correlations are the means to quantify the linear relationship between two multivariate random variables. The normal correlations can be calculated if the joint covariance matrix of the two random variables is known. Specifically, for known covariance matrix,

$$\left[\begin{array}{cc}{\mathbf{C}}_{{\mathbf{x}}_1{\mathbf{x}}_1}& {\mathbf{C}}_{{\mathbf{x}}_1{\mathbf{x}}_2}\\ {\mathbf{C}}_{{\mathbf{x}}_2{\mathbf{x}}_1}& {\mathbf{C}}_{{\mathbf{x}}_2{\mathbf{x}}_2}\end{array}\right],\text{for the variables}{\mathbf{x}}_1,{\mathbf{x}}_2$$

(86)

the normal correlation squares are given as the eigenvalues of the matrix:

$$\begin{array}{cc}{\mathbf{C}}_{{\mathbf{x}}_1{\mathbf{x}}_1}^{-1}{\mathbf{C}}_{{\mathbf{x}}_1{\mathbf{x}}_2}{\mathbf{C}}_{{\mathbf{x}}_2{\mathbf{x}}_2}^{-1}{\mathbf{C}}_{{\mathbf{x}}_2{\mathbf{x}}_1}{\mathbf{w}}_{\mathbf{x}}& =q_{{\mathbf{x}}_1{\mathbf{x}}_2}^2{\mathbf{w}}_{\mathbf{x}}\\ {\mathbf{C}}_{{\mathbf{x}}_2{\mathbf{x}}_2}^{-1}{\mathbf{C}}_{{\mathbf{x}}_2{\mathbf{x}}_1}{\mathbf{C}}_{{\mathbf{x}}_1{\mathbf{x}}_1}^{-1}{\mathbf{C}}_{{\mathbf{x}}_1{\mathbf{x}}_2}{\mathbf{w}}_{\mathbf{y}}& =q_{{\mathbf{x}}_1{\mathbf{x}}_2}^2{\mathbf{w}}_{\mathbf{y}}\end{array}$$

(87)

where w_x,y are the respective eigenvectors. The problem is to calculate the correlation coefficient between the energies of N output samples from adjacent filter banks. The covariance matrix y _i ,y_i+1 is given by (Section 3):

$$E\left(\left[\begin{array}{cc}{\mathbf{y}}_i^{\text{H}}& {\mathbf{y}}_{i+1}^{\text{H}}\end{array}\right]\left[\begin{array}{c}{\mathbf{y}}_i\\ {\mathbf{y}}_{i+1}\end{array}\right]\right)={\sigma }_s^2\left[\begin{array}{cc}{\left|c_i\right|}^2{\mathbf{I}}_N& \left|c_i\right|\left|c_{i+1}\right|\mathbf{R}\\ \left|c_i\right|\left|c_{i+1}\right|{\mathbf{R}}^{\text{H}}& {\left|c_{i+1}\right|}^2{\mathbf{I}}_N\end{array}\right]$$

(88)

Given that R is hermitian, it can be proved from (86) that the vector q y _i y_i+12 is also the vector of eigenvalues of matrix R². The correlation coefficient can be extracted as the sum of the squares of normal correlations. Since the matrix trace (Tr) is equal to the sum of the eigenvalues, it is proved that

$$\text{Tr}\left[{\mathbf{R}}^2\right]=\sum _{n=0}^{N-1}{q}_{{\mathbf{y}}_i{\mathbf{y}}_{i+1},n}^2$$

(89)

The correlation coefficient is equal to the mean of the R² eigenvalues, i.e. $p_{{\mathbf{y}}_i{\mathbf{y}}_{i+1}}{=}^{\text{Tr}\left[{\mathbf{R}}^2\right]}/{}_N$

Another important note is that since the correlation coefficient does not depend on the radio channel coefficients, it is the same for every pair of adjacent subchannel outputs. It is also noted that for non-directly adjacent subchannels, the correlation coefficient is assumed to be zero. Once again, the analysis in [36] can be followed in order to extract the metric distribution. According to [36], if ${\left\{u_i\right\}}_{i=1}^{L-1}$ is a set of L-correlated gamma variables with equal degrees of freedom and known correlation coefficients, then the PDF can be expressed analytically. Unlike the results in Sections 3 and 4 where an identity correlation matrix is considered, in this case, the positive definite matrix C_ed that expresses the correlations is given by [36]

$${\mathbf{C}}_{\text{ed}}=\left[\begin{array}{ccccc}1& \sqrt{p}& 0& \dots & 0\\ \sqrt{p}& 1& \sqrt{p}& ⋮\\ 0& \sqrt{p}& 1& \ddots & 0\\ ⋮& \ddots & \ddots & \sqrt{p}\\ 0& \dots & 0& \sqrt{p}& 1\end{array}\right]$$

(90)

The tridiagonal matrix is an L×L matrix. As a next step, the diagonal matrix D_ed is defined according to the rule:

$$\begin{array}{l}{\mathbf{D}}_{\text{ed}}=\text{diag}\left(\left[\begin{array}{ccc}\frac{{\sigma }_s^2{\left|c_0\right|}^2}{{\sigma }_s^2{\left|c_0\right|}^2+{\sigma }_w^2}& \dots & \frac{{\sigma }_s^2{\left|c_{L-1}\right|}^2}{{\sigma }_s^2{\left|c_{L-1}\right|}^2+{\sigma }_w^2}\end{array}\right]\right),\text{for}{\mathcal{\mathscr{H}}}_0\\ {\mathbf{D}}_{\text{ed}}=\text{diag}\left(\left[\begin{array}{ccc}\frac{{\sigma }_s^2{\left|c_0\right|}^2}{{\sigma }_w^2}& \dots & \frac{{\sigma }_s^2{\left|c_{L-1}\right|}^2}{{\sigma }_w^w}\end{array}\right]\right),\text{for}{\mathcal{\mathscr{H}}}_1\end{array}$$

(91)

The distributions are calculated by (55), (56), (58) and (66) using the eigenvalues ${\lambda }_i^{\text{ed}/{\mathcal{\mathscr{H}}}_0}$ and ${\lambda }_i^{\text{ed}/{\mathcal{\mathscr{H}}}_1}$ that are calculated from the matrix: A_ed=D_edC_ed. Since A_ed is tridiagonal, the eigenvalues can be calculated iteratively [38]. A distribution fit example for the weighted energy detector metric is presented in Figure 6.

Figure 6

Distribution fit example for the weighted energy detector matrix for uniform DFT filter bank.

In [26], it is mentioned that for uncorrelated, independent subchannel filter outputs, the weighted energy detector can be seen as an approximation of the matched filtering. This is accurate for the OFDM-OQAM Neyman-Pearson (NP) detector where subchannel independence is proved. More specifically, if the ‘approximately flat radio channel per subchannel’ assumption is valid and the radio channel can be adequately described by the c _i parameters, then the matched filter approximation is exceptional. As the channel frequency selectivity increases, the deviation from the matched filter performance also increases. However, in the common case of correlated filter outputs, the deviation from the matched filter performance is significant. Since the weighted energy detector metric ignores the correlation of the output samples, it fails to perfectly align the decision metrics in order to maximize probability of detection. This is achieved by the NP optimal detector of Section 3. In terms of diversity, the weighted energy detector is transformed into an non-optimal MRC that ignores the existed correlation among the diversity branches. Despite the fact that the weighted energy detector is outperformed by the NP detector, the selection of weights that depends on the SNR per subchannel provides significant advantages over the conventional energy detector. The integration of the radio channel effects in the detector with the use of the c _i parameters is essential, despite the fact that it is based on the ‘approximately flat radio channel’ approximation. Thus, the performance degradation of the weighted energy detector from the NP optimal detector is much smaller than the achieved improvement from the performance of the conventional energy detector, as presented in Figure 7.

Figure 7

ROC curve P _D vs. P _FA for detection under AWGN.

6 Coloured noise

An important weakness of many detectors is that they are not robust in the existence of coloured noise. Coloured noise may be the result of interference caused by remote PSs or CRs, it may be caused by various electronic noise sources or it may appear due to filter leakage and spurious emissions of users operating in adjacent spectrum bands [5]. In this section, additive coloured Gaussian noise is assumed. In AWGN models, the noise samples are uncorrelated. Coloured noise can be on the other hand modelled as the result of filtering AWGN samples. When the noise reaches the receiver, it is subjected also to filtering from the filter bank. Similarly to the assumption regarding the primary signal, the noise variance is considered approximately constant for the bandpass zone of each filter path. Therefore, the coloured noise modelling can be performed with the following relationships:

$$\begin{array}{l}{S}_{\mathit{\text{cw}}}\left(f\right)=S_{\mathit{\text{ww}}}\left(f\right){\left|V\left(f\right)\right|}^2{\left|H\left(f-f_i\right)\right|}^2\Rightarrow \\ S_{\mathit{\text{cw}}}\left(f\right)\simeq {\sigma }_w^2{\left|V\left(f\right)\right|}^2\end{array}$$