 Research
 Open Access
 Published:
Partial interference cancellation with maximum likelihood sequence detection in FBMC spatial multiplexing system
EURASIP Journal on Advances in Signal Processing volume 2016, Article number: 47 (2016)
Abstract
Intrinsic interference in filter bank multicarrier (FBMC) modulation prevents the maximum likelihood (ML) detection in spatial multiplexing (SM) system. This intrinsic interference is caused by the transmultiplexer impulse response in timefrequency domain. Solutions based on interference cancellation are not always effective because they may introduce error propagation. In this paper, we propose to study some receivers based on reducing the interference concerned by the cancellation. These solutions can be seen as a tradeoff between the whole Viterbi (ML) detection and the whole interference cancellation. The principle of the proposed receivers is to partially estimate and cancel the interference, and the rest of the interference is processed by a low complexity Viterbi detector. We show in this work that the receiver performance depends on the transmultiplexer impulse response and on the choice of the partial Viterbi detector.
Introduction
Nowadays, the most wellknown multicarrier modulation is the orthogonal frequency division multiplex (OFDM) modulation. Thanks to the cyclic prefix (CP) insertion, OFDM technique provides a high robustness against the multipath fading channels. However, since OFDM uses CP and also rectangular pulse shape filtering that causes some disadvantages concerning the spectrum control and efficiency, to tackle this issue, filter bank multicarrier systems with offset quadrature amplitude modulation (FBMC/OQAM) was proposed as an alternative to CPOFDM. Many works on the comparison between FBMC/OQAM and OFDM can be found in the literature such as [1]. One of the features of FBMC/OQAM is the possibility of shaping subcarrier signals with welllocalized prototype filter in time and frequency axis [1]. That is, sidelobes steeply decay and allow a better spectrum control and offer an important resistance against time and frequency misalignment. Thus, FBMC/OQAM enables asynchronous multiple access and reduces intercarrier interference since each subcarrier signal is spectrally confined in a band and has negligible interference to other bands. In cognitive radios, the filtering capability of FBMC systems makes them the perfect choice for filling in the spectrum holes [1]. In the other hand, thanks to its spectral confinements, FBMCbased systems can reduce their guard band at the frequency boundaries, thereby increasing the spectral efficiency. Moreover, the absence of CP and any guard interval in time domain also contributes to increase the spectral efficiency. Many research works have been carried out in the issues related to doubly spread channels, and it has been shown that FBMC modulations are far better choices when compared to OFDM [2–4]. For instance, the author in [1] clearly shows the outperformance of the FBMC over OFDM in terms of the resilience to the doubly dispersive channels under the same assumptions of channel mobility and symbol density.
In FBMC/OQAM, each subcarrier is modulated with an OQAM transmitting pulseamplitude modulation (PAM) symbols at each half a period (T/2). The orthogonality condition is considered only in the real field [5]. Indeed, the data at the receiver side is carried only by the real components of the signal, and the imaginary parts appear as interference terms. This interference is caused by the data symbols transmitted in the neighborhood area in the timefrequency domain. Since the interference is orthogonal to the useful data symbols, data detection is easily performed when the channel is flat fading or lowly selective [6]. However, in some situations like when the channel is highly selective, the intrinsic interference may cause performance degradation. Several works based on subchannel equalization were carried out to tackle to this issue as in [7, 8] where multitap equalization is used. In [9], the authors have performed a minimum mean square error (MMSE) per subcarrier equalization. Whereas in [10, 11] the authors propose to implement a persubcarrier decisionfeedback equalizer (DFE). The extension of the latter to the multipleinput multiple output (MIMO) spatial multiplexing (SM) system was addressed in [12]. The persubchannel equalization in FBMC with MIMO systems was also studied in [13, 14]. Some other works [15–20] address the design of MIMO precoding and decoding techniques for the FBMC modulation. For low MIMO selective channels, linear equalizations such as zero forcing (ZF) or MMSE can be applied to FBMC as shown in [21].
The presence of the inherent interference causes problems when combining FBMC/OQAM with some MIMO techniques such as Alamouti spacetime block coding (STBC) scheme and SM systems with maximum likelihood (ML) detection. Regarding the Alamouti scheme, its application in a straightforward manner to the FBMC makes an inherent interference appear that cannot be easily removed [22]. Many works have been carried out on this topic such as [22] where the authors show that Alamouti coding can be performed but only when it is combined with code division multiple access (CDMA). A pseudoAlamouti scheme was introduced in [23] where it is combined with singlecarrier FBMC using the cyclic prefix (CP). Another solution was proposed by Renfors et al. in [24] where the Alamouti coding is performed in a blockwise manner. We have proposed in [25, 26] an iterative Alamouti scheme for FBMC based on intrinsic interference cancellation. On the other side, ML detection in SM system, which is supposed to offer a diversity order equal to the number of the receive antennas [27], cannot be applied straightforwardly with FBMC due to the presence of the interference.
Full ML receivers, in principle, offer the best possible performance but require an impractically high complexity when the impulse response is long. Moreover, the intrinsic interference in FBMC is twodimensional which further complicates the detection task in maximum likelihood sense. Interference cancellation approaches generally offer the possibility of removing interference with low complexity increase and without enhancing the level of noise already present in the received signal [28]. Interference cancellation schemes are essentially based on using preliminary decisions to estimate and cancel the interference. In [29], we have addressed the possibility to cancel the interference before applying a local ML detection and proposed some receiver schemes. However, the obtained performance was limited and far from the optimum due to the error propagation. We have also proposed in [26, 30] to modify the FBMC/OQAM modulation by transmitting QAM symbols at each period of T instead of transmitting PAM symbols at each T/2. We referred to this modulation as FBMC/QAM. This modulation allows to reduce the intrinsic interference power but at the expense of the orthogonality [26]. Error propagation problem in iterative intersymbol interference (ISI) cancellation has been addressed in many research fields. In the general case, the authors in [31] have established conditions under which the cancellation scheme is effective. The idea is to consider the interference as the sum of two terms. The first one is cancelled by using tentative decisions, and the second uncancelled one is considered by a maximum likelihood sequence equalizer (MLSE).
In this work, in order to counteract the error propagation in SMFBMC and make the cancellation scheme effective, we were inspired by [31]. We apply the theory developed in [31] and propose some receiver structures that satisfy the conditions of the interference cancellation effectiveness. The receivers are based on partial interference cancellation followed by a Viterbi detector instead of an ML detector. The tentative detector is first used to only partially cancel the intrinsic interference. We have presented a part of this work in [32] where we have only focused on FBMC/OQAM with one prototype filter. In this paper, both FBMC/OQAM and FBMC/QAM modulations are addressed, and their performances are compared. Furthermore, each of the both modulations are analyzed with two different prototype filters.
The paper is organized as follows. We start in Section 2 by giving an overview on FBMC/OQAM modulation highlighting the issue of FBMC when it is combined with SMML detection. Then, in Section 3, we give a background on the use of tentative decisions to cancel the interference, and we present the principle of partial interference cancellation. Sections 4 and 5 are devoted to the analysis and adaptation of the partial interference cancellation to the FBMC/OQAM and FBMC/QAM, respectively. Simulation results of the different proposed receivers are presented in Section 6. Then, we finish by a conclusion in Section 7.
The FBMC/OQAM modulation
System model
At the transmitter side, the discrete time of the equivalent baseband FBMC signal is written as follows [5]:
where M is an even number of subcarriers, g[ m] is the prototype filter taking values in real field, and \(\frac {D}{2}\) is the delay term which depends on the length (L _{ g }) of g[ m]. For the sake of simplification, we can set, as in [5], D=L _{ g }−1 and L _{ g }=K M, where K is the overlapping factor. The transmitted symbols a _{ k,n } are realvalued symbols which are the real or the imaginary parts of QAM symbols. The additional phase term ϕ _{ k,n } is given by \(\phi _{k,n}=\frac {\pi }{2}(n+k)\pi nk\). We can rewrite (1) in a simple manner
where g _{ k,n }[ m] are the shifted versions of g[ m] in time and frequency. In the case of a perfect channel without noise, the demodulated symbol over the k ^{′}th subcarrier and the n ^{′}th instant is determined using the inner product of s[ m] and \(g_{k',n'}[\!m]\phantom {\dot {i}\!}\)
The transmultiplexer impulse response can be derived assuming null data except at one timefrequency position (k _{0},n _{0}) where a unit impulse is applied. Then, Eq. (3) becomes
where Δ n=n ^{′}−n _{0}, Δ k=k ^{′}−k _{0} and \(\Gamma _{\Delta k, \Delta n}^{(k_{0})}\) is then the transmultiplexer impulse response coefficients. It is worth noticing that the impulse response of the transmultiplexer depends on k _{0}. Indeed, because the multiplicative factor \(\phantom {\dot {i}\!}e^{j\pi k_{0}\Delta n}=(1)^{k_{0}\Delta n}\), the sign of some impulse response coefficients (with Δ n odd) depends on the parity of k _{0}.
The prototype filters are designed in a way that they provide a welllocalized spectrum and spread over only a few adjacent subcarriers. Several pulse shaping prototype filters g[ m] can be used according to their properties. A survey on prototype filters can be found in [33]. In this paper, we consider two different pulse shapes with overlapping factor set to K=4. The first one is that which was adopted in the European Physical Layer for Dynamic Access (PHYDYAS) project on FBMC [34] and is referred to as the PHYDYAS prototype filter. The second is the one based on the socalled isotropic orthogonal transform algorithm (IOTA) function [3]. IOTA filter is designed according to the energy concentration criterion, it is obtained by orthogonalizing the Gaussian function to prevent interference to neighboring points in the lattice [2]. Thus, it targets to preserve the optimum concentration property of Gaussian filter and fulfills the Nyquist criterion and can yield isotropic response in time and frequency [33]. As for PHYDYAS filter, it is designed according to the criterion of rapid decaying of the sidelobes. It is shown in [35] that, for an overlapping factor of K=4, the minimum stopband attenuation is 39.86 dB, and the sidelobes fall at ω^{−3} where ω is the angular frequency. Although the decaying rate and the stopband attenuation increase with increasing the overlapping factor K, the choice of K=4 is considered as a good tradeoff between the stopband attenuation and the system delay caused by the time spread of the filter [36]. Tables 1 and 2 depict the transmultiplexer impulse response coefficients \(\left (\Gamma _{\Delta k, \Delta n}^{(k)}\right)\) for even subcarriers k of the PHYDYAS and IOTA filter, respectively. We should note that the transmultiplexer impulse response is independent from the subcarrier number (M) as long as M is sufficiently high, M≥8. All the prototype filters g[ m] are designed to satisfy the real orthogonality condition given by [5]
Let us consider the SISO FBMC transmission. When passing through the radio channel and adding the zeromean and circular white Gaussian noise contribution b[ m], Eq. (3) becomes [37]
where \(h_{k',n'}\phantom {\dot {i}\!}\) is the channel coefficient at subcarrier k ^{′} and time index n ^{′} and the term \(\phantom {\dot {i}\!}I_{k',n'}\) is defined as an intrinsic interference. Since the prototype filters have a finite support in time and we assume that they are spectrally welllocalized, then the energy of the impulse response is confined in a finite set around the considered symbol [6]. Consequently, we can consider that the intrinsic interference term depends only on this restricted set (denoted by \(\phantom {\dot {i}\!}\Omega _{k',n'}\)). Moreover, for the sake of derivation simplicity, we consider that the channel is sufficiently lowselective such that we can assume that the channel coefficient h _{ k,n } is approximately constant at least over the summation set Ω _{ k,n }. Hence, we can write as in [37]
According to (5) and because a _{ k,n } is realvalued, the intrinsic interference \(\hat {I}_{k',n'}=ju_{k',n'}\) is pure imaginary. Thus, we can write the demodulated signal as
It is worth noting that the noise term \(\gamma _{k',n'}\phantom {\dot {i}\!}\) defined in Eq. (6) is colored, and its autocorrelation is given by
The last equality is obtained according to Eq. (4).
Problem statement
In MIMO spatial multiplexing context, when N _{ t } antennas are used to transmit N _{ t } real data symbols and N _{ r } antennas are used to collect the transmitted signals, the FBMC demodulated signal at the tth receive antenna and at a given timefrequency position (k,n) can be expressed by
We should recall that the above expression is only valid under the assumption of the lowselectivity of the channels, as assumed for (7), between the transmit and receive antennas. Finally, the matrix formulation of the system can be expressed as follows:
where H _{ k,n } is an (N _{ r }×N _{ t }) channel matrix with the element \(h_{k,n}^{(ti)}\) at the tth row and ith column, \(\mathbf {r}_{k,n}=\left [r_{k,n}^{(1)}, r_{k,n}^{(2)},\ldots,r_{k,n}^{(N_{r})}\right ]^{T}\) is the received vector, \(\mathbf {a}_{k,n}=\left [a_{k,n}^{(1)}, a_{k,n}^{(2)},\ldots,a_{k,n}^{(N_{t})}\right ]^{T}\) is the transmitted symbol vector, \(\mathbf {u}_{k,n}=\left [u_{k,n}^{(1)}, u_{k,n}^{(2)},\ldots,u_{k,n}^{(N_{t})}\right ]^{T}\) is the vector of the interference values, and \(\boldsymbol {\gamma }_{k,n}=\left [\gamma _{k,n}^{(1)}, \gamma _{k,n}^{(2)},\ldots,\gamma _{k,n}^{(N_{r})}\right ]^{T}\) is the noise vector.
Linear equalization as ZF and MMSE in FBMC/OQAM context can be performed as described in [21] where a virtually transmitted vector c _{ k,n }=a _{ k,n }+j u _{ k,n } is considered instead of the effective one. The vector r _{ k,n } represents the input of the equalizer having as output the equalized virtually transmitted vector \(\tilde {\mathbf {c}}_{k,n}\)
where G _{ k,n } is the equalization matrix based on the ZF or MMSE criterion. Then, the real part retrieval of \(\tilde {\mathbf {c}}_{k,n}\) yields the real equalized data vector \(\tilde {\mathbf {a}}_{k,n}\) [21].
As for maximum likelihood (ML) detection, the presence of the interference vector term u _{ k,n } in (11) prevents the application of ML separately at each timefrequency position (k,n). This is because the interference terms take their values in a large set and depend on the transmitted data symbols in the neighborhood around the considered position (k,n). The global ML detector is the one that considers all the transmitted data symbols within a frame. Obviously, such a receiver implementation is by far impractical due to its huge complexity.
Overview on partial interference cancellation
In [31], the authors have considered the channel model depicted in Fig. 1, where f _{0}(a _{ k },a _{ k−1},…,a _{ k−δ+1}) is a function of δ data symbols and represents the target response expected by the receiver. f _{1}(a _{ k+γ },a _{ k+γ−1},…,a _{ k−λ+1}) is a function of γ+λ data symbols and represents a small channel perturbation.
It should be noted that, in general, both f _{0} and f _{1} may be nonlinear functions. The samples of the signal at the input of the receiver are
where w _{ k } is the noise contribution. The receiver is composed by a tentative detector producing tentative decisions and a main Viterbi detector which assumes that the channel is described anly by f _{0}. Before performing the main Viterbi detector, the tentative decisions are used only to cancel the remaining ISI (RISI) represented by f _{1}. The receiver scheme is depicted in Fig. 2.
Given the correct data sequence {a _{ k }}, let us define a wrong sequence \(\left \{a_{k}^{(\epsilon)}\right \}\) where the wrong symbols \(a_{k}^{(\epsilon)}\) are determined by a given error event ε with \(a_{k}^{(\epsilon)} = a_{k} + \epsilon _{k}\). That is, for a value of k, if ε _{ k }=0, then the symbol \(a_{k}^{(\epsilon)}\) in the sequence \(\left \{a_{k}^{(\epsilon)}\right \}\) is correct. Let us define also
where K is assumed to be the total number of transmitted symbols. The authors in [31] classified the error events in terms of their distance d _{0}(ε) in the absence of RISI (f _{1} =0), which is given, in the presence of correlated noise, by
where \(\mathbf {R}=\mathbb {E}\{\mathbf {ww}^{H}\}/{\sigma _{w}^{2}}\) is the normalized noise autocovariance matrix, with \({\sigma _{w}^{2}}\) is the noise variance and w=[w _{0},w _{1},…,w _{ K−1}]^{T}. The events whose distance d _{0}(ε) is the minimum are called “firstorder” error events. Similarly, events whose distance is the second smallest are called “secondorder” error events and so on [31].
The conditions for which RISI cancellation is satisfying are summarized as follows [31]:

1.
Errors affecting the main (Viterbi detector) and the tentative detector must be statically independent.

2.
The RISI (described by f _{1}) must be small enough to guarantee that the main Viterbi detector can make relatively reliable decisions even when the tentative detector makes a decision error and such that the tentative detector also makes relatively reliable decisions in spite of the ISI.

3.
The distance of secondorder and higherorder error events that could cause error propagation must be significantly larger than that of firstorder error events.
Application to MIMOSM FBMC/OQAM
Now that the conditions for effective RISI cancellation are summarized, we will attempt to apply them to FBMC. Hence, the problem is, essentially, how to select the functions f _{0} and f _{1} such that the three conditions cited above are fulfilled. The intrinsic interference in FBMC is seen as a twodimensional intersymbol interference (2DISI). An extension of the works of Agazzi and Seshardi [31] to 2DISI channels was treated in [38] assuming that the noise samples are uncorrelated (which is not the case in FBMC). Hence, in general, the target response f _{0} may also represent a 2DISI channel. Then, a 2DViterbi algorithm is required to match with f _{0}. Designing a 2DViterbi is quite challenging. Therefore, for simplicity reasons, we opted to set the additional constraint that the target response f _{0} must be onedimensional and that f _{1} covers the rest of 2DISI.
Obviously, the receiver complexity depends essentially on the complexity of the Viterbi detector. Therefore, we have to choose a configuration with the least complex Viterbi detector that meets the conditions for effective RISI cancellation. We will select three configurations with different sizes of the target response f _{0}. According to the second condition, f _{0} must contain the largest coefficients \(\left (\Gamma _{\Delta k,\Delta n}^{(k)}\right)\) in each configuration. Hence, from Table 1, the selected target responses are [32]
and
The first configuration \(\left (f_{0}^{(1)}\right)\) corresponds to the whole ISI cancellation which has been studied in [29, 39]. Since, in FBMC, we have \(\sum _{p,q}\Gamma _{p,q}^{(k)}^{2} = 2\) [6], it is easy to calculate the power of the RISI (represented by f _{1}) for each configuration.
Regarding the first condition, it is easily satisfied when the tentative detector is different from the main one (Viterbi) [31]. We recall that we consider the case of spatial multiplexing system. Then, we chose the MIMOMMSE equalizer as the tentative detector.
The third condition concerns the spectrum distances d _{0}(ε) defined by (16). Hence, for each configuration \(\left (f_{0}^{(i)}\right.\), \(\left.{\vphantom {f^{(i)}_{0}}}i\in \{1,2,3\}\right)\), we compare the nonminimum distances to the minimum one. Then, according to (16), we have first to determine the matrix R. Since we consider that the target response are onedimensional and Viterbi algorithm is performed on each subcarrier “ k”, the noise vector w is then w= [w _{ k,n },w _{ k,n+1},…,w _{ k,n+K−1}]^{T}. Hence, the matrix \(\mathbf {R}=\mathbb {E}\{\mathbf {ww}^{H}\}/{\sigma _{w}^{2}}\) is only composed by the coefficients \(\Gamma _{0,q}^{(k)}\), \(q\in \mathbb {Z}\) and is given by
where the last equality is obtained thanks to Eq. (9).
In Tables 3 and 4, we summarize the values of the first and secondorder distances obtained by using (16) and also the power of the RISI for the three considered configurations and both the PHYDYAS and IOTA filters, respectively.
Let us denote by ε _{1} and ε _{2} the first and secondorder error events, respectively. We also define w(ε) as the number of error position in the error event ε. The values of w(ε) are also shown in the table. We remark that the difference between the secondorder and the firstorder distances is almost the same for all the configurations (0.8±0.03), so we consider (as considered in [31]) that the higherorder distances are sufficiently larger than the minimum distance for each configuration. Hence, condition 3 is fulfilled for the three configurations.
Now, we have only to determine the configuration(s) for which the second condition is satisfied. Unfortunately, the determination of the RISI power for which the cancellation starts to be effective (or equivalently, error propagation ceases) is not trivial and depends also on the noise variance σ ^{2} [31]. We will show by simulations that only the third configuration \(\left (f_{0}^{(3)}\right)\) allows to obtain effective RISI cancellation.
As for the receiver complexity, it strongly depends on that of the Viterbi detector. When we consider a spatial multiplexing system with N _{ t } transmit antennas, the Viterbi detector has to compute \(q^{i\times N_{t}}\phantom {\dot {i}\!}\) branch metrics, where q is the number of all possible symbols a _{ k,n } (constellation size) and i∈{1,2,3} is the number of the taps in \(f_{0}^{(i)}\). In order to reduce the receiver complexity, we can replace the Viterbi detection algorithm by the MAlgorithm [40] which keeps only a fixed number (J) of inner states instead of all the inner states \(\left (q^{(i1)\times N_{t}}\right)\phantom {\dot {i}\!}\). Hence, the Malgorithm has to compute only the \(\phantom {\dot {i}\!}J\times q^{N_{t}}\) branch metrics. In the rest of the paper, we call the proposed receivers “PaIC/Viterbi” (for Partial Interference Cancellation with Viterbi detector) and can be followed by an index to indicate the considered configuration.
Application of PaIC/Viterbi receivers in FBMC/QAM
Aiming to reduce the intrinsic interference power, we consider in this section the FBMC system transmitting complex QAM symbols instead of real ones (OQAM) [30]. That is, instead of transmitting real symbols a _{ k,n } at each half a period (T/2), complex symbols s _{ k,n } are transmitted at each period (T). Therefore, the equivalent baseband discrete time FBMC/QAM signal can be deduced from Eq. (1) and given by
where \(\theta _{k,n} = \phi _{k,2n}=\pi n + \frac {\pi }{2}k\). Consequently, the transmultiplexer impulse response coefficients \(\acute {\Gamma }_{p,q}\) can be also derived from Eq. (4) and expressed as
It is worthy to note that, in contrast to \({\Gamma }_{\Delta k, \Delta n}^{(k)}\), the coefficients \(\acute {\Gamma }_{\Delta k, \Delta n}\) do not depend on the subcarrier index k. The impulse response coefficients \(\acute {\Gamma }_{\Delta k, \Delta n}\) for FBMC/QAM using the PHYDYAS and IOTA filters are depicted in Tables 5 and 6, respectively.
The main principle of the PaIC/Viterbi receiver remains the same as for conventional FBMC presented in Section 4. However, the configuration of the PaIC/Viterbi receiver has to be adapted to the new FBMC/QAM context. Indeed, we have seen that the function f _{0} that matches the Viterbi detector must contain the largest coefficients \(\acute {\Gamma }_{\Delta k,\Delta n}\). According to the transmultiplexer impulse responses depicted in Tables 5 and 6, the three largest coefficients ({1,±0.239j} for PHYDYAS and {1,±0.4411j} for IOTA) in each table are aligned with the direction of the frequency axis. Therefore, we can select the following three functions \(f_{0}^{(i)}\), i=1,2,3 that determine the Viterbi target responses
and
where \(f_{0}^{(1)}\) obviously corresponds to the IICML receiver [26] where the whole interference cancellation is performed. Hence, the advantage for PaIC/Viterbi receivers in FBMC/QAM lies in the fact that the Viterbi algorithms are performed in the frequency axis direction. Whereas in FBMC/OQAM, they are performed in the time axis direction. Consequently, from an implementation point of view, the PaIC/Viterbi receivers are less complicated for implementation in FBMC/QAM than in the conventional FBMC, because only one Viterbi algorithm has to be performed for each one received multicarrier symbol, whereas in FBMC/OQAM, we need to perform a Viterbi detector simultaneously for each subcarrier.
We have seen in Section 4 that the performance of PaIC/Viterbi receivers depend on the power of the residual ISI which is concerned by the interference cancellation and also depend on the values of the spectrum distances d _{0}(ε) given by Eq. (16). To calculate the values of d _{0}(ε), we need first to determine the normalized autocovariance matrix of the noise R. Since the Viterbi algorithm is performed in the direction of the frequency axis, the noise vector w is then w=[w _{0,n },w _{1,n },…,w _{ M−1,n }]^{T}. Hence, the matrix \(\mathbf {R}=\mathbb {E}\{\mathbf {ww}^{H}\}/{\sigma _{w}^{2}}\) is only composed by the coefficients \(\acute {\Gamma _{p,0}}\), \(p\in \mathbb {Z}\) and is given by
where M is the number of subcarriers. Then, the values of d _{0}(ε) can be calculated for each single error event ε and for each one of the chosen receiver configurations (according to \(f_{0}^{(i)}\), i=1,2,3). We remind that the error events are classified according to their spectrum distance d _{0}(ε). The error events corresponding to the smallest d _{0}(ε) are called firstorder error events ε _{1}, and those corresponding to the second smallest d _{0}(ε) are called secondorder error events ε _{2}. Tables 7 and 8 give the different values of d _{0}(ε) of the first and secondorder error events for each receiver configuration, respectively, for the PHYDYAS and IOTA filters. We also show in the tables the corresponding power of the residual interference \(\left (\sigma _{\text {RISI}}^{2}\right)\) and the number of error positions (w(ε)) within the first and second error events.
It is worth noticing that in the third configuration using the IOTA filter, the values of the two smallest distances are practically the same. Moreover, the first and secondorder error events contain, respectively, 3 and 4 error positions (w(ε _{1})=3 and w(ε _{2})=4), whereas in the other cases, they only contain 1 and 2 errors, respectively. Consequently, even if the residual interference is perfectly removed, the bit error rate (BER) performance would be poor because the most likely error events are those that contain 3 or 4 errors.
Finally, we note that if we assume imperfect channel estimation, the performance of the proposed detectors (for both FBMC/OQAM and FBMC/QAM) will be affected. Indeed, channel estimation error will lead additional noise to the detected signals. However, according to the considered system model given by Eqs. (8) or (10), the channel estimation error is uncorrelated with the intrinsic interference. Hence, since the proposed configurations \(f_{0}^{(i)}\), i∈{1,2,3} are only related to the FBMC intrinsic interference, then they will not be changed in the context of imperfect channel estimation. Nevertheless, the metric used in the Viterbi algorithm might be changed to improve the detection performance under imperfect channel knowledge assumption. It is worthwhile to note that a generalized robust (against channel estimation errors) ML decoder that yields near optimal BER performance was proposed in [41].
Simulation results
In the following, we provide the simulation results concerning the three configurations of the PaIC/Viterbi receivers for both FBMC/OQAM and FBMC/QAM schemes with the PHYDYAS and IOTA filters. Since the motivation of this work is to address the problem of optimum detection in spatial multiplexing with FBMC, we have considered the simple 2×2 spatial multiplexing scheme. We assume perfect channel knowledge at the receiver side, and the four Rayleigh subchannels are spatially noncorrelated and follow the VehA channel model [42]. The parameters of the VehA channel are [42]: delays = [0 300 700 1100 1700 2500] ns, powers = [0 −1 −9 −10 −15 −20] dB, and the velocity is v=60 km/h. For the simulation settings, we considered that the sampling period is T _{ s }=100 ns, the carrier frequency is f _{ c }=1 GHz, and the number of subcarriers is M=512. Therefore, the maximum Doppler spread is given by \(f_{\mathrm {D}} = \frac {v f_{c}}{c}\), where c=3×10^{8} m/s is the light speed, thus f _{D}≈55.56 Hz. We should notice that with the above parameters, the approximation considered in Eq. (10) is still valid. The data symbols are offset quadrature phaseshift keying (OQPSK) modulated for FBMC/OQAM, and quadrature phaseshift keying (QPSK) modulated for FBMC/QAM. That is, the number of bits per data symbol is q=2 for both FBMC/OQAM and FBMC/QAM. The system performance is assessed in terms of BER as function of the signaltonoise ratio (SNR) per bit E _{ b }/N _{0} and is compared to that of the conventional CPOFDM with ML detector. The CP duration for CPOFDM is set to T _{CP}=32T _{ s }=3.2 μs. The considered SNR per bit for FBMC is given by
where \(E_{s} = \mathbb {E}\{\left s[\!m]\right ^{2}\}\) is the transmitted signal energy in each transmit antenna, N _{ t }=2 is the number of transmit antennas, and \({\sigma _{n}^{2}}\) is the noise power in each receive antenna. As for CPOFDM, we have
Therefore, the performance curves of CPOFDM exhibit an SNR loss of about 10 log10(17/16)≈0.26 dB.
FBMC/OQAM
Figure 3 depicts the performance of the MMSE equalizer (which is our tentative detector) and of the PaIC/Viterbi for the three considered configurations in FBMC/OQAM using the PHYDYAS filter. We clearly notice that PaIC/Viterbi3 exhibits almost the same performance as CPOFDM/ML, and that the RISI cancellation is effective. Hence, the value of the RISI power given in Table 3 for the third configuration is sufficiently small so that condition 2 is satisfied.
As for the first and second configurations, the performance degradation compared to CPOFDM/ML begins from about 12 dB. This relatively high degradation is due to the high values of the corresponding RISI powers causing error propagation.
Now, we consider only the third configuration \(\left (f_{0}^{(3)}\right)\) since the RISI cancellation is effective, and we assess the BER performance when using MAlgorithm instead of the Viterbi one in order to reduce the complexity. In Fig. 4, we show the obtained performance of the MAlgorithm with two different values of J (J=2 and J=4). We notice that with J=2, we have an SNR loss of about 2.5 dB compared to PaIC/Viterbi3. This SNR loss is due to the suboptimality of the MAlgorithm when J is small. Moreover, PaIC/MAlgo with J=2 provides a performance worse than the one provided by the tentative detector (MMSE) as long as the SNR is less than 12 dB. However, with J=4, we can observe that PaIC/MAlgo exhibits almost the same performance as PaIC/Viterbi3 but with much lower algorithm complexity (4 inner states instead of 16).
In the case where the IOTA filter is used, the BER performance curves are plotted in Fig. 5. We remark that no configuration among the three ones can provide the same BER performance as CPOFDM/ML. We can observe, for all configurations, an SNR loss with respect to CPOFDM/ML performance in the range where SNR is greater than 15 dB. This bad performance results are explained by the high residual interference powers presented in Table 4 that prevent the convergence to the optimal performance.
FBMC/QAM
Figure 6 depicts the BER performance of the proposed receivers in FBMC/QAM using the PHYDYAS filter. We notice that all the three PaIC/Viterbi configurations exhibit performance that converge to the CPOFDM/ML performance. This is because the conditions of effective interference cancellation are fulfilled, namely the RISI powers are sufficiently small, and the firstorder error events are separated enough from the other error events by their spectrum distance.
In the case where the IOTA filter is used, we can observe in Fig. 7 that PaIC/Viterbi3 receiver with the IOTA filter suffers from an SNR loss of about 1.2 dB with respect to CPOFDM/ML. This SNR loss is due to the errors caused by the Viterbi algorithm itself. Indeed, we have seen in Table 8 that the most likely error events are those that contain 3 or 4 errors, and the RISI power is negligible. That is, the BER performance curve is shifted up. In the other hand, since the effect of erroneous tentative decisions are practically negligible because the variance of the residual interference is very small \(\left (\sigma _{\text {RISI}}^{2}\approx 0.007\right)\), the tentative detector (MMSE equalizer) is not useful and can be withdrawn.
As for PaIC/Viterbi2 and PaIC/Viterbi1 receivers, we can notice that with the IOTA filter the BER performance does not reach the CPOFDM/ML one. We obtain an SNR loss at BER=10^{−2} with respect to CPOFDM/ML of about 2.25 dB for PaIC/Viterbi2, and 6.25 dB for PaIC/Viterbi1. Hence, both latter configurations suffer from error propagation and the interference cancellation is not effective. This performance limitation is explained by the fact that the residual interference variances are not sufficiently small; according to Table 8, we have \(\sigma _{\text {RISI}}^{2}\approx 0.2\) and \(\sigma _{\text {RISI}}^{2}\approx 0.4\) for PaIC/Viterbi2 and PaIC/Viterbi1, respectively.
Because the orthogonality condition is lost in FBMC/QAM, we can observe the effect of the interference by comparing the BERperformance of the MMSE equalizer in both cases where the PHYDYAS or IOTA filter is used. We notice in Fig. 6 that the MMSE equalizer reaches the BER value of 10^{−1} at SNR=15.5 dB, whereas in Fig. 7, we have BER=10^{−1} at SNR=9.5 dB. This is due to the fact that the BER floor in the IOTA case is much higher than the BER floor in the case where the PHYDYAS filter is used.
Conclusions
The intrinsic interference in FBMC is a 2DISI in the timefrequency plan. In order to avoid a full 2DViterbi detector, we have proposed in this paper to study a tradeoff between a whole interference cancellation receiver and a full 2DViterbi detector. The proposed receiver is composed by a tentative detector giving decisions which serve to partially cancel the interference, followed by a Viterbi detector matching to the noncancelled interference. Three configurations were treated. The first one is called PaIC/Viterbi1 and correspond to the whole interference cancellation. The second one is PaIC/Viterbi2, where the Viterbi detector matches with the two largest coefficients of the transmultiplexer impulse response. The third one is PaIC/Viterbi3 and the Viterbi detector matches with the three largest coefficients. We have shown that the performance of the PaIC/Viterbi receivers essentially depends on the power of the residual interference (RISI) which is not concerned by the Viterbi detector. When the RISI power is not sufficiently low, the receivers suffer from error propagation.
We have studied the proposed receivers in both FBMC/OQAM and FBMC/QAM. This latter can offer the best performance in some situation since the global intrinsic interference is reduced. For FBMC/OQAM, we have shown by simulations that only PaIC/Viterbi3 with the PHYDYAS filter can provide satisfactory performance compared to CPOFDM/ML one. Whereas for FBMC/QAM, all the three PaIC/Viterbi configurations with the PHYDYAS filter exhibit the same performance as CPOFDM/ML. However, the interference coefficients of the IOTA filter do not allow any PaIC/Viterbi configuration to reach the optimal BER performance. Indeed, in spite of the lowest RISI power for PaIC/Viterbi3 with the IOTA filter, the BER performance is limited because of the fact that the most likely error events in the Viterbi detector contain more than 3 errors.
Finally, since the Viterbi detection increases the computational complexity, we can replace the Viterbi detector by another one based on MAlgorithm in order to reduce the receiver complexity without performance deterioration.
As a future work, one can consider extending the present work to the context of imperfect channel state information. As we have aforementioned, in order to minimize the BER performance degradation, we should adapt the Viterbi algorithm and investigate an efficient Viterbi metric capable of achieving near optimal performance.
References
 1
B FarhangBoroujeny, OFDM versus filter bank multicarrier. IEEE Signal Proc. Mag. 28(3), 92–112 (2011).
 2
B Le Floch, M Alard, C Berrou, Coded orthogonal frequency division multiplex. Proc. IEEE. 83(6), 982–996 (1995).
 3
M Alard, Construction of a multicarrier signal. Patent WO 96/35278 (1996).
 4
T Strohmer, S Beaver, Optimal OFDM design for timefrequency dispersive channels. IEEE Trans. Commun. 51(7), 1111–1122 (2003).
 5
P Siohan, C Siclet, N Lacaille, Analysis and design of OFDM/OQAM systems based on filterbank theory. IEEE Trans. Signal Process. 50(5), 1170–1183 (2002).
 6
C Lélé, P Siohan, R Legouable, JP Javaudin, in Power Line Communications and Its Applications, 2007. ISPLC ‘07. IEEE International Symposium on. Preamblebased channel estimation techniques for OFDM/OQAM over the powerline (IEEEPiscataway, NJ, 2007), pp. 59–64.
 7
T Ihalainen, TH Stitz, M Renfors, in Circuits and Systems, 2005. ISCAS 2005. IEEE International Symposium on, 4. Efficient percarrier channel equalizer for filter bank based multicarrier systems (IEEEPiscataway, NJ, 2005), pp. 3175–3178.
 8
T Ihalainen, T Hidalgo Stitz, M Rinne, M Renfors, Channel equalization in filter bank based multicarrier modulation for wireless communications. EURASIP J. Appl. Signal Process. 2007(1), 140–140 (2007).
 9
DS Waldhauser, LG Baltar, J Nossek, in Signal Processing Advances in Wireless Communications, 2008. SPAWC 2008. IEEE 9th Workshop on. MMSE subcarrier equalization for filter bank based multicarrier systems (IEEEPiscataway, NJ, 2008), pp. 525–529.
 10
Zs Kollar, G Peceli, P Horvath, in Advances in Cognitive Radio, 2011. COCORA. IEEE First International Conference on, 7. Iterative decision feedback equalization for FBMC systems, (2011), p. 60056.
 11
LG Baltar, DS Waldhauser, JA Nossek, in Circuits and Systems, 2009. ISCAS 2009. IEEE International Symposium on. MMSE subchannel decision feedback equalization for filter bank based multicarrier systems, (2009), pp. 2802–2805.
 12
E Kofidis, AA Rontogiannis, in Personal Indoor and Mobile Radio Communications (PIMRC) 2010. IEEE 21st International Symposium on. Adaptive blast decisionfeedback equalizer for MIMOFBMC/OQAM systems, (2010), pp. 841–846.
 13
A Ikhlef, J Louveaux, in Communications, Computers and Signal Processing, 2009. PacRim 2009. IEEE Pacific Rim Conference on. Per subchannel equalization for MIMO FBMC/OQAM systems (IEEEPiscataway, NJ, 2009), pp. 559–564.
 14
T Ihalainen, A Ikhlef, J Louveaux, M Renfors, Channel equalization for multiantenna FBMC/OQAM receivers. IEEE Trans. Veh. Technol. 60(5), 2070–2085 (2011).
 15
M Caus, AI PerezNeira, Transmitterreceiver designs for highly frequency selective channels in MIMO FBMC systems. IEEE Trans. Signal Process. 60(12), 6519–6532 (2012).
 16
I Estella, A PascualIserte, M Payaro, in Future Network and Mobile Summit, 2010. OFDM and FBMC performance comparison for multistream MIMO systems (IEEEPiscataway, NJ, 2010), pp. 1–8.
 17
M Caus, AI PerezNeira, in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. Multistream transmission in MIMOFBMC systems (IEEEPiscataway, NJ, 2013), pp. 5041–5045.
 18
M Payaro, A PascualIserte, M Najar, in Wireless Conference (EW), 2010 European. Performance comparison between FBMC and OFDM in MIMO systems under channel uncertainty (IEEEPiscataway, NJ, 2010), pp. 1023–1030.
 19
M Caus, AI PerezNeira, Multistream transmission for highly frequency selective channels in MIMOFBMC/OQAM Systems. IEEE Trans. Signal Process. 62(4), 786–796 (2014).
 20
Y Cheng, P Li, M Haardt, in IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP). Coordinated Beamforming in MIMO FBMC/OQAM systems (IEEEPiscataway, NJ, 2014), pp. 484–488.
 21
M El Tabach, JP Javaudin, M Helard, in Communications, 2007 ICC ’07. IEEE International Conference on. Spatial data multiplexing over OFDM/OQAM modulations (IEEEPiscataway, NJ, 2007), pp. 4201–4206.
 22
C Lélé, P Siohan, R Legouable, The Alamouti scheme with CDMAOFDM/OQAM. EURASIP J. Adv. Sig. Proc. 2010:, 1–13 (2010).
 23
H Lin, C Lélé, P Siohan, in Signal Processing Advances in Wireless Communications, 2009 SPAWC ‘09. IEEE 10th Workshop on. A pseudo Alamouti transceiver design for OFDM/OQAM modulation with cyclic prefix (IEEEPiscataway, NJ, 2009), pp. 300–304.
 24
M Renfors, T Ihalainen, TH Stitz, in Wireless Conference (EW), 2010 European. A blockAlamouti scheme for filter bank based multicarrier transmission (IEEEPiscataway, NJ, 2010), pp. 1031–1037.
 25
R Zakaria, D Le Ruyet, in Wireless Communication Systems (ISWCS 2013), Proceedings of the Tenth International Symposium on. On interference cancellation in Alamouti coding scheme for filter bank based multicarrier systems (IEEEPiscataway, NJ, 2013), pp. 1–5.
 26
R Zakaria, D Le Ruyet, Intrinsic interference reduction in a filter bankbased multicarrier using QAM modulation. Phys. Commun. (0), (2013).
 27
R van Nee, A van Zelst, G Awater, in Vehicular Technology Conference Proceedings, 2000. VTC 2000Spring Tokyo 2000 IEEE 51st, 1. Maximum likelihood decoding in a space division multiplexing system (IEEEPiscataway, NJ, 2000), pp. 6–10.
 28
A Gersho, TL Lim, Adaptive cancellation of intersymbol interference for data transmission. Bell Syst Tech. J. 60:, 1997–2021 (1981).
 29
R Zakaria, D Le Ruyet, M Bellanger, in Wireless Conference (EW), 2010 European. Maximum likelihood detection in spatial multiplexing with FBMC (IEEEPiscataway, NJ, 2010), pp. 1038–1041.
 30
R Zakaria, D Le Ruyet, Y Medjahdi, in Wireless Communication Systems (ISWCS), 2012 International Symposium on. On ISI cancellation in MIMOML detection using FBMC/QAM modulation (IEEEPiscataway, NJ, 2012), pp. 949 –953.
 31
OE Agazzi, N Seshadri, On the use of tentative decisions to cancel intersymbol interference and nonlinear distortion (with application to magnetic recording channels). IEEE Trans. Inf. Theory. 43(2), 394–408 (1997).
 32
R Zakaria, D Le Ruyet, in Wireless Communication Systems (ISWCS), 2011 8th International Symposium on. Partial ISI cancellation with Viterbi detection in MIMO filterbank multicarrier modulation (IEEEPiscataway, NJ, 2011), pp. 322 –326.
 33
A Sahin, I Guvenc, H Arslan, A survey on multicarrier communications: prototype filters, lattice structures, and implementation aspects. IEEE Commun. Surv. Tutor. 16(3), 1312–1338 (2014).
 34
M Bellanger, PHYDYAS team, FBMC physical layer: a primer (2010). Website: http://www.ictphydyas.org. Accessed 2010.
 35
S Mirabbasi, K Martin, in Circuits and Systems, 2002. ISCAS 2002 IEEE International Symposium on, 1. Design of prototype filter for nearperfectreconstruction overlapped complexmodulated transmultiplexers (IEEEPiscataway, NJ, 2002), pp. I–821–I–824.
 36
MG Bellanger, in Acoustics, Speech, and Signal Processing, 2001 Proceedings. (ICASSP ‘01), 2001 IEEE International Conference on, 4. Specification and design of a prototype filter for filter bank based multicarrier transmission (IEEEPiscataway, NJ, 2001), pp. 2417–2420.
 37
D Katselis, E Kofidis, A Rontogiannis, S Theodoridis, Preamblebased channel estimation for CPOFDM and OFDM/OQAM systems: a comparative study. IEEE Trans. Signal Process. 58(5), 2911–2916 (2010).
 38
S Van Beneden, J Riani, JWM Bergmans, in Communications, 2006 ICC ’06. IEEE International Conference on, 7. Cancellation of linear intersymbol interference for twodimensional storage systems (IEEEPiscataway, NJ, 2006), pp. 3173–3178.
 39
R Zakaria, D Le Ruyet, in Personal Indoor and Mobile Radio Communications (PIMRC), 2011 IEEE 22nd International Symposium on. On spatial data multiplexing over coded filterbank multicarrier with ML detection (IEEEPiscataway, NJ, 2011), pp. 1391–1395.
 40
J Anderson, S Mohan, Sequential coding algorithms: a survey and cost analysis. IEEE Trans. Commun. 32(2), 169–176 (1984).
 41
B Sim Thian, A Goldsmith, in Communication, Control, and Computing (Allerton), 2011 49th Annual Allerton Conference on. Decoding for MIMO systems with correlated channel estimation errors, (2011), pp. 524–530.
 42
ITUR M.1225, Guidelines for evaluations of radio transmission technologies for IMT2000 (1997).
 43
D Tse, P Viswanath, Fundamentals of Wireless Communication (Cambridge University Press, New York, 2005).
Author information
Additional information
Competing interests
The authors declare that they have no competing interests.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Zakaria, R., Le Ruyet, D. Partial interference cancellation with maximum likelihood sequence detection in FBMC spatial multiplexing system. EURASIP J. Adv. Signal Process. 2016, 47 (2016). https://doi.org/10.1186/s136340160339x
Received:
Accepted:
Published:
Keywords
 Orthogonal Frequency Division Multiplex
 Minimum Mean Square Error
 Cyclic Prefix
 Interference Cancellation
 Viterbi Algorithm