An improved scheme based on log-likelihood-ratio for lattice reduction-aided MIMO detection

Abstract

Lattice reduction (LR)-aided detectors have been shown great potentials in wireless communications for their low complexity and low bit-error-rate (BER) performance. The LR algorithms use the unimodular transformation to improve the orthogonality of the channel matrix. However, the LR algorithms only utilize the channel state information (CSI) and do not take account for the received signal, which is also important information in enhancing the performance of the detectors. In this paper, we make a readjustment of the received signal in the LR domain and propose a new scheme which is based on the log-likelihood-ratio (LLR) criterion to improve the LR-aided detectors. The motivation of using the LLR criterion is that it utilizes both the received signal and the CSI, so that it can provide exact pairwise error probabilities (PEPs) of the symbols. Then, in the proposed scheme, we design the LLR-based transformation algorithm (TA) which uses the unimodular transformation to minimize the PEPs of the symbols by the LLR criterion. Note that the PEPs of the symbols affect the error propagation in the vertical Bell Laboratories Layered Space-Time (VBLAST) detector, and decreasing the PEPs can reduce the error propagation in the VBLAST detectors; thus, our LLR-based TA-aided VBLAST detectors will exhibit better BER performance than the previous LR-aided VBLAST detectors. Both the BER performance and the computational complexity are demonstrated through the simulation results.

1 Introduction

The multiple-input multiple-output (MIMO) technology plays an important role in increasing the spectral efficiency of wireless communications by allowing for spatial multiplexing [1]. However, with the increase of the number of antennas, the complexity of the hardware and the signal processing at both the transmitters and the receivers are increased [2]. At the receiver side, designing reliable and efficient detectors has become a critical challenge of the MIMO systems. The maximum likelihood detectors (MLD) [3–5] which are the performance-optimal detectors suffer from exponential complexity in terms of the number of the transmitted signal, and they are not practical when the number of the transmitted signal becomes large. The linear detectors (LD) [6], including the zero forcing (ZF) and the minimum mean square error (MMSE) LDs, can provide polynomial complexity. However, the bit-error-rate (BER) performance of the LDs is far inferior to the MLD. To improve the BER performance of the LDs, the decision feedback equalization, which utilizes the successive interference cancelation (SIC) scheme, can cancel the interference between different antennas and exhibit better BER performance than the LDs. To further improve the BER performance, the well-known vertical Bell Laboratories Layered Space-Time (VBLAST) detector [7–10], using both the SIC scheme and the ordering process, can provide excellent performance, while the error propagation is a critical problem in the VBLAST detector. Some other non-linear detectors were proposed to reduce the gap between the LDs and the MLD. The likelihood ascent search [11] and reactive tabu search [12] can provide well BER performance for low-order modulations, however exhibit performance degradation for higher-order quadrature amplitude modulation (QAM). The layered tabu search [13] was proposed to further improve the BER performance for higher-order QAM. However, this algorithm brings large complexity when the constellation size is large.

Recently, an efficient scheme called lattice reduction [14–18] has been shown great potential in MIMO detections. The lattice reduction (LR) algorithm attempts to change the orthogonality of the channel matrix, as the orthogonality of the channel matrix largely affects the performance of the MIMO system [15]. As the well-known LR algorithm, Lenstra, Lenstra, and Lovasz (LLL) algorithm has firstly been considered for LR-aided detection [16]. It allows suboptimum detectors, such as the LDs and the VLBAST detectors, to exploit all the available diversity [16], while it has polynomial complexity. Many LR algorithms have been developed to improve the performance of the LLL algorithm. The LR algorithm improving the minimum Euclidean distance of the LR-aided LDs, which can improve the BER performance of the LR-aided LDs, is proposed in [17]. Element-based lattice reduction (ELR) algorithm [15], a low complexity LR algorithm, was proposed to reduce the diagonal elements of the noise covariance matrix. The ELR-aided detectors show better BER performance than other LR-aided detectors, especially for large MIMO systems [15]. An improved ELR algorithm [18] is proposed to enhance the BER performance of the ELR algorithm; however, it brings large complexity increase. As it needs to solve a closest vector problem by the sphere decoding method [4, 5], which requires exponential complexity, it will be complexity expensive and impractical when the number of the transmitted signal becomes moderate and large. Moreover, [19, 20] proposed some efficient ways to reduce the complexity of the LR algorithm.

As shown above, the previous LR algorithms aim to use unimodular transformation to change the orthogonality of the channel matrix, such that the gap between the suboptimal detectors and the MLD is reduced. However, these LR algorithms do not take the received signal into consideration. In fact, the received signal is important information in the MIMO detection. It can partially reflect the received noise and is also useful in enhancing the BER performance of the LR-aided detectors. In this paper, we make a readjustment of the received signal in the LR domain and propose a scheme to improve the LR-aided detectors. The proposed scheme is based on a new criterion called the log-likelihood-ratio (LLR) criterion, which utilizes both the received signal and the channel state information (CSI). Then, by the LLR criterion, we propose our LLR-based transformation algorithm (TA) which targets to use the unimodular transformation to minimize the pairwise error probabilities (PEPs) of the symbols, while these PEPs are deduced exactly in this paper by the information of the CSI and the received signal. We show that the PEPs affect the error propagation of the VBLAST detector, and decreasing the PEPs can reduce the error propagation and enhance the BER performance of the VBLAST detector. In our proposed algorithm, a standard LR algorithm such as LLL or ELR algorithm is performed as the initial stage, where the LLL and the ELR algorithm is the classical LR algorithms as shown above, then an algorithm decreasing the PEPs of the symbols is shown. The simulation results validate that our LLR-based TA-aided VBLAST detectors can provide substantial BER performance gain over the pervious LR-aided VBLAST detectors, while only moderate computational complexity is increased.

Notation: (·)^T is the transpose of (·), and (·)^† is the pseudo-inverse of (·). We write A _i,j for the entry in the ith row and jth column of the matrix A, a _i for the ith entry in a. a _i and a ⁱ denote for the ith row and the ith column of the matrix A, respectively.

2 Preliminary

2.1 System model

Consider a MIMO system with N _t transmitted antennas and N _r received antennas as

$$ \textbf{r}^{c}=\textbf{H}^{c}\textbf{s}^{c}+\textbf{w}^{c}, $$

((1))

where $\textbf {r}^{c}\in \mathbb {C}^{N_{r}}$ is the received signal, $\textbf {H}^{c}\in \mathbb {C}^{{N_{r}}\times {N_{t}}}$ is the channel matrix. The entries of H ^c are represented as independent and identically distributed variables drawn from $\mathcal {CN}\left (0,\frac {1}{N_{t}}\right)$. $\textbf {s}^{c}\in \mathbb {C}^{N_{t}}$ is the transmitted signal independent and identically drawn from the MQAM constellation, where M is the constellation size, and the covariance matrix of s ^c is $\sigma ^{2}_{\textbf {s}^{c}}\textbf {I}$. ${\textbf {w}}^{c}\!\in \!\mathbb{C}^{N_{r}}$ is a zero-mean white Gaussian random vector with covariance matrix σ ² I. Moreover, the number of the transmitted signal is assumed to be less than or equal to the number of the received signal.

Note that (1) is equivalent to the real input-output model [14]

$$ \textbf{r}=\textbf{H s}+\textbf{w}, $$

((2))

where

$$\textbf{r}=\left[\mathfrak{R}(\textbf{r}^{c})^{T},\mathfrak{I}(\textbf{r}^{c})^{T}\right]^{T}, $$

$$\textbf{s}=\left[\mathfrak{R}(\textbf{s}^{c})^{T},\mathfrak{I}(\textbf{s}^{c})^{T}\right]^{T}, $$

$$\textbf{w}=\left[\mathfrak{R}(\textbf{w}^{c})^{T},\mathfrak{I}(\textbf{w}^{c})^{T}\right]^{T}, $$

and

$$\begin{array}{@{}rcl@{}} \textbf{H}=\left[ \begin{array}{cc} \mathfrak{R}(\textbf{H}^{c})&-\mathfrak{I}(\textbf{H}^{c})\\ \mathfrak{I}(\textbf{H}^{c})&\mathfrak{R}(\textbf{H}^{c})\\ \end{array} \right]. \end{array} $$

We know $\textbf {r}\in \mathbb {R}^{K}$, $\textbf {w}\in \mathbb {R}^{K}$, $\textbf {s}\in \mathbb {R}^{N}$, and $\textbf {H}\in \mathbb {R}^{K\times N}$, where K=2N _r and N=2N _t .

2.2 The LR-aided detectors

Note the orthogonality of the channel matrix largely affects the performance of the MIMO detection and the gap of the BER performance between the suboptimal detectors (e.g., the LDs and the VBLAST detectors) and the MLD is mainly due to the orthogonality [15]. It has been shown that the more orthogonal the channel matrix is, the smaller the gap between the suboptimal detectors and the MLD will be. The LR technique is one technique that can change the orthogonality of the channel matrix and reduce the gap between the suboptimal detectors and the MLD.

As shown above, the orthogonality of the channel matrix is crucial to the MIMO detection. The previous LR algorithms aim to find a unimodular transformation $\bar {\mathbf {H}} = \textbf {HT}$ [14], where T is a unimodular matrix (i.e., all the entries of T are integers, and the determinant of T is ±1), such that $\bar {\textbf {H}}$ is more orthogonal. In our paper, we aim to find a unimodular transformation $\bar {\textbf {H}} = \textbf {HT}$ to minimize the PEPs of the symbols.

The LR-aided detectors and our TA-aided detectors in ZF criterion based on $\bar {\mathbf {H}}$ are performed as follows [14, 15]. Note

$$ \textbf{r}=\textbf{HTT}^{-1}\textbf{s}+\textbf{w}=\bar{\textbf{H}}\bar{\textbf{s}}+\textbf{w}, $$

((3))

where $\bar {\textbf {H}}=\textbf {HT}$, and $\bar {\textbf {s}}=\textbf {T}^{-1}\textbf {s}$. To make (3) be more easily analyzed, we transform the domain of the detected symbol be consecutive integers as follows [15]. We apply scaling and shifting on s, i.e., (s−1 _N×1)/2 where $\textbf {1}_{N\times 1}\in \mathbb {R}^{N\times 1}$ is the matrix with entries 1, and the constellation set is transferred to consecutive integer sets. Then, setting x=T ⁻¹(s−1 _N×1)/2, the set of the element of x is consecutive integer set too, and we have

$$ \bar{\textbf{s}}={\textbf{T}^{-1}\textbf{s}}=2\textbf{x}+\textbf{T}^{-1}\textbf{1}_{N\times1}. $$

((4))

Then substituting (4) into (3), (3) can be written as

$$ \textbf{y}=\bar{\textbf{H}}\textbf{x}+\textbf{n}, $$

((5))

where $\textbf {y}=\left (\textbf {r}-\bar {\textbf {H}}\textbf {T}^{-1}\textbf {1}_{N\times 1}\right)/2$, n=w/2, whose variance is ${\sigma _{n}^{2}}={\sigma ^{2}}/{8}$. Considering (5), the channel matrix is more orthogonal than the channel matrix in (2), and the domain of the elements in x is consecutive integer set. Wubben and Seethaler [21] shows that it is complicated to consider the domain of x, and with no prior knowledge of x, the element of x is assumed to be drawn from the integer set $\mathbb {Z}$ with equal probability.

Considering (5), the suboptimal MIMO detectors such as the LDs or the VBLAST detectors can be used to obtain the estimation $\hat {\textbf {x}}$ of x. Once we get the estimation of x, and from (4), the estimation of $\bar {\mathbf {s}}$ can be calculated as [15]

$$ \hat{\bar{\textbf{s}}}=2\hat{\textbf{x}}+\textbf{T}^{-1}\textbf{1}_{N\times1}. $$

((6))

As $\textbf {s}=\textbf {T}\hat {\textbf {s}}$, then we can obtain the estimation $\hat {\textbf {s}}$ by quantizing $\textbf {T}\hat {\bar {\textbf {s}}}$ to the constellation set as

$$ \hat{\textbf{s}}=Q\left(\textbf{T}\hat{\bar{\textbf{s}}}\right), $$

((7))

where Q(·) denotes componentwise quantization.

As the detectors in the MMSE criterion can provide better performance than the detectors in ZF criterion, here we describe the and our TA-aided detectors in the MMSE criterion. For the MMSE detectors [15], we only need to replace the channel matrix H and the signal r in the ZF criterion by the extended matrix

$$ \textbf{H}_{MMSE}=\left(\begin{array}{c} \textbf{H} \\ \frac{1}{\rho}\textbf{I}\\ \end{array} \right), \textbf{r}_{MMSE}=\left(\begin{array}{c} \textbf{r} \\ \textbf{0} \\ \end{array} \right), $$

((8))

where ${\rho }^{2}=\frac {\sigma ^{2}_{\textbf {s}^{c}}}{\sigma ^{2}}$, $\sigma ^{2}_{\textbf {s}^{c}}$, and σ ² are defined above.

3 The LLR-based transformation algorithm

In this section, we will describe our LLR-based TA in both the ZF and the MMSE criterion. Our LLR-based TA utilizes both the CSI information and the received signal and aims to find a unimodular transformation $\bar {\textbf {H}} = \textbf {HT}$ to minimize the PEPs of the detected symbol in the VBLAST detectors, while minimizing the PEPs is helpful to enhance the BER performance of the VBLAST detectors.

3.1 LLR criterion for our transformation algorithm

Considering (5), in the first step of the ZF VBLAST detector, we should multiply (5) by the equalization matrix $\bar {\textbf {H}}^{\dag }=\textbf {T}^{-1}\textbf {H}^{\dag }$, where (·)^† is the pseudo-inverse operation, then we have

$$ \bar{\textbf{y}}=\textbf{x}+\bar{\textbf{n}}, $$

((9))

where $\bar {\textbf {y}}=\textbf {T}^{-1}\textbf {H}^{\dag }\textbf {y}$ and $\bar {\textbf {n}}=\textbf {T}^{-1}\textbf {H}^{\dag }\bar {\textbf {n}}$. Then, the covariance matrix C of $\bar {\textbf {n}}$ will be

$$\begin{array}{@{}rcl@{}} \textbf{C}&=&{\sigma_{n}^{2}}\textbf{T}^{-1}\textbf{H}^{\dag}\left(\textbf{T}^{-1}\textbf{H}^{\dag}\right)^{T}\\ &=&{\sigma_{n}^{2}}\textbf{T}^{-1}\left(\textbf{H}^{T}\textbf{H}\right)^{-1}\left(\textbf{T}^{-1}\right)^{T}. \end{array} $$

((10))

Considering the lth component of (9), we have

$$ {\bar{y}}_{l}={x}_{l}+{\bar{n}}_{l}, $$

((11))

where ${\bar {y}}_{l}$, x _l , and ${\bar {n}}_{l}$ denote the lth component of $\bar {\textbf {y}}$, x, and $\bar {\textbf {n}}$, respectively. From (10), it is easy to know that the variance of the noise ${\bar {n}}_{l},l= 1,\cdots,N$ is $\sigma ^{2}_{\bar {n}_{l}}=C_{l,l}$.

As x _l is drawn from the set $\mathbb {Z}$, then it is easy to know that

$$ \sum_{z\in \mathbb{Z}} P\left(x_{l}=z|\bar{y}_{l}\right)=1. $$

((12))

Denote

$$ {\boldsymbol\Lambda}_{l,z}\left(\textbf{C},\bar{\textbf{y}}\right)=\ln\frac{P\left({x}_{l}=\hat{x}_{l}|\bar{y}_{l}\right)}{P\left(x_{l}=z|\bar{y}_{l}\right)} $$

((13))

as the pairwise LLR [22], where z is a component in the set of x _l (i.e., $\mathbb {Z}$), and $\hat {x}_{l}$ is the estimation of the component x _l . If we want to detect the component x _l in the first step of the ZF VBLAST detector, then x _l will be detected as the component in the set of x _l that is closest to $\bar {y}$, i.e., $\hat {x}_{l}=\left \lfloor {\bar {y}}_{l}\right \rceil$, where ⌊·⌉ is the rounding function [14].

Then from (12), (13), the PEP of the component x _l , can be calculated as

$$\begin{array}{@{}rcl@{}} &&P\left(x_{l}\neq \hat{x}_{l}|\bar{y}_{l}\right)\\ &=&1-P\left(x_{l}=\hat{x}_{l}|\bar{y}_{l}\right)\\ &=&1-\frac{P\left(x_{l}=\hat{x}_{l}|\bar{y}_{l}\right)}{\sum_{z\in \mathbb{Z}} P\left(x_{l}=z|\bar{y}_{l}\right)}\\ &=&1-\frac{1}{\sum_{z\in \mathbb{Z}} \frac{P\left(x_{l}=z|\bar{y}_{l}\right)}{P\left(x_{l}=\hat{x}_{l}|\bar{y}_{l}\right)}}\\ &=&1-\frac{1}{\sum_{z\in \mathbb{Z}}\exp\left(-{\boldsymbol\Lambda}_{l,z}\left(\textbf{C},\bar{\textbf{y}}\right)\right)}. \end{array} $$

((14))

As shown in Section 2, each symbol is transmitted with equal probability. Then from [9], (13) can be written as

$$ {\boldsymbol\Lambda}_{l,z}\left(\textbf{C},\bar{\textbf{y}}\right)=\ln\frac{P\left(\bar{y}_{l}|{x}_{l}=\hat{x}_{l}\right)}{P\left(\bar{y}_{l}|x_{l}=z\right)}. $$

((15))

For $\forall z\in {\mathbb {Z}}$, we have

$$ P\left(\bar{y}_{l}|x_{l}=z\right)=\frac{1}{\sqrt{2\pi C_{l,l}}} \exp\left(-\frac{\left(\bar{y}_{l}-z\right)^{2}}{2C_{l,l}}\right). $$

((16))

Then from (16), (15) can be simplified as

$$\begin{array}{@{}rcl@{}} {\boldsymbol\Lambda}_{l,z}\left(\textbf{C},\bar{\textbf{y}}\right)&=&\ln\frac{\frac{1}{\sqrt{2\pi C_{l,l}}} \exp\left(-\frac{\left(\bar{y}_{l}-\hat{x}_{l}\right)^{2}}{2C_{l,l}}\right)} {\frac{1}{\sqrt{2\pi C_{l,l}}} \exp\left(-\frac{\left(\bar{y}_{l}-z\right)^{2}}{2C_{l,l}}\right)}\\ &=&-\frac{\left(\bar{y}_{l}-\hat{x}_{l}\right)^{2}}{2C_{l,l}} +\frac{\left(\bar{y}_{l}-z\right)^{2}}{2C_{l,l}}. \end{array} $$

((17))

Substituting (17) into (14), we can get the PEPs of the symbols x _l ,l=1,2,⋯,N.

Considering (14), it is complicated to analyze the summation in the denominator, as the summation has infinity terms. Actually, similar with [23], we can simplify the summation by only considering the dominant terms. Note that, the term $\exp \left (-{\boldsymbol \Lambda }_{l,z}\left (\textbf {C},\bar {\textbf {y}}\right)\right)$ in the summation decreases exponentially with $\left (\bar {y}_{l}-z\right)^{2}$. So it can be known that, when z is chosen to be close to $\bar {y}_{l}$, the terms $\exp \left (-{\boldsymbol \Lambda }_{l,z}\left (\textbf {C},\bar {\textbf {y}}\right)\right)$ will be the dominant terms. Set $\mathbb {D}_{l}$ as the set whose components are the B integers closest to $\bar {y}_{l}$, then the PEPs can be approximated as

$$\begin{array}{@{}rcl@{}} P\left(x_{l}\neq \hat{x}_{l}|\bar{y}_{l}\right)\approx1-\frac{1}{\sum_{z\in \mathbb{D}_{l}}\exp\left(-{\boldsymbol\Lambda}_{l,z}\left(\textbf{C},\bar{\textbf{y}}\right)\right)}. \end{array} $$

((18))

In fact, decreasing the PEPs is not beneficial to the orthogonality of the channel matrix, and the proposed LLR-based algorithm which aims to decrease the PEPs in (18) will not help to enhance the BER performance of the LR-aided ZF LD. However, for the VBLAST detector, we find that the PEPs directly affect the error probability of the first detected symbol in the VBLAST detector. Note that the error probability of the first detected symbol in the VBLAST detector is a key factor affecting the error propagation of the VBLAST detector, which is important to the BER performance of the VBLAST detector. Decreasing the PEPs will help to reduce the error propagation, and the BER performance of the VBLAST detector will be enhanced.

As minimizing the PEPs can improve the BER performance of the VBLAST detector, our target is to design a unimodular matrix such that the PEPs will be minimized. The problem can be formulated as follows.

As the largest PEP will result in symbol error with large probability, we target to minimize the largest PEP at first, i.e., we find a unimodular matrix T such that

$$\begin{array}{@{}rcl@{}} &&\min_{\textbf{T}} \max_{l} P\left(x_{l}\neq \hat{x}_{l}|\bar{y}_{l}\right)\\ s.t.&&\textbf{C}={\sigma_{n}^{2}}\textbf{T}^{-1}\left(\textbf{H}^{T}\textbf{H}\right)^{-1}\left(\textbf{T}^{-1}\right)^{T}\\ &&\bar{\textbf{y}}=\textbf{T}^{-1}\textbf{H}^{\dag}\textbf{y}. \end{array} $$

((19))

Note the solution in (19) is not unique, and there may exist many solutions in the problem (19). Secondly, we target to find a unimodular matrix to minimize the second largest PEP, which is also crucial to the BER performance, while keeping the largest PEP unchanged. Then, we can further minimize the third largest PEP, the forth one, and so on. This process continues until all the PEPs cannot be decreased.

Then, above process shows the LLR criterion for the ZF VBLAST detectors. Now, we propose the LLR criterion for the MMSE VBLAST detectors. For (5), [18] shows that in the first step of MMSE VBLAST detector, and by some manipulation, we have

$$ \bar{\textbf{y}}_{MMSE}=\textbf{x}+\bar{\textbf{n}}_{MMSE}, $$

((20))

where $\bar {\textbf {y}}_{\textit {MMSE}}=\textbf {T}^{-1}\left (\textbf {H}_{\textit {MMSE}}\right)^{\dag }\textbf {y}$, $\bar {\textbf {n}}_{\textit {MMSE}}$ is the noise, and its covariance matrix is

$$ \textbf{C}_{MMSE}={\sigma^{2}_{n}}\textbf{T}^{-1}\left(\left(\textbf{H}_{MMSE}\right)^{T}\textbf{H}_{MMSE}\right)^{-1}\left(\textbf{T}^{-1}\right)^{T}. $$

((21))

Comparing (20) with (9), the formulation in ZF and MMSE criterions are similar; then, the LLR criterion in MMSE case can be derived in the similar process as the LLR criterion in the ZF case, and we omit the process here.

3.2 The description of the LLR-based transformation algorithm

In this subsection, we propose our LLR-based TA to minimize the PEPs as described in the last subsection. The proposed algorithm consists of two stages. In the first stage, a previous LR algorithm such as LLL or ELR is performed. In the second stage, we aim to find a unimodular matrix to minimize the PEPs as described above. The advantage of the two-stage algorithm is that it can ensure performance improvement for problems of different dimensions and different modulations. Moreover, it can alleviate the computational complexity.

In fact, the solution to minimize the PEPs in the second stage cannot be obtained straightforward, and finding the global optimal solution is computationally expensive. So, we develop one iterative algorithm to find the solution as follows. The iterative algorithm is suboptimal.

Before we give our proposed algorithm, we define the row-addition operation which will be used in each iteration in the second stage of our algorithm.

Definition (row-addition operation): Let U _k,i denote the N×N matrix with a one at the (k,i)th entry and zero elsewhere. Then, a row-addition operation on the matrix T ⁻¹ is defined as

$$ \textbf{T}^{-1}\leftarrow\left(\lambda_{k,i}\textbf{U}_{k,i}+\textbf{I}\right)\textbf{T}^{-1}, $$

((22))

where λ _k,i is an integer and |λ _k,i | is called the step length.

From (22), we know after the row-addition operation, the matrix T ⁻¹ can be obtained by updating the kth row of T ⁻¹ as

$$ \textbf{t}'_{k}\leftarrow\textbf{t}'_{k}+\lambda_{k,i}\textbf{t}'_{i}, $$

((23))

where t k′, t i′(i≠k) is the kth and the ith row of T ⁻¹, respectively, while other rows of the matrix T ⁻¹ remain unchanged.

Moreover, if a row-addition operation is performed as (22), it is easy to verify that T will be updated as

$$ \textbf{T}\leftarrow\textbf{T}\left(\lambda_{k,i}\textbf{U}_{k,i}+\textbf{I}\right)^{-1}. $$

((24))

Note that

$$ \left(\lambda_{k,i}\textbf{U}_{k,i}+\textbf{I}\right)^{-1}=-\lambda_{k,i}\textbf{U}_{k,i}+\textbf{I}. $$

((25))

Then, the unimodular matrix T and the channel matrix $\bar {\textbf {H}}$ (i.e., HT) will be updated as

$$\begin{array}{@{}rcl@{}} \textbf{T}\leftarrow\textbf{T}\left(-\lambda_{k,i}\textbf{U}_{k,i}+\textbf{I}\right),\\ \bar{\textbf{H}}\leftarrow\bar{\textbf{H}}\left(-\lambda_{k,i}\textbf{U}_{k,i}+\textbf{I}\right), \end{array} $$

and similar with (23), they can be obtained by updating the ith column of T and the ith column of $\bar {\mathbf {H}}$ as

$$\begin{array}{@{}rcl@{}} \textbf{t}^{i}\leftarrow\textbf{t}^{i}-\lambda_{k,i}\textbf{t}^{k},\\ \bar{\textbf{h}}^{i}\leftarrow\bar{\textbf{h}}^{i}-\lambda_{k,i}\bar{\textbf{h}}^{k}, \end{array} $$

where t ^k and t ⁱ are the kth and the ith columns of T, and $\bar {\textbf {h}}^{k}$ and $\bar {\textbf {h}}^{i}$ are the kth and the ith columns of $\bar {\textbf {H}}$.

From the discussion above, after one row-addition operation on the matrix T ⁻¹, the matrix T is still unimodular [14]. In each iteration of our algorithm, we use one row-addition operation to update the matrix T ⁻¹ such that the PEPs of the detected symbols are minimized.

Considering (9), if we perform a row-addition operation on the matrix T ⁻¹ as (23), then (9) will be updated as

$$ \tilde{\textbf{y}}=\tilde{\textbf{x}}+\tilde{\textbf{n}}, $$

((26))

where $\tilde {\textbf {y}}=\left (\lambda _{\text {\textit {k,i}}}\textbf {U}_{\text {\textit {k,i}}}+\textbf {I}\right)\bar {\textbf {y}}$, $\tilde {\textbf {x}}=\left (\lambda _{\text {\textit {k,i}}}\textbf {U}_{\text {\textit {k,i}}}+\textbf {I}\right)\bar {\textbf {x}}$, and $\tilde {\textbf {n}}=\left (\lambda _{\text {\textit {k,i}}}\textbf {U}_{\text {\textit {k,i}}}+\textbf {I}\right)\bar {\textbf {n}}$. Then $\tilde {\textbf {y}}$, $\tilde {\textbf {x}}$ can be easily obtained by updating kth component of $\bar {\textbf {y}}$ and x in (9) as

$$\begin{array}{@{}rcl@{}} \bar{y}_{k}\leftarrow\lambda_{k,i}\bar{y}_{i}+\bar{y}_{k},\\ x_{k}\leftarrow\lambda_{k,i}x_{i}+x_{k}. \end{array} $$

The covariance matrix $\bar {\textbf {C}}$ of $\tilde {\textbf {n}}$ will be

$$\begin{array}{@{}rcl@{}} \bar{\textbf{C}}=\left(\lambda_{k,i}\textbf{U}_{k,i}+\textbf{I}\right)\textbf{C}\left(\lambda_{k,i}\textbf{U}_{k,i}+\textbf{I}\right)^{T},\\ =\left(\lambda_{k,i}\textbf{U}_{k,i}+\textbf{I}\right)\textbf{C}\left(\lambda_{k,i}\textbf{U}_{i,k}+\textbf{I}\right), \end{array} $$

((27))

where C is the covariance matrix of $\bar {\mathbf {n}}$, U _k,i , and U _i,k are defined above. The matrix $\bar {\textbf {C}}$ can be obtained by updating the matrix C as

$$\begin{array}{@{}rcl@{}} \textbf{c}_{k}\leftarrow \textbf{c}_{k}+\lambda_{k,i}\textbf{c}_{i}, \end{array} $$

((28))

$$\begin{array}{@{}rcl@{}} \textbf{c}^{k}\leftarrow \textbf{c}^{k}+\lambda_{k,i}\textbf{c}^{i}, \end{array} $$

((29))

and c _i , c ⁱ denotes for the ith row and the ith column of C, respectively. From above, the variance $\bar {C}_{j,j}$ of the noise $\tilde {n}_{j},j\neq k$ remain unchanged, and the variance of the noise $\tilde {n}_{k}$ will be updated as

$$\begin{array}{@{}rcl@{}} \sigma^{2}_{\tilde{n}_{k}}=\bar{C}_{k,k}=C_{k,k}+2\lambda_{k,i}C_{i,k}+\lambda_{k,i}^{2}C_{i,i}. \end{array} $$

((30))

Considering (26), we know that the pairwise LLR ${\boldsymbol \Lambda }_{l,z}\left (\textbf {C},\bar {\textbf {y}}\right),l\neq k$ remain unchanged, while the pairwise LLR ${\boldsymbol \Lambda }_{k,z}\left (\textbf {C},\bar {\textbf {y}}\right)$ is changed as ${\overline {\Lambda }}_{k,z}\left ({\bar {\mathbf {C}}},\tilde {\textbf {y}},\lambda _{\text {\textit {k,i}}}\right)$, and the updated pairwise LLR ${{\overline {\Lambda }}}_{k,z}\left ({\bar {\mathbf {C}}},\tilde {\textbf {y}},\lambda _{\text {\textit {k,i}}}\right)$ is defined as

$$ {{\boldsymbol\Lambda}}_{k,z}\left({\bar{\mathbf{C}}},\tilde{\textbf{y}},\lambda_{k,i}\right)=\ln\frac{P\left(\tilde{x}_{k}=\hat{\tilde{x}}_{k}|\tilde{y}_{k}\right)}{P\left(\tilde{x}_{k}=z|\tilde{y}_{k}\right)}, $$

((31))

where $\tilde {y}_{k}=\lambda _{\text {\textit {k,i}}}\bar {y}_{i}+\bar {y}_{k}$, $\hat {\tilde {x}}_{k}=\left \lfloor {\tilde {y}_{k}}\right \rceil$, and ⌊·⌉ is the rounding operation as defined above.

From Section 2, each symbol of $\tilde {x}_{k}$ is transmitted with equal probability; then from [9], we have

$$ {{\boldsymbol\Lambda}}_{k,z}\left({\bar{\mathbf{C}}},\tilde{\textbf{y}},\lambda_{k,i}\right)=\ln\frac{P\left(\tilde{y}_{k}|\tilde{x}_{k}=\hat{\tilde{x}}_{k}\right)}{P\left(\tilde{y}_{k}|\tilde{x}_{k}=z\right)}, $$

((32))

where z is a component in the set of $\tilde {x}_{k}$, and

$$ P\left(\tilde{y}_{k}|\tilde{x}_{k}=z\right)=\frac{1}{\sqrt{2\pi \bar{C}_{k,k}}}\exp\left(-\frac{\left(\tilde{y}_{k}-z\right)^{2}}{2\bar{C}_{k,k}}\right). $$

((33))

Substituting (33) into (32), we can get

$$ {{\boldsymbol\Lambda}}_{k,z}\left({\bar{\mathbf{C}}},\tilde{\textbf{y}},\lambda_{k,i}\right)=-\frac{\left(\tilde{y}_{k}-\hat{\tilde{x}}_{k}\right)^{2}}{2\bar{C}_{k,k}} +\frac{\left(\tilde{y}_{k}-z\right)^{2}}{2\bar{C}_{k,k}}. $$

((34))

After the row-addition operation, only the PEP of the kth component is changed. Similar with (14), the PEP can be formulated as

$$ P_{i}\left(\tilde{x}_{k}\neq\hat{\tilde{x}}_{k}|\tilde{y}_{k}\right)=1-\frac{1}{\sum_{z\in \mathbb{Z}}\exp\left(-{\overline{\Lambda}}_{k,z}\left({\bar{\mathbf{C}}},\tilde{\textbf{y}},\lambda_{k,i}\right)\right)}. $$

((35))

For (35), we only consider the dominant terms in the summation, which is similar with (18); then the PEP above can be written as

$$ P_{i}\left(\tilde{x}_{k}\neq\hat{\tilde{x}}_{k}|\tilde{y}_{k}\right)\approx1-\frac{1}{\sum_{z\in \mathbb{\tilde{D}}_{k}}\exp\left(-{{\overline{\Lambda}}}_{k,z}\left({\bar{\mathbf{C}}},\tilde{\textbf{y}},\lambda_{k,i}\right)\right)}, $$

((36))

where $\tilde {\mathbb {D}}_{k}$ is the set whose components are the B integers closest to $\hat {\tilde {x}}_{k}$.

Our target is to minimize the PEPs in each iteration, such that we can obtain the minimum PEPs of the detected symbols. To minimize the PEP in (36), the parameter λ _k,i in (36) should satisfy that

$$\begin{array}{@{}rcl@{}} \lambda^{*}_{k,i}&=&\arg\min_{\lambda_{k,i}\in \mathbb{Z}}P_{i}\left(\tilde{x}_{k}\neq\hat{\tilde{x}}_{k}|\tilde{y}_{k}\right)\\ &=&\arg\min_{\lambda_{k,i}\in \mathbb{Z}}\sum_{z\in \mathbb{\tilde{D}}_{k}}\exp\left(-{\overline{\Lambda}}_{k,z}\left({\bar{\mathbf{C}}},\tilde{\textbf{y}},\lambda_{k,i}\right)\right). \end{array} $$

((37))

However, the problem in (37) is complicated, and it is hard to obtain the exact solution. Now, we propose the fixed step length (FSL) method to solve it, which means the step length |λ _i,k | is constrained to be a constant c (i.e., c=1,2,⋯). Then (37) can be written as

$$ \lambda^{*}_{k,i}=\arg\min_{\left|\lambda_{k,i}\right|=c}\sum_{z\in \mathbb{\tilde{D}}_{k}}\exp\left(-{{\boldsymbol\Lambda}}_{k,z}\left({\bar{\mathbf{C}}},\tilde{\textbf{y}},\lambda_{k,i}\right)\right). $$

((38))

Substituting λ _k,i =±c into the function in (38), we can get two values, and by comparing the two values, we can obtain the solution in (38). Now, a problem is that how to choose the constant c. In fact, the constant c should not be too large. As shown above, a standard LR algorithm is used in the first stage of our algorithm. While these LR algorithms aim to find a unimodular matrix such that the channel matrix is more orthogonal. However, in the second stage, the row-addition operation will destroy the orthogonality formed in the initial stage, and a large c will seriously destroy the orthogonality of the matrix $\bar {\textbf {H}}$. Then, it will be hard to decrease the PEP if the constant c is large. The above process describes the detail of one iteration given the pair (k,i).

Now, we will describe our LLR-based algorithm. We perform the LLL or the ELR algorithm in the initial stage, and we can get (9).

Then, we use (14) to find the largest PEP $P\left (x_{k}\neq \hat {x}_{k}|\bar {y}_{k}\right)$ at first. Using the method above, and for each pair (k,i)(i≠k), we can get $P_{i}\left (\tilde {x}_{k}\neq \hat {\tilde {x}}_{k}|\tilde {y}_{k}\right)$. If the smallest one is smaller than the largest PEP, use it to update $P\left (x_{k}\neq \hat {x}_{k}|\bar {y}_{k}\right)$, and go on to decrease the largest PEP. Otherwise, begin to decrease the second largest PEP. If the second largest PEP can be decreased, update it, then return to decrease the largest PEP, and go on with the above process until the second largest PEP cannot be decreased. Then, we can further decrease the third largest PEP, the forth one, and so on. This process continues until no PEP can be decreased.

The whole process is described in Table 1.

Table 1 The description of the LLR-based TA in ZF criterion

The above process shows the LLR-based TA in ZF criterion. Now, we briefly discuss the LLR-based TA in MMSE criterion. As shown above, to obtain the LLR criterion in MMSE criterion, we need to replace the matrix H and r in the LLR criterion in ZF criterion by the extended matrix H _MMSE and r _MMSE. So replacing H and r in Table 1 by H _MMSE and r _MMSE, we can get the LLR-based TA in MMSE criterion.

4 The simulation and the analysis

In this section, we validate the performance of our LLR-based TA-aided VBLAST detectors through the computer simulations in different MIMO systems. As shown in Section 2, we simulate the figures in the uncoded systems. The channel is a Rayleigh fading channel. The entries of H are modeled as independent and identically distributed complex Gaussian variables with zero mean and $\frac {1}{N}$ variance. The elements of the transmitted signal are drawn from the MQAM constellation. The signal to noise ratio (SNR) is defined as the average received energy per information bit divided by σ ² [15, 17]. Our LLL-LLR and ELR-LLR TAs use the LLL and the ELR as the initial stage, and we set B=3. We compare the performance of the LLL-LLR and ELR-LLR-aided VBLAST detectors with the LLL [16] and the ELR-aided LDs [15] and VBLAST detectors.

4.1 The BER comparison of the VBLAST detector with SNR ordering

In this subsection, the VBLAST detector we used is shown in [8]. It is the classical VBLAST detector whose ordering process is based on the SNR. In Figs. 1, 2, 3, 4, 5 and 6, we use this kind of VBLAST detector, and it is represented as “VB1.”

Fig. 1

The comparison of the BER of different ZF detectors in 8×8 system with 16QAM

Fig. 2

The comparison of the BER of different MMSE detectors in 8×8 system with 16QAM

Fig. 3

The comparison of the BER of different ZF detectors in 8×8 system with 4QAM

Fig. 4

The comparison of the BER of different MMSE detectors in 8×8 system with 4QAM

Fig. 5

The comparison of the BER of different ZF detectors in 6×6 system with 4QAM

Fig. 6

The comparison of the BER of different MMSE detectors in 6×6 system with 4QAM

Figures 1 and 2 compare the BER performance of our ELR-LLR- and LLL-LLR-aided VBLAST detectors for different constant c with the previous LR-aided detectors in the 8×8 system with 16QAM modulation. The constant c is set to be 1 and 2. Figure 1 simulates the BER performance of these detectors in ZF criterion, while Fig. 2 simulates the BER performance in the MMSE criterion. The two figures show that all the VBLAST detectors have better BER performance than the previous LR-aided LDs, and the LLL-LLR- and the ELR-LLR-aided VBLAST detectors can provide significant BER performance gain over the LLL- and the ELR-aided VBLAST detectors, respectively, in both the ZF and the MMSE criterion. The two figures also show that our ELR-LLR- and LLL-LLR-aided VBLAST detectors for c=1 outperform the detectors for c=2. This is because with the increase of c, the orthogonality of $\bar {\textbf {H}}$ formed in the initial stage will be destroyed more seriously, and it will be harder to obtain small PEPs, which will result in worse BER performance of the VBLAST detectors. From these figures, we can find that the proposed LLL-LLR- (c=1) and the ELR-LLR-aided (c=1) VBLAST detectors achieve about 1 dB gain over the LLL- and the ELR-aided VBLAST detectors at BER = 10⁻⁵, respectively.

As shown above, the smaller the constant c is, the better the BER performance of our proposed detector will be. When the constant c is set to be 1, the BER performance of our detector will be the best. In the following, the constant c is set to be 1.

Figures 3 and 4 compare the BER performance of our ELR-LLR- and LLL-LLR-aided VBLAST detectors with the LR-aided LDs and VBLAST detectors in 8×8 system with 4QAM modulation. Figures 5 and 6 compare the BER performance of these detectors in 6×6 system with 4QAM modulation. Figures 3 and 5 compare the BER performance of different ZF detectors, while Figs. 4 and 6 compare the BER performance of different MMSE detectors. From these figures, we find that our LLR-based TA-aided VBLAST detectors still provide substantial BER performance gain than the previous LR-aided VBLAST detectors in different MIMO systems.

4.2 The BER comparison of the VBLAST detector with LLR ordering

In [9, 10], an efficient VBLAST detector whose ordering process is based on the LLR is proposed. It shows better BER performance than the VBLAST detector whose ordering process is based on the SNR, while it will bring complexity increase. It aims at minimizing the PEP in each stage of the VBLAST detector, where the PEP is calculated as Eq. (14) as shown in Section 3. In this subsection, the VBLAST detector in [9, 10] is utilized, and it is represented as “VB2” in Figs. 7 and 8.

Fig. 7

The comparison of the BER of different ZF detectors in 8×8 system with 16QAM

Fig. 8

The comparison of the BER of different MMSE detectors in 8×8 system with 16QAM

Figures 7 and 8 compare the BER performance of different detectors in 8×8 system with 16QAM modulation. Figure 7 compares the BER performance of different ZF detectors, while Fig. 8 compares the BER performance of different MMSE detectors. From these figures, we also find that our LLR-based TA-aided VBLAST detector can provide BER performance gain than the previous LR-aided VBLAST detectors in different MIMOsystems.

4.3 The simulation results of the complexity

As the LLL (ELR) is used as the initial stage of our algorithm, then the computational complexity of our LLL-LLR (ELR-LLR) will be larger than that of the LLL (ELR) algorithm. It is validated by Figs. 9 and 10. Figure 9 shows the empirical CDFs of the total number of arithmetic operations for different algorithms in ZF criterion with 8×8 16QAM system at 20 dB, while the comparison of the complexity of these algorithms in MMSE criterion with 8×8 16QAM system at 20 dB is shown in Fig. 10. The comparison in other MIMO systems is similar with the comparison in 8×8 16QAM system, and we omit the figure. As we know, the complexity of the ELR and LLL algorithms is polynomial in the MIMO system, and the polynomial complexity is acceptable in the MIMO system. From the simulation results, we find that the complexity of our ELR-LLR and LLL-LLR is no more than twice the complexity of the ELR and LLL algorithm, respectively. Then, the complexity of our algorithm is also polynomial, and the complexity of ELR-LLR and LLL-LLR has the same order as the complexity of ELR and LLL, respectively, which means the complexity of our algorithm is acceptable in the MIMO system. Then, we know that the complexity increase is moderate and acceptable in the MIMO system. Moreover, Figs. 9 and 10 also show that the computational complexity of our algorithm for c=1 is larger than that for c=2. As is shown above, it is easier to obtain small PEPs for small c, which means the number of updates will be larger, and the computational complexity will also be larger.

Fig. 9

CDFs of the number of floating point operations for different algorithms in ZF criterion (N _r =N _t =8, SNR = 20 dB)

Fig. 10

CDFs of the number of floating point operations for different algorithms in MMSE criterion (N _r =N _t =8, SNR = 20 dB)

5 Conclusions

In this paper, we made a readjustment of the received signal in the LR domain and proposed a new scheme to improve the LR algorithm. Unlike the LR algorithm which utilized the unimodular transformation to change the orthogonality of the channel matrix, the proposed scheme targeted to use the unimodular transformation to decrease the PEPs of the symbols, while the PEPs affected the error propagation in the VBLAST detectors. In our designed TA, the standard algorithm such as the LLL or the ELR algorithm was used as the initial stage. Then, a new algorithm was shown to decrease the PEPs of the detected symbols in the VBLAST detector. The simulations showed that our LLR-based TA-aided VBLAST detectors substantially improved the BER performance of the LLL- and the ELR-aided VBLAST detectors, while it only brought moderate increase in the computational complexity.