EURASIP Journal on Applied Signal Processing 2005:11, 1680–1697 c ○ 2005 N. Mysore and J. Bajcsy The Impact of Channel Estimation Errors and Co-antenna Interference on the Performance of a Coded MIMO System

This paper considers the problem of uplink transmission over multiple-input multiple-output (MIMO) channels affected by slow frequency-nonselective uncorrelated and correlated Rayleigh fading. We consider the case when channel state information, corrupted by estimation errors, is available at the receiver only. In this setting, we generalize the derivation of our previously proposed linear-complexity MIMO signal detector and derive closed-form expressions for the distribution of its soft outputs and the approximate symbol error probability. Based on this soft decision detector, we consider a turbo-coded MIMO uplink architecture with iterative processing, which enables performance within 1.6 to 2.8 dB of the ergodic capacity limit and outperforms the T-BLAST (turbo-Bell Laboratories layered space-time) system by about 10 dB at bit error rates of. The presented results illustrate that this linear-complexity MIMO signal detector is highly robust to channel estimation errors.


INTRODUCTION
The goal of next-generation wireless systems will be to provide high data rate access on both uplink and downlink transmission scenarios, while compensating for the harsh impairments introduced by the radio-frequency channel. Powerful error-correcting codes such as turbo codes [4] have already been included in the third-generation standard and will form a key component in beyond 3G systems. Through the use of spatial diversity, multiple-input multiple-output (MIMO) wireless systems have the potential of supporting very high data rates [5,6]. However, availability of channel state information only at the receiver and signal impairments (such as noise, co-antenna interference, and multipath fading) are the main obstacles in achieving reliable transmission over wireless MIMO channels. Furthermore, in most This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. practical scenarios, the occurrence of spatial correlation between antenna elements at the transmitter and receiver as well as channel estimation errors at the receiver reduces the MIMO channel capacity [7,8]. The particular case of imperfect channel state information (CSI) has also been explored and shown to reduce the performance of specific MIMO transceiver architectures in [9,10,11,12].
To achieve high data rates at low signal-to-noise ratios when CSI is available only at the receiver, a soft decision MIMO signal detector and a powerful error-correcting code are required [1,2,3,12,13,14,15,16,17,18,19,20]. MIMO signal detection should be of low complexity as signal constellation size and optimal detection complexity increase exponentially as M nT with the modulation order M and the number of transmit antennas n T . Furthermore, MIMO signal detection should be robust to channel estimation errors. These two requirements, robustness and complexity, are of significance for future wireless systems as these operate on harsh channels and are expected to use a large number of antennas at both ends of the wireless link [21].
The maximum likelihood (ML)-based method used in space-time trellis codes [13] and MAP-based MIMO signal detection used in [12,14,15] are limited to systems with a small number of antennas due to their exponential complexity. MIMO signal detection techniques of high-order polynomial complexity have been proposed in [16,17,18] based on the modified sphere detection algorithm. Alternately, lowercomplexity MIMO signal detection techniques include the coded layered space-time architecture (of quadratic complexity) [19] and T-BLAST system (of cubic complexity) [3], which combine the suboptimal nulling and canceling techniques of V-BLAST with iterative processing using convolutional codes. Recently, a linearly complex MIMO signal detector has been proposed and studied in [1,2] for a turbocoded MIMO system.
In this paper, we focus on the specific problem of achieving reliable high data rate transmission on uplink wireless channels where only the receiver possesses CSI, that is, corrupted by estimation errors. We consider an asymmetric antenna setup where the number of receive antennas at the base station exceeds the number of transmit antennas at the mobile. We generalize the linearly complex MIMO signal detector proposed in [1,2] to channels with estimation errors. We incorporate this detector into a turbo-coded MIMO system and we observe that it achieves near ergodic capacity performance and outperforms T-BLAST [3] by about 10 dB on selected slow frequency-nonselective Rayleigh fading channels. Section 2 of this paper discusses and reviews the considered MIMO channel models under different levels of channel correlation and imperfect CSI, the ergodic capacity limits, and the coded transmitter architecture. Section 3 focuses on the derivation of the proposed MIMO signal detector and on its incorporation into an iterative receiver. Section 4 contains simulation results for the investigated turbo-coded MIMO systems, while in Section 5, we analyze the proposed detector's outputs. We conclude in Section 6.

SYSTEM MODEL
In this section, we consider the general MIMO channel model that includes channel estimation errors and present the utilized spatial correlation models. Furthermore, we review the evaluation of the ergodic channel capacity and discuss the turbo-coded MIMO transmitter under consideration. Figure 1 illustrates the MIMO system model in which the transmitter sends complex symbols from n T antennas and the receiver utilizes n R antennas. The received vector is given by

The MIMO channel model
where ε is an n R by n T matrix due to the channel estimation errors. We assume that channel matrix H and the error matrix ε are uncorrelated and the elements of ε are i.i.d. complex Gaussian random variables with zero mean and variance σ 2 ε /2 in each dimension [8]. The variance σ 2 ε indicates the quality of channel estimation and is assumed to be known at the receiver.

The spatial correlation model
The amplitude of the complex path gain H i, j is assumed to be Rayleigh distributed and the phase is uniform. We can introduce spatial correlation via [22] where the elements of G are i.i.d. complex zero mean Gaussian random variables with variance 1/2 in each dimension.
The correlation matrices C TX and C RX are real, symmetric, and reflect the correlation between the elements of a uniformly spaced antenna array. If the antennas are spaced sufficiently apart and there are many scatterers near the transmitter or receiver, then C TX and C RX are given by the identity matrix I. We assume Jakes' correlation model for the mobile [23], that is, the (i, j)th element is given by C TX (i, j) = J 0 (2πd TX i, j /λ), where d TX i, j is the antenna spacing between the ith and jth transmit antenna, λ is the wavelength of the carrier frequency, and J 0 is the Bessel function of the zeroth kind. Figure 2 plots the power correlation (the square of C TX (i, j)) versus the antenna spacing. We use the Pedersen correlation model for the base station, and, if the signals are impinging from broadside, the (i, j)th element of C RX is adopted from [24] where J 2m is the Bessel function of the 2mth kind, d RX i, j is the antenna spacing between the ith and jth receive antenna, and β is the angular spread measured at the base station. The angular spread in a typical outdoor macrocellular urban environment for a carrier frequency of 2.1 GHz is between 7 and 12 degrees [25]. The power correlation (the square of C RX (i, j)) versus antenna spacing is plotted in Figure 2 for β = 7, 10, and 12 degrees.

Ergodic capacity evaluation with channel estimation errors
The performance of the considered coded MIMO systems will be compared to the ergodic capacity limit. Consequently, in this section, we briefly review the evaluation of this quantity and the effects of imperfect channel state information. We can express the rate of transmission of a coded MIMO system as R = n T R m R c (in bits per channel use), where R m is the modulation rate and the channel coding rate R c is given by k/n when a k-bit message is represented as an n-bit coded sequence. Furthermore, the transmitted bit energy-to-noise ratio, which allows comparison to the ergodic channel capacity, is given by E b /N 0 = P/(2Rσ 2 ), where P is the average transmitted power and σ 2 = N 0 /2 is the noise variance in each dimension. The ergodic capacity for a MIMO channel with estimation errors is derived in [8] and is given by where (•) H is the Hermitian operator and E is the expectation on the random matrix H. Since the channel is assumed to be ergodic, we approximate the statistical expectation in (5) by an average over many realizations of H. Figures 3 and 4 illustrate the impact of channel estimation errors on the channel capacity for (n T , n R ) = (2, 10) and (4,20) antenna configurations (averaged over 100 000 channel realizations) in slow frequency-nonselective uncorrelated and correlated Rayleigh fading channels, respectively. In general, spatial correlation slightly reduces the ergodic capacity curves from the uncorrelated scenario, whereas the capacity curve for imperfect CSI at the receiver (σ 2 ε = 0.1 or 10 percent) saturates in the high SNR region, while the capacity for perfect CSI at the receiver (σ 2 ε = 0) continues to increase. Conversely, at low SNRs, which is our region of interest, the introduction of channel estimation errors slightly shifts the capacity curve to the right with respect to the perfect CSI case.
In terms of SNR performance degradation for actual coded MIMO systems, the capacity results give the performance degradation (in terms of SNR and/or data rates) between the "best" possible coded system without CSI errors and the "best" possible coded system for given level of CSI errors. If, on the other hand, one considers a specific coded MIMO system, the performance degradation versus CSI error causes the following two limiting cases. For very high values of SNR (σ 2 σ 2 ε ), the channel estimation errors dominate the AWGN noise terms and the BER system performance saturates for increasing SNR values. (A potentially low error floor due to the estimation error will occur.) Conversely, for very low SNR values (σ 2 σ 2 ε ), the system performance is noise limited, so the CSI error term will result in a minor SNR degradation. Since the channel estimation error causes an additional signal-dependent noise term corrupting the signal (cf. (1) and (2)), a more precise analysis is not elementary and strongly depends on the numerical stability of the specific MIMO decoding algorithm(s). Figure 5 illustrates a schematic block diagram of the considered MIMO transmitter. The source is assumed to produce independent, equiprobable bits that are passed to the channel encoder, which is a turbo-code consisting of two constituent recursive systematic convolutional encoders as described in [4,26]. We can describe the encoding process as message symbols m ∈ {0, 1} k being mapped into codeword symbol of length n given by c = ( f 1 (m), f 2 (π(m))), where f 1 (·) and f 2 (·) denote the constituent encoders that are separated by an interleaver π. The coded symbols are then interleaved and mapped (using a lookup table) into an M-ary modulation 10 20 10 20 10 20 10 20   symbol. These symbols are demultiplexed onto n T streams, channel interleaved, and transmitted. The role of the channel interleaver and deinterleaver is to break up channel fades affecting consecutively transmitted symbols in coded packets.

THE RECEIVER ARCHITECTURE
In this section, we first derive a linearly complex MIMO signal detector in the presence of channel estimation errors. The detector tries to separate n T transmitted symbols from the received vector r and provides soft-decision outputs on the modulation symbols. These soft decisions are then used by a channel decoder, which in our case is a turbo decoder. In addition, a particular low-complexity iterative processing scheme improves the overall system performance. ( Figure 6 illustrates the overall schematic block diagram of the iterative space-time receiver under consideration.)

Derivation of the MIMO signal detector
We will assume the discrete-time MIMO channel model from (1). The proposed detector tries to recover channel observations on each transmitted modulation symbol, thus reducing the dimensionality of the problem from n R to n T , where for uplink transmission scenarios, n T < n R . The following paragraphs outline the general operation of the detector and are followed by an illustrative numerical example. As the first step, the estimated channel matrix H is decomposed into two matrices, H = S A, where the matrix A is a diagonal matrix containing the column norms of H along its main diagonal, and S is composed of the normalized columns of H, (Please note this decomposition is not related to any of the traditional matrix decompositions used in signal processing, e.g., Cholesky, QR, LU, SVD, etc.) Consequently, we obtain estimates of the transmitted symbol vector b by filtering the vector of received observations r by a bank of parallel filters represented by the matrix S, that is, where R = S H S is the antenna correlation matrix. The elements of the n T -by-one vector y are channel observations on the transmitted vector b which are corrupted by co-antenna interference due to R A, as well as the channel estimation error in S H εb and the filtered noise n. The jth element of y is given by where R j,k = R * k, j = s H j s k (conjugate is denoted by (•) * ), R j, j = 1, and the filtered noise elements n j have zero means and covariance matrix 2σ 2 R.
In order to compute the likelihood (soft decision) for the transmitted symbol b j being the lth modulation symbol, that is, P( y j |b j = Q l ), we approximate in (9) the sum of the coantenna interference nT k=1 k = j R j,k A k,k b k , the channel estimation error ( S H εb) j , and the filtered noise n j as a two-dimensional Gaussian random variable. As shown in Appendix A, the mean µ j of this Gaussian random variable is given by and covariance matrix K j of this complex Gaussian random variable is where {•} and {•} denote the real and imaginary parts and E is the expectation on the discrete random variable Q, which can be one of M possible modulation symbols.
Finally, we can express the likelihood for the transmitted symbol from the jth antenna b j being the lth modulation symbol Q l as follows: where Assume that the receiver estimates the channel with an estimation error of σ 2 ε = 0.1 and that the channel matrix is estimated as and the Gaussian noise variance in each dimension is σ 2 = 1. The received vector r was observed to be Step 1 (matrix decomposition). The detector decomposes the channel matrix into the two submatrices according to (6) and (7): Step 2 (acquiring channel observations). We filter the received vector r by the matrix S to lower the dimensionality of the problem (from n R = 4 to n T = 2) and acquire channel observations on the transmitted symbols contained in y. The channel observation vector y (using (8)) is given by Note that this reduction in dimensionality is interesting for the case of large number of receive antennas (at the base station) when compared to the number of transmit antennas (at the mobile), for example, practical prototype systems with n R = 60 have already been explored, as mentioned in [21].
Step 3 (computing statistics for the Gaussian approximation). The co-antenna correlation matrix R = S H S is given by and hence the statistics for the Gaussian approximation for the co-antenna interference, channel estimation error, and the filtered noise from (10) and (11) for the first transmit antenna are µ 1 = 0, where the mean vector µ 1 and K 1 (1, 2), K 1 (2, 1) are zero as the constellation is centered around zero and all points are equiprobable.
Step 4 (determination of the soft decisions). Using (12), the likelihoods for the symbol from the first transmit antenna are given by P( y 1 |b 1 = −1) = 0.022 and P( y 1 |b 1 = 1) = 0.033 repeating the above calculation for the second transmit antenna, P( y 2 |b 2 = −1) = 0.077 and P( y 2 |b 2 = 1) = 0.006. (Please note that these likelihoods are not normalized as a posteriori probabilities would be.)

Turbo decoding and iterative processing
The demapper in Figure 6 translates the soft decisions of the detector into a posteriori probabilities on the codeword symbols. For instance, the estimate for the pth codeword bit in the jth modulation stream is given by The estimates Λ p are assumed to be the channel observations on the bits of codeword symbols.
In the ith iteration, the iterative loop between the decoder and demapper can be formally described as where the soft decision demapper and decoding functions are represented by φ and ϕ 1 , ϕ 2 , respectively. The vectors u and w i 1 , w i 2 denote the soft outputs of the linear detector and the demapper, respectively in Figure 6. The decoding functions ϕ 1 and ϕ 2 evaluate logarithmic arrays of a posteriori probabilities for message symbols and codeword symbols of the constituent encoders of the turbo-code. The extrinsic information vectors on the codeword symbols (z i 1 and z i 2 ) and on the message bits (x i 1 and x i 2 ) are initially (for i = 0) set to a constant and are treated as independent coordinatewise observations of these symbols or bits, so that, for instance, P(m j = k|x) is proportional to exp(x k, j ). The decisions on the message symbols are formed by thresholding outputs of the first decoder ϕ 1 (w i 1 , x i 2 ) after each iteration. Turbo equalization was first proposed in [27] and involves the recomputation of the probabilities on the transmitted modulation symbols from the a posteriori probabilities on the codeword symbols. These computed probabilities on the modulation symbols become the extrinsic information for the detector and hence will result in better estimates on the codeword symbols. Although this method may be feasible in single-antenna systems, the transmitted signal vector constellation quickly becomes too large for a coded MIMO system as it increases as M nT for M-ary modulation with n T antennas. In the approach under consideration, we reassemble the codeword symbol probabilities into probabilities on the modulation symbols P(b j = Q l ) and recompute the mean vector and covariance matrix of the Gaussian approximation using (10) and (11). As the system iterates, the mean of the co-antenna interference will approach the true interference value and its variance becomes zero, thus the effect of the interference is removed from the received symbol.

SIMULATION RESULTS
In this section, we present simulation results of the proposed linear-complexity detector in two uplink coded MIMO systems. These simulation results are evaluated for slow, frequency-nonselective Rayleigh fading channels when perfect and imperfect channel state information is available at the receiver only. As discussed in the introduction, we consider the uplink transmission scenario where it is possible to have a large number of receive antennas, for example, 10 and 20. The message block size was chosen to be 32000 bits and the channel code is a rate 1/2 turbo-code, which is composed of two eight state quaternary recursive systematic encoders (rate 2/3) adopted from [26]. The turbo encoder contains a symmetric S-rand interleaver [28], where S = 80. The coded bits are mapped into 16-QAM modulation symbols, whose lookup table has been adopted from [29]. We use the spatial correlation model, described in Section 2.2, where the antenna spacing at the mobile and base station is assumed to be λ/2 and λ, respectively, and the angular spread β is assumed to be 10 degrees. The ergodic capacity limits are evaluated numerically by averaging over 1 million channel realizations and the variance of the estimation errors σ 2 ε is set to be 10 percent. The turbo decoder in the receiver performs 10 decoding iterations and uses the BCJR algorithm [30] in the constituent decoders. Figures 7 and 8 compare the performance of the coded MIMO system utilizing the proposed detector (of linear complexity) to the T-BLAST system [3] (of cubic MIMO signal detection complexity) for the (2,10) and (4,20) antenna configurations in an uncorrelated Rayleigh fading channel with perfect CSI at the receiver. Both systems operate at equivalent rates (the T-BLAST using a 16-PSK modulation), use the same antenna setups and perform 10 decoding iterations. Our proposed system outperforms the T-BLAST system by about 10 dB in both cases at a bit error rate of 10 −5 and achieves performance within 1.5 dB of the ergodic capacity limit. Figure 9 illustrates the performance of the considered coded MIMO system with iterative processing for (n T , n R ) = (2, 10) antennas (R= 4 bits per channel use) and (4,20) antennas (R= 8 bits per channel use) in a slow frequencynonselective uncorrelated Rayleigh fading channel. The coded MIMO system with imperfect CSI for both antenna configurations performs within 1.6 dB of the ergodic capacity limit and 1 dB worse than the corresponding system with perfect CSI. Figure 10 illustrates the performance of the coded MIMO system with iterative processing for (n T , n R )= (2, 10) and (4,20) antennas in a slow frequency-nonselective correlated Rayleigh fading channel. Since we consider a strongly correlated environment, channel estimation errors    have a larger impact than in the uncorrelated case. That is, the performance of the coded MIMO system with (n T , n R )= (2, 10) and (4,20) antennas is within 2.1 and 2.8 dB of the ergodic capacity limit, respectively. These systems perform 1.  configurations, respectively, when compared to the corresponding systems which have perfect CSI at the receiver. An estimation error variance of 10% represents a large value and the works in [9,10,11] indicate that the error variance is usually much smaller, that is, lower than 1 percent. In these scenarios, the performance loss due to channel estimation errors in our considered coded MIMO system is expected to be smaller, if not negligible. Figure 11 illustrates reduction of the co-antenna interference during the iterative decoding process for different CSI/ correlation scenarios and values of SNR. In all four cases, at low SNRs, the variance does not converge to zero, whereas for slightly higher SNRs, the variance reaches zero in less than ten iterations. In correlated channels, the initial variance of the co-antenna interference is about three times larger than in the uncorrelated case. Furthermore, for the variance of the co-antenna interference to converge, channel estimation errors require an additional SNR increase of 1 dB and 1.6 dB in the uncorrelated (Figures 11a, 11b) and correlated fading (see Figures 11c, 11d) scenarios, respectively.

ANALYSIS OF THE PROPOSED DETECTOR
Bit error rate (versus SNR) performance analysis of the proposed detector in a coded MIMO system with an iterative receiver is difficult due to the nature of the MIMO fading channels and the process of iterative detection/decoding. Current union-bound-based techniques provide approximations that  are loose by about 2 dB at low SNRs [31]. The other classes of techniques are either limited to low-density parity-check codes [32] or provide only the SNR value at which the turbo decoder starts to converge [33]. Consequently, as an initial step towards the desired BER performance analysis, this section derives closed-form expressions for the proposed detector's soft-output statistics and symbol error rates after the detector.

Analysis of co-antenna interference in Rayleigh fading channels
To better characterize the co-antenna interference, we will determine the mean and variance of the column norms and correlations between the normalized columns of the n R by n T random channel matrix H. In the following, we assume σ 2 ε = 0, that is, perfect CSI is available at the receiver and drop the use of the "hat" symbol. To permit mathematical tractability, we assume the base station antennas are correlated and mobile antennas are uncorrelated. Aj,j = 2Γ(n R /2 + 1)/Γ(n R /2) − µ 2 Aj,j ≈ 1/2 for reasonable values of n R , (n R < 100), an angular spread β of ten degrees, and antenna spacing at the receiver given by λ.

Justification of Claim 1 Using (3), the norm of the jth column norm of H A j, j is given by
where in the fourth step we performed the eigenvalue decomposition on the real and symmetric matrix C RX resulting in the real orthogonal and diagonal matrices V and Λ, respectively. The lth element of e j is a realization of a complex Gaussian random variable with zero mean and variance Λ l,l /2 in the real and imaginary dimensions, l = 1, 2, . . . , n R . Further expanding the last line of (22), where W l and X l are independent and Gaussian distributed with zero mean and variance Λ l,l /2, l = 1, 2, . . . , n R . Consequently, the random variable A 2 j, j is chi-squared distributed with 2n R degrees of freedom (as it is the sum of 2n R chisquared random variables [34]), and its mean and variance are given by Since by (4), the elements of C RX depend on the random parameter β (angular spread measured at the receiver), the quantities in (24) are random variables. For the specific case of β = 10 degrees and an antenna spacing at the receiver is λ, the mean and the variance in (24) become approximately n R and 2n R , respectively. Hence, A j, j is a chi random variable with n R degrees of freedom that can be expressed as follows: where Θ l are i.i.d. Gaussian random variables with zero mean and unit variance for l = 1, 2, . . . , n R . The mean of A j, j is given by µ Aj,j = √ 2Γ(n R /2 + 0.5)/Γ(n R /2) and its variance is σ 2 Aj,j = 2Γ(n R /2 + 1)/Γ(n R /2) − µ 2 Aj,j [35]. The mean can be approximated as √ n R , which grows slowly with increasing number of receive antennas, and the variance of A j, j is close to 1/2 for reasonable n R (e.g., up to 100). Claim 2. The (j, k)th element of the antenna correlation matrix R j,k is a zero-mean complex Gaussian random variable with variance 1/n R in each dimension for an angular spread β of ten degrees and an antenna spacing at the receiver of λ.

Justification of Claim 2
Assuming the same parameters as before for the spatial correlation, that is, β = 10 0 , and the antenna spacing at the receiver given by λ, we can express the correlation between the jth and kth normalized columns of H as where the column norms are approximated by √ n R , as the variance is reasonably small (approximately 1/2). Using the eigenvalue decomposition on C RX , where W l , X l , Y l , and Z l are independent Gaussian product variables with zero mean and variance Λ 2 l,l /4 for l = 1, 2, . . . , n R . Since nR l=1 Λ 2 l,l /4 ≈ n R /2, we can apply the central limit theorem on each summation, and hence express R j,k as a complex Gaussian random variable with zero mean and variance 1/n R in each dimension. Finally, the uncorrelated fading scenario is a special case of the correlated case, where C RX = I. Hence, W l and X l in (23) are i.i.d. zero-mean Gaussian random variables with variance 1/2 for l = 1, 2, . . . , n R . Therefore, the norm of the jth column of H, A j, j , is a chi random variable with 2n R degrees of freedom and its mean is µ Aj,j = Γ(n R +0.5)/Γ(n R ) and variance is σ 2 Aj,j = Γ(n R + 1)/Γ(n R ) − µ 2 Aj,j . The mean can still be approximated as √ n R and, for reasonably large n R (e.g., up to 100), the variance of A j, j is close to 1/4. Similarly, W l , X l , Y l , and Z l in (27) are i.i.d Gaussian product random variables with zero mean and variance 1/2 for l = 1, 2, . . . , n R . Using the central limit theorem, we can approximate the summations in the real and imaginary dimensions of R j,k in (27) as a Gaussian random variable with zero mean and variance 2n R /(4n 2 R ) = 1/(2n R ).

Performance analysis of the MIMO detector
We consider the average and approximate probability of symbol error of the proposed detector for the cases of perfect and imperfect CSI at the receiver in slow frequency-nonselective uncorrelated and correlated Rayleigh fading channels. The average probability of symbol error considers the full correlation model (C TX and C RX ), while (to permit mathematical tractability) the approximate symbol error only considers spatial correlation at the base station only. The transmitter is assumed to use a square QAM constellation with M points and a minimum distance of 2B between signal points. Furthermore, the transmitted symbols from each antenna are assumed to be equiprobable and independent of transmitted symbols from other antennas. Claim 3. Let the channel observation vector in the presence of channel estimation errors be given by y = RAb + n, where R = S H S. Then the probability of symbol error conditioned on the estimated channel H for the first transmit antenna is given by where D = − nT k=2 R 1,k A k,k Q lk is the co-antenna interference for j = 1 transmit antenna and for the case of perfect CSI at the receiver, R = S H S, and R 1,1 = 1.
(Note that for a time-varying channel, the average probability of symbol error can be determined by averaging the above expression over many channel realizations as the channel is assumed to be stationary and ergodic.)

Justification of Claim 3
We can reexpress the channel observation vector y from (8) as where H = SA, R = S H S, n = S H v, and y 1 is given by where R 1,1 is a loss factor corresponding to the imperfect CSI being available at the receiver. The probability of symbol error conditioned on the estimated channel matrix is given by (31) where the minimum distance decision region Z l corresponds to the lth QAM symbol Q l . If we further condition P[ y 1 ∈ Z l |b 1 = Q l ] on the remaining transmit antennas, that is, let b k = Q lk , k = 2, 3 . . . , n T , then the co-antenna interference term in (30) becomes a deterministic quantity. Therefore, the QAM symbol transmitted from the first antenna b 1 is only corrupted by additive Gaussian noise and hence assuming the real and complex parts of y 1 are independent the probability of error in (31) is given by the weighted sum of the probability of error when b 1 = Q l is a corner, middle, or side point of the constellation, where D = − nT k=2 R 1,k A k,k Q lk is the co-antenna interference for the first ( j = 1) transmit antenna and {n 1 }, {n 1 } are the real and imaginary components of the noise n 1 .
We can evaluate (31) by using (32) and the Q-function If we have perfect CSI at the receiver, then R = S H S and R 1,1 = 1. For a time-varying H, we can average the symbol error probability in (33) over many channel realizations as the channel is assumed to be stationary and ergodic. In the following, we derive the approximate performance of the detector as a closed-form expression.
Claim 4. Let the channel observation vector in the presence of channel estimation errors be given by y = RAb + n, where R = S H S. Then the approximate probability of symbol error for j = 1 transmit antenna is given by where R 1,1 is the average loss factor due to imperfect CSI at the receiver and the variance of the co-antenna interference in each dimension is given by σ 2 CAI = nT k=2 Q lk 2 /2 and σ 2 CAI = nT k=2 Q lk 2 for the uncorrelated and correlated fading scenario, respectively. For the case of perfect CSI at the receiver, R = S H S and R 1,1 = 1.

Justification of Claim 4
As mentioned earlier, to permit mathematical tractability, we assume spatial correlation at the base station only. Without loss of generality, we consider the j = 1 transmit antenna, where the channel observation is given in (30) and the corresponding probability of symbol error is given by If we condition P[ y 1 ∈ Z l |b 1 = Q l ] on the remaining transmit antennas, that is, let b k = Q lk , k = 2, 3, . . . , n T , then the co-antenna interference (CAI) term in (30) will be dependent on two random variables A k,k and R 1,k . Using the results of the previous section, we can approximate A k,k , k = 1, 2, . . . , n T , by its mean √ n R as its variance is small (1/4 or 1/2) and since the elements of ε are much smaller than the elements of the channel matrix, the correlation coefficient R 1,k (k = 1) will be assumed to remain as a zeromean complex Gaussian random variable with a variance of 1/(2n R ) and 1/n R in each dimension for the uncorrelated and correlated fading scenarios, respectively. Hence, the CAI term is a complex Gaussian random variable with zero mean and variance σ 2 CAI = nT k=2 Q lk 2 /2 and σ 2 CAI = nT k=2 Q lk 2 in each dimension and the probability of error in (35) is given by the classical expression for a QAM symbol corrupted by additive Gaussian noise. As mentioned previously, the probability of symbol error is the weighted sum of the three expressions for the probability of error for b 1 = Q l is a corner, middle, or side point of the constellation where we assumed the real and complex parts of y 1 are independent and n 1 is the sum of the Gaussian noise n 1 and the CAI term ( and refer to the real and imaginary components). Since R 1,1 is random variable with an unknown density function, we can approximate it by its value averaged over many channel realizations R 1,1 , where for instance, R 1,1 is equal to 0.95 for an estimation error variance of 10% (averaged over 100 000 channel realizations). Therefore, the approximate probability of error for the first transmit antenna using (36) and the Q-function [36] is given by where for perfect CSI at the receiver R = S H S and hence R 1,1 = 1 in (37).

Numerical results
We provide numerical results to support the probabilistic analysis of and correlated Rayleigh fading channels. As mentioned in Section 5.1, we will only consider spatial correlation at the base station, that is, using Pedersen's model (4) with an antenna spacing of λ and an angular spread of β = 10 • . In Figures 12a and 12b, we plot the derived chi density function of A 1,1 and compare it against the simulated density function. In both the uncorrelated and correlated cases, the mean is approximately 4.5 (≈ √ n R ) and the variance is 1/4 and 1/2, respectively, which agrees with their predicted values from Section 5.1. In Figures 13a and 13b, we plot the derived and simulated density functions of the real component of R 1,2 for 1 million channel realizations. In Section 5.1, we showed that by the central limit theorem, the derived density function is approximately Gaussian and this is confirmed by the simulated density function. In both uncorrelated and correlated cases, the simulated density function has zero mean and variance, 0.025 and 0.05 respectively, which agrees with the derived mean and variance. Since

Performance results of the detector
We consider the performance of the linear MIMO signal detector in slow frequency-nonselective uncorrelated and correlated Rayleigh fading channels. The spatial correlation model is as described in Section 2.2 with an antenna spacing of λ/2 and λ at mobile and base station, respectively and the angular spread β is assumed to be 10 degrees. Figures 14 and  15 illustrate the average symbol error rate of the MIMO signal detector averaged over 100 000 channel realizations (using (33)) and the corresponding approximate symbol error rate (using (37)) for (n T , n R )= (2, 10) and (4,20) antennas, respectively using a 16-QAM symbol constellation with perfect and imperfect channel state information (σ 2 ε = 10%) being available at the receiver. In the uncorrelated cases (Figures 14a and 15a), there is a close match between the approximate and average performance. However, in the correlated cases (Figures 14b and 15b), there exists a small difference between the approximate and average symbol error rate as the approximate performance does not include the spatial correlation between transmit antennas. For comparison, the performance of the optimal detector is also shown for the same system parameters.
Although the proposed detector is co-antenna interference limited at high SNRs, it only performs slightly worse than the exponentially complex optimal detector at very low SNRs, for example, −5 to −10 dB, which is the region corresponding to the performance of the coded MIMO systems as shown in Figures 9 and 10. Furthermore, Figures 14 and  15 illustrate that the performance of the detector is robust to channel estimation errors.

CONCLUSION
Low-complexity soft-decision MIMO signal detectors are required to exploit the capacity of MIMO channels in beyond 3G wireless systems. These detectors must be robust, as in    Figure 14: Performance analysis of the proposed and optimal detector for (n T , n R )= (2, 10) antenna system in slow frequency-nonselective (a) uncorrelated and (b) correlated Rayleigh fading channels.  most practical scenarios, channel estimation errors occur at the receiver, especially in the uplink transmission scenarios due to power limitations at the mobile. In this paper, we generalized the linear detector proposed in prior work [1,2] to accommodate channel estimation errors and considered its robustness in coded MIMO systems on slow frequencynonselective correlated Rayleigh fading channels. We consider an uplink transmission scenario, where it is feasible to have a larger number of receive antennas at the base station than the number of transmit antennas at the mobile. In an uncorrelated Rayleigh fading channel with perfect CSI at the receiver, the proposed detector in the turbocoded MIMO system outperforms the T-BLAST system [3] by about 10 dB while performing within 1.5 dB of the ergodic capacity limit (at a bit error rate of 10 −5 ) for the (n T , n R )= (2, 10) and (4,20) antenna configurations. Furthermore, for a channel estimation error variance of 10%, we demonstrated that the turbo-coded MIMO system with (n T , n R ) = (2, 10) (4 bits per channel use) and (4,20) antennas (8 bits per channel use) operates within 1.6 dB of the ergodic capacity limit at a bit error rate of 10 −5 for uncorrelated channels, whereas in spatially correlated channels and the same estimation error variance, the performance of the (n T , n R )= (2, 10) and (4,20) antenna systems is within 2.1 dB and 2.8 dB of the ergodic capacity limit at a bit error rate of 10 −5 , respectively.
We characterized the co-antenna interference at the output of the proposed detector through density functions of the column norm and the correlation between the columns of the random channel matrix H. By doing so, we determined the approximate closed-form expressions for the performance of the detector and compared this to an average based on many channel realizations. Although the linearly complex detector was found to be co-antenna interference limited at high SNRs, at very low SNRs, that is, −5 to −10 dB, which is the region corresponding to the coded MIMO system performance, it performs only marginally worse than the exponentially complex optimal detector. Furthermore, the linear detector is robust to channel estimation errors in correlated and uncorrelated Rayleigh fading channels.
Future work includes a BER performance analysis of the coded MIMO system that features the proposed detector and turbo decoder, extending the system to frequency-selective channels and a multiuser uplink scenario.

A. APPROXIMATING THE OUTPUT STATISTICS OF THE PROPOSED MIMO SIGNAL DETECTOR
We would like to analyze the output statistics of the MIMO signal detector from Section 3.1. The jth element of the channel observation vector y (from (9)) is given by y j = A j, j b j + nT k=1 k = j R j,k A k,k b k + S H εb j + n j , (A.1) where the transmitted symbol b j is corrupted by the coantenna interference, channel estimation error, and filtered noise. We can approximate these three complex impairments as a two-dimensional Gaussian random variable n j . The mean of n j is given by The expectation can be applied on each term in the braces {·}. The mean of n j is zero as the expectation on the filtered noise vector is zero, that is, where the kth symbol b k , k = 1, 2, . . . , n T , k = j, can be described as a discrete random variable Q taking one of M possible values (as we are using M-ary signaling). The three impairments are independent of each other, and hence the variance of the two-dimensional Gaussian random variable n j in each dimension is the sum of the individual variances. The variance of n j is the ( j, j)th entry of the covariance matrix of n = S H v which is E[nn H ] = S H E[vv H ] S = 2σ 2 R. Since the main diagonal of R is composed of ones, the variance of n j remains the same as the variance of the elements of the white Gaussian noise vector v, that is, σ 2 in each dimension. Similarly, the variance of ( S H εb) j is the ( j, j)th entry of the covariance matrix given by Since the main diagonal of R is composed of ones, the variance of the jth element of S H εb remains the same as the variance of the elements of ε, that is, σ 2 ε /2 in each dimension. In the second step, we used E[bb H ] = 1/n T I and the elements of the random product matrix εε H have a mean of n T σ 2 ε along the main diagonal and zero for the off-diagonal terms [35]. Therefore, we can express the elements of the covariance matrix of n j as where the transmitted symbols b k , k = 1, 2, . . . , n T , k = j, are assumed to be independent of each other at any time instant. The off-diagonal entries K j (1, 2) and K j (2, 1) do not include terms from the filtered noise and channel estimation error as the real and imaginary dimensions of n j and ( S H εb) j are independent of each other and E[n j ] = 0 and E[( S H εb) j ] = 0.

B. COMPARISON BETWEEN THE PROPOSED MIMO AND DS-CDMA SIGNAL DETECTION
The discrete representation of a MIMO system in (1) and further exemplified in (8) has parallels to a synchronous DS-CDMA system. The formulation of the detector in Section 3.1 reveals that the jth transmit antenna corresponds to the jth user and the receive antennas correspond to the chips of a user's spreading sequences in a synchronous DS-CDMA system. As seen in (9), the other (n T − 1) antennas result in co-antenna interference which is analogous to multiple-access interference. The proposed detector is best suited for an uplink scenario (mobile to base station), where it is feasible to have more receive antennas than transmit antennas, that is, it corresponds to a typical underloaded CDMA system [36]. Finally, the filtering by the matrix S in (8) and the Gaussian approximation for the filtered noise and the co-antenna interference is analogous to that performed by the conventional CDMA detector based on a bank of match filters. Due to the fundamental change in the problem definition, there exist many differences between the two scenarios. One of the apparent differences is the use of multilevel modulation in MIMO transmission, which has not been studied extensively, for example, see [37,38,39,40] for synchronous DS-CDMA systems. Another key difference is that elements of the correlation matrix R are functions of random spreading sequences (i.e., the normalized columns of H) and, as such, cannot be designed. Finally, the issue of channel estimation errors has no parallel in synchronous DS-CDMA systems as the spreading sequences are perfectly known by the receiver.