Joint channel and phase noise estimation for mmWave full-duplex communication systems

Abstract

Full-duplex (FD) communication at millimeter-wave (mmWave) frequencies suffers from a strong self-interference (SI) signal, which can only be partially canceled using conventional RF cancelation techniques. This is because current digital SI cancellation techniques, designed for microwave frequencies, ignore the rapid phase noise (PN) variation at mmWave frequencies, which can lead to large estimation errors. In this work, we consider a multiple-input multiple-output mmWave FD communication system. We propose an extended Kalman filter-based estimation algorithm to track the rapid variation of PN at mmWave frequencies. We derive a lower bound for the estimation error of PN at mmWave and numerically show that the mean square error performance of the proposed estimator approaches the lower bound. We also simulate the bit error rate performance of the proposed system and show the effectiveness of a digital canceler, which uses the proposed estimator to estimate the SI channel. The results show that for a 2×2 FD system with 64−QAM modulation and PN variance of 10⁻⁴, the residual SI power can be reduced to − 25 dB and − 40 dB, respectively, for signal-to-interference ratio of 0 and 15 dB.

1 Introduction

The next generation of wireless communication technologies, known as 5G, are expected to offer multi-gigabit data rates to mobile users [1, 2]. This has prompted wireless service providers to seek higher bandwidth at less crowded millimeter-wave (mmWave) frequencies. The short wave lengths of mmWave frequencies also allow for practical implementations of base stations with large number of antennas known as massive multiple-input multiple-output (MIMO) system, which is another promising technology for 5G networks [3]. Given these capabilities offered by mmWave communication, these systems have become increasingly popular in academia and industry. While MIMO systems can fully benefit from the capabilities offered by communication at mmWave frequencies [3], due to large peak-to-average power ratio, orthogonal frequency division modulation (OFDM) is not popular for mmWave communication [4]. Since there is still an open debate about the modulation type at mmWave frequencies [5], we do not consider OFDM in this work.

Full-duplex (FD) communication has also emerged recently as a promising wireless technology, which allows for efficient use of bandwidth by enabling in-band transmission and reception [6–9]. The major obstacle in exploiting the full potential of FD communication is the self-interference (SI) signal, which is significantly stronger than the desired communication signal [10, 11]. The power of the SI signal can be reduced via two different suppression techniques: (i) passive suppression, where transmit and receive antennas are physically isolated to reduce the leakage of the transmit signal into the RF front end of the receiver chain, and (ii) active suppression, where the SI signal is suppressed via subtracting the analog replica of SI signal from the received signal [6]. The experimental results at microwave frequencies show that the successive combination of passive and active suppression can reduce the SI signal power to the receiver noise floor [12]. For this reason, in the majority of the radio architectures proposed for FD communication at microwave frequencies, the residual SI signal at baseband is treated as noise [6, 13–15]. The cancelation techniques that treat the SI signal as noise suffer from two fundamental problems: (i) they assume Gaussian distribution of the SI signal. However, as explained in [16, 17], the SI signal has a strong line of sight (LoS) component, and hence, it is not Gaussian, and (ii) treating SI as noise requires statistical knowledge of the SI channel, which might not be available.

Recently, SI channel measurements have been carried out for FD communication at mmWave frequencies [17, 18]. The measurements indicate that, as opposed to the microwave frequency band, the SI channel at mmWave has a non-line-of-sight (NLoS) component, which cannot be canceled using passive and active suppression techniques. This partial suppression of the SI signal results in a large residual SI signal at baseband, which is still significantly higher than the receiver noise floor [17]. Another challenge for mmWave FD communication systems is that the oscillator phase noise (PN) is large and rapidly changing [19]. Thus, the majority of the existing techniques for residual SI signal cancelation at baseband, which assume a very steady oscillator PN [20–23] cannot be used for FD mmWave communication. We note that the important aspect of mmWave communication considered in this paper is the estimation of fast varying PN. This fast variation of PN is in the order of symbol time at mmWave [24].

In this work, we consider the problem of joint channel and PN estimation for a mmWave FD MIMO system communication system. The main contributions of this work are as follows:

• 1. We construct a state vector for the joint estimation of the channel and PN and propose an algorithm based on extended Kalman filtering technique to track the fast PN variation at mmWave band.

• 2. We derive the lower bound on the estimation error of the proposed estimator and numerically show that the proposed estimator reaches the performance of the lower bound. We also show the effectiveness of a digital SI cancelation, which uses the proposed estimation technique to estimate the SI channel.

• 3. We present simulation results to show the mean square error (MSE) and bit error rate (BER) performance of a mmWave FD MIMO system with different PN variances and signal-to-interference ratios (SIR). The results show that for a 2×2 FD system with 64−QAM modulation and PN variance of 10⁻⁴, the residual SI power can be reduced to − 25 dB and − 40 dB, respectively, for signal-to-interference ratio of 0 and 15 dB.

Notation: The following notation is used in this paper. Superscripts (·)^† and (·)^T are the conjugate and the transpose operators, respectively. Bold face small letters, e.g., x are used for vectors, bold face capital letters, e.g., X are used for matrices. e^jθ is the multivariate complex exponential function. |·| is the absolute value operator, and ∠x is the phase of the complex variable x. ⊙ is the Hadamard (element-wise) product. diag(x) creates a matrix with elements of vector x on the main diagonal. trace(·) is the trace of a matrix, which sums up all the diagonal elements of a given matrix. $x \in \mathcal {A}$, means x is an element of set $\mathcal {A}$. 0_N is N by 1 vector of all zeros, and I_N is N by N identity matrix. 1_N×N is N×N matrix of all 1. $\mathbb {E}[\cdot ]$ is the expectation operator. $\mathfrak {R}\{\cdot \}$ returns the real part of a complex quantity. Finally, in Table 1, we present the important symbols used in the mathematical representation of the system model. In general, if x is used in the mathematical representation of the system model, then $\bar {x}$ is used for the mathematical representation of the system model needed for joint channel and PN estimation, and $\hat {x}$ is an estimate of x.

Table 1 Important symbols used in this paper

2 System model

We consider the MIMO communication system between two mmWave FD nodes a and b, each with N_t transmit and N_r receive antennas as illustrated in Fig. 1. The considered communication system can be a model for backhaul communication for cellular systems [24]. In this work, we make the following assumptions:

• 1. The same number of transmit and receive antennas for both nodes: We assume both nodes in the considered FD communication system have the same number of transmit and receive antennas.

  Fig. 1

  

  System model block diagram of FD communication, where AC stands for analog SI cancelation, DC stands for digital SI cancelations, $e^{j\theta ^{[r/r_{b}/t/\text {SI}]}_{l}}$ represents the PN at the lth antenna

• 2. Modeling of RF impairments: RF impairments due to imperfect transmitter and receiver chain electronics have been shown to significantly degrade the performance of the analog cancelation techniques [25, 26]. Since the focus of this work is residual SI cancelation, we only include PN in our model and assume that the other hardware impairments are dealt by a RF canceler. Such an assumption is also made in [20, 21, 27, 28].

• 3. Assumptions on oscillators: We make two assumptions about the oscillators. First, we assume that free-running oscillators are used. The assumption of using free-running oscillators for mmWave communications has also been made in [24, 29]. Second, we assume each transmit and receive antenna is equipped with an independent oscillator.

• 4. Quasi-static flat-fading channel assumption: The SI measurement results of [17] show that even with omnidirectional dipole antennas, the delay spread of the channel does not exceed 800 ns. This delay is significantly smaller than the proposed symbol durations for 5G communication [30, 31], which are in order of μs. Hence, not only can the channel be assumed flat but it can also be assumed to remain constant over transmission of one block of data (quasi-static). Similarly, measurement results of the desired communication channel show that the channel delays are relatively small compared to the symbol durations [32].

• 5. Synchronized transmission and reception: Although synchronizing transmission and reception of analog desired communication signal with the reception of analog SI signal is an important practical problem and requires its own detailed investigation, the synchronized FD communication assumption is widely used in the literature of channel and PN estimation for digital SI cancelation (DC) [20, 21, 33].

2.1 Mathematical representation of received vector

In this subsection, we present a mathematical model for the received vector of a FD MIMO communication system at mmWave frequencies. The received vector at node a and at time instant n is y(n) and is given by

$$\begin{array}{*{20}l} \mathbf{y}(n)=\mathbf{H}(n)\mathbf{x}(n)+\mathbf{H}^{\text{SI}}(n)\mathbf{x}^{\text{SI}}(n)+\mathbf{w}(n), \end{array} $$

(1)

where $\mathbf {y}(n)\triangleq [y_{1}(n),\cdots,y_{N_{r}}(n)]^{T}$, and y_i(n) is the received symbol at the ith antenna. For i∈{1,⋯,N_r} and k∈{1,⋯,N_t}, the element in the ith row and kth column of N_r×N_t channel matrix H(n) is given by $h_{i,k}e^{j\left (\theta ^{[r]}_{i}(n)+\theta ^{[t]}_{k}(n)\right)}$, where h_i,k is the communication channel between the kth transmit antenna of node b to the ith receive antenna of node a, for m∈{r,t,SI}, $\theta ^{[m]}_{j}(n)$ is the oscillator PN at the jth antenna and m determines the type of antenna such that m=r indicates a receive antenna, m=t means a transmit antenna, and m=SI indicates an interfering antenna. Furthermore, PN variation of a free-running oscillator follows a Wiener process [34], i.e.,

$$\begin{array}{*{20}l} \theta^{[m]}_{j}(n)= \theta^{[m]}_{j}(n-1)+\delta(n), \end{array} $$

(2)

where δ(n) is Gaussian noise with mean 0 and variance $\sigma ^{2}_{[m]}$, i.e., $\delta (n)\sim \mathcal {N}\left (0,\sigma ^{2}_{[m]}\right)$.

Similarly, the element in the ith row and kth column of N_r×N_t SI channel matrix H^SI(n) is given by $h^{\text {SI}}_{i,k} e^{j\left (\theta ^{[r]}_{i}(n)+\theta ^{[\text {SI}]}_{k}(n)\right)}$, where $h^{\text {SI}}_{i,k}$ is the interference channel between the kth transmit antenna and the ith receive antenna of node a.

In addition, the kth elements of N_t×1 vectors x(n) and x^SI(n) are given by x_k(n) and $x^{\text {SI}}_{k}(n)$, respectively, which are the transmitted symbols from the kth transmit antenna of nodes b and a, respectively.

Finally, $\mathbf {w}(n) \triangleq [w_{1}(n),\cdots,w_{N_{r}}(n)]^{T}$, where w_i(n) is the complex Gaussian noise, i.e., $w_{i}(n)\sim \mathcal {CN}(0,\sigma ^{2})$.

2.2 Mathematical representation for joint channel and PN estimation

For received vector y(n) and noise vector w(n) in (1), a useful mathematical model for joint channel and PN estimation is of the form ([35], Ch. 13, pp.450, Eq. 13.66)

$$\begin{array}{*{20}l} \mathbf{y}(n)=\bar{\mathbf{H}}(n)f\left(\boldsymbol{\beta}(n)\right)+\mathbf{w}(n), \end{array} $$

(3)

where $\bar {\mathbf {H}}(n)$ is the state transition model matrix, f is a nonlinear function, and β(n) is the state vector to be estimated.

A fundamental step in the problem of joint channel and PN estimation is the construction of the state vector and the state transition matrix based on the system model given by (1). The state vector and the state transition matrix for the joint PN and channel estimation in the presence of SI signal are given by (4) and (5), respectively.

• The state vector:

  $$\begin{array}{*{20}l} \boldsymbol{\beta}(n)\triangleq[\boldsymbol{\bar{\beta}}_{1}(n),\cdots,\boldsymbol{\bar{\beta}}_{N_{r}}(n)]^{T} \end{array} $$

  (4)

• The state transition matrix:

  $$\begin{array}{*{20}l} \bar{\mathbf{H}}(n)\triangleq \left[ \begin{array}{ccc} \bar{\mathbf{h}}_{1} & 0 & 0 \\ 0 & \ddots & 0 \\ 0 & 0 & \bar{\mathbf{h}}_{N_{r}} \\ \end{array} \right] \end{array} $$

  (5)

where

$$\begin{array}{*{20}l}&\boldsymbol{\bar{\beta}}_{i}(n)\triangleq [\bar{\beta}_{i,1},\cdots, \bar{\beta}_{i,{2N_{t}}}], \end{array} $$

(6a)

$$\begin{array}{*{20}l} &\bar{\beta}_{i,\bar{k}}(n)\triangleq \left\{ \begin{array}{ll} \theta^{[r]}_{i}(n)+\theta^{[t]}_{k}(n), & \bar{k} \text{ is odd;} \\ \theta^{[r]}_{i}(n)+\theta^{[\text{SI}]}_{k}(n), & \bar{k} \text{ is even} \end{array}\right., \end{array} $$

(6b)

$$\begin{array}{*{20}l} &\bar{\mathbf{h}}_{i}\triangleq [\bar{h}_{i,1},\cdots,\bar{h}_{i,2N_{t}}]\odot [\bar{x}_{1}(n),\cdots,\bar{x}_{2Nt}(n)], \end{array} $$

(6c)

$$\begin{array}{*{20}l} &\bar{h}_{i,\bar{k}} \triangleq \left\{ \begin{array}{ll} h_{i,k}, & \bar{k} \text{ is odd;} \\ h^{\text{SI}}_{i,k}, & \bar{k} \text{ is even} \end{array}\right., \end{array} $$

(6d)

$$\begin{array}{*{20}l} &\bar{x}_{\bar{k}}(n)\triangleq \left\{ \begin{array}{ll} x_{k}(n), & \bar{k} \text{ is odd;} \\ x^{\text{SI}}_{k}(n), & \bar{k} \text{ is even} \end{array}\right., \end{array} $$

(6e)

$$\begin{array}{*{20}l} &\bar{k}=\{1,\cdots,2N_{t}\} \end{array} $$

(6f)

$$\begin{array}{*{20}l} & k=\left\{ \begin{array}{ll} \bar{k}, & {\bar{k}<N_{t} ;} \\ \bar{k}-N_{t}, & {\bar{k}>N_{t}} \end{array}\right.. \end{array} $$

(6g)

The principle idea behind the design of the state vector β(n) and the state transition matrix $\bar {\mathbf {H}}(n)$ as given by (4) and (5), respectively, is the fact that the PN noise is the only random variable that varies from one symbol to another and needs to be tracked. On the other hand, because of the quasi-static nature of the communication and SI channels, they remain constant over transmission of a single data packet. Therefore, these channels need to be estimated only once at the beginning of data transmission. This initial channel estimation for the constant channels can be done using pilot transmission.

Furthermore, we note that at each receive antenna, there are 2N_t parameters that need to be estimated, N_t parameters for the communication channel, and N_t parameters for the SI channel. This explains the existence of index $\bar {k}\in \{1,\cdots,2N_{t}\}$.

Finally, with the state vector β(n) and the state transition matrix $\bar {\mathbf {H}}(n)$ given by (4) and (5), the discrete-time received vector at time instant n and at the baseband of node a is given by

$$\begin{array}{*{20}l} \mathbf{y}(n)=\bar{\mathbf{H}}(n)e^{j\boldsymbol{\beta}(n)}+\mathbf{w}(n). \end{array} $$

(7)

3 Joint channel and PN estimation

In this section, we use the state vector (4) and the state transition matrix (5) and present a joint channel and PN estimator based on the concept of extended Kalman filtering (EKF) [35]. The observation vector of EKF is given by y(n) in (7), which is a nonlinear function of the states β(n). The EKF state equation is given by

$$\begin{array}{*{20}l} \boldsymbol{\beta}(n)=\boldsymbol{\beta}(n-1)+\mathbf{u}(n), \end{array} $$

(8)

where u(n) is Gaussian with mean zero and covariance $\mathbf {Q}\triangleq \mathbb {E}[\boldsymbol {\beta }(n)\boldsymbol {\beta }^{T}(n)]$, i.e., $\mathbf {u}(n)\sim \mathcal {N}(\boldsymbol {0}_{2N_{r}N_{t}}, \mathbf {Q})$. The 2N_tN_r×2N_tN_r covariance matrix Q is given by

$$\begin{array}{*{20}l} \mathbf{Q}\triangleq\mathbb{E}\left[\boldsymbol{\beta}(n)\boldsymbol{\beta}^{T}(n)\right]=\left[ \begin{array}{ccc} \mathbf{R}_{1,1}&\cdots & \bar{\mathbf{R}}_{1,N_{r}} \\ \vdots & \vdots &\vdots\\ \mathbf{R}_{N_{r},1}&\cdots & \bar{\mathbf{R}}_{N_{r},N_{r}} \\ \end{array} \right], \end{array} $$

(9)

where, for m,n∈{1,⋯,N_r}, R_m,n is 2N_tN_t matrix given by (10), where $\sigma ^{2}_{r}$, $\sigma ^{2}_{t}$, and $\sigma ^{2}_{\text {SI}}$ are PN variances due to receive, transmit, and SI antennas, respectively.

$$ {\begin{aligned} \mathbf{R}_{m,n}=\left\{ \begin{array}{ll} \sigma^{2}_{r}\boldsymbol{1}_{2N_{t}\times2N_{t}}+ \text{diag}\left(\underbrace{\sigma^{2}_{t},\cdots,\sigma^{2}_{t}}_{2N_{t}}\right),& m=n \text{ and\ is\ odd}\\ \sigma^{2}_{r}\boldsymbol{1}_{2N_{t}\times2N_{t}}+\text{diag}\left(\underbrace{\sigma^{2}_{\text{SI}},\cdots,\sigma^{2}_{\text{SI}}}_{2N_{t}}\right),& m=n \text{ and is even}\\ \text{diag}\left(\underbrace{\sigma^{2}_{t},\cdots,\sigma^{2}_{t}}_{2N_{t}}\right),& m \neq n \text{ and is odd}\\ \text{diag}\left(\underbrace{\sigma^{2}_{\text{SI}},\cdots,\sigma^{2}_{\text{SI}}}_{2N_{t}}\right),& m \neq n \text{ and is even}\\ \end{array}\right. \end{aligned}} $$

(10)

The EKF state update equations are given by [35]

$$\begin{array}{*{20}l} \boldsymbol{\hat{\beta}}(n|n)&=\boldsymbol{\hat{\beta}}(n|n-1) \\ &\quad+ \mathfrak{R}\left\{\mathbf{K}(n)\left(\mathbf{y}(n)-\bar{\mathbf{H}}(n)e^{j\boldsymbol{\hat{\beta}}(n|n-1)}\right)\right\}, \end{array} $$

(11)

$$\begin{array}{*{20}l} \boldsymbol{\hat{\beta}}(n|n-1)&=\boldsymbol{\hat{\beta}}(n-1|n-1), \end{array} $$

(12)

$$\begin{array}{*{20}l} \mathbf{K}(n)&=\mathbf{M}(n|n\!\,-\,1)\mathbf{D}^{\dag}(n) \\&\quad\times\left(\sigma^{2}\mathbf{I}_{N_{r}}+\mathbf{D}(n)\mathbf{M}(n|n-1)\mathbf{D}^{\dag}(n)\right)^{-1}, \end{array} $$

(13)

$$\begin{array}{*{20}l} \mathbf{M}(n|n-1)&=\mathbf{M}(n-1|n-1)+\mathbf{Q}, \end{array} $$

(14)

$$\begin{array}{*{20}l} \mathbf{M}(n|n)&=\mathfrak{R}\left\{\left(\mathbf{I}_{N_{r}}-\mathbf{K}(n)\mathbf{D}(n)\right)\mathbf{M}(n|n-1)\right\}, \end{array} $$

(15)

where, for k∈{1,⋯,N_t},

$$\begin{array}{*{20}l} &\mathbf{D}(n)=\frac{\partial\bar{\mathbf{H}}(n)e^{j\boldsymbol{\beta}(n)}}{\partial\boldsymbol{\beta}^{T}(n)}=\left(\begin{array}{ccc} \mathbf{z}_{1} & \boldsymbol{0}^{T}_{2Nt} & \boldsymbol{0}^{T}_{2Nt} \\ \boldsymbol{0}^{T}_{2Nt} & \ddots & \vdots \\ \boldsymbol{0}^{T}_{2Nt} & \boldsymbol{0}^{T}_{2Nt} & \mathbf{z}_{N_{r}} \\ \end{array} \right), \end{array} $$

(16)

$$\begin{array}{*{20}l} &\mathbf{z}_{i}=\left\{\begin{array}{ll} h_{i,k}x_{k}(n)e^{j\hat{\beta}_{i,k}(n|n-1)} & {k} \text{ is even} \\ h^{\text{SI}}_{i,k}x^{\text{SI}}_{k}(n)e^{j\hat{\beta}_{i,k}(n|n-1)} & {k} \text{ is odd} \end{array}\right., \end{array} $$

(17)

and $\hat {\beta }_{i,k}(n|n-1)$ is the 2(i−1)N_t+k element of vector $\boldsymbol {\hat {\beta }}(n|n-1)$.

Remark 1

We note that the state vector, as given by (8), is a real vector. This is because the state vector only contains the phases, which are real numbers. The complex channel coefficients are estimated using this estimated real vector and using the complex exponential function as given by (7). Since the states are all real, when updating the mean of the states in EKF, we can safely discard the imaginary part of the updated mean as in (11).

3.1 Symbol detection

The EKF Eq. (17) shows that z_i requires the knowledge of the constant channels h_i,k, $h_{i,k}^{\text {SI}}$ and the transmitted symbols. Note that $x^{\text {SI}}_{k}$, the SI symbol is perfectly known at the receiver.

The knowledge of the constant channels can be obtained using pilot-based estimation during the initial half-duplex (HD) phase of the communication. In addition, the transmitted symbols at time n are detected using the initial channel estimates and the estimates of the state vector β at time n−1. This is because at time n of the EKF algorithm, β(n−1) has been successfully estimated. This procedure is shown in Fig. 2.

Fig. 2

Time diagram of modified EKF

3.2 Lower bound of estimation error

In this section, we derive a lower bound on the estimation error of the estimator proposed in the previous subsection. We first note that the mean square error (MSE) for estimating the state vector β(n) is given by

$$\begin{array}{*{20}l} \text{MSE}=\text{trace}\left(\mathbb{E}\left[\left(\boldsymbol{\beta}(n)-\boldsymbol{\hat{\beta}(n)}\right)\left(\boldsymbol{\beta}_{n}-\boldsymbol{\hat{\beta}_{n}}\right)^{T}\right]\right) \end{array} $$

(18)

With the above definition of the MSE vector, we present the following proposition.

proposition 1

MSE of the EKF is lower bounded by trace(Q), i.e.,

$$\begin{array}{*{20}l} \text{MSE}\geq trace\left(\boldsymbol{Q}\right), \end{array} $$

(19)

where Q is the state covariance matrix given by (9).

Proof

See Appendix 5. □

Remark 2

We note that (19) shows that the lower bound on the estimation error increases as the sum of diagonal elements of the covariance matrix of the states increases. Furthermore, (9) indicates that the diagonal elements of the state covariance matrix are the function of PN variance. Consequently, increasing the PN variance will result in worse estimation error. Since the residual SI cancelation is performed using the estimated SI channel, increasing the PN variance will result in worse SI cancelation performance. It is also worth to note that [34] shows that the PN variance is a monotonic increasing of function of carrier frequency. This means that the estimation error increases with increasing the carrier frequency and vice versa.

3.3 Complexity analysis of EKF

For the complexity analysis of the proposed joint channel and PN estimation technique, we take the approach used by [36, 37] and count the number of multiplications and additions used in each step of EKF algorithm. Table 2 shows the complexity of each step of EKF algorithm using $\mathcal {O}-$notation. The corresponding complexity calculations for this table can be found in Appendix 5.

Table 2 Complexity of each step of EKF algorithm

Remark 3

According to Table (2), the EKF has a polynomial complexity as a function of number of transmit N_t and receive N_r antennas. We can justify the increased complexity as follows. In [20], the authors propose an algorithm for channel estimation with linear complexity. However, the algorithm in [20] assumes a constant PN for a block of data. This could be an acceptable scenario in microwave communication but does not suit mmWave communication. Hence, the increased complexity of the proposed algorithm is justified because of fast variation of PN, i.e., PN variation over symbol time.

4 Simulation results

In this section, we present simulation results for MIMO FD systems at 60 GHz frequency, which corresponds to mmWave frequency band [3]. For each simulation run, we assume a communication packet is 40 symbol long, i.e., N=40. This communication packet is transmitted after the training packet, which is 2N_t symbols long, and is used for estimating the constant channels for EKF initialization as described in Section 3.1. We then use 10,000 simulation runs to obtain the desired simulation results.

Moreover, we use the assumptions presented in Section 2 to generate the random noise and PN. As summarized in [38], there are many mmWave channel models available for mmWave systems. In this work, similar to a large number of existing works in [24, 29, 39–41], we adopt a general Rician model. Note that the proposed estimator is independent of the adopted model. A performance comparison of the different mmWave channel models is outside the scope of this work.

We generate the random SI and communication channel (H_SI/COM) using Rician distribution as follows:

$$\begin{array}{*{20}l} \mathbf{H}_{\text{SI/COM}}=\sqrt{\frac{K}{K+1}}\mathbf{H}_{\text{LoS}}+\sqrt{\frac{1}{K+1}}\mathbf{H}_{\text{NLoS}}, \end{array} $$

(20)

where K is the Rician distribution K-factor; H_LoS is the LoS component of the channel, and is generated assuming uniform distribution for angle of arrival, using the approach presented in [24]; H_NLoS is the NLoS component of the channel; and for both SI and communication channel is generated assuming Rayleigh fading. Furthermore, for both the SI and communication channel, we set the K-factor to 2 dB.

We note that the SI and communication channels have different power intensities, i.e., $\mathbb {E}\left [\mathbf {H}_{\text {SI}}\mathbf {H}^{\dag }_{\text {SI}}\right ] \neq \mathbb {E}\left [\mathbf {H}_{\text {COM}}\mathbf {H}^{\dag }_{\text {COM}}\right ]$. Assuming that the LoS power of the residual SI (SI signal after the passive and analog cancelation) is the same as the LoS power of the communication signal, the signal to interference ratio (SIR) is given by:

$$\begin{array}{*{20}l} \text{SIR}=\frac{\sigma^{2}_{\text{COM}}}{\sigma^{2}_{\text{SI}}}, \end{array} $$

(21)

where $\sigma ^{2}_{\text {COM}}$ and $\sigma ^{2}_{\text {SI}}$ are the variances of NLoS components of the communication and SI channels, respectively.

In addition, SNR is defined as

$$\begin{array}{*{20}l} \text{SNR}\triangleq \frac{\mathbb{E}[E_{s}]}{\sigma^{2}}, \end{array} $$

(22)

where E_s is the symbol energy, $\mathbb {E}[E_{s}]=1$, and σ² is the noise variance.

Finally, we use the MSE for the state vector at time N=40. This MSE is given by rewriting (18) in terms of the Euclidean norm of a vector, i.e., ||·||₂,

$$\begin{array}{*{20}l} \mathbb{E}\left[\left|\left|\boldsymbol{\beta}(N)-\boldsymbol{\hat{\beta}}(N)\right|\right|_{2}\right]. \end{array} $$

(23)

In what follows, we first present the MSE results for different FD MIMO communication systems. We then investigate the residual SI power after digital cancelation and the bit error rate (BER) performance of these systems with the proposed PN estimation technique.

4.1 MSE performance

In this section, we investigate the MSE performance of the proposed PN estimation technique for a 2×2 FD MIMO system and assume that SIR=0 dB, i.e., the SI signal is as strong as the desired communication signal.

Figure 3 shows the MSE performance of the proposed system against the derived theoretical bound in Section 3.2 for different quadrature amplitude modulations (QAM) and different PN variances. Firstly, as discussed in Remark 2, with increasing PN variance, the estimation performance degrades. Secondly, it can be observed from this figure that lower order modulations have better performance compared to the higher order modulations. This is because as shown in Section 3.1, the EKF algorithms require to detect the transmitted symbols. Hence, the MSE of EKF is affected by the detection error. Finally, Fig. 3 shows that at high SNRs, the MSE performance of the proposed estimator approaches the lower bound.

Fig. 3

MSE performance for PN variances $\sigma ^{2}_{r}=\sigma ^{2}_{t}=\sigma ^{2}_{\text {SI}}=10^{-4},10^{-5}$ and different QAM modulations for a 2×2 FD MIMO system with SIR =0 dB

In Fig. 3, we also plot the MSE result of the state-of-the-art pilot-based phase noise estimator in [20, 23] for microwave frequency. As expected, this estimator does not perform well compared to our proposed estimator. This is because it assumes that the PN variations are small, which is not applicable for the case for mmWave frequency. Note that we only show the MSE result of the estimator in [20, 23] for 64−QAM modulation since the MSE performance is invariant with respect to the modulation order (the estimator uses pilots and does not require detection).

4.2 Comparison with unscented Kalman filter

We compare the performance of the proposed EKF estimator with unscented Kalman filter (UKF). UKF provides an alternative for linearizing the observations. The detailed implementation of the UKF is provided in Appendix 5. Figure 4 shows the performance of the EKF and UKF estimators for 8−QAM modulation, SIR =0 dB, and different PN variances. We can see that the MSE performance of the proposed EKF estimator is better than the UKF estimator. This is because (i) UKF estimator works with the sigma point approximation of the mean of the state process, while EKF tracks the PN based on the true mean of the linear state vector; (ii) while the MSE performances of both EKF and UKF are degraded because of the detection error, this error affects UKF algorithm more than EKF. This is because the sigma points calculation are affected more by the error due to the symbol detection (Section 3.1); and (iii) UKF is inherently more suitable for the systems which experience high nonlinearities, i.e., both the state and process models are nonlinear and noise is nonlinear too. In our case, only the process model in (7) is nonlinear.

Fig. 4

MSE performance of the UKF and proposed EKF for PN variances $\sigma ^{2}_{r}=\sigma ^{2}_{t}=\sigma ^{2}_{\text {SI}}=10^{-4},10^{-5}$ and 8–QAM modulation for a 2×2 FD MIMO system with SIR =0 dB

4.3 Residual SI power

In this section, we numerically investigate the remaining SI power after digital cancelation for a 2×2 MIMO FD system with 64−QAM modulation, assuming the PN variance for all the oscillators is 10⁻⁴. This residual power is given by

$$\begin{array}{*{20}l} P_{\text{SI}}=\left|\left|\left(\mathbf{H}^{\text{SI}}(n)-\bar{\mathbf{H}}^{\text{SI}}(n)\right)\mathbf{x}^{\text{SI}}(n)\right|\right|_{2}, \end{array} $$

(24)

where ||·||₂ is the Euclidean norm of a vector, and $\bar {\mathbf {H}}^{\text {SI}}(n)$ is an estimate of the SI channel using the proposed EKF estimator. Figure 5 shows the residual SI power for different SIR values, where a SIR value of 0 dB indicates that passive and analog cancelation stages have managed to reduce the SI power to the same level as the desired signal power.

Fig. 5

The residual SI power P_SI after digital cancelation

The numerical result of Fig. 5 shows that the performance of digital canceler depends on the residual SI power after passive and analog cancelation stages. As the residual SI power after passive and analog cancelation decreases, so does the residual SI power after the digital cancelation. The results show that the residual SI power can be reduced to − 25 and − 40 dB for SIR of 0 and 15 dB, respectively. This is important as it shows the effectiveness of digital SI cancelation after passive and analog cancelation.

4.4 BER performance

Finally, in this section, we present the BER results of a 2×2 FD MIMO system with different QAM modulations, assuming that PN variance for all oscillators is 10⁻⁴. Figure 6 shows the BER performance of the system for different values of SIR. The results are consistent with the results of the residual SI power in Fig. 5, i.e., the higher the SIR, the better the BER results. Furthermore, 8−QAM system performs better than the 64−QAM system, which is consistent with the results of Fig. 6.

Fig. 6

BER performance of the proposed system for a 2×2 MIMO FD system with different QAM modulations

5 Conclusion

In this paper, we considered a MIMO FD system for mmWave communication and proposed a joint channel and PN estimation algorithm¹. We also derived a lower bound on the estimation error and numerically showed that the MSE of the proposed estimator approaches the error bound. Furthermore, we investigated the residual SI power after the digital cancelation and showed that the digital canceler, which uses the estimated SI channel can reduce the SI power to − 25 to − 40 dB. These results indicate the effectiveness of digital cancelation after passive and analog cancelation stages.

6 A lower bound of the estimation error

In this section, we derive the lower bound of the estimation error. We start the proof by expanding $\mathbb {E}\left [\left (\boldsymbol {\beta }(n)-\boldsymbol {\hat {\beta }}(n)\right)\left (\boldsymbol {\beta }(n)-\boldsymbol {\hat {\beta }}(n)\right)^{T}\right ]$.

$$ {\begin{aligned} &\mathbb{E}\left[\left(\boldsymbol{\beta}(n)-\boldsymbol{\hat{\beta}}(n)\right)\left(\boldsymbol{\beta}(n)-\boldsymbol{\hat{\beta}}(n)\right)^{T}\right]=\mathbb{E}\left[\boldsymbol{\beta}(n)\boldsymbol{\beta}^{T}(n)\right]\\ &\quad+\mathbb{E}\left[\boldsymbol{\hat{\beta}}(n)\boldsymbol{\hat{\beta}}^{T}(n)\right] -\mathbb{E}\left[\boldsymbol{\beta}(n)\boldsymbol{\hat{\beta}}^{T}(n)\right] -\mathbb{E}\left[\boldsymbol{\hat{\beta}}(n)\boldsymbol{\beta}^{T}(n)\right] \end{aligned}} $$

(25)

Next, we show that the last two terms of (25) are zero. We do this by showing only $\mathbb {E}\left [\boldsymbol {\beta }(n)\boldsymbol {\hat {\beta }}^{T}(n)\right ]=0$ as a similar approach can be used to show that $\mathbb {E}\left [\boldsymbol {\hat {\beta }}(n)\boldsymbol {\beta }^{T}(n)\right ]=0$.

We first note that β(n) given by (8) is a Gaussian autoregressive model (AR) with mean zero, i.e., $\mathbb {E}\left [\boldsymbol {\beta }(n)\right ]=0$. Hence,

$$ {\begin{aligned} \mathbb{E}\left[\boldsymbol{\beta}(n)\boldsymbol{\hat{\beta}}^{T}(n)\right]&=\int\int \boldsymbol{\beta}(n)\boldsymbol{\hat{\beta}(n)}~p(\boldsymbol{\beta}(n),\mathbf{y}(n))~d\boldsymbol{\beta}(n)~d\mathbf{y}(n)\\&=\int\boldsymbol{\hat{\beta}(n)}\int \boldsymbol{\beta}(n)~p(\boldsymbol{\beta}(n))~d\boldsymbol{\beta}(n)~p(\mathbf{y}(n)|\boldsymbol{\beta}(n))~d\mathbf{y}(n)\\ &=\int\boldsymbol{\hat{\beta}(n)}\mathbb{E}\left[\boldsymbol{\beta}(n)\right]~p(\mathbf{y}(n)|\boldsymbol{\beta}(n))~d\mathbf{y}(n)=0. \end{aligned}} $$

(26)

Consequently, we can rewrite (25) as follows:

$$ \begin{aligned} &\mathbb{E}\left[\left(\boldsymbol{\beta}(n)-\boldsymbol{\hat{\beta}}(n)\right)\left(\boldsymbol{\beta}(n)-\boldsymbol{\hat{\beta}}(n)\right)^{T}\right]\\ &=\mathbb{E}\left[\boldsymbol{\beta}(n)\boldsymbol{\beta}^{T}(n)\right] +\mathbb{E}\left[\boldsymbol{\hat{\beta}}(n)\boldsymbol{\hat{\beta}}^{T}(n)\right] \end{aligned} $$

(27)

It is easy to show that $\mathbb {E}\left [\boldsymbol {\hat {\beta }}(n)\boldsymbol {\hat {\beta }}^{T}(n)\right ]$ is a positive semi-definite matrix and hence

$$\begin{array}{*{20}l} \mathbb{E}\left[\left(\boldsymbol{\beta}(n)-\boldsymbol{\hat{\beta}}(n)\right)\left(\boldsymbol{\beta}(n)-\boldsymbol{\hat{\beta}}(n)\right)^{T}\right]\geq \mathbb{E}\left[\boldsymbol{\beta}(n)\boldsymbol{\beta}^{T}(n)\right] \end{array} $$

(28)

Furthermore, the properties of trace allows us to write

$$ {{\begin{aligned} \text{trace}\left(\mathbb{E}\left[\left(\boldsymbol{\beta}(n)\,-\,\boldsymbol{\hat{\beta}}(n)\right)\left(\boldsymbol{\beta}(n)\,-\,\boldsymbol{\hat{\beta}}(n)\right)^{T}\right]\right)\geq \text{trace}\left(\mathbb{E}\left[\boldsymbol{\beta}(n)\boldsymbol{\beta}^{T}(n)\right]\right) \end{aligned}}} $$

(29)

Finally, using (29) and the definitions of Q and MSE in (9) and (18), we can establish the proof of the proposition.

7 B Complexity analysis of EKF

In this section, we provide the complexity analysis of the EKF algorithm by counting the number of multiplications and additions. However, before we proceed, it can easily be shown that every entry of product of a K×L matrix by a L×M matrix requires L multiplications and L−1 additions, and hence, the whole matrix requires KML multiplications and KM(L−1) additions, where KM is the size of the resulting matrix. Furthermore, it is known that matrix inversion has the same complexity in terms of additions and multiplication as the matrix multiplication, up to a multiplicative constant γ [42]. We can now proceed with calculating the complexity of EKF algorithm in (30) to (32).

$$ {\begin{aligned} &\boldsymbol{\hat{\beta}}(n|n)=\!\underbrace{\boldsymbol{\hat{\beta}}(n|n-1) \underbrace{\mathfrak{R}\left\{\mathbf{K}(n)\underbrace{\left(\mathbf{y}(n)- \underbrace{\bar{\mathbf{H}}(n)e^{j\boldsymbol{\hat{\beta}}(n|n-1)}}_{N_{r}(2N_{t}N_{r})+N_{r}(2N_{t}N_{r}-1)} \right)}_{N_{r}+N_{r}(2N_{t}N_{r})+N_{r}(2N_{t}N_{r}-1)} \right\}}_{2N_{t}N^{2}_{r}+2N_{t}N_{r}(N_{r}-1)+N_{r}+N_{r}(2N_{t}N_{r})+N_{r}(2N_{t}N_{r}-1)}}_{ \begin{aligned} &2N_{t}N_{r}\,+\,2N_{t}N^{2}_{r}\,+\,2N_{t}N_{r}(N_{r}-1)\,+\,N_{r}\,+\,N_{r}(2N_{t}N_{r})\,+\,N_{r}(2N_{t}N_{r}-1) \end{aligned}}, \end{aligned}} $$

(30)

$$ {{\begin{aligned} &\mathbf{K}(n)=\mathbf{M}(n|n-1)\mathbf{D}^{\dag}(n) \\ &\underbrace{{{\left({\sigma^{2}\mathbf{I}_{N_{r}}+{\mathbf{D}(n){\mathbf{M}(n|n-1)\mathbf{D}^{\dag}(n)}}}\right)^{-1}}}}_{\begin{aligned} &4N_{t}^{2}N_{r}^{3}+2N_{t}N_{r}^{2}(2N_{t}N_{r}-1)+2N_{t}N_{r}^{3}+2N_{t}N_{r}^{2}(N_{r}-1)\\&+\gamma N_{r}^{3}+\gamma N_{r}^{2}(N_{r}-1)+N_{r}^{2}+2N_{t}N_{r}^{2}+N_{r}(2N_{t}N_{r}-1)\\&+4N_{t}^{2}N_{r}^{3}+2N_{t}N_{r}^{2}(2N_{t}N_{r}-1)\end{aligned}}, \end{aligned}}} $$

$$\begin{array}{*{20}l} &\mathbf{M}(n|n-1)=\underbrace{\mathbf{M}(n-1|n-1)+\mathbf{Q}}_{4N_{t}^{2}N_{r}^{2}}, \end{array} $$

(31)

$$ {\begin{aligned} &\mathbf{M}(n|n)=\mathfrak{R}\left\{\underbrace{\mathbf{M}(n|n-1)\underbrace{\left(\mathbf{I}_{2N_{t}N_{r}}-\underbrace{\mathbf{K}(n)\mathbf{D}(n)}_{2N_{t}N_{r}^{3}+2N_{t}N_{r}^{2} (N_{r}-1)}\right)}_{4N_{t}^{2}N_{r}^{2}+2N_{t}N_{r}^{3}+2N_{t}N_{r}^{2} (N_{r}-1)}}_{8N_{t}^{3}N_{r}^{3}+4N_{t}^{2}N_{r}^{2}(2N_{t}N_{r}-1)+2N_{t}N_{r}^{3}+2N_{t}N_{r}^{2} (N_{r}-1)}\right\}, \end{aligned}} $$

(32)

8 C Unscented Kalman filter (UKF)

Unscented Kalman filter (UKF) provides an alternative to EKF for nonlinear state vector estimation. In UKF instead of linearizing the observation vector, the probability distributions of states and observations are approximated using sigma points [43]. UKF can solve a very general class of problems, where both state process and observations are nonlinear. However, the joint channel and PN estimation problem, as given by the observation vector (7) and the state vector (8), has a linear state process and additive noise. This allows for the use of non-augmented state vectors for UKF [44]. For the state vector β(n) in (8), the sigma points $\mathcal {B}(i,n)$ are given by

$$\begin{array}{*{20}l} \mathcal{B}(0,n|n-1)&=\mathcal{B}(0, n-1), \end{array} $$

(33a)

$$\begin{array}{*{20}l} \mathcal{B}_(i, n|n-1)&=\mathcal{B}(i, n-1)+\left(\sqrt{\left\{\left(L+\lambda\right)\mathbf{Q}\right\}}\right)_{i},\\ \quad i&=1,\cdots, L, \end{array} $$

(33b)

$$\begin{array}{*{20}l} \mathcal{B}(i, n|n-1)&=\mathcal{B}(i, n-1)-\left(\sqrt{\left\{\left(L+\lambda\right)\mathbf{Q}\right\}}\right)_{i},\\ \quad i&=L+1,\cdots, 2L\, \end{array} $$

(33c)

where, $\sqrt {\{\cdot \}}$ is the matrix square root, (·)_i is the ith column of the matrix, L=4N_tN_r, λ=α²L−L, where α=10⁻³ [43], and Q is the state covariance matrix given by (9). Subsequently, the mean of the sigma points, which is used as an approximate to the true mean of the probability distribution of states, is given by

$$\begin{array}{*{20}l} \bar{\boldsymbol{\beta}}(n)=\sum_{i=0}^{2l}\mathcal{W}^{m}_{i} \mathcal{B}(i, n|n-1), \end{array} $$

(34)

where,

$$\begin{array}{*{20}l} &\mathcal{W}^{m}_{0}=\frac{\lambda}{L+\lambda} \end{array} $$

(35)

$$\begin{array}{*{20}l} &\mathcal{W}^{m}_{i}=\frac{1}{2(L+\lambda)}, \quad i=1,\cdots,2L. \end{array} $$

(36)

Similarly, the covariance of the state vector based on the sigma points approximation is given by

$$\begin{array}{*{20}l} \bar{\mathbf{P}}_{n}&=\sum\limits_{i=0}^{2L}\mathcal{W}^{m}_{i} \left[\mathcal{B}_(i, n|n-1)-\bar{\boldsymbol{\beta}}(n)\right]\\ &\quad\left[\mathcal{B}_(i, n|n-1)-\bar{\boldsymbol{\beta}}(n)\right]^{*} \end{array} $$

(37)

Moreover, the sigma points for the observations, and the corresponding approximate mean of probability distribution of observations are given by

$$\begin{array}{*{20}l} &\mathcal{Y}(n|n-1)=\bar{\mathbf{H}}(n)e^{j \mathcal{B}(i, n|n-1)}, \end{array} $$

(38a)

$$\begin{array}{*{20}l} &\bar{\mathbf{y}}(n)=\sum\limits_{i=0}^{2l}\mathcal{W}^{c}_{i} \mathcal{Y}(n|n-1), \end{array} $$

(38b)

where $\mathcal {W}^{c}_{0}=\mathcal {W}^{m}_{0}+(1-\alpha ^{2}+\beta)$, β=2, and $\mathcal {W}^{c}_{i}=\mathcal {W}^{m}_{i}$ for i=1,⋯,2L. Once the state and the process models are approximated by the sigma points using (33a)–(33c), and (38a), respectively, the updated mean $\boldsymbol {\hat {\beta }}(n)$ and variance $\hat {\mathbf {P}}_{n}$ can be calculated as follows:

$$\begin{array}{*{20}l} &\boldsymbol{\hat{\beta}}(n)=\bar{\boldsymbol{\beta}}(n)+\mathcal{K}\left(\mathbf{y}(n)-\bar{\mathbf{y}}(n)\right), \end{array} $$

(39)

$$\begin{array}{*{20}l} &\hat{\mathbf{P}}_{n}=\bar{\mathbf{P}}_{n}-\mathcal{K}\mathbf{P}_{y,y}\mathcal{K}^{T}, \end{array} $$

(40)

where

$$\begin{array}{*{20}l} \mathcal{K}&=\mathbf{P}_{x,y}\mathbf{P}^{-1}_{y,y}, \end{array} $$

(41)

$$\begin{array}{*{20}l} \mathbf{P}_{x,y}&\,=\,\sum\limits_{i=0}^{2L}\mathcal{W}_{i}^{c}\left[\mathcal{B}_(i, n|n-1)-\bar{\boldsymbol{\beta}}(n)\right]\left[\mathcal{Y}(n|n\,-\,1)-\bar{\mathbf{y}}(n)\right]^{*}, \end{array} $$

(42)

$$\begin{array}{*{20}l} \mathbf{P}_{y,y}&=\sum\limits_{i=0}^{2L}\mathcal{W}_{i}^{c}\left[\mathcal{Y}(n|n-1)-\bar{\mathbf{y}}(n)\right]\left[\mathcal{Y}(n|n-1)-\bar{\mathbf{y}}(n)\right]^{*}. \end{array} $$

(43)

Algorithm 1 summarizes the UKF joint channel and PN estimation algorithm.