Skip to main content

Transmit and receive hybrid beamforming design for OFDM dual-function radar-communication systems

Abstract

This article investigates the design of hybrid digital and analog transmit/receive beamformers for orthogonal frequency division multiplexing dual-function radar-communication system. Specifically, we model the hybrid beamforming design by simultaneously optimizing the spectrum efficiency of communication and approximate output signal-to-interference noise ratio of radar sensing, through a flexible performance tradeoff. Due to the coupling relationship among the beamformers in the hybrid structure and the independence of signals in different frequency bands, the resultant optimization problem is NP-hard. Hence, a consensus alternating direction method of multipliers approach is designed to tackle the difficulty. The transmit hybrid beamformer is obtained by introducing auxiliary variables and exploiting the block continuous upper bound minimization method. Then, the receive beamformer can be optimized via generalized eigenvalue decomposition. Numerical simulations are provided to demonstrate the effectiveness of the proposed scheme.

1 Introduction

Radar and communications represent fundamentally important use of the electromagnetic (EM) spectrum. Due to the growing demand for higher radar sensing capability and communication data rates, greater bandwidth is needed for radar and communications, as well as many other services. Undoubtedly, this has led to the congestion of EM spectrum, which is known as precious resource. As an example, the spectrum allocated to world interoperability for microwave access (WiMAX) and long-term evolution (LTE) (2500–2690 MHz band) is adjacent to the air traffic control radar (2700–2900 MHz) [1]. Moreover, the millimeter-wave band used by automotive radar and high-resolution imaging radar [2] has also been shared with 5G network [3]. Therefore, DFRC systems (also named as RadCom in the literature), which share hardware platforms and frequency bands, have become the main objectives guiding active sensing and wireless technologies [4].

In the earlier work [5], it is argued that DFRC systems can allow both environmental sensing and vehicle to everything (V2X) communications to be performed more efficiently. A unique waveform design concept has been presented in [6] to allow for simultaneously performing both digital beamforming radar and multiple-input multiple-output (MIMO) communication operations. To increase the degree of freedom, the MIMO-DFRC system that embeds the amplitude-shift keying signals into the radar emission is discussed in [7]. In addition, by modulating the radar emission pulse into the communication signal, the radar-embedded communication system with a low probability of interception has been proposed in [8] and [9]. To realize dual-function beampattern, in [10], the dual-function waveform design is modeled as a trade-off problem. These DFRC systems are investigated by adopting fully digital beamformers, which could result in unfavorable hardware cost and power consumption. Therefore, hybrid beamforming architectures are considered in [11], to develop DFRC systems with OFDM modulation. Apart from traditional waveforms, some new waveforms are investigated for both sensing and communications to improve the overall performance of DFRC systems [12] and [13]. The orthogonal time frequency space (OTFS) waveform multiplexes information symbols in the delay-Doppler domain [12] and [14]. From the perspective of communications, the OTFS has the ability to deal with large Doppler shift and obtain both time and frequency diversities in doubly selective channels. Moreover, it can achieve similar sensing performance with OFDM as shown in [13] and [14].

Typically, the emission energy should be focused towards the target angle for radar sensing. Nevertheless, the method of minimizing the difference between the transmitted waveform and the predetermined waveform is not suitable for the case of uncertain beampatterns. Consequently, it is essential to consider the different performance indicators in the DFRC system. For radar systems, various performance parameters are used in radar beamforming design, such as mutual information maximization [15,16,17] and radar ambiguity function minimization [18]. Since the performance of target parameter estimation relies greatly on the radar SINR [19], the radar transmission waveform is often designed via SINR maximization [20,21,22].

Note that, the above-mentioned studies focus on the radar receive filter of the DFRC system in the narrowband case, and the DFRC system was developed by adopting fully digital beamforming. So far, the design problem for wideband DFRC system has not been well investigated. Further, it is necessary to investigate radar receive filters with hybrid digital and analog beamforming architecture. This motivates us to carry out the study on hybrid beamforming transceivers for OFDM-DFRC systems in this paper. The proposed method models the hybrid beamforming of the DFRC system as a tradeoff problem, which is solved by updating the hybrid beamforming transmitter, hybrid beamforming radar receiver, and user receiver in an alternating manner.

The main contributions of this work can be summarized as follows:

  • We propose to jointly design the hybrid beamforming transmitter, hybrid beamforming radar receiver, and user receiver in an OFDM-DFRC system. Specifically, the hybrid beamforming design is formulated as a tradeoff problem composed of spectral efficiency (SE) of communication and the approximate output SINR for radar sensing.

  • We derive an alternating optimization method to tackle the nonconvex weighted problem. More specifically, the communication and radar sensing performance metrics are normalized to deal with their different ranges. The hybrid radar receive filter is updated after the hybrid beamforming transmitter and user receiver optimization. To optimize the hybrid beamforming transmitter and user receiver, the block successive upper-bound minimization method (BSUM) [23] is employed. The resulting problem can be solved in a distributed manner based on the consensus alternating direction method of multipliers (CADMM) [24].

  • We convert the design problem of analog precoding into a phase-only problem and solve it with the variable step size gradient projection (VSGP) algorithm [23]. In order to validate the proposed design, various numerical results are provided.

The remainder of this paper is organized as follows. Section 2 introduces the system model. In Sect. 3, the performance indicator and optimization problems for the DFRC system are presented. The alternating optimization method based on CADMM is proposed in Sect. 4. Section 5 presents various numerical simulations and Sect. 6 concludes the paper.

Notations: Unless otherwise specified, vectors are denoted by bold lowercase letters \({\textbf{a}}\) and matrices are represented by bold uppercase letters \(\textbf{A}\), \((\cdot )^T\), \((\cdot )^H\) and \((\cdot )^*\) are stand for the transpose, Hermitian transpose and complex conjugate of the matrices, respectively. The sets of n-dimensional complex-valued vectors and \(N\times N\) complex-valued matrices are denoted by \(\mathbb {C}^{n}\) and \(\mathbb {C}^{N\times N}\). \(\nabla\) and \(\angle\) denotes the gradient and angle operator, respectively. \(\textbf{I}_N\) denotes the \(N\times N\) identity matrix and \(\mathbb {E}\{\cdot \}\) denotes the statistical expectation. \(\Vert \cdot \Vert _2\) denotes the \(l_2\) norm and \(\vert \cdot \vert\) represent determinant or absolute value depending on context.

2 Background

Consider a wideband OFDM-DFRC system that sends communication symbols to U users equipped with \(N_{cr}\) antennas, while performing radar detection. A hybrid digital and analog beamforming architecture is adopted at the transmitter, where the number of radio frequency (RF) chains \(N_{{\text{RF}}_t}\) is much less than the number of transmitter antennas \(N_t\). Further, let \(N_{rr}\) and \(N_{\text{RF}_r}\) be the numbers of receiver antennas and RF chains, respectively. The data symbols are modulated and spread across K subcarriers, the symbol width is \(\Delta t\) and the subcarrier frequency step is \(\Delta f\). Moreover, we assume \(\Delta f =1/\Delta t\) to ensure the orthogonality of subcarriers, and each antenna array is a uniform linear array (ULA).

2.1 Transmit model

Let \(\textbf{s}_k\in \mathbb {C}^U\) be the vector composed of data symbols at the k-th subcarrier, i.e., each user transmits a communication symbol on each subcarrier, and further assume that \(\mathbb {E}\{\textbf{s}_k\textbf{s}_k^H\}=\textbf{I}_U\), \(k=1,...,K\). In the hybrid beamforming system, the data symbols are first processed by the low-dimensional digital beamformer \(\textbf{F}_{\text{BB}_k}=[\textbf{f}_{\text{BB}_{k,1}},...,\textbf{f}_{\text{BB}_{k,U}}]\in \mathbb {C}^{N_{\text{RF}_t}\times U}\). Then, after \(N_{\text{RF}_t}\) K-point inverse fast Fourier transform (IFFT) and adding a cyclic prefix of length D, the data symbols are converted from the frequency domain to the time domain. Finally, an analog beamformer \(\textbf{F}_{\text{RF}}\in \mathbb {C}^{N_t\times N_{\text{RF}_t}}\) is used to generate the final transmission signal. Therefore, the final transmitted signal at subcarrier k can be written as

$$\begin{aligned} \textbf{x}_{k} = \textbf{F}_{{\text{RF}}}\textbf{F}_{\text{BB}_{k }}\textbf{s}_{k}. \end{aligned}$$
(1)

To investigate the performance upper bound of the method, we consider a hybrid beamformer with a fully connected structure. Specifically, each RF chain is connected to all the antenna elements via a set of phase shifters, i.e., \(|\textbf{F}_{\text{RF}}(i,j) |= 1\), \(\forall i, j\). Furthermore, we assume the power of transmitter for each user on each subcarrier satisfies the following condition:

$$\begin{aligned} \Vert \textbf{F}_{{\text{RF}}}\textbf{f}_{{\text{BB}_{k,u}}}\Vert _2^2=\frac{\mathcal {P}_k}{U}. \end{aligned}$$
(2)

2.2 Communication model

2.2.1 Receive signal model

Assume that the user uses a fully digital beamformer, and for the u-th user on the k-th subcarrier, the signal arriving through the channel \(\textbf{H}_{k,u}\in \mathbb {C}^{N_{cr}\times N_t}\) is mixed with additive white Gaussian noise (AWGN). After removing the cyclic prefix and \(N_{cr}\) K-point FFT, the frequency domain signal is processed by the digital combiner \(\textbf{u}_{k,u}\) to obtain the communication symbol of the user u on the subcarrier k as

$$\begin{aligned} \hat{y}_{k, u}&= \textbf{u}_{k, u}^{H}\textbf{H}_{k, u} \textbf{F}_{\text{RF}} \textbf{F}_{\text{BB}_k} \textbf{s}_{k}+\textbf{u}_{k, u}^{H}\hat{\textbf{n}}_{k, u} \nonumber \\&=\underbrace{\textbf{u}_{k, u}^{H} \textbf{H}_{k, u} \textbf{F}_{\text{RF}} \textbf{f}_{\text{BB}_{k, u}} s_{k, u}}_{\text{ desired } \text{ signal } } +\underbrace{\sum _{i \ne u}^{U} \textbf{u}_{k, u}^{H} \textbf{H}_{k, u} \textbf{F}_{\text{RF}} \textbf{f}_{\text{BB}_{k, i}} s_{k, i}}_{\text {multi-user interference }(\text{MUI})}+\textbf{u}_{k, u}^{H} \hat{\textbf{n}}_{k, u}, \end{aligned}$$
(3)

where \(\hat{\textbf{n}}_{k, u}\) denotes the AWGN of the u-th user at the k-th subcarrier with zero mean and covariance matrix \(\sigma _{\hat{n}}^2\textbf{I}_{N_{cr}}\).

2.2.2 Channel model

Following the previous works [25,26,27,28], in this paper we consider a geometric wideband channel model with \(N_{cl}\) clusters and each cluster contributes with \(N_{ray}\) rays between the transmitter and each user. The delay d wideband mmWave channel matrix for u-th user on k-th subcarrier can be modeled as

$$\begin{aligned} \textbf{H}_{k,u}[d] = \sum _{i=1}^{N_{cl}}\sum _{l=1}^{N_{ray}} \alpha _{i,l}\textbf{a}_{r}(\theta _{i,l},k)\textbf{a}_t^H(\phi _{i,l},k)p(dT_s-\tau _i), \end{aligned}$$
(4)

where \(\alpha _{i,l}\sim \mathcal{C}\mathcal{N}(0,\frac{N_tN_{cr}}{N_{cl}N_{ray}})\) denotes the complex path gain of l-th ray in the i-th cluster, and \(p(dT_s-\tau _i)\) is the pulse-shaping function for \(T_s\)-spaced signaling at time delay \(\tau _{i}\). Further, \(\textbf{a}_{r}(\theta _{i,l},k)\) and \(\textbf{a}_t(\phi _{i,l},k)\) are the receive and transmit array response vectors at k-th subcarrier, where \(\theta _{i,l}\) and \(\phi _{i,l}\) represent the angles of arrival and departure (AoAs/AoDs), respectively. For a ULA with N antennas, the steering vector at k-th subcarrier is expressed as:

$$\begin{aligned} \textbf{a}(\theta ,k) = \frac{1}{\sqrt{N}}\left[ 1,e^{j\pi \frac{f_k}{f_{\text{cent}}}\text{sin}\theta },...,e^{j\pi \frac{f_k}{f_{\text{cent}}}(N-1)\text{sin}\theta }\right] ^T, \end{aligned}$$
(5)

where \(f_{\text{cent}}\) is the center frequency, and \(f_k = f_{\text{cent}}+(k-\frac{K+1}{2})\Delta f\). Given the delay d channel matrix (4), the channel of u-th user at k-th subcarrier can be expressed as

$$\begin{aligned} \textbf{H}_{k,u} = \sum _{d=1}^D \textbf{H}_{k,u}[d]e^{-j\frac{2\pi k}{K}d}. \end{aligned}$$
(6)

2.3 Radar model

Denoting the steering matrix from the transmitter to the target (or interference) to the radar receiver as \(\textbf{A}(\varphi ,k)=\textbf{a}_r(\varphi ,k)\textbf{a}^T_t(\varphi ,k)\), \(k=1,...,K\). Therefore, the radar received signal at the k-th subcarrier can be modeled as

$$\begin{aligned} \tilde{\textbf{y}}_{r,k}&=\underbrace{\xi _g\textbf{A}(\varphi _g,k)\textbf{F}_{\text {RF}}\textbf{F}_{\text{BB}_{k}}\textbf{s}_{k}}_{\text{target}}+\underbrace{\sum _{j=1}^J\xi _j\textbf{A}(\varphi _j,k)\textbf{F}_{\text {RF}}\textbf{F}_{\text {BB}_{k}}\textbf{s}_{k}}_{\text {interference}}+\bar{\textbf{n}}_{k}, \end{aligned}$$
(7)

where \(\xi _g\) and \(\xi _j\), respectively, denote the reflection coefficients of the target and the j-th interference source, and they are assumed to be independent zero mean random variables with variances \(\mathbb {E}\{\vert \xi _g \vert ^2 \}=\sigma _g^2\) and \(\mathbb {E}\{\vert \xi _j \vert ^2 \}=\sigma _j^2\), respectively. \(\varphi _g\) and \(\varphi _j\), respectively represent the angles of the target and the j-th interference source, and \(\bar{\textbf{n}}_{k}\) denotes the AWGN of k-th subcarrier with zero mean and covariance matrix \(\sigma _{\bar{n}}^2\textbf{I}_{N_{rr}}\).

Do the reverse operations of transmitted signal to received signal, the received signal of the k-th subcarrier is first processed by an analog beam combiner \(\textbf{W}_{\text{RF}}\in \mathbb {C}^{N_{rr}\times N_{\text{RF}_r}}\) implemented by analog phase shifters, i.e., \(|\textbf{W}_{\text{RF}}(i,j) |= 1\), \(\forall i, j\). After removing the cyclic prefix and \(N_{rr}\) K-point fast Fourier transformation (FFT), the signal is converted to the frequency domain. At each subcarrier k, a digital combiner \(\textbf{w}_{\text{BB}_k}\in \mathbb {C}^{N_{\text{RF}_r}}\) is applied to estimate the radar target as

$$\begin{aligned} \bar{y}_{r,k}&=\underbrace{\sum _{u=1}^U\textbf{w}_{\text{BB}_k}^H\textbf{W}^H_{\text{RF}}\xi _g\textbf{A}(\varphi _g,k)\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}s_{k,u}}_{\text{target}} \nonumber \\&\quad +\underbrace{\sum _{u=1}^U\textbf{w}_{\text{BB}_k}^H\textbf{W}^H_{\text{RF}}\sum _{j=1}^J\xi _j\textbf{A}(\varphi _j,k)\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}s_{k,u}}_{\text {interference}} +\frac{1}{U}\sum _{u=1}^U\textbf{w}_{\text{BB}_k}^H\textbf{W}^H_{\text{RF}}\bar{\textbf{n}}_{k}. \end{aligned}$$
(8)

2.4 Performance indicators

In the trade-off problem, it is difficult to achieve the purpose of trade-off by setting the trade-off coefficient in the problem for two indicators with different ranges. Hence, it is better to select performance indicators with similar variation ranges for the problem to achieve fair performance trade-off. In this paper, we design a hybrid beamforming transceiver and user receiver to achieve the trade-off between radar and communication performance. For communication, we consider SE as a performance parameter of the communication function, which is defined as

$$\begin{aligned} \text{SE} = \frac{1}{K}\sum \limits _{k=1}^K\sum \limits _{u=1}^U\text{SE}_{k,u}, \end{aligned}$$
(9)

where \(\text{SE}_{k,u}\) is the spectrum efficiency of the u-th user on the k-th subcarrier as

$$\begin{aligned} \text{SE}_{k,u} = \text{log}_2\left( 1+\frac{|\textbf{u}^H_{k,u}\textbf{H}_{k,u}\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}|^2 }{\sum \limits _{i\ne u}^U|\textbf{u}^H_{k,u}\textbf{H}_{k,u}\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,i}}|^2+\sigma _{\hat{n}}^2\textbf{u}^H_{k,u}\textbf{u}_{k,u}}\right) . \end{aligned}$$
(10)

For radar sensing, the SINR can be used to measure the effect of transmitter and radar filter on estimating the target and suppressing interference. For the ease of design, we use the following formula to define the approximate SINR:

$$\begin{aligned} \text{SINR} = \frac{1}{KU}\sum \limits _{k=1}^K\sum _{u=1}^U\text{SINR}_{k,u}, \end{aligned}$$
(11)

where the \(\text{SINR}_{k,u}\) is expressed as

$$\begin{aligned} \text{SINR}_{k,u} = \frac{\mathbb {E}\{|\textbf{v}^H_{_k}\xi _g\textbf{A}(\varphi _g,k)\textbf{p}_{k,u} |^2\} }{\mathbb {E}\left\{ |\textbf{v}^H_k\sum \limits _{j=1}^J\xi _j\textbf{A}(\varphi _j,k)\textbf{p}_{k,u} |^2\right\} +\frac{\sigma ^2_{\bar{n}}}{U}\Vert \textbf{v}_k\Vert _2^2}. \end{aligned}$$
(12)

where \(\textbf{p}_{k,u}=\textbf{F}_{\text{RF}} \textbf{f}_{\text{BB}_{k, u}} s_{k,u}\) and \(\textbf{v}_{_k}=\textbf{W}_{\text{RF}}\textbf{w}_{\text{BB}_k}\). Let \({\varvec{\Theta }}_{g,k}\) and \({\varvec{\Theta }}_{in,k}\) be defined, respectively, as

$$\begin{aligned} {\varvec{\Theta }}_{g,k}= & {} \sigma ^2_g\textbf{A}^H(\varphi _g,k)\textbf{v}_k\textbf{v}_k^H\textbf{A}(\varphi _g,k), \end{aligned}$$
(13)
$$\begin{aligned} {\varvec{\Theta }}_{in,k}= & {} \sum \limits _{j=1}^J\sigma ^2_j\textbf{A}^H(\varphi _j,k)\textbf{v}_k\textbf{v}_k^H\textbf{A}(\varphi _j,k)+\frac{\sigma _{\bar{n}}^2}{\mathcal {P}_k}\Vert \textbf{v}_k \Vert _2^2\textbf{I}_{N_t}. \end{aligned}$$
(14)

Then, the SINR in (12) can be re-expressed as

$$\begin{aligned} \text{SINR}_{k,u}=\frac{\textbf{f}^H_{\text{BB}_{k,u}}\textbf{F}^H_{\text{RF}}{\varvec{\Theta }}_{g,k}\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}}{\textbf{f}^H_{\text{BB}_{k,u}}\textbf{F}^H_{\text{RF}}{\varvec{\Theta }}_{in,k}\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}}. \end{aligned}$$
(15)

It is worth noting that the optimization of SINR needs the knowledge of angles as well as the variances of reflection coefficients of the target and interferences. These parameters can be obtained from a cognitive paradigm [29,30,31] and in this work they are assumed to be known for simplicity.

2.5 Normalization

In this paper, the objective function is established as the weighted summation of the communication SE and radar SINR, which is defined as

$$\begin{aligned} \eta \text{SINR}+(1-\eta )\text{SE}, \end{aligned}$$
(16)

where \(0\le \eta \le 1\) is the weight parameter, which is used to indicate the relative importance of the two objective functions.

The weighting method indicates the relative importance of the two functions in the integrated system by setting the weight factor. However, the effectiveness of the above method is based on the condition that the two objective functions have the same range of variation. In other words, for two objective functions with inconsistent variation ranges, it is difficult to find suitable weights to compensate for the difference in amplitude and indicate relative importance. According to [32, 33], the way to deal with this defect is normalizing the objective function, which can be mathematically expressed as

$$\begin{aligned} \frac{\eta }{KU}\sum \limits _{k=1}^K\sum \limits _{u=1}^U\frac{\text{SINR}_{k,u}-\mathrm {SINR_{min}}}{\mathrm {SINR_{max}}-\mathrm {SINR_{min}}} +\frac{(1-\eta )}{K}\sum \limits _{k=1}^K\sum \limits _{u=1}^U\frac{\text{SE}_{k,u}-\mathrm {SE_{min}}}{\mathrm {SE_{max}}-\mathrm {SE_{min}}}, \end{aligned}$$
(17)

where the maximum and minimum values of SINR and SE of each user on each subcarrier, i.e., \(\mathrm {SINR_{max}}\), \(\mathrm {SE_{min}}\), \(\mathrm {SINR_{min}}\) and \(\mathrm {SE_{max}}\), are calculated by setting the tradeoff parameter to 0 and 1, respectively.

With the normalization, the problem of hybrid beamforming design for OFDM-DFRC is modeled as

$$\begin{aligned} \min \limits _{\begin{array}{c} \textbf{F}_{\text{RF}},\textbf{f}_{\text{BB}_{k,u}}\\ \textbf{u}_{k,u},\textbf{W}_{\text{RF}},\textbf{w}_{\text{BB}_k} \end{array}}&\sum \limits _{k=1}^K\sum \limits _{u=1}^U\beta _r\text{SINR}_{k,u}+\beta _c\text{SE}_{k,u}\\ \mathrm {s.t.}\,\,&\Vert \textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}\Vert ^2_2=\frac{\mathcal {P}_k}{U}, \forall k,u\\&\vert \textbf{F}_{\text {RF}}(i,j) \vert =1,\forall i,j,\\&\vert \textbf{W}_{\text {RF}}(i,j) \vert =1,\forall i,j, \end{aligned}$$
(18)

where \(\beta _r=-\eta /KU(\mathrm {SINR_{max}}-\mathrm {SINR_{min}})\), \(\beta _c=-(1-\eta )/K(\mathrm {SE_{max}}-\mathrm {SE_{min}})\). Obviously, since the non-convex objective function and the coupling of the digital and analog precoder in the constraint terms, the problem (18) is difficult to solve. To effectively tackle the problem, a CADMM algorithm is developed.

3 Methods

In this paper, the hybrid beamforming transmitter, hybrid beamforming radar receiver, and user receiver are jointly optimized to minimize the objective function in problem (18). Due to the objective function containing multiple variables, this joint optimization problem is complicated. This motivates us to apply alternate optimization strategies [24, 34, 35], to decouple and simplify this multi-variable joint optimization problem. Specifically, we first alternately optimize the hybrid beamforming transmitters \(\textbf{F}_{\text{RF}}\) and \(\textbf{F}_{\text{BB}_{k}}\) and user receivers \(\textbf{u}_{k,u}\) by assuming fixed hybrid beamforming radar receiver. Next, we design the hybrid beamforming radar receivers \(\textbf{W}_{\text{RF}}\) and \(\textbf{w}_{\text{BB}_{k}}\) for given transmitter and user receiver.

3.1 Hybrid transmitter and user receiver design

In this section, we update \(\textbf{F}_{\text{RF}}\), \(\textbf{F}_{\text{BB}_{k}}\) and \(\textbf{u}_{k,u}\), based on the given hybrid beamforming radar receiver \(\textbf{W}_{\text{RF}}\) and \(\textbf{w}_{\text{BB}_{k}}\), which removes the constraint \(\vert \textbf{W}_{\text {RF}}(i,j) \vert =1\). Furthermore, we introduce auxiliary variables \(\textbf{f}_{k,u}=\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}},\forall k,u\) to decouple the analog precoder and digital precoder, and then add it as coupling constraints (19b) to the problem (18):

$$\begin{aligned}&\min \limits _{\begin{array}{c} \textbf{F}_{\text{RF}},\textbf{f}_{\text{BB}_{k,u}} \\ \textbf{u}_{k,u},\textbf{f}_{k,u} \end{array}}&\sum \limits _{k=1}^K\sum \limits _{u=1}^U\beta _r\text{SINR}_{k,u}+\beta _c\text{SE}_{k,u}, \end{aligned}$$
(19a)
$$\begin{aligned}&\mathrm {s.t.}&\textbf{f}_{k,u}= \textbf{F}_{\text{RF}}\textbf{f}_{\text {BB}_{k,u}}, \forall k,u, \end{aligned}$$
(19b)
$$\begin{aligned}{} & {} \Vert \textbf{f}_{k,u}\Vert ^2_2=\frac{\mathcal {P}_k}{U}, \forall k,u, \end{aligned}$$
(19c)
$$\begin{aligned}{} & {} \vert \textbf{F}_{\text {RF}}(i,j) \vert =1,\forall i,j, \end{aligned}$$
(19d)

where \(\text{SINR}_{k,u}\) and \(\text{SE}_{k,u}\) can be expressed as

$$\begin{aligned}&\text{SINR}_{k,u}=\frac{\textbf{f}^H_{k,u}{\varvec{\Theta }}_{g,k}\textbf{f}_{k,u}}{\textbf{f}^H_{k,u}{\varvec{\Theta }}_{in,k}\textbf{f}_{k,u}}, \end{aligned}$$
(20)
$$\begin{aligned}&\text{SE}_{k,u} = \text{log}_2\left( 1+\frac{\textbf{f}^H_{k,u}\overline{\textbf{H}}_{k,u}\textbf{f}_{k,u}}{\sum \limits _{i\ne u}^U\textbf{f}^H_{k,i}\overline{\textbf{H}}_{k,u}\textbf{f}_{k,i}+\sigma ^2_{\hat{n}}\textbf{u}^H_{k,u}\textbf{u}_{k,u}}\right) , \end{aligned}$$
(21)

with \(\overline{\textbf{H}}_{k,u}\) being defined as

$$\begin{aligned} \overline{\textbf{H}}_{k,u}=\textbf{H}_{k,u}^H\textbf{u}_{k,u}\textbf{u}_{k,u}^H\textbf{H}_{k,u}. \end{aligned}$$
(22)

To deal with the resulting problem, the CADMM algorithm [24] is employed. Specifically, the augmented Lagrangian function of (19) is given in (23) shown as follows:

$$\begin{aligned} \mathcal {L}(\textbf{f}_{k,u},\textbf{u}_{k,u},\textbf{f}_{\text{BB}_{k,u}},\textbf{F}_{\text{RF}}, \textbf{d}_{k,u})=\sum \limits _{k=1}^K\sum \limits _{u=1}^U\mathcal {L}_{k,u}(\textbf{f}_{k,u}, \textbf{u}_{k,u}, \textbf{f}_{\text{BB}_{k,u}},\textbf{F}_{\text{RF}},\textbf{d}_{k,u}), \end{aligned}$$
(23)

and

$$\begin{aligned} \mathcal {L}_{k,u}(\textbf{f}_{k,u}, \textbf{u}_{k,u}, \textbf{f}_{\text{BB}_{k,u}},\textbf{F}_{\text{RF}},\textbf{d}_{k,u}) =F(\textbf{f}_{k,u})+\frac{\rho }{2}\Vert \textbf{f}_{k,u}-\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}+\textbf{d}_{k,u} \Vert _2^2, \end{aligned}$$
(24)

where \(F(\textbf{f}_{k,u})=\beta _r\text{SINR}_{k,u}+\beta _c\text{SE}_{k,u}\), \(\textbf{d}_{k,u}\in \mathbb {C}^{N_t}\) and \(\rho >0\) are the dual variables and corresponding penalty parameters, respectively. Under the CADMM framework, we first update the original variable to minimize the Lagrangian function, and then update the dual variable and penalty parameters [36]. More precisely, at the \((m+1)\)-th iteration, the update produces of the corresponding CADMM are given below:

$$\begin{aligned} \textbf{f}^{(m+1)}_{k,u}= & {} \mathop {\text{argmin}}_{\textbf{f}_{k,u}\in \mathcal {D}_{\textbf{f}_{k,u}}}\mathcal {L}_{k,u}(\textbf{f}_{k,u}, \textbf{u}^{(m)}_{k,u},\textbf{f}^{(m)}_{\text{BB}_{k,u}},\textbf{F}^{(m)}_{\text{RF}}, \textbf{d}^{(m)}_{k,u}), \end{aligned}$$
(25a)
$$\begin{aligned} \textbf{u}^{(m+1)}_{k,u}= & {} \mathop {\text{argmin}}_{\textbf{u}_{k,u}}\mathcal {L}_{k,u}(\textbf{f}^{(m+1)}_{k,u},\textbf{u}_{k,u},\textbf{f}^{(m+1)}_{\text{BB}_{k,u}},\textbf{F}^{(m)}_{\text{RF}},\textbf{d}^{(m)}_{k,u}), \end{aligned}$$
(25b)
$$\begin{aligned} \textbf{f}^{(m+1)}_{\text{BB}_{k,u}}= & {} \mathop {\text{argmin}}_{\textbf{f}_{\text{BB}_{k,u}}}\mathcal {L}_{k,u}(\textbf{f}^{(m+1)}_{k,u},\textbf{u}^{(m)}_{k,u}, \textbf{f}_{\text{BB}_{k,u}},\textbf{F}^{(m)}_{\text{RF}},\textbf{d}^{(m)}_{k,u}), \end{aligned}$$
(25c)
$$\begin{aligned} \textbf{F}^{(m+1)}_{\text{RF}}= & {} \mathop {\text{argmin}}_{\textbf{F}_{\text{RF}}\in \mathcal {D}_{\textbf{F}_{\text{RF}}}}\mathcal {L}(\textbf{f}^{(m+1)}_{k,u},\textbf{u}^{(m+1)}_{k,u}, \textbf{f}^{(m+1)}_{\text{BB}_{k,u}},\textbf{F}_{\text{RF}},\textbf{d}^{(m)}_{k,u}), \end{aligned}$$
(25d)
$$\begin{aligned} \textbf{d}^{(m+1)}_{k,u}= & {} \textbf{d}^{(m)}_{k,u}+\textbf{f}^{(m+1)}_{k,u}-\textbf{F}^{(m+1)}_{\text{RF}}\textbf{f}^{(m+1)}_{\text{BB}_{k,u}}, \end{aligned}$$
(25e)

where the sets \(\mathcal {D}_{\textbf{f}_{k,u}}\) and \(\mathcal {D}_{\textbf{F}_{\text{RF}}}\) are, respectively, defined as

$$\begin{aligned} \mathcal {D}_{\textbf{f}_{k,u}}= & {} \left\{ \textbf{f}_{k,u}\mid \Vert \textbf{f}_{k,u}\Vert ^2_2=\frac{\mathcal {P}_k}{U}\right\} ,\\ \mathcal {D}_{\textbf{F}_{\text{RF}}}= & {} \{ \textbf{F}_{\text{RF}} \mid \vert \textbf{F}_{\text {RF}}(i,j) \vert =1 \}. \end{aligned}$$

The above equations are repeated until some termination conditions are satisfied. For example, when the maximum number of iterations \(N_{max}\) is reached. Next, the solutions to the sub-problems are presented.

3.1.1 Update of \(\{\textbf{f}_{k,u}\}\)

Given \(\textbf{u}^{(m)}_{k,u}\), \(\textbf{f}^{(m)}_{\text{BB}_{k,u}}\), \(\textbf{F}^{(m)}_{\text{RF}}\) and \(\textbf{d}^{(m)}_{k,u}\), the sub-problem with respect to \(\textbf{f}_{k,u}\) is given by

$$\begin{aligned} \min \limits _{\textbf{f}_{k,u}}\,\,&F(\textbf{f}_{k,u})+\frac{\rho }{2}\Vert \textbf{f}_{k,u}-\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}+\textbf{d}_{k,u} \Vert _2^2 \\ \mathrm {s.t.}\,\,&\Vert \textbf{f}_{k,u}\Vert ^2_2=\frac{\mathcal {P}_k}{U} \end{aligned}$$
(26)

which is difficult to solve due to the non-convex function \(F(\textbf{f}_{k,u})\). In this work, the problem is tackled by minimizing a upper bound function of \(F(\textbf{f}_{k,u})\), inspired by the BSUM method [23]. More precisely, the quadratic upper bound function of \(F(\textbf{f}_{k,u})\) at the current point \(\textbf{f}_{k,u}^{(m)}\) can be written as

$$\begin{aligned} f(\textbf{f}_{k,u})&=F(\textbf{f}^{(m)}_{k,u})+2Re\{\nabla ^HF(\textbf{f}_{k,u}^{(m)})(\textbf{f}_{k,u}-\textbf{f}^{(m)}_{k,u})\}\\&\quad +\frac{1}{2}\begin{pmatrix}\textbf{f}_{k,u}-\textbf{f}^{(m)}_{k,u}\\ \textbf{f}^*_{k,u}-\textbf{f}^{*(m)}_{k,u}\end{pmatrix}^H{\varvec{\Psi }}_{k,u}^{(m)}\begin{pmatrix}\textbf{f}_{k,u}-\textbf{f}^{(m)}_{k,u}\\ \textbf{f}^*_{k,u}-\textbf{f}^{*(m)}_{k,u}\end{pmatrix}, \end{aligned}$$
(27)

where \(\nabla F(\textbf{f}_{k,u})\) is the partial derivative of \(\textbf{f}^*_{k,u}\) and \({\varvec{\Psi }}_{k,u}\) is a positive semi-definite matrix which should be selected such that \({\varvec{\Psi }}_{k,u}-\mathcal {H}_{k,u}\) is also positive semi-definite. Here, \(\mathcal {H}_{k,u}\) denotes the Hessian matrix of \(F(\textbf{f}_{k,u})\) at the current point \(\textbf{f}_{k,u}^{(m)}\). \(\nabla F(\textbf{f}_{k,u})\) and \(\mathcal {H}_{k,u}\) are deviated in Appendix A. In order to make sure that \({\varvec{\Psi }}_{k,u}-\mathcal {H}_{k,u}\) is positive semi-definite, we can choose \({\varvec{\Psi }}_{k,u}=\lambda _{\text{max}}\textbf{I}\), and \(\lambda _{\text{max}}\) denotes the largest eigenvalue of \(\mathcal {H}_{k,u}\). Furthermore, (27) can be converted into the following form which only contains \(\textbf{f}_{k,u}\):

$$\begin{aligned} f(\textbf{f}_{k,u})&=F(\textbf{f}^{(m)}_{k,u})+2Re\{\nabla ^HF(\textbf{f}_{k,u}^{(m)})(\textbf{f}_{k,u}-\textbf{f}^{(m)}_{k,u})\}\\&\quad +\frac{\lambda ^{(m)}_{\text{max}}}{2}\Vert \textbf{f}_{k,u}-\textbf{f}^{(m)}_{k,u} \Vert _2^2+ \frac{\lambda ^{(m)}_{\text{max}}}{2}\Vert \textbf{f}^*_{k,u}-\textbf{f}^{*(m)}_{k,u} \Vert _2^2\\&=F(\textbf{f}^{(m)}_{k,u})+2Re\{\nabla ^HF(\textbf{f}_{k,u}^{(m)})(\textbf{f}_{k,u}-\textbf{f}^{(m)}_{k,u})\} +\lambda ^{(m)}_{\text{max}}\Vert \textbf{f}_{k,u}-\textbf{f}^{(m)}_{k,u} \Vert _2^2. \end{aligned}$$
(28)

Replacing \(F(\textbf{f}_{k,u})\) in (26) by (28) yields the optimization problem as

$$\begin{aligned}{} & {} \min \limits _{\textbf{f}_{k,u}}\,\,\, 2Re\{\nabla ^HF(\textbf{f}_{k,u}^{(m)})(\textbf{f}_{k,u}-\textbf{f}^{(m)}_{k,u})\} +\lambda ^{(m)}_{\text{max}}\Vert \textbf{f}_{k,u}-\textbf{f}^{(m)}_{k,u} \Vert _2^2\nonumber \\{} & {} \qquad +\frac{\rho }{2}\Vert \textbf{f}_{k,u}-\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}+\textbf{d}_{k,u} \Vert _2^2, \end{aligned}$$
(29a)
$$\begin{aligned}{} & {} \mathrm {s.t.}\,\,\,\Vert \textbf{f}_{k,u}\Vert ^2_2=\frac{\mathcal {P}_k}{U}. \end{aligned}$$
(29b)

which can be further transformed into the following optimization problem (details can be found in Appendix B):

$$\begin{aligned} \min \limits _{\textbf{f}_{k,u}}&\frac{\rho }{2}\left\| \textbf{f}_{k,u}-\left( \textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}-\textbf{d}_{k,u}-\frac{2\nabla F(\textbf{f}_{k,u})}{\rho }\right) \right\| _2^2 +\lambda ^{(m)}_{\text{max}}\Vert \textbf{f}_{k,u}-\textbf{f}^{(m)}_{k,u}, \Vert _2^2\\ \mathrm {s.t.}&\Vert \textbf{f}_{k,u}\Vert ^2_2=\frac{\mathcal {P}_k}{U}. \end{aligned}$$
(30)

It is seen that the above problem have a closed-form optimal solution, which can be obtained by utilizing Karush–Kuhn–Tucker (KKT) conditions, as

$$\begin{aligned} \textbf{f}_{k,u}^{(m+1)}=\sqrt{\frac{\mathcal {P}_k}{U}}\frac{\tilde{\textbf{f}}_{k,u}^{(m+1)}}{\Vert \tilde{\textbf{f}}_{k,u}^{(m+1)} \Vert _2}, \end{aligned}$$
(31)

where \(\tilde{\textbf{f}}_{k,u}^{(m+1)}\) is defined as

$$\begin{aligned} \tilde{\textbf{f}}_{k,u}^{(m+1)} = \frac{\frac{\rho }{2}\left( \textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}-\textbf{d}_{k,u}-\frac{2\nabla F(\textbf{f}_{k,u})}{\rho }\right) +\lambda ^{(m)}_{\text{max}}\textbf{f}_{k,u}^{(m)}}{\frac{\rho }{2}+\lambda ^{(m)}_{\text{max}}}. \end{aligned}$$
(32)

3.1.2 Update of \(\{\textbf{u}_{k,u}\}\)

The subproblem (25b) is to updating \(\textbf{u}_{k,u}\) to maximize the SE. In fact, the problem of maximizing SE is the same as the problem of optimizing the SINR of the user’s received signal. Based on this observation, we can obtain \(\textbf{u}_{k,u}\) by solving the following problem

$$\begin{aligned}&\min \limits _{\textbf{u}_{k,u}}&\frac{\textbf{u}^H_{k,u}\varvec{\Xi }_{k,u}\textbf{u}_{k,u}}{\textbf{u}^H_{k,u}\overline{\varvec{\Xi }}_{k,u}\textbf{u}_{k,u}}, \end{aligned}$$
(33)

where \(\varvec{\Xi }_{k,u}\) and \(\overline{\varvec{\Xi }}_{k,u}\) are defined as

$$\begin{aligned}\varvec{\Xi }_{k,u}& = \textbf{H}_{k,u}\textbf{F}_{\text{RF}}\textbf{f}_{\text {BB}_{k,u}}\textbf{f}^H_{\text {BB}_{k,u}}\textbf{F}^H_{\text{RF}}\textbf{H}_{k,u}^H, \\ \overline{\varvec{\Xi }}_{k,u} &= \sum \limits _{i\ne u}^U\textbf{H}_{k,u}\textbf{F}_{\text{RF}}\textbf{f}_{\text {BB}_{k,i}}\textbf{f}^H_{\text {BB}_{k,i}}\textbf{F}^H_{\text{RF}}\textbf{H}_{k,u}^H+\sigma ^2_{\hat{n}}\textbf{I}_{N_{cr}}. \end{aligned}$$
(34)

Obviously, problem (33) is a unconstrained maximization problem, whose optimal solution is the maximum generalized eigenvalue of \((\varvec{\Xi }_{k,u},\overline{\varvec{\Xi }}_{k,u})\) or the principal eigenvector of \(\overline{\varvec{\Xi }}^{-1}_{k,u}\varvec{\Xi }_{k,u}\).

3.1.3 Update of \(\{\textbf{f}_{\text{BB}_{k,u}}\}\)

When the analog precoder \(\textbf{F}_{\text{RF}}\), auxiliary variables \(\textbf{f}_{k,u}\) and user receiver \(\textbf{u}_{k,u}\) are all fixed, the minimization of (25c) with respect to \(\textbf{f}_{\text{BB}_{k,u}}\) becomes

$$\begin{aligned}&\min \limits _{\textbf{f}_{\text{BB}_{k,u}}}&\Vert \textbf{f}_{k,u}-\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}+\textbf{d}_{k,u} \Vert _2^2, \end{aligned}$$
(35)

whose closed-form solution is

$$\begin{aligned} \textbf{f}^{(m+1)}_{\text{BB}_{k,u}}=(\textbf{F}^H_{\text{RF}}\textbf{F}_{\text{RF}})^{-1}\textbf{F}^H_{\text{RF}}(\textbf{d}_{k,u}+\textbf{f}_{k,u}). \end{aligned}$$
(36)

3.1.4 Update of \(\textbf{F}_{\text{RF}}\)

The analog precoder \(\textbf{F}_{\text{RF}}\) appears only in the second term of the objective (24) with other fixed blocks. Therefore, the sub-problem with respect to \(\textbf{F}_{\text{RF}}\) is given by

$$\begin{aligned} \min \limits _{\textbf{F}_{\text{RF}}}\,\,&\sum \limits _{k=1}^K\sum \limits _{u=1}^U\Vert \textbf{f}_{k,u}-\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}+\textbf{d}_{k,u}\Vert _2^2,\\ \mathrm {s.t.}\,\,&\vert \textbf{F}_{\text{RF}}(i,j)\vert =1,\forall i,j. \end{aligned}$$
(37)

Since the contribution of each row element of \(\textbf{F}_{\text{RF}}\) to the objective function is independent, we can transform the problem into the following form:

$$\begin{aligned} \min \limits _{\textbf{F}_{\text{RF}}}\,\,&\sum \limits _{i=1}^{N_t}\sum \limits _{k=1}^K\sum \limits _{u=1}^U\Vert \textbf{b}_{k,u}(i)-\textbf{F}_{\text{RF}}(i,:)\textbf{f}_{\text{BB}_{k,u}}\Vert _2^2,\\ \mathrm {s.t.}\,\,&\vert \textbf{F}_{\text{RF}}(i,j)\vert =1,\forall i,j, \end{aligned}$$
(38)

where \(\textbf{b}_{k,u}=\textbf{f}_{k,u}+\textbf{d}_{k,u}\). The above problem can be divided into \(N_t\) parallel sub-problems, the i-th sub-problem is expressed as

$$\begin{aligned} \min \limits _{\textbf{F}^T_{\text{RF}}(i,:)}\,\,&\sum \limits _{k=1}^K\sum \limits _{u=1}^U\Vert \textbf{b}_{k,u}(i)-\textbf{f}^T_{\text{BB}_{k,u}}\textbf{F}^T_{\text{RF}}(i,:)\Vert _2^2,\\ \mathrm {s.t.}\,\,&\vert \textbf{F}^T_{\text{RF}}(i,j)\vert =1,\forall i,j. \end{aligned}$$
(39)

Problem (39) is similar to the constant modulus waveform design problem, which can be solved by a phase-only beamforming algorithm. First, we rewrite the objective function into the following form:

$$\begin{aligned}&\sum \limits _{k=1}^K\sum \limits _{u=1}^U\Vert \textbf{b}_{k,u}(i)-\textbf{f}^T_{\text{BB}_{k,u}}\textbf{F}^T_{\text{RF}}(i,:)\Vert _2^2\\&\quad =\sum \limits _{k=1}^K\sum \limits _{u=1}^U(\textbf{b}_{k,u}(i)-\textbf{f}^T_{\text{BB}_{k,u}}\textbf{F}^T_{\text{RF}}(i,:))^H(\textbf{b}_{k,u}(i)-\textbf{f}^T_{\text{BB}_{k,u}}\textbf{F}^T_{\text{RF}}(i,:))\\&\quad =\sum \limits _{k=1}^K\sum \limits _{u=1}^U\textbf{b}^H_{k,u}(i)\textbf{b}_{k,u}(i)+\textbf{F}^*_{\text{RF}}(i,:)\sum \limits _{k=1}^K\sum \limits _{u=1}^U\textbf{f}^*_{\text{BB}_{k,u}}\textbf{f}^T_{\text{BB}_{k,u}}\textbf{F}^T_{\text{RF}}(i,:)\\&\quad -\sum \limits _{k=1}^K\sum \limits _{u=1}^U\textbf{b}^T_{k,u}(i)\textbf{f}^T_{\text{BB}_{k,u}}\textbf{F}^T_{\text{RF}}(i,:)-\textbf{F}^*_{\text{RF}}(i,:)\sum \limits _{k=1}^K\sum \limits _{u=1}^U\textbf{f}^*_{\text{BB}_{k,u}}\textbf{b}_{k,u}(i)\\&\quad =\textbf{f}_{\text{RF}}^H\textbf{P}\textbf{f}_{\text{RF}}-\textbf{q}^H\textbf{f}_{\text{RF}}-\textbf{f}_{\text{RF}}^H\textbf{q}+r, \end{aligned}$$

where \(\textbf{f}_{\text{RF}}=\textbf{F}^T_{\text{RF}}(i,:)\), \(\textbf{P}=\mathop {\sum }\limits _{k=1}^{K}\mathop {\sum }\limits _{u=1}^{U}\textbf{f}^{*}_{\text{BB}_{k,u}}\textbf{f}^T_{\text{BB}_{k,u}}\), \(\textbf{q}=\mathop {\sum }\limits _{k=1}^{K}\mathop {\sum }\limits _{u=1}^{U}\textbf{f}^{*}_{\text{BB}_{k,u}}\textbf{b}_{k,u}(i)\) and \(r=\sum \limits _{k=1}^K\sum \limits _{u=1}^U\textbf{b}^H_{k,u}(i)\textbf{b}_{k,u}(i)\). Furthermore, problem (39) can be recast as

$$\begin{aligned}&\min \limits _{\textbf{f}_{\text{RF}}}&\Vert \textbf{P}^{\frac{-1}{2}}\textbf{q}-\textbf{P}^{\frac{1}{2}}\textbf{f}_{\text{RF}}\Vert _2^2, \end{aligned}$$
(40a)
$$\begin{aligned}&\mathrm {s.t.}&\vert \textbf{f}_{\text{RF}}(n)\vert =1,\quad n=1,...,N_{\text{RF}_t}. \end{aligned}$$
(40b)

Obviously, (40) is a constant modulus least squares problem, and its optimal solution can be obtained by the VSGP algorithm [37], the successive closed forms algorithm [38], or the fixed step size gradient projection algorithm [39]. In this article, we use the VSGP algorithm. According to the above steps, the proposed CADMM algorithm for designing the hybrid transmitter and user receiver is summarized in Algorithm 1.

figure a

3.2 Hybrid radar receiver design

In this section, we update the radar hybrid receivers \(\textbf{W}_{\text{RF}}\) and \(\textbf{w}_{\text{BB}_k}\) for fixed hybrid transmitters. In this case, the sub-problem of \(\textbf{W}_{\text{RF}}\) and \(\textbf{w}_{\text{BB}_k}\) can be formulated as a non-weighted problem, i.e.

$$\begin{aligned}&\max \limits _{\textbf{W}_{\text{RF}},\textbf{w}_{\text{BB}_k}}&\sum \limits _{k=1}^K\text{SINR}_k, \end{aligned}$$
(41a)
$$\begin{aligned}&\mathrm {s.t.}&\vert \textbf{W}_{\text{RF}}(i,j)\vert =1,\forall i,j. \end{aligned}$$
(41b)

Due to the objective function of problem (41) is a nonconvex function, this implies that it is impossible to directly optimize \(\textbf{W}_{\text{RF}}\) and \(\textbf{w}_{\text{BB}_k}\). Therefore, we first use generalized eigenvalue decomposition to obtain the optimal digital receiver \(\textbf{v}_k\) for each subcarrier, and then update \(\textbf{W}_{\text{RF}}\) and \(\textbf{w}_{\text{BB}_k}\) with the least square method. For the definition \(\textbf{v}_{k}=\textbf{W}_{\text{RF}}\textbf{w}_{\text{BB}_k}\), \(\forall k\), problem (41) can be formulated as

$$\begin{aligned} \max \limits _{\textbf{v}_{k}}\sum \limits _{k=1}^K\frac{\textbf{v}^H_{k}{\varvec{\Upsilon }}_{g,k}\textbf{v}_{k}}{\textbf{v}^H_{k}{\varvec{\Upsilon }}_{in,k}\textbf{v}_{k}}, \end{aligned}$$
(42)

Where \({\varvec{\Upsilon }}_{g,k}\) and \({\varvec{\Upsilon }}_{in,k}\) are defined as follows

$$\begin{aligned} {\varvec{\Upsilon }}_{g,k} =&\sigma ^2_g\textbf{A}(\varphi _g,k)\textbf{F}_{\text{RF}}\textbf{F}_{\text{BB}_k}\textbf{F}_{\text{BB}_k}^H\textbf{F}_{\text{RF}}^H\textbf{A}^H(\varphi _g,k), \end{aligned}$$
(43)
$$\begin{aligned} {\varvec{\Upsilon }}_{in,k}=&\sum \limits _{j=1}^J\sigma ^2_j\textbf{A}(\varphi _j,k)\textbf{F}_{\text{RF}}\textbf{F}_{\text{BB}_k}\textbf{F}_{\text{BB}_k}^H\textbf{F}_{\text{RF}}^H\textbf{A}^H(\varphi _j,k)+\sigma ^2_{\bar{n}}\textbf{I}_{N_{rr}}. \end{aligned}$$
(44)

Obviously, we can easily obtain the optimal solution of the problem (42) by the maximum generalized eigenvalue of \(({\varvec{\Upsilon }}_{g,k},{\varvec{\Upsilon }}_{in,k})\) or the principal eigenvector of \({\varvec{\Upsilon }}^{-1}_{in,k}{\varvec{\Upsilon }}_{g,k}\).

Then, we update \(\textbf{W}_{\text{RF}}\) and \(\textbf{w}_{\text{BB}_k}\) by solving the following problem

$$\begin{aligned}&\min \limits _{\textbf{W}_{\text{RF}},\textbf{w}_{\text{BB}_k}}&\sum \limits _{k=1}^K\Vert \textbf{v}_k-\textbf{W}_{\text{RF}}\textbf{w}_{\text{BB}_k} \Vert _2^2, \end{aligned}$$
(45a)
$$\begin{aligned}&\mathrm {s.t.}&\vert \textbf{W}_{\text{RF}}(i,j)\vert =1,\forall i,j. \end{aligned}$$
(45b)

For the above problem, we can design the analog combiner \(\textbf{W}_{\text{RF}}\) and the digital combiner \(\textbf{w}_{\text{BB}_k}\) with an alternate optimization algorithm.

3.2.1 Update of \(\{\textbf{w}_{\text{BB}_{k}}\}\)

We first consider the design of the digital combiner \(\textbf{w}_{\text{BB}_k}\) based on the fixed analog combiner \(\textbf{W}_{\text{RF}}\). Therefore, problem (45) can be rewritten as an unconstrained problem, i.e.

$$\begin{aligned}&\min \limits _{\textbf{W}_{\text{RF}},\textbf{w}_{\text{BB}_k}}&\sum \limits _{k=1}^K\Vert \textbf{v}_k-\textbf{W}_{\text{RF}}\textbf{w}_{\text{BB}_k} \Vert _2^2, \end{aligned}$$
(46)

whose optimal solution is

$$\begin{aligned} \textbf{w}_{\text{BB}_k}=(\textbf{W}^H_{\text{RF}}\textbf{W}_{\text{RF}})^{-1}\textbf{W}_{\text{RF}}\textbf{v}_k. \end{aligned}$$
(47)

3.2.2 Update of \(\textbf{W}_{\text{RF}}\)

Similar to the analog beamformer \(\textbf{F}_{\text{RF}}\), with given other beamformers, the sub-problem to \(\textbf{W}_{\text{RF}}\) can be recast as a parallel sub-problem. Specifically, for the i-th row of \(\textbf{W}_{\text{RF}}\), the sub-problem can be expressed as

$$\begin{aligned}&\min \limits _{\textbf{W}_{\text{RF}}(i,:)}&\sum \limits _{k=1}^K\Vert \textbf{v}_k(i)-\textbf{w}^T_{\text{BB}_k}\textbf{W}^T_{\text{RF}}(i,:) \Vert _2^2, \end{aligned}$$
(48a)
$$\begin{aligned}&\mathrm {s.t.}&\vert \textbf{W}_{\text{RF}}(i,j)\vert =1,\forall i,j. \end{aligned}$$
(48b)

Further, this problem can be rewritten as

$$\begin{aligned}&\min \limits _{\textbf{w}_{\text{RF}}}&\Vert \overline{\textbf{P}}^{\frac{-1}{2}}\overline{\textbf{q}}-\overline{\textbf{P}}^{\frac{1}{2}}\textbf{w}_{\text{RF}}\Vert _2^2, \end{aligned}$$
(49a)
$$\begin{aligned}&\mathrm {s.t.}&\vert \textbf{w}_{\text{RF}}(n)\vert =1,\quad n=1,...,N_{\text{RF}_r}, \end{aligned}$$
(49b)

where \(\overline{\textbf{q}}=\sum \limits _{k=1}^K\textbf{w}_{\text{BB}_k}^*\textbf{v}_{k}(i)\) and \(\overline{\textbf{P}}=\sum \limits _{k=1}^K\textbf{w}_{\text{BB}_k}^*\textbf{w}_{\text{BB}_k}^T\). Problem (49) can also be solved by the VSGP algorithm.

figure b
figure c

Finally, the proposed alternating optimization algorithm for the hybrid radar receiver is summarized in Algorithm 2. In fact, the SINR of the radar received signal is affected by the hybrid transmitter and hybrid receiver, which implies that we need to alternately optimize the hybrid transmitter and hybrid receiver. Therefore, the final algorithm for the five variables is summarized in Algorithm 3.

4 Performance analysis

4.1 Convergence analysis

Let \(g^k\) be the sum of SE and SINR values obtained by Algorithm 3 at the k-th iteration. Since the update of each block in the method with other variable fixed, the objective in (19) monotonically non-increasing after each iteration, it also means that the sum of SINR and SE is monotonically non-decreasing. i.e.

$$\begin{aligned} g^k \ge g^{k-1} . \end{aligned}$$
(50)

Then, we can conclude the proposed algorithm will converge to at least a local optimum since the value of the objective in (19) is bounded. This will be verified by simulations in the next section.

4.2 Complexity analysis

The main computational complexities of consensus-ADMM method for hybrid transmitter are contributed by solving subproblems (30), (35) and (37). More precisely, solving (32) needs updating \(\textbf{u}_{k,u}\) and \(\textbf{f}_{k,u}\). The update of \(\textbf{u}_{k,u}\) need \(\mathcal {O}\left( N_{cr}^{3}\right)\), and the update of \(\textbf{f}_{k,u}\) according to (32) needs to compute \(\Delta F(\textbf{f}_{k,u})\) and \({\varvec{\Psi }}_{k,u}\). Their complexities are \(\mathcal {O}\left( N_t^{2}\right)\) and \(\mathcal {O}\left( N_t^{2}\right)\), so the complexities of \(\textbf{f}_{k,u}\) is \(\mathcal {O}\left( N_t^{2}\right)\). Solving (35) needs a complexity of \(\mathcal {O}\left( N_t^{3}\right)\). Solving (37) with VSGP method needs a complexity of \(\mathcal {O}\left( I_1 N_{\text{t}} N_{\text{RF}}^2\right)\), where \(I_1\) s the number of iterations required in VSGP method. To summarize, the overall complexity of the consensus-ADMM method is \(\mathcal {O}\left( I_0(N_{cr}^3+N_t^3 + I_1N_{\text{t}} N_{\text{RF}}^2)\right)\), where \(I_0\) is the number of consensus-ADMM iterations. Now we discuss the complexity of the hybrid receiver design method, which consists of three steps. The update of \(\textbf{v}_{k}\) need \(\mathcal {O}\left( N_{rr}^{3}\right)\), the update of \(\textbf{w}_{\text{BB}_k}\) need \(\mathcal {O}(N_{rr}^{3})\) and the update of \(\textbf{W}_{\text{RF}}\) with aid of VSGP method takes a complexity of \(\mathcal {O}(I_2 N_{\text{rr}} N_{\text{RF}_r}^2)\), \(I_2\) s the number of iterations required in VSGP method. Therefore, the overall complexity of the hybrid receiver design method is \(\mathcal {O}(I_3(N_{rr}^3+I_2N_{\text{rr}} N_{\text{RF}_r}^2))\), where \(I_3\) is the number of outer iterations. As a result, the overall complexity of the proposed algorithm is given by \(\mathcal {O}(I_4(I_0(N_{cr}^3+N_t^3 + I_1N_{\text{t}} N_{\text{RF}}^2)+I_3(N_{rr}^3+I_2N_{\text{rr}} N_{\text{RF}_r}^2)))\), where \(I_4\) is the number of outer iterations.

5 Results and discussion

In this section, we will provide numerical simulations to verify the performance of the proposed algorithms. The simulation parameters are set as follows unless specified otherwise. We consider that the DFRC system has \(N_t=64\) transmit antennas and \(N_{\text{RF}_t}=16\) RF chains. This transmitter serves \(U=4\) users, and each user is equipped with \(N_{cr}=4\) receive antennas. We assume that the receive antennas of the radar hybrid combiner is \(N_{rr}=32\) and \(N_{\text{RF}_r}=4\) RF chains. The center frequency of the DRFC system is set to \(f_{\text{cent}}=10\) GHz, and the frequency step of OFDM is \(\Delta f=20\) MHz. The number of subcarriers in the OFDM system is set to \(K=128\), and \(U=4\) data streams are transmitted on each subcarrier. Furthermore, we consider that per subcarrier channel with 5 clusters and 10 scatters per cluster in which the angels of arrival (departure) are generated according to Laplacian distribution with random mean cluster angels \(\bar{\theta }_i\in [0,2\pi )\) (\(\bar{\phi }_i\in [0,2\pi )\)) and angular spreads of 10 degrees within each cluster. The number of cyclic prefix lengths is \(D=16\). The delay \(\tau _i\) of each cluster is uniformly distributed in \([0, DT_s]\), with \(T_s=1\). We assume that the power per user of the transmitter on each subcarrier is the same, and the power on each subcarrier \(P_k=1\). Further, we assume that the radar target is located at \(\varphi _g=20^{\circ }\) and the power is \(\sigma _g^2=20\) dB. There are 2 interferences in the detection area, their directions are \(\varphi _1=-50^{\circ }\) and \(\varphi _2=40^{\circ }\), and the power of each interference is \(\sigma _j^2=30\) dB. The noise variance of the user and radar receiver are assumed to be \(\sigma _{\hat{n}}^2=0\) dB and \(\sigma _{\bar{n}}^2=0\) dB, respectively. For the proposed method, the initial penalty parameter and convergence tolerance are set to be \(\rho _{k,u}=0.3,\forall k,u\), \(\epsilon _{\text{CADMM}}=10^{-4}\) and \(\epsilon _{\text{ADMM}}=10^{-3}\), respectively. In addition, We set the convergence tolerance of VSGP to \(\epsilon _{\text{RF}}=10^{-5}\).

Fig. 1
figure 1

The convergence behaviors of the consensus-ADMM when \(N_t=64\), \(N_{\text{RF}}=16\) \(U=4\), and \(\eta =0.5\)

Fig. 2
figure 2

The communication spectral efficiency and radar signal-to-noise ratio versus different penalty coefficients

Fig. 3
figure 3

The normalized spectral efficiency and radar SINR versus tradeoff parameter \(\eta\) with the communication \(\text{SNR}=0\) dB

Fig. 4
figure 4

The convergence behaviors of the CADMM when\(N_t=64\), \(N_{\mathrm {RF_t}}=16\), \(N_{{cr}}=4\), \(U=4\) and \(N_{{rr}}=32\), \(N_{\mathrm {RF_r}}=4\)

Fig. 5
figure 5

The SINR of radar versus the for different numbers of radar RF chains with \(U=4\), \(N_{{t}}=32\), \(N_{\mathrm {RF_r}}=8\), \(N_{rr}=32\), and \(\eta =0.5\)

Fig. 6
figure 6

The transmit beampattern versus tradeoff parameter \(\eta\) with the \(\text{SNR}=0\) dB

Fig. 7
figure 7

The beampattern behaviors for different number of interference

Fig. 8
figure 8

The averaged spectral efficiency and radar SINR versus the number of users with \(N_t=64\), \(N_{\mathrm {RF_t}}=8\), \(N_{{cr}}=4\), \(N_{{rr}}=32\), \(N_{\mathrm {RF_r}}=4\), \(\eta =0.5\) and \(\text{SNR}=0\) dB

Fig. 9
figure 9

The spectral efficiency and radar SINR versus the number of RF chains with \(N_t=64\), \(U=4\), \(N_{{rr}}=32\), \(N_{\mathrm {RF_r}}=4\), \(\eta =0.5\) and \(\text{SNR}=0\) dB

Figure 1 reveals the sum of SE and SINR values versus the iteration number. The result shows that the objective value obtained by the consensus-ADMM decrease with the iteration number increasing. This verifies the effectiveness of the proposed consensus-ADMM. In addition, Fig. 1 also displays two direct criteria concerning communication and radar functions (i.e., SE and radar SINR) versus the iteration number. The result reveals that both communication and radar performance becomes better as the iteration number increasing. This illustrates the proposed method works well for the OFDM-DFRC system. This illustrates the proposed method works well for the OFDM-DFRC system.

Figure 2 shows the communication spectral efficiency and radar signal-to-noise ratio versus different penalty coefficients. From the figure, we can see that the system performance obtained under different penalty coefficients is same. Hence, when the value of the penalty coefficient is reasonable, the penalty coefficient has no effect on the algorithm.

Figure 3 analyzes the impact of the trade-off parameter \(\eta\) on the performance of the DFRC system. For the hybrid transmit beamformer, hybrid receive beamformer, and fully digital receiver, we, respectively, calculated the normalized radar receive SINR and communication SE. As shown in the figure, the curve of communication spectrum efficiency decreases as \(\eta\) increases, and the curve of radar receive SINR increases as the trade-off coefficient increases. Further, we can choose the appropriate trade-off parameter according to the intersection of the two lines.

Figure 4 compares the convergence performance of the algorithm based on the phase-only method and the algorithm using the block coordinate descent (BCD) method by considering \(\eta =0.5\). The result reveals that compared with the BCD method, the phase-only method has a significant convergence performance advantage, and the target performance achieved by the phase-only method is close to the performance achieved by the BCD method. In this method, we normalize the performance index, and the penalty coefficient \(\rho\) is constant. In the iterative process, the increase of the weight ratio of the penalty term leads to the change of \(\textbf{F}_{\text{RF}}\) and \(\textbf{F}_{\text{BB}}\), while SE is sensitive to the change of \(\textbf{F}_{\text{RF}}\) and \(\textbf{F}_{\text{BB}}\), resulting in the continuous rise of the SE curve.

In Fig. 5, we aim to show the spectral efficiency of the DFRC system versus signal-to-noise ratios (\(SNR=\mathcal {P}_k/\sigma ^2_{\hat{n}}\)) for different trade-off parameters \(\eta\). For comparison purposes, the fully digital beamformer is also provided. It can be observed in Fig. 3 that, the spectral efficiency decreases as the trade-off parameter \(\eta\) increases from 0 to 1.

Figure 6 demonstrates the synthesized transmit beam patterns for different trade-off parameters \(\eta\). From the results, we note that as the tradeoff parameter \(\eta\) increases, the achieved beam pattern has a higher energy concentration on the target angle. Combining the analysis of the influence of the above-mentioned trade-off parameters \(\eta\) on the spectrum efficiency, we should select an appropriate trade-off parameter to achieve the balance of communication and radar performance.

Figure 7 describes the transmit beam pattern under different number of interference. Further, we assume that the radar target is located at \(\varphi _{g}=20^{\circ }\) and the power is \(\sigma _{g}^{2}=20 \,\text{dB}\). There are 3 interferences in the detection area, their directions are \(\varphi _{1}=-50^{\circ }\), \(\varphi _{2}= 40^{\circ }\) and \(\varphi _{3}=-40^{\circ }\), each interference is \(\sigma _{j}^{2}=30 \,\text{dB}\). From the figure, we can conclude that the power of the transmit beam pattern at the target angle is not sensitive to the amount of interference items, because the performance indicator in the radar is SINR, which causes the transmit power to concentrate on the target azimuth.

In Figure 8, we describe the effect of the number of users on the performance of the fully digital beamformer and proposed method when \(N_t=64\), \(N_{\mathrm {RF_t}}=8\), \(N_{{cr}}=4\), \(N_{{rr}}=32\), \(N_{\mathrm {RF_r}}=4\), \(\eta =0.5\) and \(\text{SNR}=0\) dB. In this work, we focus on designing a hybrid beamformer that approaches the performance of a fully digital precoder while reducing the number of RF chains. It can be considered that the performance obtained by the fully digital precoder is the upper bound of the performance that hybrid beamforming can achieve. Specifically, we introduce the average spectrum efficiency (i.e., SE/U) to replace the spectrum efficiency per subcarrier. It can be observed in Fig. 8 that, as the U increases, the average spectral efficiency of the DFRC system for users becomes worse. This is because the increase in the number of users leads to a stronger MUI received by each user. Furthermore, we noticed that increasing U will cause deterioration of radar SINR.

Next, we display the impact of the transmitter RF chains \(N_{\text {RF}_t}\) on radar SINR and communication spectrum efficiency. It can be seen in Fig. 9 that, increasing the number of RF chains \(N_{\text {RF}_t}\) can improve the communication spectrum efficiency and radar SINR. In addition, it can be observed that as the number of RF chains increases, the performance gap between the proposed method and the fully digital method gradually decreases.

6 Conclusion

We considered the wideband OFDM-DFRC system with a hybrid beamforming transceiver and the user receiving filter in this work. For the joint optimization problem of multiple beamformers, a trade-off optimization problem consisting of communication spectrum efficiency and radar signal SINR is modeled. To solve the non-convex optimization problem, a CADMM method based on block successive upper-bound minimization method is proposed to update the hybrid beamforming transmitter and user receiver. Furthermore, an approach based on the phase-only algorithm is proposed to optimize the hybrid analog precoder. Numerical results show that the proposed method guarantees radar and communication performance. In addition, our simulation results show that the performance of the two systems can be well balanced by setting appropriate trade-off parameters. In addition, the results also show that as the number of RF chains increases, the performance gap between the fully connected hybrid beamformer and the all-digital beamformer gradually decreases.

Availability of data and materials

Not applicable.

Notes

  1. The real-valued forms of complex-valued vector \(\textbf{a}\) and matrix \(\textbf{A}\) are \(\textbf{a}_r=[\mathcal {R}\{a\}^T,\mathcal {I}\{a\}^T]^T\) and \(\textbf{A}_r=\begin{bmatrix} \mathcal {R}\{\textbf{A}\} &{}-\mathcal {I}\{\textbf{A}\} \\ \mathcal {I}\{\textbf{A}\} &{}\mathcal {R}\{\textbf{A}\} \end{bmatrix}^T\), respectively.

Abbreviations

ODFM:

Orthogonal frequency division multiplexing

DFRC:

Dual-function radar-communication system

SINR:

Signal-to-interference noise ratio

CADMM:

Consensus alternating direction method of multipliers

BSUM:

Block continuous upper bound minimization method

EM:

Electro-magnetic

WiMAX:

World interoperability for microwave access

LTE:

Long-term evolution

V2X:

Vehicle to everything

MIMO:

Multiple-input multiple-output

OTFS:

Orthogonal time frequency space

SE:

Spectral efficiency

VSGP:

Variable step size gradient projection

RF:

Radio frequency

ULA:

Uniform linear array

IFFT:

Inverse fast Fourier transform

FFT:

Fast Fourier transform

AWGN:

Additive white Gaussian noise

AoAs:

Angles of arrival

AoDs:

Angles of departure

KKT:

Karush–Kuhn–Tucker

BCD:

Block coordinate descent

References

  1. H. Griffiths, L. Cohen, S. Watts, E. Mokole, C. Baker, M. Wicks, S. Blunt, Radar spectrum engineering and management: technical and regulatory issues. Proc. IEEE 103(1), 85–102 (2015). https://doi.org/10.1109/JPROC.2014.2365517

    Article  Google Scholar 

  2. J. Choi, V. Va, N. Gonzalez-Prelcic, R. Daniels, C.R. Bhat, R.W. Heath, Millimeter-wave vehicular communication to support massive automotive sensing. IEEE Commun. Mag. 54(12), 160–167 (2016). https://doi.org/10.1109/MCOM.2016.1600071CM

    Article  Google Scholar 

  3. W. Roh, J.-Y. Seol, J. Park, B. Lee, J. Lee, Y. Kim, J. Cho, K. Cheun, F. Aryanfar, Millimeter-wave beamforming as an enabling technology for 5g cellular communications: theoretical feasibility and prototype results. IEEE Commun. Mag. 52(2), 106–113 (2014). https://doi.org/10.1109/MCOM.2014.6736750

    Article  Google Scholar 

  4. A. Hassanien, M.G. Amin, Y.D. Zhang, F. Ahmad, Dual-function radar-communications: Information embedding using sidelobe control and waveform diversity. IEEE Trans. Signal Process. 64(8), 2168–2181 (2016). https://doi.org/10.1109/TSP.2015.2505667

    Article  MathSciNet  MATH  Google Scholar 

  5. C. Sturm, W. Wiesbeck, Waveform design and signal processing aspects for fusion of wireless communications and radar sensing. Proc. IEEE 99(7), 1236–1259 (2011). https://doi.org/10.1109/JPROC.2011.2131110

    Article  Google Scholar 

  6. C. Sturm, W. Wiesbeck, Waveform Design for Joint Digital Beamforming Radar and MIMO Communications Operability. Principles of Waveform Diversity and Design. (SciTech Publishing, 2011)

  7. A. Hassanien, M.G. Amin, Y.D. Zhang, F. Ahmad, A dual function radar-communications system using sidelobe control and waveform diversity. In: Proc. IEEE Radar Conf. (RadarCon), pp. 1260–1263 (2015). https://doi.org/10.1109/RADAR.2015.7131188

  8. S.D. Blunt, P. Yatham, J. Stiles, Intrapulse radar-embedded communications. IEEE Trans. Aerosp. Electron. Syst. 46(3), 1185–1200 (2010). https://doi.org/10.1109/TAES.2010.5545182

    Article  Google Scholar 

  9. S.D. Blunt, J.G. Metcalf, C.R. Biggs, E. Perrins, Performance characteristics and metrics for intra-pulse radar-embedded communication. IEEE J. Sel. Areas Commun. 29(10), 2057–2066 (2011). https://doi.org/10.1109/JSAC.2011.111215

    Article  Google Scholar 

  10. F. Liu, L. Zhou, C. Masouros, A. Li, W. Luo, A. Petropulu, Toward dual-functional radar-communication systems: optimal waveform design. IEEE Trans. Signal Process. 66(16), 4264–4279 (2018). https://doi.org/10.1109/TSP.2018.2847648

    Article  MathSciNet  MATH  Google Scholar 

  11. Z. Cheng, Z. He, B. Liao, Hybrid beamforming design for OFDM dual-function radar-communication system. IEEE J. Sel. Topics Signal Process. (2021). https://doi.org/10.1109/JSTSP.2021.3118276

    Article  Google Scholar 

  12. P. Raviteja, Y. Hong, E. Viterbo, E. Biglieri, Effective diversity of otfs modulation. IEEE Wirel. Commun. Lett. 9(2), 249–253 (2020). https://doi.org/10.1109/LWC.2019.2951758

    Article  Google Scholar 

  13. C. Liu, S. Liu, Z. Mao, Y. Huang, H. Wang, Low-complexity parameter learning for otfs modulation based automotive radar. In: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8208–8212 (2021). https://doi.org/10.1109/ICASSP39728.2021.9414107

  14. P. Raviteja, K.T. Phan, Y. Hong, E. Viterbo, Interference cancellation and iterative detection for orthogonal time frequency space modulation. IEEE Trans. Wirel. Commun. 17(10), 6501–6515 (2018). https://doi.org/10.1109/TWC.2018.2860011

    Article  Google Scholar 

  15. Y. Yang, R.S. Blum, MIMO radar waveform design based on mutual information and minimum mean-square error estimation. IEEE Trans. Aerosp. Electron. Syst. 43(1), 330–343 (2007). https://doi.org/10.1109/TAES.2007.357137

    Article  Google Scholar 

  16. Y. Chen, Y. Nijsure, C. Yuen, Y.H. Chew, Z. Ding, S. Boussakta, Adaptive distributed MIMO radar waveform optimization based on mutual information. IEEE Trans. Aerosp. Electron. Syst. 49(2), 1374–1385 (2013). https://doi.org/10.1109/TAES.2013.6494422

    Article  Google Scholar 

  17. S. Sen, A. Nehorai, OFDM MIMO radar with mutual-information waveform design for low-grazing angle tracking. IEEE Trans. Signal Process. 58(6), 3152–3162 (2010). https://doi.org/10.1109/TSP.2010.2044834

    Article  MathSciNet  MATH  Google Scholar 

  18. Y. Li, S.A. Vorobyov, V. Koivunen, Ambiguity function of the transmit beamspace-based MIMO radar. IEEE Trans. Signal Process. 63(17), 4445–4457 (2015). https://doi.org/10.1109/TSP.2015.2439241

    Article  MathSciNet  MATH  Google Scholar 

  19. G. Cui, H. Li, M. Rangaswamy, MIMO radar waveform design with constant modulus and similarity constraints. IEEE Trans. Signal Process. 62(2), 343–353 (2014). https://doi.org/10.1109/TSP.2013.2288086

    Article  MathSciNet  MATH  Google Scholar 

  20. A. Aubry, A. DeMaio, A. Farina, M. Wicks, Knowledge-aided (potentially cognitive) transmit signal and receive filter design in signal-dependent clutter. IEEE Trans. Aerosp. Electron. Syst. 49(1), 93–117 (2013). https://doi.org/10.1109/TAES.2013.6404093

    Article  Google Scholar 

  21. S.M. Karbasi, A. Aubry, A. De Maio, M.H. Bastani, Robust transmit code and receive filter design for extended targets in clutter. IEEE Trans. Signal Process. 63(8), 1965–1976 (2015). https://doi.org/10.1109/TSP.2015.2404301

    Article  MathSciNet  MATH  Google Scholar 

  22. P. Stoica, H. He, J. Li, Optimization of the receive filter and transmit sequence for active sensing. IEEE Trans. Signal Process. 60(4), 1730–1740 (2012). https://doi.org/10.1109/TSP.2011.2179652

    Article  MathSciNet  MATH  Google Scholar 

  23. M. Hong, M. Razaviyayn, Z.-Q. Luo, J.-S. Pang, A unified algorithmic framework for block-structured optimization involving big data: with applications in machine learning and signal processing. IEEE Signal Process. Mag. 33(1), 57–77 (2016). https://doi.org/10.1109/MSP.2015.2481563

    Article  Google Scholar 

  24. S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  25. O.E. Ayach, S. Rajagopal, S. Abu-Surra, Z. Pi, R.W. Heath, Spatially sparse precoding in millimeter wave MIMO systems. IEEE Trans. Wirel. Commun. 13(3), 1499–1513 (2014). https://doi.org/10.1109/TWC.2014.011714.130846

    Article  Google Scholar 

  26. A. Alkhateeb, R.W. Heath, Frequency selective hybrid precoding for limited feedback millimeter wave systems. IEEE Trans. Commun. 64(5), 1801–1818 (2016). https://doi.org/10.1109/TCOMM.2016.2549517

    Article  Google Scholar 

  27. S. Park, A. Alkhateeb, R.W. Heath, Dynamic subarrays for hybrid precoding in wideband mmwave MIMO systems. IEEE Trans. Wirel. Commun. 16(5), 2907–2920 (2017). https://doi.org/10.1109/TWC.2017.2671869

    Article  Google Scholar 

  28. H. Li, M. Li, Q. Liu, A.L. Swindlehurst, Dynamic hybrid beamforming with low-resolution pss for wideband mmwave MIMO-OFDM systems. IEEE J. Sel. Areas Commun. 38(9), 2168–2181 (2020). https://doi.org/10.1109/JSAC.2020.3000878

    Article  Google Scholar 

  29. A. Aubry, A. De Maio, B. Jiang, S. Zhang, Ambiguity function shaping for cognitive radar via complex quartic optimization. IEEE Trans. Signal Process. 61(22), 5603–5619 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  30. S.M. Karbasi, A. Aubry, V. Carotenuto, M.M. Naghsh, M.H. Bastani, Knowledge-based design of space-time transmit code and receive filter for a multiple-input-multiple-output radar in signal-dependent interference. IET Radar Sonar Navig. 9(8), 1124–1135 (2015)

    Article  Google Scholar 

  31. Z. Cheng, Z. He, B. Liao, M. Fang, Mimo radar waveform design with papr and similarity constraints. IEEE Trans. Signal Process. 66(4), 968–981 (2018). https://doi.org/10.1109/TSP.2017.2780052

    Article  MathSciNet  MATH  Google Scholar 

  32. R.T. Marler, J.S. Arora, Survey of multi-objective optimization methods for engineering. Struct. Multidiscipl. Optim. 26(6), 369–395 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  33. R.T. Marler, J.S. Arora, The weighted sum method for multi-objective optimization: new insights. Struct. Multidiscipl. Optim. 41(6), 853–862 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  34. F. Sohrabi, W. Yu, Hybrid digital and analog beamforming design for large-scale antenna arrays. IEEE J. Sel. Topics Signal Process. 10(3), 501–513 (2016). https://doi.org/10.1109/JSTSP.2016.2520912

    Article  Google Scholar 

  35. F. Sohrabi, W. Yu, Hybrid analog and digital beamforming for mmwave OFDM large-scale antenna arrays. IEEE J. Sel. Areas Commun. 35(7), 1432–1443 (2017). https://doi.org/10.1109/JSAC.2017.2698958

    Article  Google Scholar 

  36. S. Boyd, L. Vandenberghe, Convex Optimization (Cambridge University Press, 2004)

  37. M. Zhang, J. Li, S. Zhu, X. Chen, Fast and simple gradient projection algorithms for phase-only beamforming. IEEE Trans. Veh. Technol. 70(10), 10620–10632 (2021). https://doi.org/10.1109/TVT.2021.3110192

    Article  Google Scholar 

  38. O. Aldayel, V. Monga, M. Rangaswamy, Tractable transmit mimo beampattern design under a constant modulus constraint. IEEE Trans. Signal Process. 65(10), 2588–2599 (2017). https://doi.org/10.1109/TSP.2017.2664040

    Article  MathSciNet  MATH  Google Scholar 

  39. J. Tranter, N.D. Sidiropoulos, X. Fu, A. Swami, Fast unit-modulus least squares with applications in beamforming. IEEE Trans. Signal Process. 65(11), 2875–2887 (2017). https://doi.org/10.1109/TSP.2017.2666774

    Article  MathSciNet  MATH  Google Scholar 

  40. K. Kreutz-Delgado, The Complex Gradient Operator and the cr-calculus. arXiv preprint arXiv:0906.4835 (2009)

Download references

Acknowledgements

This work was supported in part by Guangdong Basic and Applied Basic Research Foundation under Grant 2022A1515010188, and in part by the National Natural Science Foundation of China under Grants 62171292 and 62211530432.

Funding

The funding was provided by Guangdong Basic and Applied Basic Research Foundation (Grant No. 2022A1515010188) and National Natural Science Foundation of China (Grant Nos. 62211530432, 62171292).

Author information

Authors and Affiliations

Authors

Contributions

JZ developed the method presented in this paper and performed the experiments. BL wrote the manuscript. All authors discussed the results and implications and commented on the manuscript at all stages. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Bin Liao.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1 Calculation of \(\nabla F(\textbf{f}_{k,u})\) and \(\mathcal {H}_{k,u}\)

Note: \(F(\textbf{f}_{k,u})=F_0(\textbf{f}_{k,u})+F_1(\textbf{f}_{k,u})\), with \(F_0(\textbf{f}_{k,u})\) and \(F_1(\textbf{f}_{k,u})\) are, respectively, defined as

$$\begin{aligned} F_0(\textbf{f}_{k,u})= & {} \beta _r\text{SINR}_{k,u} \end{aligned}$$
(51)
$$\begin{aligned} F_1(\textbf{f}_{k,u})= & {} \beta _c\text{SE}_{k,u} \end{aligned}$$
(52)

Furthermore, \(\nabla F(\textbf{f}_{k,u})\) can be calculated by

$$\begin{aligned} \nabla F(\textbf{f}_{k,u})=\nabla F_0(\textbf{f}_{k,u})+\nabla F_1(\textbf{f}_{k,u}) \end{aligned}$$
(53)

where \(\nabla F_0(\textbf{f}_{k,u})\) and \(\nabla F_1(\textbf{f}_{k,u})\) can be computed by

$$\begin{aligned}&\nabla F_0(\textbf{f}_{k,u})=\frac{\alpha ({\varvec{\Theta }}_{g,k}-F_0(\textbf{f}_{k,u}){\varvec{\Theta }}_{in,k})\textbf{f}_{k,u}}{\textbf{f}^H_{k,u}{\varvec{\Theta }}_{in,k}\textbf{f}_{k,u}} \end{aligned}$$
(54)
$$\begin{aligned}&\nabla F_1(\textbf{f}_{k,u})=\frac{\beta \overline{\textbf{H}}_{k,u}\textbf{f}_{k,u}}{\left( \sum \limits _{i=1}^U\textbf{f}^H_{k,i}\overline{\textbf{H}}_{k,u}\textbf{f}_{k,i}+\sigma ^2_{\hat{n}}\textbf{u}^H_{k,u}\textbf{u}_{k,u}\right) \text{ln}2} \end{aligned}$$
(55)

Further, the complex Hessian matrix can be defined as

$$\begin{aligned} \mathcal {H}&= \left( \begin{array}{ll} \mathcal {H}_{\textbf{f}\textbf{f}} &{} \mathcal {H}_{\textbf{f}^*\textbf{f}} \\ \mathcal {H}_{\textbf{f}\textbf{f}^*} &{} \mathcal {H}_{\textbf{f}^*\textbf{f}^*} \end{array}\right) \end{aligned}$$
(56)

which is, however, complicated to directly calculate the complex Hessian matrix because the Hessian matrix contains four second-order complex derivatives. In this work, the complex Hessian matrix is transformed from the real Hessian matrix since the relationship between the second-order terms in the complex and real forms as shown below.

$$\begin{aligned} \mathcal {H}&= \frac{1}{4} \textbf{J}\overline{\mathcal {H}}\textbf{J}^{H} \end{aligned}$$
(57)

where \(\overline{\mathcal {H}}\) is the real Hessian matrix, and \(\textbf{J}\) is defined as

$$\begin{aligned} \textbf{J}&= \left( \begin{array}{cc} \textbf{I}_{N_t} &{} j \textbf{I}_{N_t} \\ \textbf{I}_{N_t} &{} -j \textbf{I}_{N_t} \end{array}\right) \end{aligned}$$
(58)

In addition (57) shows that the eigenvalues of the real Hessian matrix are twice the size of the eigenvalues of the complex Hessian matrix [40]. This means that the maximum eigenvalue of the complex Hessian matrix can be obtained by calculating the maximum eigenvalue of the real Hessian matrix.

To calculate the real Hessian matrix \(\overline{\mathcal {H}}_{k,u}\), we first transform \(F(\textbf{f}_{k,u})\) into a real-value form as \(F(\textbf{f}_{k,u,r})=F_0(\textbf{f}_{k,u,r})+F_1(\textbf{f}_{k,u,r})\), with \(F_0(\textbf{f}_{k,u,r})\) and \(F_1(\textbf{f}_{k,u,r})\) are, respectively, defined as

$$\begin{aligned}&F_0(\textbf{f}_{k,u,r})= \alpha \frac{\textbf{f}^T_{k,u,r}{\varvec{\Theta }}_{g,k,r}\textbf{f}_{k,u,r}}{\textbf{f}^T_{k,u,r}{\varvec{\Theta }}_{in,k,r}\textbf{f}_{k,u,r}} \end{aligned}$$
(59)
$$\begin{aligned}&F_1(\textbf{f}_{k,u,r})= \beta \text{log}_2\left( 1+\frac{\textbf{f}^T_{k,u,r}\overline{\textbf{H}}_{k,u,r}\textbf{f}_{k,u,r}}{\sum \limits _{i\ne u}^U\textbf{f}^H_{k,i,r}\overline{\textbf{H}}_{k,u,r}\textbf{f}_{k,i,r}+\sigma ^2_{\hat{n}}\textbf{u}^H_{k,u,r}\textbf{u}_{k,u,r}}\right) \end{aligned}$$
(60)

where \(\textbf{f}_{k,i,r}\), \(\overline{\textbf{H}}_{k,u,r}\), \(\textbf{u}_{k,u,r}\), \({\varvec{\Theta }}_{g,k,r}\) and \({\varvec{\Theta }}_{in,k,r}\) are the real-valued form of \(\textbf{f}_{k,i}\), \(\overline{\textbf{H}}_{k,u}\), \(\textbf{u}_{k,u}\), \({\varvec{\Theta }}_{g,k}\) and \({\varvec{\Theta }}_{in,k}\), respectively.Footnote 1

Then, we have

$$\begin{aligned} \overline{\mathcal {H}}_{k,u} = \overline{\mathcal {H}}^0_{k,u} + \overline{\mathcal {H}}^1_{k,u} \end{aligned}$$
(61)

where \(\overline{\mathcal {H}}^0_{k,u}\) and \(\overline{\mathcal {H}}^1_{k,u}\) can be computed by

$$\begin{aligned} \overline{\mathcal {H}}^0_{k,u}=&\frac{2(\alpha {\varvec{\Theta }}_{g,k,r}-F_0(\textbf{f}_{k,u,r}){\varvec{\Theta }}_{in,k,r})}{\textbf{f}^T_{k,u,r}{\varvec{\Theta }}_{in,k,r}\textbf{f}_{k,u,r}}\nonumber \\&\quad -\frac{4\alpha ({\varvec{\Theta }}_{g,k,r}\textbf{f}_{k,u,r}\textbf{f}_{k,u,r}^T{\varvec{\Theta }}_{g,k,r}+{\varvec{\Theta }}_{in,k,r}\textbf{f}_{k,u,r}\textbf{f}^T_{k,u,r}{\varvec{\Theta }}_{g,k,r})}{(\textbf{f}^T_{k,u,r}{\varvec{\Theta }}_{in,k,r}\textbf{f}_{k,u,r})^2}\nonumber \\&\quad +\frac{8F_0(\textbf{f}_{k,u,r}){\varvec{\Theta }}_{in,k,r}\textbf{f}_{k,u,r}\textbf{f}^T_{k,u,r}{\varvec{\Theta }}_{in,k,r}}{(\textbf{f}^T_{k,u,r}{\varvec{\Theta }}_{in,k,r}\textbf{f}_{k,u,r})^2} \end{aligned}$$
(62)

and

$$\begin{aligned} \overline{\mathcal {H}}^1_{k,u}&=\frac{2\beta \overline{\textbf{H}}_{k,u,r}}{\left( \sum \limits _{i= 1}^U\textbf{f}^H_{k,i,r}\overline{\textbf{H}}_{k,u,r}\textbf{f}_{k,i,r}+\sigma ^2_{\hat{n}}\textbf{u}^H_{k,u,r}\textbf{u}_{k,u,r}\right) \text {ln}2}\nonumber \\&\quad -\frac{4\beta \overline{\textbf{H}}_{k,u,r}\textbf{f}_{k,u,r}\textbf{f}_{k,u,r}^T\overline{\textbf{H}}_{k,u,r}}{\left( \sum \limits _{i= 1}^U\textbf{f}^H_{k,i,r}\overline{\textbf{H}}_{k,u,r}\textbf{f}_{k,i,r}+\sigma ^2_{\hat{n}}\textbf{u}^H_{k,u,r}\textbf{u}_{k,u,r}\right) ^2\text {ln}2} \end{aligned}$$
(63)

Finally, we can get the complex Hessian matrix based on (57), and get the maximum eigenvalue of the complex Hessian matrix.

1.2 Derivation of (30)

Define \(\nabla F(\textbf{f}^{(m)}_{k,u})=\textbf{t}_{k,u}=\textbf{t}_{re}+j\textbf{t}_{im}\), \(\textbf{f}_{k,u}=\textbf{f}_{re}+j\textbf{f}_{im}\) and \(\textbf{q}_{k,u} = \textbf{F}_{{\text{RF}}}\textbf{f}_{{{\text{BB}}_{k,u}}}-\textbf{d}_{k,u} = \textbf{q}_{re}+\textbf{q}_{im}\).

$$\begin{aligned}&F(\textbf{f}^{(m)}_{k,u})+2Re\left\{ \nabla ^HF(\textbf{f}^{(m)}_{k,u})(\textbf{f}_{k,u}-\textbf{f}^{(m)}_{k,u})\right\} \nonumber \\&\quad +\frac{\rho _{k,u}}{2}\Vert \textbf{f}_{k,u}-\textbf{F}_\text{RF}\textbf{f}_{\text{BB}_{k,u}}+\textbf{d}_{k,u} \Vert _2^2\nonumber \\&\quad = F(\textbf{f}^{(m)}_{k,u})-2Re\left\{ \nabla ^HF(\textbf{f}^{(m)}_{k,u})\textbf{f}^{(m)}_{k,u}\right\} +2Re\left\{ \nabla ^HF(\textbf{f}^{(m)}_{k,u})\textbf{f}_{k,u}\right\} \nonumber \\&\quad +\frac{\rho _{k,u}}{2}\Vert \textbf{f}_{k,u}-\textbf{F}_\text{RF}\textbf{f}_{\text{BB}_{k,u}}+\textbf{d}_{k,u} \Vert _2^2\nonumber \\&\quad =F(\textbf{f}^{(m)}_{k,u})-2Re\left\{ \nabla ^HF(\textbf{f}^{(m)}_{k,u})\textbf{f}^{(m)}_{k,u}\right\} +2(\textbf{t}_{re}^T\textbf{f}_{re}+\textbf{t}_{im}^T\textbf{f}_{im})\nonumber \\&\quad +\frac{\rho _{k,u}}{2}\left[ (\textbf{f}_{re}-\textbf{q}_{re})^T(\textbf{f}_{re}-\textbf{q}_{re})+(\textbf{f}_{im}-\textbf{q}_{im})^T(\textbf{f}_{im}-\textbf{q}_{im})\right] \nonumber \\&\quad =F(\textbf{f}^{(m)}_{k,u})-2Re\left\{ \nabla ^HF(\textbf{f}^{(m)}_{k,u})\textbf{f}^{(m)}_{k,u}\right\} +2Re\left\{ \textbf{t}^H_{k,u} \textbf{q}_{k,u}\right\} -\frac{2}{\rho _{k,u}}\Vert \textbf{t}_{k,u} \Vert _2^2\nonumber \\&\quad +\frac{\rho _{k,u}}{2}\left[ \left( \textbf{f}_{re}-\left( \textbf{q}_{re}-\frac{2\textbf{t}_{re}}{\rho _{k,u}}\right) \right) ^T\left( \textbf{f}_{re}-\left( \textbf{q}_{re}-\frac{2\textbf{t}_{re}}{\rho _{k,u}}\right) \right) \right. \nonumber \\&\quad \left. +\left( \textbf{f}_{im}-\left( \textbf{q}_{im}-\frac{2\textbf{t}_{im}}{\rho _{k,u}}\right) \right) ^T\left( \textbf{f}_{im}-\left( \textbf{q}_{im}-\frac{2\textbf{t}_{im}}{\rho _{k,u}}\right) \right) \right] \nonumber \\&\quad =\frac{\rho _{k,u}}{2}\Vert \textbf{f}_{k,u}-(\textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}-\textbf{d}_{k,u}-\frac{2\nabla F(\textbf{f}_{k,u}^{(m)})}{\rho _{k,u}}) \Vert _2^2\nonumber \\&\quad + F(\textbf{f}^{(m)}_{k,u})-2Re\left\{ \nabla ^HF(\textbf{f}^{(m)}_{k,u})\textbf{f}^{(m)}_{k,u}\right\} -\frac{2}{\rho _{k,u}}\Vert \nabla F(\textbf{f}_{k,u}^{(m)}) \Vert _2^2 \nonumber \\&\quad +2Re\left\{ \nabla F(\textbf{f}_{k,u}^{(m)})(\textbf{F}_\text{RF}\textbf{f}_{\text{BB}_{k,u}}-\textbf{d}_{k,u})\right\} \end{aligned}$$
(64)

Since the problem (29) w.r.t. \(\textbf{f}_{k,u}\), the terms that do not contain \(\textbf{f}_{k,u}\) in (64) can be regarded as constant terms when optimizing \(\textbf{f}_{k,u}\). Therefore, the optimization problem for \(\textbf{f}_{k,u}\) can be recast into the following form.

$$\begin{aligned}&\min \limits _{\textbf{f}_{k,u}}&\left\| \textbf{f}_{k,u}-\left( \textbf{F}_{\text{RF}}\textbf{f}_{\text{BB}_{k,u}}-\textbf{d}_{k,u}-\frac{2\nabla F(\textbf{f}_{k,u})}{\rho _{k,u}}\right) \right\| _2^2 \end{aligned}$$
(65a)
$$\begin{aligned}&\mathrm {s.t.}&\Vert \textbf{f}_{k,u}\Vert ^2_2=\frac{\mathcal {P}_k}{U} \end{aligned}$$
(65b)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeng, J., Liao, B. Transmit and receive hybrid beamforming design for OFDM dual-function radar-communication systems. EURASIP J. Adv. Signal Process. 2023, 37 (2023). https://doi.org/10.1186/s13634-023-00996-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13634-023-00996-2

Keywords