 Research
 Open Access
 Published:
Joint window and filter optimization for new waveforms in multicarrier systems
EURASIP Journal on Advances in Signal Processingvolume 2018, Article number: 63 (2018)
Abstract
One of the demands of the nextgeneration wireless communication systems is being supportive to asynchronous traffic types. In order to meet this demand, many waveform candidates for nextgeneration wireless communication systems have low outofsubband emissions (OOSBE) of the transmitted signal, and the popular ones deploy either filtering or windowing at the transmitter. In this paper, a joint windowing and filtering multicarrier waveform based on a generalized orthogonal frequency division multiplexing (OFDM) system is proposed. A joint window and filter optimization problem to minimize OOSBE is also formulated. The optimization problem is first divided into two solvable subproblems using interiorpoint methods, and then an iterative algorithm is proposed to obtain the optimal window and filter pair. Simulation results suggest that the proposed joint windowing and filtering waveform has an advantage in suppressing OOSBE or enhancing spectral efficiency, and is more robust to frequency asynchronism compared to waveforms deploying only either filtering or windowing.
Introduction
New transmission waveforms beyond orthogonal frequency division multiplexing (OFDM) have been studied to a great extent recently, in order to meet the demands of nextgeneration communication systems [1]. Many new waveform proposals aim at suppressing sidelobe power or outofsubband emissions (OOSBE) in generalized OFDM systems, since waveforms with this property may provide advantages such as low interference among asynchronous users accessing adjacent frequencies, spectral efficiency increase through reuse of guard frequencies, and enabling multiservice systems using mixed numerologies [2–7].
Among the many waveform proposals, filter bank multicarrier (FBMC) systems [8] possess perhaps the best sidelobe suppression performance when their associated pulseshaping filters are properly designed. However, filters with this great property usually have a long impulse response, typically in the order of several symbol durations. This causes severe intersymbol interference (ISI) and results in difficulties in receiver designs especially when the signal is transmitted over multipath channels [9]. With a similar transmitter structure, a windowingbased waveform, commonly known as weighted overlapandadd (WOLA) [4], has been proposed to suppress sidelobe powers without using long pulse shaping. Nevertheless, an extended guard interval, such as a cyclic prefix (CP), is required for eliminating ISI caused by multipath channels and the window of WOLA [10, 11], which leads to reduced spectral efficiency. In [12, 13], windowing schemes that require no guard interval extension were proposed to suppress sidelobe powers. However, the receiver structure in [12] causes signaltonoise ratio (SNR) loss and leads to a degraded data reception performance, whereas the receiver structure in [13] leads to intercarrier interference (ICI) between subcarriers for arbitrary window coefficients with multipath fading channels.
In [14–16], precodingbased waveforms were proposed to suppress OOSBE by designing a precoder for transmitting data symbols. Generalized frequency division multiplexing (GFDM) [14] is able to suppress sidelobe power by using a circularly pulseshaped precoder. In general, the GFDM precoder is represented by a nonunitary matrix[17]. The nonunitariness of such precoding matrix could lead to noise enhancement at the receiver, and thus, it degrades the performance of data reception. Spectrally precoded OFDM [15] and a power leakagesuppressing precoder [16] both possess unitary precoding matrices so that there is no noise enhancement at the receiver. However, the precodingbased waveforms require a number of null subcarriers, called virtual carriers (VCs), to be inserted in each subband, which will degrade spectral efficiency.
Recently, some new filteringbased waveforms were proposed with the objective to shorten the impulse response of the pulse shaping filters while maintaining a similar level of OOSBE suppression [18–25]. These waveforms take advantage of persubband filtering to obtain a much shorter filter than FBMC, which executes persubcarrier filtering. Nevertheless, all filteringbased waveforms increase the delay spread of the equivalent channel and inevitably suffer from either a decreased spectral efficiency due to guard interval extension [21] or a receiver with a larger complexity for equalization of ISI effects. To trade a better OOSBE suppression performance, inserting a small number of VCs between adjacent subbands as additional guard bands is also considered [2, 22–24]. However, such insertion also leads to decreased spectral efficiency.
In this paper, a new waveform that jointly utilizes windowing in [12] and filtering at the transmitter is proposed. To suppress OOSBE, a subbandbased joint window and filter optimization method is proposed. The proposed waveform using the proposed optimization method has an advantage in OOSBE suppression in contrast with previous filteringbased and windowingbased waveforms. Alternatively, the proposed waveform is more spectrally efficient for suppressing OOSBE to a specific level because it requires less VCs. Simulation results confirm the above advantages of the proposed waveform.
Related waveforms
The waveforms related to this research, including the aforementioned windowingbased and filteringbased waveforms, are further presented in the following.
Windowingbased waveforms: WOLA deploys nonmemoryless windowing at either the transmitter or the receiver [4]. To mitigate OOSBE, WOLA performed at the transmitter is introduced. A Nyquist window (e.g., raisedcosine) is usually adopted by current WOLA systems. However, an extended CP is required for ISI elimination. A waveform using memoryless windowing at the transmitter was proposed in [12]. This waveform has low OOSBE compared to the conventional OFDM and needs no extra CP. The objective of optimizing the window coefficients in [12] is to minimize the power leakage out of a subcarrier. Nevertheless, the receiver that assures orthogonality between subcarriers induces SNR loss caused by noise enhancement. A memoryless windowingbased waveform was also proposed in [13], which adopted a different receiver structure from that in [12]. Specifically, it gives up on assuring orthogonality between subcarriers for multipath fading channels; thus, it results in ICI and endurable ISI.
Filteringbased waveforms: Universalfiltered multicarrier (UFMC) [18–20] was first proposed with the idea of using persubband filtering to suppress OOSBE. Such subbandbased filtering incurs mitigated ISI compared to FBMC. A UFMC system with an extended CP was proposed in [21] to further eliminate the ISI. With the similar idea of UFMC, filteredOFDM (fOFDM) [22, 23] considers a longer filter (up to a half symbol duration) to trade off even better OOSBE suppression at the expense of some manageable ISI. A UFMC system that employs WOLAbased windowing and specifically targets at lowlatency applications has been proposed in [25].
Contributions
The major contributions of this paper are as follows:

A persubbandbased waveform that exploits both windowing and filtering operations at the transmitter is proposed. This waveform is suitable for performing joint processing because there are two sets of design parameters. Unlike [25] which utilizes WOLAbased windowing, the windowing structure used in this paper is inspired by that in [12], which helps the CP length to be further reduced in contrast with the previous filteringbased waveforms. The proposed waveform exploits a subbandbased filtering and windowing design, whereas a persubcarrier design is executed in [12]. This design makes the proposed waveform more suitable for multisubband systems (e.g., in [2]).

A joint window and filter optimization method is proposed to suppress OOSBE of the proposed waveform. The proposed method enables the proposed waveform to possess advantages in spectral efficiency and OOSBE suppression over the previous filteringbased and windowingbased waveforms. The proposed method also facilitates the proposed waveform to induce much less SNR loss than the windowingbased waveform in [12] and to allow the use of a shorter CP for ISI elimination compared to WOLA and the previous filteringbased waveform to suppress OOSBE to a specific level.
Organization and notations
The remainder of this paper is organized as follows. Section 2 presents the system model. Section 3 addresses the joint window and filter design problem. The proposed optimization method is then described in Section 4. Simulation results are provided in Section 5, and Section 6 offers a conclusion.
Notations: Boldface lowercase letters represent column vectors; boldface uppercase letters represent matrices. The superscripts (·)^{∗}, (·)^{T}, and (·)^{H} denote the conjugate, transpose, and transposeconjugate operations, respectively. The expected value operation is denoted as E{·}. The vector d(θ) of an Mlength vector \( \boldsymbol {\theta } \triangleq [ \ \theta _{1} \ \cdots \ \theta _{M}\ ]^{T} \) is [ cosθ_{1} ⋯ cosθ_{M} sinθ_{1} ⋯ sinθ_{M} ]^{T}. Operations ∥·∥ and diag(·) denote the vector norm and vector/matrix diagonalization, respectively. Hadamard and Kronecker products between vectors v_{1}, v_{2} are denoted as v_{1}∘v_{2} and v_{1}⊗v_{2}, respectively. An identity matrix of size M is denoted as I_{M}; a zero matrix of size M×N is denoted as 0_{M×N}. An Nlength vector with all of its entries be equal to one is denoted as 1_{N}; a zero column vector of size M is denoted as 0_{M}. A column vector v_{L}(z) of length L is written as v_{L}(z)=[ 1 z ⋯ z^{L−1} ]^{T}. An M×M circulant matrix C whose first column is [ c_{0} c_{1} ⋯ c_{M−1} ]^{T} is written as
A normalized discrete Fourier transform (DFT) matrix of size M is W_{M}. The (m,n)th entry of the DFT matrix is \([\textbf {W}_{M}]_{m,n} = \frac {1}{\sqrt {M}}e^{j\frac {2\pi }{M}(m1)(n1)}\). The domain of function f is denoted as dom f. The sets of nonnegative and positive vectors of size M are denoted as \(\mathsf {R}_{+}^{M} \) and \( \mathsf {R}_{++}^{M} \), respectively.
System model
Transmitter structure
Consider a multicarrier system with M subcarriers. Suppose that a communication device is allocated to a subband consisting of P adjacent subcarriers. Figure 1 depicts the block diagram of the transmitter. Let s_{P}[ n] denote the transmit symbol vector and assume
where E_{s} denotes the symbol energy. Now, the transmit symbol vector is precoded by an IDFT matrix \(\textbf {W}_{M}^{H}\), resulting in a vector \(\textbf {u}_{M}[\!n] = \textbf {W}_{M}^{H} \textbf {s}[\!n]\) where \( \textbf {s}[\!n] = \left [\boldsymbol {0} \ \textbf {s}_{P}^{T}[\!n] \ \boldsymbol {0} \ \right ]^{T} \).
To eliminate ISI and suppress the OOSBE, the vector u[ n] is obtained by adding a CP of length L to u_{M}[ n] and multiplying a matrix consisting of window coefficients. Specifically, the windowed signal vector is expressed as u[ n]=diag([w_{−L} ⋯ w_{0} ⋯ w_{M−1}])Ru_{M}[ n], where w_{m},m=−L,⋯,M−1 are the window coefficients, and matrix R, which represents adding the CP, is written as
Now, define
as the Fourier transform of the window coefficients, then the power spectral density (PSD) of signal u[ n], after paralleltoserial (P/S) conversion of vector u[ n], is [26]
where N=M+L, \( \Omega _{k} = \frac {2\pi k}{M} \) and \(\mathcal {K}\) denotes the set of subcarriers that are actually used in a subband. Let K denote the index of the subcarrier at the left of the allocated subband, then
In conventional OFDM systems, the window coefficients are usually chosen as w_{m}=1,∀m=−L,⋯,M−1 referred to as a rectangular window. In this paper, the coefficients are considered to be more general, with a constraint that the power of the signal using any arbitrary coefficients is equal to that using the rectangular window. That is,
Note that the slope of sidelobe power decay of conventional OFDM signals is approximately proportional to the frequency distance to the center of the mainlobe [27]. The slow decay of sidelobe power in conventional OFDM systems results in large OOSBE.
An effective way of suppressing OOSBE is deploying persubband filtering at the transmitter, which was first proposed in [18]. Subsequently, the windowed signal u[ n] is filtered by an FIR filter F(z), which is defined as
where f[ n] is the impulse response and L_{f} denotes the filter length, which is assumed to be much smaller than the number of subcarriers, i.e., L_{f}≪M, and f=[ f[ 0] f[ 1] ⋯ f[L_{f}−1]]^{T}. Recall that the definition of \(\textbf {v}_{L_{f}}(z)\) is given in Section 1.3. Then, the PSD of the transmitted signal x[ n] is written as
The filter function f[ n],n=0,⋯,L_{f}−1, is chosen such that the expected value of the energy of the filtered signal x[ n] is equivalent to that of the windowed signal u[ n]. Based on Parseval’s relation and (3),
Effective channel
As illustrated in Fig. 1, the filtered signal x[ n] is sent over the channel H(z). It is assumed that H(z) is an FIR channel with a maximum order L_{h}, i.e.,
where h[ n] denotes the impulse response of the FIR channel. By assuming that the transmitter and the receiver are perfectly synchronized, the signal u[ n] can be viewed as passing through an effective channel, whose impulse response is written as
where L_{c}=L_{f}+L_{h} is the length of the effective channel. Here, the CP length is assumed to be no smaller than the effective channel length, i.e., L≥L_{c}, so that the ISI can be eliminated at the receiver [21].
Receiver structure
The modified zeroforcing receiver presented in [12] is adopted. Specifically, channel equalization in this paper is only performed on the subcarriers in the allocated subband, rather than on all the subcarriers, as presented in [12]. The block diagram of the receiver is depicted in Fig. 2. As suggested in [21], receiver filtering in the original filteringbased waveform is not considered here in order to scale down the complexity of the receiver.
The received signal after serialtoparallel (S/P) and CP removal is multiplied by the DFT matrix W_{M}, and then channel equalization for signals on the used subcarriers is performed by multiplying the inverse of the diagonal matrix \(\boldsymbol {\Lambda }(\mathcal {K})\). After that, the equalized signal is multiplied by the dewindowing matrix \( \textbf {F}(\mathcal {K}) \) in order to restore the orthogonality between subcarriers in the subband. In [12], it has been proven that the dewindowing matrix \( \textbf {F}(\mathcal {K}) \) is independent of the effective channel if the window satisfies the cyclicprefixed property: w_{m}=w_{m+M},m=−1,−2,⋯,−L. Based on this property, the windowed signal vector u[ n] is mathematically equivalent to:
where D=diag(w) with \(\textbf {w} \triangleq \left [\ w_{0} \ w_{1} \ \cdots \ w_{M1} \ \right ]^{T} \). Without regarding the signals of users allocated to other subbands, it can be shown that the vector y_{M}[n], after serialtoparallel conversion and CP removal, is
where C is an M×M circulant matrix whose first column is \(\left [c[0] \ c[1] \ \cdots \ c[L_{c}1] \ \boldsymbol {0}_{(ML_{c}1)} \right ]^{T}\) and \(\textbf {e}_{M}[\!n] \sim CN\left (\boldsymbol {0}_{M}, \sigma _{e}^{2} \textbf {I}_{M}\right)\) is the additive white Gaussian noise vector [28]. The circulant matrix can be diagonalized by the DFT matrix:
where \( \mathbf {\Lambda } = \text {diag}\left (\left [\ C\left (e^{j0}\right) \ C\left (e^{j\frac {2\pi }{M}}\right) \ \cdots \ C\left (e^{j\frac {2\pi }{M}(M1)}\right) \ \right ]\right) \) with C(e^{jω}) denoting the effective channel frequency response. Since the user is allocated to the subband consisting of subcarriers in \( \mathcal {K} \), equalization for the effective channel is only performed on the allocated subcarriers. Assuming that the receiver has perfect knowledge of effective channel responses of the allocated subcarriers, the equalized vector z_{P}[ n] is expressed as
where \(\textbf {E}(\mathcal {K}) = [\!\begin {array}{ccc}\boldsymbol {0}_{P\times (K1)} & \textbf {I}_{P} & \boldsymbol {0}_{P\times (MPK+1)} \end {array}]\), and \( \boldsymbol {\Lambda }(\mathcal {K}) \) is the P×P diagonal matrix whose diagonal elements are the effective channel frequency responses of subcarriers in \( \mathcal {K} \). That is, the diagonal matrix is written as \( \boldsymbol {\Lambda }(\mathcal {K}) = \text {diag}\left (\left [ \ C\left (e^{j\frac {2\pi }{M}K}\right) \ C\left (e^{j\frac {2\pi }{M}(K+1)}\right) \ \cdots \ C\left (e^{j\frac {2\pi }{M}(K+P1)}\right)\right ] \right)\).
After equalization, to restore the orthogonality between subcarriers in the subband, dewindowing is performed so that the decoded symbol vector \( \hat {\mathbf {s}}_{P}[\!n] \) is written as
where \(\textbf {F}(\mathcal {K}) \triangleq \textbf {E}(\mathcal {K})\textbf {F}\textbf {E}^{T}(\mathcal {K})\). The matrix F is designed in a zeroforcing sense, which can be interpreted as transforming the equalized signal into a timedomain signal, by multiplying a diagonal matrix whose diagonal elements are the inverse of window coefficients, and then taking DFT on the product [12], i.e.,
Such dewindowing operation will cause SNR loss at the receiver with an arbitrary w other than a constantmodulus window, e.g., w=1_{M}.
A different receiver structure that adopts dewindowing before channel equalization was proposed in [13]. This structure is not considered here because the adopted one assures orthogonality between subcarriers while data reception is executed at the receiver, and ISI is eliminated given that L≥L_{c}. Moreover, the adopted structure is more suitable for performing window optimization because SNR loss caused by noise enhancement can be quantified explicitly, which will be presented in Section 2.4.
Noise analysis
In this subsection, the SNR loss caused by dewindowing is derived, which plays an important role in window optimization. The derived SNR loss can be approximated as the same as that quantified in [12]. By substituting (8), (9), and \( \textbf {F}(\mathcal {K}) \) in (11) into (10), the correlation matrix of vector z_{P}[ n], after CP removal, DFT, and channel equalization, is written as
where \(\mathbf {\Sigma }_{z} = \text {diag}\left (\left [ \ \sigma _{z_{1}}^{2} \ \sigma _{z_{2}}^{2} \ \cdots \ \sigma _{z_{P}}^{2}\right ]\right) \) is the correlation matrix of noise after DFT and channel equalization. The ith diagonal element of Σ_{z} is written as
where \(\sigma _{i}^{2} \triangleq \sigma _{e}^{2}/  H\left (e^{j\frac {2\pi }{M}(i+K1)}\right) ^{2} \) is the resulting noise variance of the conventional OFDM system. After dewindowing is performed as in (11), the correlation matrix of \( \hat {\textbf {s}}_{P}[\!n] \) is written as \(\textbf {R}_{\hat {s}} = E_{s}\textbf {I}_{P} + \textbf {F}(\mathcal {K}) \mathbf {\Sigma }_{z} \textbf {F}^{H}(\mathcal {K}).\) The output noise power of an entry in \( \hat {s}_{P}[n] \) is the corresponding diagonal element of the latter term. After the dewindowing operation in (12), the total output noise power is written as
As a special case of the proposed waveform, filteringbased waveforms apply rectangular windowing. The resulting total output noise power is \(\mathcal {N}_{\mathrm {F}} = \sum _{i=1}^{P} \sigma _{z_{i}}^{2}\), which is obtained by substituting w_{m}=1,∀m∈{0,⋯,M−1} into (14). By letting \(F\left (e^{j\frac {2\pi }{M}(i+K1)}\right) = 1, \forall i \in \{ 1, \cdots, P \}\) and adopting rectangular windowing, the total output noise of the conventional OFDM system is obtained as \( \mathcal {N}_{\mathrm {C}} = \sum _{i=1}^{P} \sigma _{i}^{2} \).
To quantify the noise enhancement effect caused by dewindowing at the receiver compared to the conventional OFDM system, the SNR loss is defined as
As suggested in [20], a large variation of \(F\left (e^{j\Omega _{k}}\right)^{2}, k \in \mathcal {K}\) will degrade the bit error rate (BER) performance. Therefore, the passband ripple of transmit filtering is constrained tightly so that \( F(e^{j\Omega _{k}})^{2} \approx 1, \forall k \in \mathcal {K} \). Accordingly, the ratio of total noise power of filteringbased waveforms to the total noise power of the conventional OFDM is approximated as
since \( \sigma _{i}^{2} \approx \sigma _{z_{i}}^{2}, i = 1, \cdots, P \). As a result, the SNR loss in (15) is approximated as
In the sequel, it is assumed that the approximation in (16) holds given a tight passband ripple constraint. The reasons are that (i) this approximation is accurate for most of the channel responses and (ii) the approximation error does not affect the BER performance of the proposed waveform, as will be shown later in Sections 5.3 and 5.7.
Problem statement
The goal of this paper is to jointly design the filter f and the window w so that the OOSBE can be suppressed with a controllable SNR loss, and the passband ripple caused by filtering is restrained. The design is based on a minimax criterion. In [29], it has been pointed out that a wide transition bandwidth (TBW) is beneficial to suppressing stopband magnitude for minimaxbased filter design with specified filter length and passband ripple. This phenomenon suggests that the OOSBE suppression performance of a filteringbased waveform is limited with a fixed number of VCs, filter length, and passband ripple. Here, windowing is introduced to further improve the OOSBE suppression performance over the filteringbased waveforms.
Suppose a user is allocated with a subband \(\left (\omega _{\mathrm {p}}^{(l)}, \omega _{\mathrm {p}}^{(r)}\right)\). Then, the specification of the transmit PSD is illustrated in Fig. 3. Let the stopband and the passband of the filter and the window be defined as \(B_{\mathrm {s}} \triangleq \left [\pi, \omega _{\mathrm {s}}^{(l)}\right ] \cup [\omega _{\mathrm {s}}^{(r)}, \pi)\) and \(B_{\mathrm {p}} \triangleq \left (\omega _{\mathrm {p}}^{(l)}, \omega _{\mathrm {p}}^{(r)}\right)\), respectively. Neighboring subbands allocated to other users lie in the stopband such that the ICI between users can be mitigated. Define \( B_{\mathrm {t}} \triangleq \left (\omega _{\mathrm {s}}^{(l)}, \omega _{\mathrm {p}}^{(l)}] \cup [\omega _{\mathrm {p}}^{(r)}, \omega _{\mathrm {s}}^{(r)}\right) \) as the transition band. The subcarriers in the transition band are treated as VCs. Let η denote the oneside normalized TBW, which is expressed as half the TBW over the subcarrier spacing (i.e., 2π/M), then the set of left and the right edges of the stopband are
where \(\left (\omega _{\mathrm {p}}^{(l)}, \omega _{\mathrm {p}}^{(r)}\right)=(\omega _{\mathrm {c}}  P\pi /M, \omega _{\mathrm {c}} + P\pi /M)\). For convenience, η can be thought of as the number of VCs on one side of the transition band.
Our design objective is to minimize the maximum (weighted) PSD in the stopband. The minimization is subject to constraints of SNR loss and passband ripple. In addition, for fair comparison, the signals after windowing and filtering should not have any power amplification effect. Therefore, the constraints (3) and (5) should be satisfied. Based on the above requirements, the optimization problem is expressed as
where α and γ denote controllable parameters of passband ripple and SNR loss, respectively, and G(e^{jω}) is the weighting function of the stopband in order to meet a specified spectrum mask restriction. For simplicity, it is assumed that G(e^{jω})=1,∀ω∈B_{s} in the following derivations. When a more general G(e^{jω}) is considered, however, it may not impose too much difficulty in generalizing the derivations.
Here, problem (19) is formulated as a more compact form. Based on the cyclicprefix window property, the Fourier transform of window coefficients in (1) can be written as \( W\left (e^{j\omega }\right) = \textbf {v}_{N}^{H}\left (e^{j\omega }\right) \textbf {R} \textbf {w} \). Consequently, the PSD of signal u[ n] is expressed as a quadratic form:
where the positive semidefinite matrix B(e^{jω}) is
With this formulation, the original problem (19) is written as
This problem is nonconvex due to, e.g., constraints (22b), (22c), or (22e) [30]. Therefore, the welldeveloped convex optimization techniques [30] can not be applied directly to solve this problem. Instead of optimizing w and f simultaneously, a method is proposed to optimize w and f iteratively.
Proposed method
In this section, an alternative optimization method to solve problem (19) is proposed. First, the problem (22) is divided into two subproblems, which are (i) optimization of filter: optimize the filter function f, subject to filter constraints with a given window, and (ii) optimization of window: optimize the cyclicprefixed window coefficients w_{m},m=−L,⋯,M−1, subject to window constraints with a given filter. Then, an iterative algorithm is proposed to solve the original optimization problem. The convergence and computational complexity of the proposed method, and the complexity of the transceiver are analyzed.
Optimization of filter function
The filter design method presented in [20] is used by relaxing the nonconvex problem into a convex one using spectral factorization [31]. The relaxed problem can be solved by using an interiorpoint method (IPM) in [30]. Given any w that satisfies (22d) and (22e), problem (22) becomes a filter optimization problem written as
This problem is nonconvex due to the lowerbounded constraint (23b) and the equality constraint (23c). This issue is overcome by using a technique presented in [32]. Let the autocorrelation function of f[ n] be defined as
Then, the squared magnitude of F(e^{jω}) can be written as
where r_{f}=[r_{f}(−L_{f}+1) ⋯ r_{f}(0) ⋯ r_{f}(L_{f}−1)]^{T} and
Note that r_{f}(m) is conjugatesymmetric, i.e., \( r_{f}(m) = r_{f}^{*}(m)\). With (24), the PSD in (4) can be rewritten as
With this equation and some manipulations, the problem (23) is modified as
where b(e^{jω})=w^{H}B(e^{jω})wa^{T}(e^{jω}) and the vector q is given by [q]_{i}=w^{H}R^{T}Q_{i}Rw with
Details of the derivation of constraint (27c) from (23c) are described in Appendix 1. The problem (27) is semiinfinite, which means that there are infinite number of constraints. A straightforward way of relaxing this problem is to discretize ω by sampling a finite set of frequencies in [−π,π) such that
where J denotes the number of samples. This paper adopts uniformly spaced sampling, i.e., \(\check {\omega }_{i} = \pi +2\pi i/J, i = 0, 1, \cdots, J1\). A previous work [33] recommended that J≈30L_{f} is sufficiently large to approximate the problem (27). With this discretization, the filter optimization problem (27) is modified as
where \(\check {B}_{\mathrm {s}} = \left \{\check {\omega }_{i} \big  \check {\omega }_{i} = \pi +{2\pi i}/{J}, i = 0, 1, \cdots, J1,\right. \left. \text {and} \ \check {\omega }_{i} \in B_{\mathrm {s}}\right \}\) and \(\check {B}_{\mathrm {a}} = \left \{\check {\omega }_{i} \big \check {\omega }_{i} = \pi +{2\pi i}/{J}, i = 0,\right. \left. 1, \cdots, J1 \right \}\). This problem can be solved efficiently by existing toolboxes (e.g., cvx [34]). To obtain f from the optimal r_{f}, spectral factorization in [31] can be applied using an efficient implementation based on inverse fast Fourier transform (IFFT) and fast Fourier transform (FFT).
Optimization of window coefficients
A persubband optimization method is proposed that minimizes the maximum stopband PSD subject to window constraints in (22), whereas persubcarrier optimization was proposed in [12]. Specifically, the window coefficients are restricted as real, and then the window optimization problem is relaxed as a convex one. By adopting the technique presented in Section 4.1, problem (22) is reformulated as
Given any fixed r_{f} that satisfies (31b), (31d), and (31e). The constraint (31c) is removed and the problem (22) is rewritten as
where U(e^{jω})=B(e^{jω})a^{T}(e^{jω})r_{f}. The reasons that the constraint (31c) is removed are that (31c) is nonconvex in w in general, and the relaxation does not prevent us from finding the optimal w and f for problem (22), as will be explained in Section 4.3.
In the literature, a common way of dealing with a complex value optimization problem is to reformulate it as a realvalued one via separating the real part and imaginary part of the parameters (e.g., [35, 36]). Define \(\hat {\textbf {w}} = \left [\text {Re}\left \{ \textbf {w}^{T} \right \} \ \text {Im}\left \{ \textbf {w}^{T} \right \}\right ]^{T},\) then the SNR loss in (32b) is expressed as a function of \(\hat {\textbf {w}}\):
where \( \hat {w}_{m} \triangleq [\! \hat {\textbf {w}}]_{m} \). Subsequently, problem (32) is reformulated via separating the real and imaginary parts of w, and relaxing (32c) as an inequality constraint so that this problem becomes a convex one. Then,
where \( \hat {\textbf {R}} \triangleq \boldsymbol {1}_{2} \otimes \textbf {R} \) and
The convexity of this problem depends on constraint (34b), since the objective function and constraint (34b) are convex. The convexity of \( \hat {\sigma }(\hat {\textbf {w}}), \text {dom}\,\hat {\sigma } \subseteq \mathsf {R}_{+}^{2M} \) is shown by the following theorem.
Theorem 1
Convexity of \( \hat {\sigma } (\hat {\textbf {w}}) \):

The function \( \hat {\sigma }(\hat {\textbf {w}}) \) in (33) is convex if \( \text {dom}\,\hat {\sigma } = \mathsf {D}(\boldsymbol {\theta }) \triangleq \left \{(\boldsymbol {1}_{2} \otimes \boldsymbol {\tau }) \circ \textbf {d}(\boldsymbol {\theta }) \big \boldsymbol {\tau } \in \mathsf {R}_{++}^{M}\right \} \) for any θ=[θ_{1} ⋯ θ_{M} ]^{T} that satisfies θ_{1},⋯,θ_{M}∈[−π,π).

The function \( \hat {\sigma } \) is nonconvex if \( \mathsf {dom}\,\hat {\sigma } = \mathsf {R}_{++}^{2M} \).
Proof
See Appendix 2. □
Theorem 1 suggests that problem (34) is convex if \(\text {dom}\, \hat {\sigma } = \mathsf {D}(\boldsymbol {\theta }) \) for any θ_{1},⋯,θ_{M}∈[−π,π). For simplicity, restricting θ=0_{M} is adopted, in which case the window coefficients are positivevalued. This restriction implies that the original SNR loss function \( \sigma (\textbf {w}), \text {dom}\,\sigma = \mathsf {R}_{++}^{M} \), is convex, which is suggested by the proof of Theorem 1(a). By applying frequency discretization as in (29) and (30), problem (34) is approximated as
where \( \textbf {w} \in \mathsf {R}_{++}^{M} \). This problem is a convex optimization problem with finite constraints, which can be solved by using cvx.
Proposed iterative method for joint optimization
An iterative method that optimizes window and filter is proposed. In an iteration of the proposed method, the window optimization and filter optimization presented in the previous subsections are performed based on given filter function and given window coefficients, respectively. Specifically, window optimization is performed based on the optimal filter’s autocorrelation function obtained in the previous iteration, and then filter optimization is performed based on the optimal window of the current iteration.
A detailed description of the proposed method is shown in Algorithm 1. In each iteration, the window optimization in (35) and the filter optimization (30) are performed sequentially. Let w(t) and r_{f}(t) denote the optimal solutions of (35) and (30) in iteration t, respectively. The squared differences of the optimal filter’s correlation function vector and optimal window between iterations t and t−1 are, respectively, written as ξ_{w}(t)=∥w(t)−w(t−1)∥^{2} and ξ_{r}(t)=∥r_{f}(t)−r_{f}(t−1)∥^{2}. Since the proposed method is convergent (the proof will be given in Section 4.4), the squared differences ξ_{w}(t) and ξ_{r}(t) will decrease as the number of iterations increases. Therefore, the stopping criteria of the proposed iterative algorithm is that both ξ_{w}(t) and ξ_{r}(t) are smaller than predetermined tolerances ε_{w} and ε_{r}, respectively. Let \(\hat {t}\) denote the termination iteration index. Then, the optimal window is \(\textbf {w}(\hat {t})\), and the optimal filter is obtained by performing spectral factorization [31] on \(\textbf {r}_{f}(\hat {t})\). For convenience, the starting point is given as \( \textbf {r}_{f}(0)= \left [\boldsymbol {0}_{L_{f}1}^{T} \ 1 \ \boldsymbol {0}_{L_{f}1}^{T}\right ]^{T}\), which means that no transmit filtering is deployed. Other starting points have been tried, and the resulting performances in OOSBE suppression are close to or worse than that of the current one. Finding the optimal starting point is an open problem for alternative optimization algorithms [37, 38], which is left for our future work.
In any iteration of the algorithm, the pair (w(t),r_{f}(t)) must satisfy the constraint (27c) after filter optimization. Accordingly, the constraint (31c) is not necessary to be met by the window w(t) based on r_{f}(t−1). Therefore, it is removed from (31) in window optimization (32).
Convergence analysis of the proposed algorithm
It is necessary to characterize whether the proposed algorithm is convergent, since it is operated iteratively. To justify that the convergence of the proposed algorithm is guaranteed, it is sufficient to show that the iteration of updating w and r_{f} converges and the objective is lowerbounded. A similar argument can be found in [37, 38]. In each iteration, the optimal w is first obtained by solving (35). The optimal r_{f} is then computed by solving (30) based on the determined w. Consequently, the objective function \( \max _{\check {\omega }_{i} \in \check {B}_{\mathrm {s}}} \ \textbf {w}^{H} \textbf {B}\left (e^{j\check {\omega }_{i}}\right) \textbf {w} \textbf {a}^{T}\left (e^{j\check {\omega }_{i}}\right) \textbf {r}_{f} \) decreases in each iteration. Clearly, the objective function is lowerbounded because the stopband PSD is nonnegative. Therefore, the proposed algorithm is convergent.
Complexity analysis of the proposed algorithm
The computational complexity of the IPM [30] for problem (30) is expressed as \(O\left (L_{f}^{3.5}\right)\) [36]. The complexity order of spectral factorization for obtaining the optimal filter from its autocorrelation function using an FFTbased implementation is O(L_{f} log2L_{f}) [31]. Similar to filter optimization, the computational complexity of the IPM for window optimization is expressed as O(M^{3.5}) [39]. As a result, the computational complexity in each iteration of the proposed iterative algorithm is \(O\left (M^{3.5}+ L_{f}^{3.5}+L_{f}\log _{2}L_{f} \right)\), which is equivalent to O(M^{3.5}) since L_{f}≪M. Observe that the number of iterations for satisfying ξ_{w}(t)≤ε_{w} and ξ_{f}(t)≤ε_{r} is independent from M. Therefore, the complexity order of the proposed method can also be expressed as O(M^{3.5}).
Complexity analysis of the transceiver
At the transmitter, the number of arithmetic operators of cyclicprefixed windowing is expressed as O(M) because the windowed signal can be represented by (7); the number of arithmetic operators of filtering is expressed as O(NL_{f})=O(N^{2}) since L_{f} is usually an integer equal to N divided by a power of two and can possibly be reduced to O(N log2N) [40]. Dewindowing at the receiver in (11) can be implemented by using FFT, IFFT, and M additional multiplications. Therefore, the computational complexity of receiver with dewindowing is O(3M log2M+M)=O(M log2M), which is of the same order as the receiver of conventional OFDM.
Results and discussion
In this section, some numerical results are provided for demonstrating the advantages of the proposed waveform.
Parameter settings
The parameters used in our simulations are listed below:

Total number of subcarriers: M=128

Number of subcarriers in a subband: P=12

Maximum channel order: L_{h}=M/8=16

Filter length: L_{f}=17

CP length: L=M/4=32
The values of the channel impulse response in (6) are assumed to be i.i.d. complex Gaussian random variables:
There are 10^{7} channel realizations used in our simulations. It is assumed that all subcarriers in the subband are used, i.e., \(\mathcal {K} = \{0, 1, \cdots, P1 \}\). The center frequency of the allocated subband is ω_{c}=(P−1)π/M. The passband set \(\left (\omega _{\mathrm {p}}^{(l)}, \omega _{\mathrm {p}}^{(r)}\right)\), and the pair stopband edges, \( \omega _{\mathrm {s}}^{(l)} \) and \( \omega _{\mathrm {s}}^{(r)} \), are computed as Eq. (18) with arbitrary normalized TBW η. In the following simulations, the proposed waveform is referred to as “joint windowing and filtering” (joint WF).
Convergence of the proposed algorithm
Figure 4 verifies the convergence of the proposed optimization method under different configurations. The normalized TBW used in this figure is η=3.5. The left and right plots demonstrate that the objective function in (22) decreases and converges rapidly in each iteration after window optimization and filter optimization, respectively.
Approximation of SNR loss
To examine the accuracy of the approximation in (17) with respect to various passband ripple constraints, the histogram of the ratio \(\mathcal {R}\) in (16), based on 10^{7} arbitrary channel realizations, is depicted in Fig. 5, and the probability of the ratio falling in a range is shown in Table 1. The normalized TBW used here is also η=3.5. One can observe that the ratios with α=0.3 dB and α=0.5 dB both have peak probabilities very close to \(\mathcal {R}=1\). In addition, over 96% of the ratios with α=0.3 dB and α=0.5 dB distribute in the range of (−0.93,1.07). These observations suggest that the approximation is tight for most of the channel realizations, and the approximation error imposes a slight affect on the BER performance, as will be shown in Section 5.7. On the contrary, the ratio with α=1 dB leads to inaccurate approximation since its distribution is scattered in a much wider range and does not have a peak probability close to \(\mathcal {R}=1\). In summary, α=0.5 dB is chosen for the following simulations because it results in an accurate SNR loss approximation and has a better OOSBE suppression performance than the case with α=0.3 dB.
Related waveforms for comparison
To highlight the advantages of the proposed joint WF waveform, the following related waveforms that use either windowing or filtering are introduced for comparison in our simulations.

Pure filtering: The waveform deploys optimum filtering in [20] at the transmitter with rectangular windowing.

Pure windowing: The waveform deploys optimum windowing without filtering at the transmitter. The optimal window is obtained through solving (35).

WOLA at the transmitter: Transmitter windowing of WOLA is performed on some of the CP symbol and an additional cyclic suffix (CS), and then the weighted CS is overlapped with the weighted CP of the next symbol. The transceiver structure in ([26], [Ch. 9]) is adopted for simulations. The raisedcosine (RC) window is usually used by current WOLA systems, which is expressed as \( \textbf {w}_{\text {rc}} = \left [ \begin {array}{ccccccc} p_{0} & \cdots & p_{\nu 1} & \boldsymbol {1}_{N\nu }^{T} & p_{\nu } & \cdots & p_{2\nu 1} \end {array} \right ]^{T} \) where
$$\begin{array}{*{20}l} {\kern.4cm}p_{n} = 0.5 \left(1 \cos\left(\frac{n+1 }{\nu}\pi \right) \right), n = 0, \cdots, \nu1, \end{array} $$and p_{n}=1−p_{n−ν},n=ν,⋯,2ν−1 [41] with ν denoting the CS length. The receiver of WOLA is the same as that of the conventional OFDM system. To eliminate ISI introduced at the receiver, the CP length of WOLA should satisfy L≤L_{h}+ν. This paper assumes that ν=L_{f} for fair comparison.
Normalized PSD
Here, the PSDs of the transmitted signals modulated by the aforementioned waveforms are demonstrated. The PSD is normalized so that its value on an allocated subband approaches 0 dB. The normalized PSD is written as
Figures 6 and 7 depict the normalized PSDs of the proposed and the other waveforms. In Fig. 6, the normalized TBWs of the pure filtering, pure windowing, and the proposed waveforms are η=8.5, which is as wide as that of WOLA. The resulting maximum stopband PSD of the proposed waveform with SNR loss γ=1.2 dB is about −67 dB, which is the best performance compared to the others. With a little bit less SNR loss, the proposed waveform with γ=0.5 dB has a maximum stopband PSD performance of approximately 4 dBlarger than that of the one with γ=1.2 dB, and it still outperforms other waveforms in suppressing maximum stopband PSD. In Fig. 7, the normalized TBWs of the proposed, pure filtering, and pure windowing waveforms are chosen as η=3.5 so that less VCs are required. The proposed waveform using different SNR losses also outperforms other waveforms in suppressing maximum stopband PSD. Although WOLA has good stopband suppression performance for frequencies far from the passband, it will cause larger PSD at some frequencies near the stopband. Therefore, WOLA using RC window will require more VCs to avoid possible ICI from other subbands compared to the proposed waveform.
Bandwidth efficiency
The proposed joint WF waveform is more spectrally efficient than the other waveforms due to its capability of suppressing OOSBE to a specific level with a narrower guard band. As depicted in Fig. 3, the TBW is treated as the guard band between two adjacent subbands and is shared by two different users that are allocated to these subbands separately. Accordingly, the bandwidth efficiency is defined as
Note that η denotes the normalized TBW over the bandwidth of a subcarrier, and P denotes the number of subcarriers in a subband.
The bandwidth efficiency performance of the various waveforms is listed in Table 2. These waveforms possess similar BER performances, which will be shown later. It is observed that the proposed waveform outperforms all the other waveforms at the price of approximately 0.5 dB SNR loss. Moreover, the proposed and the pure filtering waveforms can obtain better performances by relaxing the passband ripple constraint. These observations not only show the superiority of the proposed waveform in suppressing OOSBE spectralefficiently but also suggest that the proposed optimization method offers more flexibility in designing the transmission waveform.
Singleuser BER simulations
BER is considered as a metric for the performance of data transmission and reception in our simulations. QPSK or 16QAM without channel coding [42] is adopted as the modulation scheme with graycoded symbols. Here, the simulations are executed based on the following cases.

ISIfree case: The CP length is sufficient to eliminate ISI, and the transmitter is perfectly synchronized with the receiver. In this case, the proposed waveform, compared to OFDM, WOLA, and the pure filtering waveform, has a 0.5 dB BER disadvantage, which is roughly equal to the SNR loss. This phenomenon suggests that the proposed waveform trades a slight SNR loss for the advantageous bandwidth efficiency as shown in Table 2.

ISIincurring case: The CP length is insufficient so that there is ISI incurred at the receiver, and the transmitter is perfectly synchronized with the receiver. In this case, the proposed waveform is less prone to ISI compared to the pure filtering waveform (and WOLA) because it requires a shorter filter to achieve the same OOSBE suppression level. Therefore, the proposed waveform can use a shorter CP to mitigate ISI, which makes it more spectrally efficient than the pure filtering waveform and WOLA.
The results of the ISIfree case are shown in Figs. 8 and 9. In the simulations, the normalized TBW is set as η=3.5. It is observed that the pure filtering waveform and WOLA have BER performances similar to that of OFDM. The pure windowing waveform has an exact 1.2 dB disadvantage. Although the proposed waveform has an approximately 0.5 dB disadvantage (approximately equal to the SNR loss), it possesses the best OOSBE suppression performance as shown in Figs. 6 and 7. This advantage will be further demonstrated in Section 5.8.
In Figs. 10 and 11, the results of the ISIincurring case are demonstrated. Here, the CP length is set as L=23. The filter lengths of the pure filtering and the proposed waveforms are set as L_{f}=L=23 and L_{f}=17, respectively. They are chosen this way so that the two waveforms possess a similar maximum stopband PSD of approximately −44 dB. The CS length of WOLA is ν=17, and the resulting OOSBE suppression performance is −26.56 dB, and that of the pure windowing waveform with 1.2 dB SNR loss is −29.83 dB. In both plots, one can observe that the proposed waveform outperforms the pure filtering waveform and WOLA in the presence of ISI at the receiver. Although the BER performance of the pure windowing waveform is not be affected by ISI, its performance in suppressing OOSBE is poor, as demonstrated in Figs. 6 and 7. These results not only suggest that the proposed waveform requires a shorter filter length than the previous filteringbased waveform to achieve similar OOSBE suppression performances, but also indicate that the proposed waveform is less prone to ISI while possessing good capability of suppressing OOSBE compared to WOLA and the pure filtering waveform.
Asynchronous multiuser BER simulations
Here, BER simulations are performed based on an asynchronous uplink scenario with multiple users modulated by the same waveform. The proposed waveform has the best performance especially in this scenario.
As suggested in [1–3], the requirement of relaxed frequency synchronization is based on the necessity of deploying a waveform with reduced OOSBE compared to conventional OFDM. Therefore, frequency asynchronism caused by carrier frequency offsets (CFOs) is introduced in our simulations. In [2], it was pointed out that the CFO of a user can be estimated and compensated at the receiver without requiring any information feedback from the user. Some example methods of estimating CFOs can be found in [43, 44]. However, ICI will still occur because various CFOs of users are unable to be compensated for at the same time.
Suppose there are three users transmitting data symbols simultaneously, as illustrated in Fig. 3. The CFO of the user allocated to subband \(\mathcal {K}\) is assumed to be perfectly estimated and compensated for at the receiver, while the other two users are allocated to neighboring subbands and are asynchronous to the receiver, i.e., the CFOs of the neighboring users are not compensated for. Specifically, the two neighboring users, indexed user 1 and user 2, are allocated to subbands \(\mathcal {K}_{1} \triangleq \{ P\eta, P\eta 1, \cdots, 1\eta \} \) and \( \mathcal {K}_{2} \triangleq \{ P+\eta, P+\eta +1, \cdots, 2P+\eta 1 \} \), respectively. The data symbols of the two users are denoted as \( \textbf {s}_{P}^{(1)}[\!n] \), and \( \textbf {s}_{P}^{(2)}[\!n] \), respectively. The precoded data symbols of user l before adding CP are denoted as \( \textbf {u}_{M}^{(l)}[\!n] = \textbf {W}_{M}^{H} \textbf {E}^{T}(\mathcal {K}_{l})\textbf {s}_{P}^{(l)}[\!n], l = 1, 2 \). The window coefficients deployed by each user are the same; the filter functions used by user 1 and user 2, denoted f^{(1)}[ n] and f^{(2)}[ n], respectively, are the frequencyshifted versions of the optimal filter function to the center frequencies of the allocated subbands, i.e.,
Let h_{l}[n]∼CN(0,1),n=0,1,⋯,L_{h} denote the channel impulse response of user l, then the resulting effective channel is c_{l}[n]=h_{l}[n]∗f^{(l)}[n] where ∗ denotes the operation of convolution. For simplicity, the CP length is assumed to be sufficient to eliminate ISI. In the presence of CFO, the received signal vector after serialtoparallel conversion and CP removal is expressed as
where \(\mathbf {\Phi }({\varepsilon }_{l}) \triangleq \text {diag}\left (\left [1{e}^{j\frac {2\pi }{M}{\varepsilon }_{l}} \cdots {e}^{j\frac {2\pi }{M}(M1){\varepsilon }_{l}} \right ]\right)\) is the matrix representing phase shift incurred by the CFO, ε_{l} denotes the normalized CFO of user l, and C_{l} is the circulant matrix consisting the effective channel’s impulse response of user l [44]. The decoding of s_{P}[n] is the same as performed in (10) and (11).
In Figs. 12 and 13, the BER simulation results of the user allocated to subband \(\mathcal {K}\) are demonstrated. Here, the normalized CFOs are expressed as i.i.d. uniformly distributed random variables in the range of [−ϕ,ϕ], as suggested in [44], where ϕ=0.1. The normalized TBW is η=1. The other parameters are set as presented in Section 5.1. The maximum stopband PSD of the proposed waveform (SNR loss γ=0.5 dB), the pure filtering waveform, and the pure windowing waveform are −28.67 dB, −24.51 dB, and −20.87 dB, respectively. One can observe that the proposed waveform outperforms other waveforms significantly when \(E_{b}/\sigma _{e}^{2}\) is larger than 12.5 dB. This is because the proposed waveform has the best capability of suppressing OOSBE so that ICI introduced by CFOs can be greatly mitigated. Due to rectangular windowing and inferior capabilities in OOSBE suppression, the BER performances of OFDM, WOLA, and pure filtering waveform are similar. The pure windowing waveform has the worst BER performance since it will cause severe ICI due to destroying the orthogonality between subcarriers. In regimes of lower transmitting energy, i.e., \( E_{b}/\sigma _{e}^{2} \leq 12.5\) dB, the proposed waveform has a slight performance disadvantage compared to WOLA, the pure filtering waveform, and the conventional OFDM. This is because the BER performance of the zeroforcing receiver is more prone to noise rather than ICI in such regimes. These results suggest that the proposed waveform is more capable of suppressing ICI for asynchronous transmissions than other waveforms.
BER simulations with ETU channel model
BER simulations are executed based on the extended typical urban (ETU) channel model [45]. The results are demonstrated in Figs. 14 and 15. In these figures, the sampling frequency is 1.92 MHz, the velocity of the UE is 3 km/h, the number of subcarriers is M=128, and η=1. The filter length of the proposed joint WF method is L_{f}=9, and ν=L_{f} for WOLA in both figures. In Fig. 14, the CP lengths of all the waveforms are set as L=16, and the filter length of the pure filtering one is L_{f}=9. One can observe that the joint WF method also has a 0.5 dB performance disadvantage comparing with OFDM, WOLA, and the pure filtering method, which is the same as what was demonstrated in Fig. 9. In Fig. 15, the CP lengths of all the waveforms are set as L=8, and the filter length of the pure filtering one is L_{f}=9. The filter lengths are chosen so that the pure filtering and the proposed waveforms possess a similar maximum stopband PSD of approximately −31 dB. In this figure, it is also observed that the proposed waveform outperforms the pure filtering waveform and WOLA in the presence of ISI at the receiver, as demonstrated in Fig. 11.
Conclusions
In this paper, a joint windowing and filtering multicarrier waveform was proposed. An iterative method for joint optimization was also proposed to minimize the OOSBE with controllable SNR loss at the receiver. In contrast with the previous pure filtering and pure windowing waveforms, the proposed waveform has an advantage in suppressing OOSBE to a lower level with similar BER performances. This advantage indicates that the proposed waveform is more spectrally efficient when suppressing OOSBE than the other waveforms. Moreover, the proposed waveform requires a shorter filter length and induces less SNR loss for achieving an OOSBE suppression specification compared to previous pure filtering and pure windowing waveforms, respectively. The detailed comparison of the proposed waveform with OFDM, the pure filtering waveform, the pure windowing waveform, and WOLA is presented in Table 3. Simulation results support the above advantages of the proposed waveform.
Channel estimation and peaktoaverage power ratio (PAPR) reduction remain as open problems. Extending the proposed waveform to a multipleinputmultipleoutput system is also of interest.
Appendix 1: Derivation of constraint (27c)
By substituting (24) into (4), the PSD of the transmitted signal x[n] is expressed as
where \(\textbf {V}(e^{j\omega })= \sum _{k\in \mathcal {K}} \textbf {v}_{N}\left (e^{j(\omega \Omega _{k})}\right)\textbf {v}_{N}^{H}\left (e^{j(\omega \Omega _{k})}\right) \). By integrating (37) from π to −π,
Next, denote the ith entry of vector a(e^{jω}) as \( \left [\textbf {a}(e^{j\omega })\right ]_{i}=e^{j(L_{f}i)\omega } \). Then, the ith entry of vector q is written as
Let the (m,n)th entry of matrix V(e^{jω}) be \( \left [ \textbf {V}\left (e^{j\omega }\right) \right ]_{m,n} = \sum _{k\in \mathcal {K}} e^{j(mn)(\omega \Omega _{k})} \). Then, the N×N matrix Q_{i} is given by
By substituting Eq. (38) into (5), constraint (27c) is obtained.
Appendix 2: Proof of Theorem 1

First, prove that D(θ) is a convex set. Let \(\hat {\textbf {w}}_{1} = (\boldsymbol {1}_{2} \otimes \boldsymbol {\tau }) \circ \textbf {d}(\boldsymbol {\theta })\) and \(\hat {\textbf {w}}_{2} = \left (\boldsymbol {1}_{2} \otimes \boldsymbol {\tau }^{\prime }\right) \circ \textbf {d}(\boldsymbol {\theta })\) be two vectors in D(θ), then
$${} \lambda \hat{\mathbf{w}}_{1} + (1\lambda) \hat{\mathbf{w}}_{2} = \left\{ \boldsymbol{1}_{2} \otimes[ \lambda \boldsymbol{\tau} + (1\lambda) \boldsymbol{\tau}^{\prime} ] \right\} \circ \mathbf{d}(\boldsymbol{\theta}) \in \mathsf{D}(\boldsymbol{\theta}) $$for all m=1,⋯,M and λ∈[0,1]. Therefore, D(θ) is a convex set.
Followed by the argument of \(\text {dom}\,\hat {\sigma }\), define
$$\begin{array}{*{20}l} g(\boldsymbol{\tau}) &= \hat{\sigma}(\hat{\mathbf{w}})\big_{\hat{\mathbf{w}}= (\boldsymbol{1}_{2} \otimes \boldsymbol{\tau}) \circ \mathbf{d}(\boldsymbol{\theta})} \\ &= \frac{1}{M} \sum_{m=1}^{M} \frac{1}{\tau_{m}^{2}}, \text{dom}\,g = \mathsf{R}_{++}^{M}, \end{array} $$where \( \tau _{m} \triangleq [\boldsymbol {\tau }]_{m} \). This definition implies that \(\hat {\sigma }\) is convex if g is convex. The convexity of g is shown by its secondorder condition [30]. By taking the secondorder derivatives of g, the Hessian is written as
$$\nabla^{2} g(\boldsymbol{\tau}) = \frac{6}{M} \text{diag}\left(\left[ \ \tau_{1}^{4} \ \tau_{2}^{4} \ \cdots \ \tau_{M}^{4} \right]\right), $$which is a positive definite matrix. Therefore, \(\hat {\sigma }(\hat {\textbf {w}})\) is convex.

By taking secondorder derivatives of \(\hat {\sigma }\), the Hessian is given by
$$\left[\nabla^{2} \hat{\sigma}(\hat{\mathbf{w}})\right]_{m,n} = \left\{\begin{array}{ll} \frac{8}{M} \hat{w}_{m}^{2}\hat{\sigma}_{m}^{3}  \frac{2}{M}\hat{\sigma}_{m}^{2}, & m = n \\ \frac{8}{M}\hat{w}_{m} \hat{w}_{n} \hat{\sigma}_{m}^{3}, & mn=M \\ 0, & \text{else} \end{array} \right. $$where \( \hat {\sigma }_{m} = 1/\left (\hat {w}_{m}^{2} + \hat {w}_{m+M}^{2}\right) \) for 1≤m≤M, and \( \hat {\sigma }_{m} = 1/\left (\hat {w}_{m}^{2} + \hat {w}_{mM}^{2}\right) \) for M+1≤m≤2M. The Hessian matrix \(\nabla ^{2} \hat {\sigma }(\hat {\textbf {w}})\) is not positive semidefinite. This property can be shown by considering the following example: \(\hat {\textbf {w}} = \frac {1}{\sqrt {2}} \boldsymbol {1}_{2M}\). The resulting minimum eigenvalue of the Hessian is −2/M. Therefore, the function \(\hat {\sigma }\) is nonconvex.
Abbreviations
 CFO:

Carrier frequency offset
 CP:

Cyclic prefix
 CS:

Cyclic suffix
 DFT:

Discrete fourier transform
 fOFDM:

FilteredOFDM
 FBMC:

Filter bank multicarrier
 FFT:

Fast fourier transform
 FIR:

Finite impulse response
 GFDM:

Generalized frequency division multiplexing
 ICI:

Intercarrier interference
 IFFT:

Inverse FFT
 IPM:

Interiorpoint method
 ISI:

Intersymbol interference
 OFDM:

Orthogonal frequency division multiplexing
 OOSBE:

Outof subband emissions
 PAPR:

Peaktoaverage power ratio
 PSD:

Power spectral density
 RC:

Raisedcosine
 SNR:

Signaltonoise ratio
 TBW:

Transition bandwidth
 UFMC:

Universalfiltered multicarrier
 VC:

Virtual carrier
 WOLA:

Weighted overlapandadd
References
 1
J. G. Andrews, S. Buzzi, W. Choi, S. V. Hanly, A. Lozano, A. C. K. Soong, J. C. Zhang, What will 5G be?. IEEE J. Sel. Areas Commun.32(6), 1065–1082 (2014).
 2
G. Wunder, P. Jung, M. Kasparick, T. Wild, F. Schaich, Y. Chen, S. T. Brink, I. Gaspar, N. Michailow, A. Festag, L. Mendes, N. Cassiau, D. Ktenas, M. Dryjanski, S. Pietrzyk, B. Eged, P. Vago, F. Wiedmann, 5GNOW: nonorthogonal, asynchronous waveforms for future mobile applications. IEEE Commun. Mag.52(2), 97–105 (2014).
 3
X. Zhang, L. Chen, J. Qiu, J. Abdoli, On the waveform for 5G. IEEE Commun. Mag.54(11), 74–80 (2016).
 4
A. A. Zaidi, R. Baldemair, H. Tullberg, H. Bjorkegren, L. Sundstrom, J. Medbo, C. Kilinc, I. D. Silva, Waveform and numerology to support 5G services and requirements. IEEE Commun. Mag.54(11), 90–98 (2016).
 5
L. Zhang, A. Ijaz, P. Xiao, A. Quddus, R. Tafazolli, Subband filtered multicarrier systems for multiservice wireless communications. IEEE Trans. Wirel. Commun.16(3), 1893–1907 (2017).
 6
L. Zhang, A. Ijaz, P. Xiao, R. Tafazolli, Multiservice system: an enabler of flexible 5G air interface. IEEE Commun. Mag.55(10), 152–159 (2017).
 7
P. Guan, D. Wu, T. Tian, J. Zhou, X. Zhang, L. Gu, A. Benjebbour, M. Iwabuchi, Y. Kishiyama, 5G field trials: OFDMbased waveforms and mixed numerologies. IEEE J. Sel. Areas Commun.35(6), 1234–1243 (2017).
 8
B. FarhangBoroujeny, OFDM versus filter bank multicarrier. IEEE Signal Proc. Mag.28(3), 92–112 (2011).
 9
X. Mestre, M. Majoral, S. Pfletschinger, An asymptotic approach to parallel equalization of filter bank based multicarrier signals. IEEE Trans. Signal Process.61(14), 3592–3606 (2013).
 10
M. Pauli, P. Kuchenbecker, in Proc. IEEE Int. Conf. Commun., vol. 3. On the reduction of the outofband radiation of OFDMsignals (IEEEAtlanta, 1998), pp. 1304–1308.
 11
Y. Medjahdi, R. Zayani, H. Shaïk, D. Roviras, in 2017 IEEE International Conference on Communications Workshops ICC Workshops). WOLA processing: a useful tool for windowed waveforms in 5G with relaxed synchronicity (IEEEParis, 2017), pp. 393–398.
 12
Y. P. Lin, S. M. Phoong, Window designs for DFTbased multicarrier systems. IEEE Trans. Signal Process.53(3), 1015–1024 (2005).
 13
D. Roque, C. Siclet, Performances of weighted cyclic prefix OFDM with lowcomplexity equalization. IEEE Trans. Commun.17(3), 439–442 (2013).
 14
N. Michailow, M. Matthé, I. S. Gaspar, A. N. Caldevilla, L. L. Mendes, A. Festag, G. Fettweis, Generalized frequency division multiplexing for 5th generation cellular networks. IEEE Trans. Commun.62(9), 3045–3061 (2014).
 15
H. M. Chen, W. C. Chen, C. D. Chung, Spectrally precoded OFDM and OFDMA with cyclic prefix and unconstrained guard ratios. IEEE Trans. Wirel. Commun.10(5), 1416–1427 (2011).
 16
M. Ma, X. Huang, B. Jiao, Y. J. Guo, Optimal orthogonal precoding for power leakage suppression in DFTbased systems. IEEE Trans. Commun.59(3), 844–853 (2011).
 17
A. Farhang, N. Marchetti, L. E. Doyle, Lowcomplexity modem design for GFDM. IEEE Trans. Signal Process.64(6), 1507–1518 (2016).
 18
V. Vakilian, T. Wild, F. Schaich, S. ten Brink, J. F. Frigon, in 2013 IEEE Globecom Workshops (GC Wkshps). Universalfiltered multicarrier technique for wireless systems beyond LTE (IEEEAtlanta, 2013), pp. 223–228.
 19
T. Wild, F. Schaich, Y. Chen, in 2014 19th International Conference on Digital Signal Processing. 5G air interface design based on universal filtered (UF)OFDM (IEEEHong Kong, 2014), pp. 699–704.
 20
M. F. Tang, B. Su, in 2016 IEEE International Conference on Communications Workshops (ICC). Filter optimization of low outofsubband emission for universalfiltered multicarrier systems (IEEEKuala Lumpur, 2016), pp. 468–473.
 21
L. Zhang, P. Xiao, A. Quddus, Cyclic prefixbased universal filtered multicarrier system and performance analysis. IEEE Trans. Signal Process.23(9), 1197–1201 (2016).
 22
J. Abdoli, M. Jia, J. Ma, in 2015 IEEE 16th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). Filtered OFDM: a new waveform for future wireless systems (IEEEStockholm, 2015), pp. 66–70.
 23
X Zhang, M Jia, L Chen, J Ma, J Qiu, in 2015 IEEE Global Communications Conference (GLOBECOM). Filteredofdm  enabler for flexible waveform in the 5th generation cellular networks (IEEESan Diego, 2015), pp. 1–6.
 24
P. Weitkemper, J. Bazzi, K. Kusume, A. Benjebbour, Y. Kishiyama, in 2016 IEEE International Conference on Communications Workshops (ICC). Adaptive filtered OFDM with regular resource grid (IEEEKuala Lumpur, 2016), pp. 462–467.
 25
D. J. Han, J. Moon, D. Kim, S. Y. Chung, Y. H. Lee, Combined subbandsubcarrier spectral shaping in multicarrier modulation under the excess frame length constraint. IEEE J. Sel. Areas Commun.35(6), 1339–1352 (2017).
 26
Y. P. Lin, S. M. Phoong, P. Vaidyanathan, Filter bank transceivers for OFDM and DMT systems (Cambridge University Press, Cambridge, United Kingdom, 2010).
 27
J. A. Zhang, X. Huang, A. Cantoni, Y. J. Guo, Sidelobe suppression with orthogonal projection for multicarrier systems. IEEE Trans. Commun.60(2), 589–599 (2012).
 28
P. P. Vaidyanathan, Multirate systems and filter banks (PrenticeHall, New Jersey, 1993).
 29
K. Ichige, M. Iwaki, R. Ishii, Accurate estimation of minimum filter length for optimum FIR digital filters. IEEE Trans. Circ. Syst. II: Analog Digit. Signal Process.47(10), 1008–1016 (2000).
 30
S. Boyd, L. Vandenberghe, Convex optimization (Cambridge university press, Cambridge, United Kingdom, 2004).
 31
A. Papoulis, Signal analysis vol. 191 (McGrawHill, New York, 1977).
 32
S. P. Wu, S. Boyd, L. Vandenberghe, in Proceedings of 35th IEEE Conference on Decision and Control, vol. 1. FIR filter design via semidefinite programming and spectral factorization (IEEEKobe, 1996), pp. 271–276.
 33
A. S. Alkhairy, K. G. Christian, J. S. Lim, Design and characterization of optimal FIR filters with arbitrary phase. IEEE Trans Signal Process.41(2), 559–572 (1993).
 34
M. Grant, S. Boyd, CVX: MATLAB Software for disciplined convex programming, version 2.1 (2014). http://cvxr.com/cvx.
 35
R. G. Lorenz, S. P. Boyd, Robust minimum variance beamforming. IEEE Trans. Signal Process.53(5), 1684–1696 (2005).
 36
Z. L. Yu, W. Ser, M. H. Er, Z. Gu, Y. Li, Robust adaptive beamformers based on worstcase optimization and constraints on magnitude response. IEEE Trans. Signal Process.57(7), 2615–2628 (2009).
 37
S. W. Peters, R. W. Heath, in 2009 IEEE International Conference on Acoustics, Speech and Signal Processing. Interference alignment via alternating minimization (IEEETaipei, 2009), pp. 2445–2448.
 38
L. Kong, S. Han, C. Yang, Hybrid Precoding With Rate and Coverage Constraints for Wideband Massive MIMO Systems. IEEE Trans. Wirel. Commun.17(7), 4634–4647 (2018).
 39
M. Colombo, J. Gondzio, Further development of multiple centrality correctors for interior point methods. Comput. Optim. Appl.41(3), 277–305 (2008).
 40
T. Wild, F. Schaich, in 2015 IEEE 81st Vehicular Technology Conference (VTC Spring). A reduced complexity transmitter for UFOFDM (IEEEGlasgow, 2015), pp. 1–6.
 41
G. Cuypers, K. Vanbleu, G. Ysebaert, M. Moonen, P. Vandaele, in Communications, 2003. ICC ’03. IEEE International Conference On, vol. 4. Combining raised cosine windowing and per tone equalization for RFI mitigation in DMT receivers (IEEEAnchorage, 2003), pp. 2852–28564.
 42
J. G. Proakis, Digital communications, 4th edn (McGrawHill, New York, 2001).
 43
H. T. Hsieh, W. R. Wu, Blind maximumlikelihood carrierfrequencyoffset estimation for interleaved OFDMA uplink systems. IEEE Trans. Veh. Technol.60(1), 160–173 (2011).
 44
X. N. Zeng, A. Ghrayeb, Joint CFO and channel estimation for OFDMA uplink: an application of the variable projection method. IEEE Trans. Wirel. Commun.8(5), 2306–2311 (2009).
 45
S. Sesia, M. Baker, I. Toufik, LTE  the UMTS long term evolution: from theory to practice (John Wiley & Sons, New Jersey, 2011).
Acknowledgements
The authors would like to thank Dr. Gordon L. Stüber, Joseph M. Pettit Chair Professor of ECE department, Georgia Institute of Technology, for his valuable comments that significantly improved the quality of this paper.
Funding
This work is supported by the Ministry of Science and Technology, Taiwan, under contracts MOST 1062221E002034.
Availability of data and materials
Not available online. Please contact author for data requests.
Author information
Affiliations
Contributions
MFT made the main contributions to the joint window and filter optimization algorithms’ design and experiments, as well as drafting the manuscript. BCS checked the manuscript and offered critical suggestions to design the algorithm. Both authors read and approved the final manuscript.
Corresponding author
Correspondence to MingFu Tang.
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Received
Accepted
Published
DOI
Keywords
 Filteringbased waveform
 Windowingbased waveform
 Alternative optimization
 Joint window and filter optimization
 Multicarrier modulation
 Outofsubband emissions