Approximate equiangular tight frames for compressed sensing and CDMA applications

Abstract

Performance guarantees for recovery algorithms employed in sparse representations, and compressed sensing highlights the importance of incoherence. Optimal bounds of incoherence are attained by equiangular unit norm tight frames (ETFs). Although ETFs are important in many applications, they do not exist for all dimensions, while their construction has been proven extremely difficult. In this paper, we construct frames that are close to ETFs. According to results from frame and graph theory, the existence of an ETF depends on the existence of its signature matrix, that is, a symmetric matrix with certain structure and spectrum consisting of two distinct eigenvalues. We view the construction of a signature matrix as an inverse eigenvalue problem and propose a method that produces frames of any dimensions that are close to ETFs. Due to the achieved equiangularity property, the so obtained frames can be employed as spreading sequences in synchronous code-division multiple access (s-CDMA) systems, besides compressed sensing.

1 Introduction

Recent theoretical and practical results in mathematics, computer science, and engineering highlight that frame theory is fundamental in many research areas, and frames are useful in numerous applications requiring redundancy [1]. Among all frames, equiangular unit norm tight frames (ETFs) have a special structure that makes them particularly important in many fields like signal processing [2, 3], quantum information theory [4, 5], and communications [6–9].

From a geometrical perspective, an ETF is a set of unit norm vectors in a Hilbert space forming equal angles, therefore, having identical correlation, which is also the smallest possible [6]. Besides small column correlation, ETFs exhibit row orthogonality, a property known as tightness. These properties are important when considering overcomplete spanning systems, as they lead to frames that are close to orthonormal bases. However, ETFs either do not exist for arbitrary frame dimensions or their construction is difficult [6, 10], thus, in many problems, closely related frames are used as substitutes.

According to well-known results from frame and graph theory [6, 7], an equiangular frame can be defined up to unitary equivalence by its so called signature matrix. Considering real equiangular frames, the corresponding signature matrix is a symmetric matrix with zero diagonal and ± 1’s off-diagonal entries, and it can be thought of as the Seidel matrix of a graph. It is known that the signature matrix corresponding to a real ETF has exactly two distinct eigenvalues [7]. Therefore, the problem of designing an ETF can be reduced to an inverse eigenvalue problem, that is, the construction of a matrix with specific structure and spectrum consisting of two distinct eigenvalues.

Many signature matrices that correspond to ETFs are known and constructions of ETFs based on signature matrices have been proposed in [6]. These techniques impose certain restrictions on frame dimensions. In this paper, we consider frames of arbitrary dimensions and construct a symmetric matrix with spectrum that approximates the spectrum of the corresponding signature matrix. The obtained matrix is then used for the construction of frames that are close to ETFs. The produced frames are almost tight, with frame vectors forming angles that approximate the optimal value.

Small column correlation as well as tightness are important when designing sensing matrices for compressed sensing (CS) [11, 12]. CS is a new acquisition technique for sparse signals that can reduce the sampling rate drastically, as long as the matrix implementing the sensing mechanism satisfies the aforementioned properties. These properties play an important role in the performance of the numerical algorithms deployed to recover sparse signals [13]. Therefore, any improvement concerning the sensing matrix results in higher reconstruction accuracy and reduction of the number of necessary measurements.

Tight frames are considered the best candidates when designing spreading sequences in code-division multiple access (CDMA) systems. Equal norm tight frames [14], also known as Welch bound equality (WBE) sequences [15], are considered capacity optimal spreading sequences and minimize the interference experienced by the individual users. However, it was shown in [9] that when the number of active users in the system changes, only ETFs result in the same interuser interference. Our simulations show that if an ETF construction is not available, nearly equiangular nearly tight frames can reduce interuser interference substantially when used as spreading sequences.

The rest of the paper is organized as follows: Section 2 reviews some results of frame theory and the connection of frames to graphs. In Section 3, we present two algorithms for constructing signature matrices, which we use to obtain nearly equiangular frames. In Section 4, we use the proposed frames as sensing matrices for compressed sensing and spreading sequences for synchronous CDMA systems. Conclusions are drawn in Section 5.

2 Frames review

2.1 Finite frames basics

A finite frame in a real or complex m-dimensional Hilbert space $\mathbb {H}^{m}$ is a sequence of N≥m vectors $\left \{f_{k} \right \}_{k=1}^{N}$, $f_{k} \in \mathbb {H}^{m}$, satisfying the following condition

$$ \alpha\left\|f\right\|_{2}^{2} \leq \sum\limits_{k=1}^{N} \left|\left\langle f,f_{k} \right\rangle \right|^{2} \leq \beta \left\|f\right\|_{2}^{2} \quad \forall f\in \mathbb {H}^{m}, $$

(1)

with positive constants α and β [1]. We refer to α, β as the lower and upper frame bounds, respectively. The m×N matrix F=[f ₁ f ₂ …f _N ], with columns the frame vectors f _k , is usually identified with the frame itself.

The following notions are related to a frame $\left \{f_{k} \right \}_{k=1}^{N}$ [1]:

• (a) The ratio ρ=N/m is referred to as the redundancy of the frame and is a “measure of overcompleteness” of the frame.

• (b) When α=β, we say that the frame is α-tight.

• (c) A frame is called uniform or equal norm, when ∥f _k ∥=C, C>0, for all k∈{1,…,N}, and unit norm, when ∥f _k ∥=1 for all k∈{1,…,N}.

• (d) For a unit norm frame, the absolute value of the inner product between two frame vectors equals the cosine of the acute angle between the lines spanned by the two vectors. If there is a constant c>0 for which |〈f _k ,f _ℓ 〉|=c, k≠ℓ, then the frame is called equiangular.

An important operator of frames is the Grammian operator [1]. The matrix representation of the Grammian of a frame is called the Gram matrix. The Gram matrix is the N×N matrix R=F ^∗ F, where F ^∗ denotes the conjugate transpose of F. It is known [1] that F is an m×N frame if and only if the Gram matrix is a self adjoint N×N projection with rank m. Frames $F = \{f_{k}\}_{k=1}^{N}$ and $G = \{g_{k}\}_{k=1}^{N}$ are unitarily equivalent, if there is a unitary transformation $U:\mathbb {H}^{m}\rightarrow \mathbb {H}^{m}$ with F=UG:={Uf _k }, k∈{1,…,N}. A frame is determined by its Gram matrix up to unitary equivalence.

2.1.1 Tight frames

When α=β, we obtain an α-tight frame, that is

$$ f = \frac{1}{\alpha} \sum\limits_{k=1}^{N} \left\langle f,f_{k} \right\rangle f_{k} \quad \forall f\in \mathbb{H}^{m}. $$

(2)

In this case, the rows of α ^−1/2 F form an orthogonal family, each with norm $\sqrt {\alpha }$. It follows immediately that FF ^∗=α I _m , where I _m is the m×m identity matrix. Considering the spectral properties of an α-tight frame, the following proposition summarizes well-known results.

Proposition 1

(Spectral properties of tight frames [1])

A frame is α-tight if and only if one of the following conditions holds:

• (a) The non-zero eigenvalues of the Gram matrix equal α.

• (b) The non-zero singular values of F equal $\sqrt {\alpha }$.

• (c) The spectral norm of F equals $\sqrt {\alpha }$.

Constructing a tight frame is straightforward; we take an orthonormal basis and select the desired number of rows. For example, m×N harmonic tight frames are obtained by deleting (N−m) rows of an N×N DFT matrix.

2.1.2 Unit norm tight frames

Finite frames that are both tight and normalized are called unit norm tight frames (UNTFs) and possess a significant structure. A UNTF can be thought of as a sequence that retains the decomposition properties of orthonormal bases while relaxing the need to be a basis. There is only one choice for the frame bound of a UNTF of N vectors for $\mathbb {H}^{m}$, which is given by the following theorem.

Theorem 1

([16]) If $\{f_{k}\}_{k=1}^{N}$ is a finite unit norm α-tight frame for an m-dimensional Hilbert space $\mathbb {H}^{m}$, then α=N/m.

Therefore, a UNTF in a finite dimensional space is an m×N matrix such that (a) the rows are orthogonal, (b) each row has norm $\sqrt {N/m}$, (c) each column has norm 1. The spectral properties of a UNTF are obtained by Proposition 1 by setting α=N/m. The value of the spectral norm of a UNTF is the lowest possible bound for m×N frames and is often used as a measure of how close a given frame is to a UNTF.

2.1.3 Equiangular tight frames

When a unit norm tight frame has vectors forming equal angles, we obtain an equiangular tight frame. ETFs exhibit equal column correlation, which is also the smallest possible [6]; thus, they are maximally incoherent equiangular frames. ETFs are arguably the most important class of finite-dimensional frames, and they are the natural choice when one tries to combine the advantages of orthonormal bases with the concept of redundancy provided by frames.

The maximal correlation between different normalized frame vectors, also known as mutual coherence, is defined as μ(F)= max1≤k,ℓ≤Nk≠ℓ|〈f _k ,f _ℓ 〉|. Mutual coherence is bounded according to

$$ \mu(F) \geq \sqrt{\frac{N-m}{m(N-1)}}, $$

(3)

which is referred to as Welch bound [6]. Equality holds, if and only if F is an equiangular tight frame [6]. As unit norm tight frames with dimensions m,N exist for a specific tightness parameter (α=N/m), an ETF is an equiangular N/m-tight frame.

Despite their important properties and their numerous practical applications, there is no explicit way of constructing ETFs. This problem is connected with other important problems such as packings in Grassmannian spaces and antipodal spherical codes. It has also connections to graph theory, equiangular line sets, and coding theory. The techniques reported in [3, 6, 17–19] construct only a few of the existent frames.

2.2 Connection between ETFs and graphs

Studies concerning the connection of frames with graphs have shown that the existence of an ETF in a real Hilbert space depends on the existence of a matrix Q with zero diagonal and ± 1’s off-diagonal entries. This matrix corresponds to the Seidel matrix of a special type of strongly regular graphs [6]. Recall that the Seidel matrix of a graph with N vertices is an N×N matrix with the (i,j) entry defined to be 1 if the i and j vertices are adjacent, −1 if the i and j vertices are not adjacent, and 0 if i=j. From [6, 7] we obtain the following definition.

Definition 1

Given an m×N ETF F=[f ₁ f ₂ …f _N ], the Gram matrix can be written in the form

$$ R = I+cQ, $$

(4)

where I is the N×N identity matrix and c is the Welch bound given by (3). The N×N matrix Q, is known as the signature matrix of the frame F.

The main results about signature matrices are summarized in the following theorem.

Theorem 2

([7]) Let Q be a self-adjoint N×N matrix, with q _i,i =0 for all i and |q _i,j |=1 for all i≠j. Then the following are equivalent:

• i. Q is the signature matrix of an m×N ETF.

• ii. Q ²=(N−1)I+ν Q for some necessarily real number ν.

• iii. Q has exactly two distinct eigenvalues, denoted as λ ₁<λ ₂.

When any of the above conditions hold, the parameters m,N,ν,λ ₁,λ ₂ are related with certain equations [8], implying that for many values of m,N ETFs do not exist. It can be shown that [19]

$$ \begin{aligned} &{\lambda}_{1} = -\sqrt{\frac{m(N-1)}{N-m}}, \text{with multiplicity}~N-m, \\ &{\lambda}_{2} = \sqrt{\frac{(N-m)(N-1)}{m}}, \text{with multiplicity}~ m. \end{aligned} $$

(5)

According to [7], there are finitely many possible N×N signature matrices and finitely many real equiangular frames of N vectors. For more details about the connection between graphs and frames, the reader is referred to [6–8, 19, 20].

3 Construction of nearly equiangular frames

3.1 Previous work and our contribution

When designing a frame, the design specifications arise from the application of interest. As a result, there exist a large number of construction methods, as diverse as the applications requiring a frame. Usually, the constructions that come to address specific requirements are difficult to generalize to solve different types of frame design problems. On the other hand, more general constructions [3, 6, 17–19, 21] impose certain restrictions on the frame dimensions.

Recently, the construction of equiangular tight frames has gained the interest of the sparse modeling community, as ETFs are maximally incoherent. Thus, a category of algorithms that produce incoherent frames has been developed in the latest years [22–30]. Most of the existing algorithms are inspired by the work of [17] and are based on a “shrinkage” operation on the Gram matrix. For example, the algorithm proposed in [22] starts from an arbitrary m×N frame that has full rank, and sequentially “shrinks” the absolute values of the off-diagonal entries of the Gram matrix in order to reduce frame’s column correlation. Then, truncated SVD is used to obtain a frame with rank m.

Our motivation in frame design originates from the application of incoherent frames in sparse recovery and compressed sensing. However, in contrast to the existing methods that operate on the absolute values of the Gram matrix entries, the method proposed here is inspired by the results presented in Theorem 2 and aims at finding the right signs of these entries. While the approach in this paper involves some operations on the Gram matrix, the proposed methodology is based on the construction of an appropriate “signature” matrix. Compared to existing algorithms that produce incoherent frames, that is, frames exhibiting small maximum column correlation, the proposed frames not only exhibit small column correlation but the degree of correlation between different pairs of columns is similar. Thus, the obtained frames are approximately equiangular.

Before proceeding, we need to define the signature matrix of an arbitrary real frame. Suppose we are given an ETF with dimensions m, N. From Eq. (4) we see that we can derive the N×N signature matrix from the corresponding Gram matrix by keeping the signs of the off-diagonal entries and zeroing the diagonal. In the same manner, we can obtain an N×N symmetric matrix with ± 1’s off-diagonal entries and zero diagonal from the Gram matrix of an arbitrary m×N frame. Therefore, we obtain the following definition.

Definition 2

The signature matrix Q of an arbitrary m×N real frame F=[f ₁ f ₂ …f _N ] is the N×N matrix with entries derived from the corresponding Gram matrix, R=F ^T F, according to

$$ q_{ij}= \left\{ \begin{array}{llll} & \text{sgn}~(r_{ij}), \quad && {i\neq j}, \\ & 0, \quad && {i=j}, \end{array} \right. $$

(6)

where r _ij is the (i,j) entry of R.

Obviously, the eigenvalues of an arbitrary signature matrix do not satisfy (5).

According to Theorem 2 the construction of an m×N ETF reduces to finding the corresponding N×N signature matrix. Therefore, if we construct a signature matrix that approximates the signature matrix of an ETF, using Eq. (4), we can obtain a frame that is close to an ETF. The method we propose here directly operates on the signature matrix of an initial m×N frame, in order to obtain a matrix that approximates the spectral and structural properties defined in Theorem 2. Then, the Gram matrix calculated by Eq. (4) yields a frame with small mutual coherence, almost tight, with the additional property of approximate equiangularity, that is, the frame columns form similar angles. To our knowledge, this property is not achieved by any other method in the literature. Approximate equiangularity makes the produced frames appropriate for other applications besides compressed sensing. In our experimental results, we demonstrate the employment of these frames as spreading sequences for s-CDMA systems.

Construction of ETFs based on signature matrices has been proposed by the frame community in [6]. The authors of [6] have proved that an explicit construction of an ETF of dimensions m×N can be obtained, if N=2m and N=p ^α+1, where p is an odd prime number and $\alpha \in \mathbb {N}$. For frame dimensions satisfying these constraints, an N×N symmetric signature matrix, the so called conference matrix, can be obtained through a recursive process. The algorithm we propose here produces signature matrices that yield frames close to ETFs without imposing any restrictions on frame dimensions.

3.2 Problem formulation

A problem concerning the construction of a matrix from prescribed spectral data is an inverse eigenvalue problem (IEP). A large category of IEPs includes structured inverse eigenvalue problems (SIEPs), where given a set $\mathcal {N}$ of specially structured matrices and a set of scalars $\left \{\lambda _{i}\right \}_{i=1}^{N}$, $\lambda _{i} \in \mathbb {R}$, corresponding to the desired spectrum, we want to find a matrix $X \in \mathcal {N}$ such that $\sigma (X) = \left \{\lambda _{i}\right \}_{i=1}^{N}$, where σ(X) denotes the spectrum of X [31]. SIEPs are difficult to solve and most of the existing algorithms have been designed to solve problems of special type [31, 32].

The signature matrix of an ETF is a symmetric matrix with zero diagonal, ± 1’s off-diagonal entries, and spectrum containing the eigenvalues given by (5). The problem we need to solve to find a signature matrix is formulated as follows:

Signature matrix inverse eigenvalue problem (SMIEP). Considering a set of two real numbers, λ ₁,λ ₂, given by (5), find a symmetric N×N matrix with zero diagonal, ± 1’s off-diagonal entries and spectrum

$$ \sigma = \{ \underbrace{{\lambda}_{1},\dots,{\lambda}_{1},}_{N-m} \underbrace{{\lambda}_{2}, \dots,{\lambda}_{2}}_{m}\}, \quad m < N. $$

(7)

Therefore, the signature matrix problem is a special type of SIEP.

A similar problem addressed in [32] is the symmetric non-negative inverse eigenvalue problem (SNIEP), that is, finding a symmetric matrix with non-negative entries and specific spectrum. The numerical method for the solution of SNIEP presented in [32] utilizes alternating projection ideas, proposing an algorithm in which, first, the eigenvalue decomposition is used to impose the desired spectrum, and, then, every negative entry of the obtained matrix is set to zero to obtain a non-negative matrix. The numerical method we propose here for the construction of a signature matrix is inspired by the work of [32].

3.3 Construction of signature matrices

A solution to SMIEP would be a matrix satisfying the structure of a signature matrix and the spectrum defined by (7). Let $\mathcal {A}$ denote the set of N×N real symmetric matrices with zero diagonal and ± 1’s off-diagonal entries

$$ \mathcal{A} = \left\{Q \in \mathbb{R}^{N\times N} | q_{ij}=\pm 1, i \neq j, ~\text{and}~q_{ii}=0 \right\}. $$

(8)

Let $\mathcal {B}$ denote the set of real symmetric matrices with spectrum σ,

$$ \mathcal{B} = \left\{Q \in \mathbb{R}^{N\times N} | Q = P \Sigma P^{-1}\right\}, $$

(9)

where Σ=diag (σ) and P is some orthogonal matrix. The signature matrix problem (SMIEP) can be stated as

$$ \text{Find}~ X \in \mathcal{A}\cap\mathcal{B}. $$

(10)

The numerical method we propose alternatively projects between $\mathcal {A}$ and $\mathcal {B}$. The projection onto $\mathcal {A}$ is straightforward. Concerning the projection onto $\mathcal {B}$, we use Theorem 3.2 found in [32]. Given a matrix $M \in \mathbb {R}^{N \times N}$, with M=Pdiag(μ ₁,μ ₂,…,μ _N )P ⁻¹, where P is a real orthogonal matrix and μ ₁≥μ ₂≥⋯≥μ _N , a projection onto $\mathcal {B}$ is given by P Σ P ⁻¹.

Therefore, starting from an initial symmetric matrix Q ₀ of dimensions N×N, with zero diagonal, ± 1’s off-diagonal entries and arbitrary spectrum, in the k-th iteration we do the following:

• Step 1. Compute the eigenvalue decomposition Q _k−1=P Λ P ⁻¹, where Λ is a diagonal matrix containing the eigenvalues of Q _k−1 and P is the matrix of the corresponding eigenvectors. Then, project onto $\mathcal {B}$ applying $\widetilde {Q}_{k}= P \Sigma P^{-1}$, where Σ:=diag(σ) is the diagonal matrix with entries the desired eigenvalues.

• Step 2. Project onto $\mathcal {A}$ obtaining a matrix Q _k with the desired structure that is approximant to $\widetilde {Q}_{k}$, by keeping the signs of the off-diagonal entries of $\widetilde {Q}_{k}$ and set the diagonal to zero,

  $$ q_{ij}= \left\{ \begin{array}{llll} & \text{sgn}~(\tilde{q}_{ij}), \quad && {i\neq j}, \\ & 0, \quad && {i=j}. \end{array} \right. $$

  (11)

Step 1 replaces the eigenvalues of the given matrix with the requested ones, thus, it yields a matrix with the desired spectrum, impairing the matrix structure. Step 2 yields a matrix exhibiting the requested structure, impairing the matrix spectrum. The above steps bring up Algorithm 1. Note that, due to small numerical inaccuracy, $\widetilde {Q}_{k}$ from step 1 may not be perfectly symmetric, thus, we perform the following operation: $\widetilde {Q}_{k} := 0.5 \cdot (\widetilde {Q}_{k}^{T}+\widetilde {Q}_{k})$.

Studying the convergence of the proposed algorithm is not a trivial task. Well-known results from alternating projections cannot be applied here because convexity conditions for the employed sets are not satisfied, and in case the corresponding ETF does not exist, SMIEP is not solvable. Therefore, the proposed numerical method for SMIEP does not always produce an exact solution. However, it can produce an approximate solution satisfying structural constraints and approximating spectral constraints. Although such a matrix is not the signature matrix of an ETF, it can be used to obtain a frame that is close to an ETF as we will see next.

Before proceeding to the construction of ETFs, let us see some experimental results concerning the produced signature matrices. First, we use Algorithm 1 to find signature matrices of ETFs that are known to exist. Our experiments have shown that the algorithm can produce the signature matrices of ETFs with dimensions m×(m+1) in a few iterations. When the algorithm is used to construct ETFs of other dimensions, e.g., 5×10, 6×16, it may need a few trials (with different starting matrices) to find the corresponding signature matrices. A possible explanation for this is that the algorithm may converge locally, thus, finding a solution depends on the starting matrix. Considering that the construction of ETFs with the aforementioned dimensions is a problem that has already been solved (see [33] for a list of known ETFs), the most important results of Algorithm 1 concern finding the signature matrices of nearly equiangular frames of arbitrary dimensions. To test the proposed algorithm, we employ an initial signature matrix obtained from a randomly generated frame. We use frames of various dimensions. After a few iterations, the proposed algorithm produces a signature matrix with the requested structure and significantly improved spectrum that approximates (7); therefore, Algorithm 1 yields an approximate solution to SMIEP. Figure 1 demonstrates results concerning the spectrum of a signature matrix before and after applying Algorithm 1. The initial signature matrix was obtained by a random Gaussian 64×128 matrix.

Fig. 1

The spectrum of the signature matrix of a 64×128 random Gaussian matrix before and after processing the matrix with Algorithms 1 and 2. The black dotted line stands for the spectrum of the signature matrix corresponding to a 64×128 ETF

Our experiments with Algorithm 1 have shown that, even though the proposed processing improves the signature matrix spectrum substantially, it becomes ineffective after a few iterations. To further improve our results, we propose to modify the second step as follows. Before changing the value of a matrix entry according to (11), we examine its distance from 1 (off-diagonal) or 0 (diagonal). To avoid a significant spectrum impairment, if this distance is greater than a threshold t, we keep the entry unchanged, that is

$$ q_{ij}= \left\{ \begin{array}{lllll} & \text{sgn}(\tilde{q}_{ij}), \quad && {\text{if} \quad \left|1-\left|\tilde{q}_{ij}\right|\right|<t}, \qquad & {i\neq j}, \\ & 0, \quad && {\text{if} \quad \left|\tilde{q}_{ij}\right|<t}, \qquad & {i=j},\\ & {\tilde{q}_{ij}}, \quad && {\text{otherwise.}} \end{array} \right. $$

(12)

According to (12), an off-diagonal positive (negative) entry $\tilde {q}_{ij}$ is projected on +1 (−1), if the distance between the current entry and the target value is less than t. Similarly, a diagonal entry is projected on 0, if its distance from 0 is smaller than t. In the experiments presented in this paper, we choose t=1/2, but fine tuning is possible depending on the frame dimensions. Thus, the second step of the algorithm does not actually project on $\mathcal {A}$, but on a broader set; equivalently, the steps towards $\mathcal {A}$ are smaller compared to Algorithm 1.

This way the k-th iteration does not produce a matrix having the appropriate entries, but structure is improved gradually. After a number of iterations is reached, we apply (11) to finally produce a matrix with the desired structure. Thus, we obtain Algorithm 2. Experimental results showing the improvement achieved with Algorithm 2 are presented in Fig. 1.

Algorithm 2 is a modified alternating projections method. Instead of taking full projection steps, it moves slowly towards the target set $\mathcal {A}$ by selectively projecting on {− 1,0,+ 1} the entries of the signature matrix that satisfy a threshold requirement. The idea to move slowly towards the target sets when applying alternating projections is not new. In [34, 35], a method referred to as relaxed alternating projections replaces the unrelaxed projection steps by underrelaxed versions. According to this method, the projection operators $P_{\mathcal {A}}$ and $P_{\mathcal {B}}$ providing the projections of the signature matrix on the sets $\mathcal {A}$ and $\mathcal {B}$, respectively, may be replaced by $(1-\lambda)I_{N} - \lambda P_{\mathcal {A}}$ and $(1-\mu)I_{N} - \mu P_{\mathcal {B}}$, where λ,μ∈(0,1] are the relaxation parameters and I _N , the N×N identity matrix. Moving only partway towards the constraint set may enlarge the regions of convergence. Applying relaxed alternating projections in our problem did not yield better results than Algorithm 1. In our experiments, we have also tried other relaxation schemes; the projections expressed by (12) yielded the best results. Algorithm 2 can be viewed as a modified version of relaxed alternating projections and its convergence will be studied in our future work.

3.4 Nearly equiangular frames based on signature matrices

The signature matrix obtained by Algorithm 2 will be used next to construct a nearly equiangular frame. First, we construct the Gram matrix R according to (4). A symmetric N×N matrix obtained by (4) corresponds to an m×N frame if it is of rank m. Thus, a rank reduction step follows. Using singular value decomposition (SVD), we keep the m largest eigenvalues and set the rest to zero. The matrix produced after rank reduction may not have ones in the diagonal, therefore, a normalization step follows to ensure that the Gram matrix corresponds to a unit norm frame. Finally, using SVD, we obtain an m×N frame, which is unit norm, almost tight, with the frame vectors forming angles near the optimal value. More particularly, let R=USV ^T be the singular value decomposition of the obtained Gram matrix, after rank reduction and normalization. Considering that R is a symmetric matrix, thus U=V, we can obtain a frame F determined by R according to $F=\sqrt {S}V$, where $\sqrt {S}$ is a diagonal matrix with entries the square roots of the singular values of R. The above steps bring up Algorithm 3. Recall that the frame obtained this way is unique up to unitary equivalence.

Some results of the produced frames are presented in Fig. 2. The demonstrated results include the frame vectors’ correlation for a 64×128 and a 96×128 frame. The results show that the angles formed by the frame vectors have values that are concentrated near the optimal bound corresponding to an ETF, that is, the obtained frames are approximately equiangular. For a 96×128 frame that exhibits a lower redundancy the results are more impressive. The proposed constructions are compared with random Gaussian matrices and incoherent tight frames produced with [26] and [30]. Both [26] and [30] attain small mutual coherence; however, they do not exhibit the approximate equiangularity property like the proposed constructions.

Fig. 2

Correlation distribution of frame vectors of 64×128 (left) and 96×128 (right) frames constructed with [26], [30] and proposed method. The optimal lowest bound (Welch bound) for these dimensions is μ _opt=0.0887 and μ _opt=0.0512, respectively

3.5 Nearly equiangular, nearly tight frames based on signature matrices

Algorithm 3 produces frames of any dimensions with the frame vectors forming angles near the optimal value. Even though the obtained frames exhibit good spectral properties, they are not exactly tight, a characteristic that is important for many applications. One way to improve tightness is the following well-known theorem.

Theorem 3

([36]) Given a matrix $F \in {\mathbb R}^{m \times N}$, N≥m, suppose F has singular value decomposition UΣV ^∗. With respect to the Frobenius norm, a nearest α-tight frame F ^′ to F is given by $\sqrt {\alpha }\cdot UV^{*}$. Assume in addition that F has full row-rank. Then $\sqrt {\alpha }\cdot UV^{*}$ is the unique α-tight frame closest to F. Moreover, one may compute UV ^∗ using the formula (FF ^∗)^−1/2 F.

Having produced a nearly equiangular m×N frame with Algorithm 3, we apply Theorem 3 with α=N/m. As tightness opposes unit-normness, we must carry out a few iterations, alternating between these two properties according to Algorithm 4 to obtain a nearly equiangular, nearly tight unit norm frame.

A metric to evaluate how close the obtained frame is to a tight frame is the spectral norm. Recall that the spectral norm of a tight frame equals the lowest possible bound $\sqrt {N/m}$. To see the improvement of tightness achieved by Algorithm 4 we construct frames of various dimensions and compute their spectral norm. The results presented in Table 1 are averaged over 500 frame samples and concern m×N frames with m=32:16:96 and N=128.

Table 1 Spectral norm of m×N frames with m=32:16:96 and N=128 obtained with Algorithm 3 and Algorithm 4

While Algorithm 4 reduces the spectral norm of the obtained frames, it also affects the frame vectors’ correlation. Figure 3 demonstrates results of the frame vectors’ correlation for a 64×128 and a 96×128 frame produced by Algorithm 4. Comparing Fig. 3 to Fig. 2, we observe a slight deterioration of correlation’s distribution, as a price of the improvement of tightness. Therefore, the choice between Algorithm 3 and Algorithm 4 for the construction of nearly equiangular frames, depends on the specific requirements of the related application.

Fig. 3

Correlation distribution of frame vectors produced with Algorithm 4. Results are demonstrated for a 64×128 (left) and 96×128 (right) frame. Optimal lowest bound (Welch bound) is μ _opt=0.0887 and μ _opt=0.0512, respectively

3.6 Comparison with existing methods

In this section, we discuss the differences between the proposed work and existing approaches, while enlightening some aspects of our method. Considering the one-to-one correspondence between ETFs and signature matrices, we will compare the different algorithms using as a performance criterion the spectrum of the signature matrix, i.e., we will graphically examine how close the spectrum is of an obtained signature matrix to the ideal spectrum given by Eq. (5). Our comparison includes the work presented in [22] and [26]; similar results were obtained by comparing with other existing algorithms [23, 24, 30]. Figure 4 demonstrates typical results obtained with [22] and [26] for a 64×128 frame.

Fig. 4

The spectrum of the signature matrix of a 64×128 random Gaussian matrix, a frame obtained with the proposed Algorithm 4, and frames obtained with [22], [26]. The black dotted line stands for the spectrum of the signature matrix corresponding to a 64×128 ETF

The algorithm presented in [22] focuses on incoherence. The iterative operations applied on the Gram matrix include shrinkage and rank reduction. Denoting with r _ij the (i,j) entry of the Gram matrix, the following modification is applied:

$$ \widehat{r}_{ij} =\left\{ \begin{array}{llllll} & {\gamma r_{ij}}, \quad && && \left|r_{ij}\right| \geq t, \\ & {\gamma t \cdot \text{sgn}(r_{ij})}, \quad && \ \ t> &&\left|r_{ij}\right| \geq \gamma t, \\ & {r_{ij}}, \quad && \gamma t > &&\left|r_{ij} \right|, \end{array} \right. $$

(13)

where γ,t are appropriate parameters. For the results presented in Fig. 4, we used γ=0.7,t=0.5. Eq. 13 leaves the signs of the Gram entries unchanged; therefore, it does not modify the signature matrix. However, the shrinkage operation impairs the matrix rank, and a rank reduction step must follow to yield an acceptable Gram. Rank reduction implicitly affects the signature matrix. The spectrum demonstrated in Fig. 4 reflects these changes. The method of [22] is an alternating projections method. The involved sets include the set of N×N symmetric matrices with bounded absolute values (the bounds are defined according to (13)) and the set of N×N symmetric rank m matrices, that is, the set of matrices with m non-zero eigenvalues. Nevertheless, the imposed constraints do not improve the signature matrix sufficiently as another important property of ETFs, namely, tightness is not addressed by the optimization algorithm.

The optimization process proposed in [26] leads to a better spectrum compared to [22]. The goal of [26] is to find a frame that satisfies both incoherence and tightness, thus, it produces frames that are closer to ETFs. Concerning the corresponding Gram matrix, incoherence involves explicit operations on the absolute values, while tightness involves operations on the eigenvalues. Therefore, the averaged projections method proposed in [26] concerns one more constrained set, i.e., the set of N×N symmetric matrices with specific eigenvalues (see Proposition 1). Similarly to rank reduction, operations on the eigenvalues may change the signs of the Gram entries, implicitly improving the signature matrix.

The Gram matrix of an ETF is a matrix with specific structural and spectral properties. Structural properties are related to incoherence; structural constraints involve bounds on the absolute values of the Gram entries. Spectral properties are related to tightness and involve restrictions on the eigenvalues. The methods presented in [22] and [26] are based on a joint optimization of the Gram matrix with respect to both structural and spectral constraints. In this case, the (implicit) optimization of the signature matrix resulting from the application of spectral constraints is subjected to incoherence constraints. Eq. 4 provides a representation of the Gram matrix of an ETF that enables a decoupling optimization strategy: the signature matrix can be directly optimized without being subjected to incoherence constraints. The incoherence constraints are applied to the Gram matrix in a single step by multiplying the signature matrix Q with the Welch bound c. A direct optimization of the signature matrix that does not involve incoherence constraints is more efficient. The proposed construction of approximate ETFs assigns the Gram entries identical absolute values equal to the Welch bound in a single step and results in frames that exhibit approximate equiangularity, a property that is not achieved by any other method. Such a strict incoherence constraint cannot be applied with alternating projections methods; experimental results show that both [22] and [26] do not converge. Therefore, in this paper, the joint optimization of the Gram matrix with respect to both structural and spectral constraints is replaced by an equivalent two-fold problem, the optimization of the signature matrix and the absolute Gram entries. These problems can be tackled separately, resulting in a more efficient optimization strategy that leads to a different set of solutions, namely, approximate equiangular frames.

We would like to underline that, from a theoretical perspective, the decoupling strategy described above only holds for the ideal case, i.e., perfect ETFs. When dealing with approximate ETFs, and suppose that we have computed an optimal signature matrix, the application of (4) does not necessarily lead to the desired Gram; usually it must be followed by a rank reduction step, which may not preserve the properties (i.e., the spectrum) of the optimal signature matrix. Even though there is no theoretical evidence that a decoupling strategy will perform well for the case of approximate ETFs, we assert that the experimental results presented here are promising. Nevertheless, a theoretical study about the relation between an approximate solution of the signature matrix problem (especially when an accurate solution does not exist) and the corresponding constrained Gram remains a challenge.

We continue with a comment on the complexity of the tested algorithms. The work of [26] and [22] applies explicit operations on the Gram matrix to impose structural and spectral constraints. These operations involve projections on sets of matrices with specific structural and spectral properties; the projections are performed iteratively and both algorithms exhibit similar complexity. In our algorithm, the explicit construction of the Gram matrix is a two step non-iterative procedure: first we apply (4) and then we perform rank reduction. However, the construction of the signature matrix involved in (4) is performed through an iterative process which imposes structural and spectral constraints on a matrix of the same dimensions of the Gram matrix. Therefore, we cannot argue that the proposed algorithm is less complex compared to [26] and [22].

Before ending this discussion, we would like to note that the major contribution of the proposed work is the introduction of a different strategy towards the optimization of the Gram matrix of an approximate ETF, which, for the first time, addresses the construction of the corresponding signature matrix. Taking into account that the frames obtained with the proposed method, besides incoherence and tightness, exhibit the additional property of approximate equiangularity, we conclude that our optimization strategy is more effective compared to the state of the art algorithms.

4 Applications of nearly equiangular frames

As we have already mentioned in the introduction, frames that are close to ETFs have many applications in signal processing and communications. In this section, first, we employ the proposed constructions to acquire sparse synthetic signals with compressed sensing and study the reconstruction performance of Orthogonal Matching Pursuit (OMP) [37] in this setting. Then, we use nearly equiangular nearly tight frames to obtain spreading sequences for synchronous CDMA systems and show their superiority against WBE sequences when the number of users in the system changes.

4.1 Sensing matrices for compressed sensing

Compressed sensing is a novel theory [11, 12] that exploits sparsity to recover signals that have been sampled at a drastically smaller rate than the conventional Shannon/Nyquist theorem imposes. Undersampling implies that the number of measurements m is much smaller than the dimension N of the signal. Let α be a sparse signal with a few non-zero coefficients, that is, ∥α∥₀=T, T≪N, where ∥·∥₀ is the so-called ℓ ₀ norm (which is actually not a norm) counting the non-zero coefficients of the respective signal. We obtain m measurements according to

$$ y = \varPhi \alpha, $$

(14)

where $\varPhi \in {\mathbb R}^{m \times N}$ is a matrix describing the sensing mechanism. Equation (14) defines an underdetermined linear system with more unknowns than equations. A sparse solution satisfying (14) can be computed numerically, as long as the matrix Φ satisfies the restricted isometry property (RIP) [38]. We can loosely say that when a matrix obeys RIP of order s, then all subsets of s columns are nearly orthogonal. Random Gaussian or Bernoulli matrices are known to satisfy RIP with high probability [39], however, research still goes on aiming either at the improvement of performance guarantees or at the discovery of sensing operators that are more appropriate for practical applications [40].

A very coarse estimate on a matrix RIP bounds is the mutual coherence of a matrix. According to [13], reconstruction algorithms such as OMP and basis pursuit (BP) [41] can recover a sparse signal acquired with CS as long as the sensing matrix forms an incoherent UNTF. ETFs are optimal CS matrices with respect to mutual coherence as they achieve the Welch bound. The nearly equiangular frames produced in this paper with Algorithms 3 and 4 have frame vectors exhibiting correlation that approximates the optimal value of an ETF. Next, we investigate the behavior of the proposed frames in CS.

Considering synthetic sparse signals of length N=120, with T=4 non-zero coefficients, we obtain measurements according to (14). The obtained measurements are used to find the unknown sparse signal, using OMP, a well-known algorithm for sparse signal recovery. The experiments are realized using random Gaussian matrices, incoherent UNTFs produced with [26], [30], and the frames produced with Algorithms 3 and 4. Varying the number of measurements m={15,20,25,30,35}, for every value of m, we perform 10,000 experiments. The quality of the recovered signal is measured computing the mean squared error (MSE). The results demonstrated in Table 2 include average values.

Table 2 MSE for sparse signals of length N=120 obtained with CS, for variable number of measurements, m=15:5:35, and various types of sensing matrices

According to Table 2, the proposed nearly equiangular frames outperform random Gaussian matrices, and yield results similar to incoherent UNTFs. In order to explain the obtained results of sparse reconstruction, next, we present results regarding the mutual coherence of the matrices employed in our CS experiments. Table 3 includes average values over 10,000 realizations for sensing matrices with dimensions 20×120 and 30×120. According to these results, the superiority of the proposed frames against random Gaussian matrices is obviously expected, considering their incoherence. We cannot assert the same comparing nearly equiangular frames to incoherent UNTFs obtained with [26] and [30]. Frames produced with [26] and [30] exhibit mutual coherence that is smaller than the proposed frames, especially when the frame dimensions yield high redundancy. However, the smallest mutual coherence does not necessarily lead to the highest reconstruction accuracy.

Table 3 Properties of sensing matrices employed in CS experiments

The above results are not that surprising if we take into account that many authors have argued that mutual coherence may not express well the effectiveness of a matrix in sparse signal recovery [22, 42]. Clearly, other properties of the sensing matrix such as average coherence and spectral norm seem to determine the effectiveness of the employed matrix as well. The role of the spectral norm in sparse recovery was studied in [13]. The notion of average coherence was defined in [42] as

$$ \mu_{g}(F) = \frac{1}{N(N-1)} \sum\limits_{i=1}^{N}\sum\limits_{\substack{j=1 \\ i\neq j}}^{N}|\langle f_{i}, f_{j} \rangle |^{2}. $$

(15)

The authors of [42] conjecture its important role in sparse recovery based on experimental results. While the frames obtained with [30] exhibit the smallest mutual coherence, the values of average coherence computed by (15) are identical for all matrix constructions except for random Gaussian matrices. We conclude that the results obtained in Tables 2 and 3 indicate that the effectiveness of a matrix involved in sparse recovery seems to depend on all the aforementioned properties, with average coherence playing a rather important role.

Before proceeding to another application of nearly equiangular frames, we would like to make a comment concerning Algorithms 3 and 4. While Algorithm 4 produces frames with smaller spectral norm, the achieved improvement slightly affects the reconstruction performance of OMP. Taking into account the additional computational cost introduced by Algorithm 4 and the fact that the matrices employed in CS are practically of large dimensions, we suggest that Algorithm 3 is the best choice for the construction of sensing matrices, considering both effectiveness and computational cost. Comparison between Algorithm 3 and [26], [30] leads to a similar conclusion, strengthening our preference to Algorithm 3 especially when the application necessitates limitation of resources.

4.2 Spreading sequences for CDMA systems

In synchronous CDMA systems, the users share the entire bandwidth and each user is distinguished from the others by its spreading sequence or code. (the term signature is also extensively used, but we avoid using it here as it might confuse the reader. Clearly, a signature sequence is not related to a signature matrix by any means). The capacity region defined as the set of information rates at which users can transmit while retaining reliable transmission is characterized as a function of the spreading sequences and average input power constraints of the users. Capacity optimal sequences are functions of codebook length as well as the number of users [14, 15].

Suppose that x ₁,x ₂,…,x _N is a set of vectors in ${\mathbb R}^{m}$ corresponding to N possible users of an s-CDMA system. These vectors form a set of sequences of length m. Optimal spreading sequences have been characterized in [14] to be the WBE sequences, that is, equal norm tight frames. WBE sequences minimize the total squared correlation, that is,

$$ \text{TSC} = \sum\limits_{i=1}^{N}\sum\limits_{j=1}^{N}\left|\left\langle x_{i}, x_{j}\right\rangle\right|^{2}, $$

(16)

which results in that the interference experienced by any user is exactly the same.

The design of WBE sequences for s-CDMA has gained a lot of interest in the last decades [15, 43, 44]. However, WBE sequences do not perform well when the number of users in the cell changes. If the number of the active users is smaller than N, then a code set designed for N users is no longer optimal and new codes should be assigned to all users [9].

The interference experienced by the j-th user in the system depends on the term [14]

$$ I(j) = \sum\limits_{i\neq j}\left|\left\langle x_{i}, x_{j}\right\rangle\right|^{2}. $$

(17)

Consider a system with K<N active users. In [9] it was shown that all users experience the same interference, which depends only on K, the current number of active users, if and only if the code set is an equiangular sequence set. As we have mentioned in Section 3.1, the authors of [9] use conference matrices to construct ETFs that are employed as spreading sequences. However, construction techniques for conference matrices impose restrictions on frame dimensions.

Algorithm 4 may produce nearly equiangular nearly tight frames of any dimensions, offering flexibility when designing codes for an s-CDMA system. The experiments presented here involve the computation of interuser interference when employing the proposed frame vectors as codes of a CDMA system with variable number of users. The performance of the proposed frames in s-CDMA is compared to a method presented in [44] for spreading sequence design. The authors of [44] produce WBE sequences from an appropriate Gram matrix. The construction of the Gram matrix is addressed as an inverse eigenvalue problem, which is solved using finite-step algorithms from matrix theory. The code set we use in our experiments is designed for 128 users with codes of length 64, obtained by a 64×128 frame produced with Algorithm 4. For each subset of K≤128 active users, we randomly choose K codes and examine the interference term given by (17). As a measure of how close we are to the target that all users experience the same interference, we compute the standard deviation of the interference term for every active subset of users. The results are averaged over a series of random trials.

Figure 5 demonstrates results when the number of users in the cell varies from 64 to 128. The performance of the proposed frame when used as a spreading sequence set is also compared to an incoherent UNTF (WBE sequence set) produced by our previous work [26]. It is obvious that the proposed nearly equiangular nearly tight frame, outperforms WBE sequence sets, when the system works with a load up to 85% its total load.

Fig. 5

Standard deviation of the interference term for variable number of active users in a s-CDMA system designed for 128 users

Next, we demonstrate results for total squared correlation for comparison with other work on spreading sequence design based on TSC. Total squared correlation is related to the total mean squared error (MSE) in the system, that is, the sum of MSEs observed by all users. In [43] it is shown that for a link with additive Gaussian noise of zero power and spectral density σ ², assuming that the users receive signals of equal power p, the total MSE for K users in the system is given by

$$ \text{MSE} = p\sum\limits_{i=1}^{K}\sum\limits_{j=1}^{K} \left\langle x_{i}, x_{j}\right\rangle^{2} - \left(2\sqrt{p}-\sigma^{2}\right)\sum\limits_{i=1}^{K}x_{i}^{T}x_{j} + K. $$

(18)

For unit norm codes x _i , we receive

$$ \text{MSE} = p\text{TSC} + (1+\sigma^{2} -2\sqrt{p})K. $$

(19)

From (19) it is obvious that minimizing TSC is equivalent to minimizing MSE. Algorithms like the one presented in [43] produce spreading sequences that minimize TSC.

Table 4, presents averaged values of TSC observed in the above scenarios of active subset of users. For K={64,80,96,112,128} users in the system, we randomly choose K codes and calculate TSC. The results are averaged over a series of random trials. The measured values are similar for all methods showing that the proposed construction yields results similar to [26] and [44]. Therefore, compared to WBE sequences, the proposed frames reduce the interference experienced by the users without affecting the total MSE in the system.

Table 4 Average total squared correlation (TSC) for variable number of active users

5 Conclusions

Signature matrices are matrices with specific structure and spectrum, and define ETFs up to unitary equivalence. While signature matrices of ETFs exist for certain frame dimensions, in this paper, we propose a numerical method that produces signature matrices of nearly equiangular frames of arbitrary dimensions. These frames are almost tight, with frame vectors exhibiting near optimal correlation. Using the obtained frames as spreading sequences for CDMA systems, we reduce the interference experienced by the individual users, when the number of active users in the system changes. We also employ nearly equiangular frames to obtain sparse signals with CS, achieving high reconstruction performance of sparse recovery algorithms. The proposed frame constructions, which are based on signature matrices, rely on the solution of a structured inverse eigenvalue problem. Any progress towards the solution of similar inverse eigenvalue problems could improve the proposed constructions substantially.