# Canonical polyadic decomposition of third-order semi-nonnegative semi-symmetric tensors using LU and QR matrix factorizations

- Lu Wang
^{1, 2, 4}, - Laurent Albera
^{1, 2, 4, 5}Email author, - Amar Kachenoura
^{1, 2, 4}, - Huazhong Shu
^{3, 4}and - Lotfi Senhadji
^{1, 2, 4}

**2014**:150

https://doi.org/10.1186/1687-6180-2014-150

© Wang et al.; licensee Springer. 2014

**Received: **31 March 2014

**Accepted: **10 September 2014

**Published: **8 October 2014

## Abstract

Semi-symmetric three-way arrays are essential tools in blind source separation (BSS) particularly in independent component analysis (ICA). These arrays can be built by resorting to higher order statistics of the data. The canonical polyadic (CP) decomposition of such semi-symmetric three-way arrays allows us to identify the so-called mixing matrix, which contains the information about the intensities of some latent source signals present in the observation channels. In addition, in many applications, such as the magnetic resonance spectroscopy (MRS), the columns of the mixing matrix are viewed as relative concentrations of the spectra of the chemical components. Therefore, the two loading matrices of the three-way array, which are equal to the mixing matrix, are nonnegative. Most existing CP algorithms handle the symmetry and the nonnegativity separately. Up to now, very few of them consider both the semi-nonnegativity and the semi-symmetry structure of the three-way array. Nevertheless, like all the methods based on line search, trust region strategies, and alternating optimization, they appear to be dependent on initialization, requiring in practice a multi-initialization procedure. In order to overcome this drawback, we propose two new methods, called ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$, to solve the problem of CP decomposition of semi-nonnegative semi-symmetric three-way arrays. Firstly, we rewrite the constrained optimization problem as an unconstrained one. In fact, the nonnegativity constraint of the two symmetric modes is ensured by means of a square change of variable. Secondly, a Jacobi-like optimization procedure is adopted because of its good convergence property. More precisely, the two new methods use LU and QR matrix factorizations, respectively, which consist in formulating high-dimensional optimization problems into several sequential polynomial and rational subproblems. By using both LU and QR matrix factorizations, we aim at studying the influence of the used matrix factorization. Numerical experiments on simulated arrays emphasize the advantages of the proposed methods especially the one based on LU factorization, in the presence of high-variance model error and of degeneracies such as bottlenecks. A BSS application on MRS data confirms the validity and improvement of the proposed methods.

## Keywords

## Introduction

Higher order (HO) arrays, commonly called tensors, play an important role in numerous applications, such as chemometrics [1], telecommunications [2], and biomedical signal processing [3]. They can be seen as HO extensions of vectors (one-way arrays) and matrices (two-way arrays). In many practical situations, the available data measurements cannot be arranged into a tensor form directly, that is to say, the observation diversity is insufficient either in time or frequency. However, if the latent data satisfies the statistical independence assumption, which is reasonable in many applications, meaningful HO arrays can be built by resorting to HO statistics (HOS) of the data [4]. In this instance, the HO arrays are partially symmetric or Hermitian due to the special algebraic structure of the basic HOS, such as moments and cumulants. In independent component analysis (ICA), the latent physical phenomena which are assumed to be statistically independent can be revealed by decomposing the HO array into factors. There exists several ways to decompose a given HO array, such as the Tucker model [5, 6]. Among the existing reliable HO array decomposition models, the canonical polyadic (CP) decomposition model has attracted much attention. Indeed, its uniqueness can be ensured under the sufficient conditions established by Kruskal [7]. In addition, unlike the HO singular value decomposition (HOSVD) [6], the CP model does not impose any orthogonality constraint on its factors.

Theoretically, a polyadic decomposition exactly fits an array by a sum of rank-one terms [8]. A CP decomposition is defined as a polyadic decomposition with a minimal number of rank-one terms which are needed to exactly fit a given HO array. Currently, the CP decomposition is gaining importance in several applications, for example, in exploratory data analysis [9], sensor array processing [10], telecommunications [11, 12], ICA [13], and in multiple-input multiple-output radar systems [14]. A multitude of methods were developed to compute the CP decomposition. They include the iterative alternating least squares (ALS) procedure [15], which gains popularity due to its simplicity of implementation and low numerical complexity. Uschmajew proved the local convergence property of ALS under some conditions [16]. However, this convergence can be slow. Therefore, an enhanced line search (ELS) procedure was proposed by Rajih et al. [17] to cope with the slow convergence problem of ALS. Other approaches were also proposed, such as the conjugate gradient algorithm [18] and joint eigenvalue decomposition-based algorithms [19, 20], to cite a few. Some HO arrays enjoy certain properties, such as i) *symmetry* and ii) *nonnegativity*, which cannot be simply handled by the aforementioned general CP decomposition methods. Therefore, special CP models become more and more important.

The first special form of the CP model for three-way arrays that are symmetric in two modes brings the concept of individual differences in scaling (INDSCAL) analysis [21]. On one hand, INDSCAL analysis has been studied as a way of multiple factor analysis [22] with applications to chemometrics, psychology, and marketing research. On the other hand, in the domain of signal processing, and more particularly in blind source separation (BSS), the INDSCAL analysis is widely known as the joint diagonalization of a set of matrices by congruence (JDC). During the past two decades, many successful JDC methods have been proposed, such as Yeredor’s alternating columns and diagonal center (ACDC) algorithm [23], the joint approximate diagonalization (JAD) algorithm proposed by Cardoso and Souloumiac [24], the fast Frobenius diagonalization (FFDIAG) algorithm proposed by Ziehe et al. [25], Afsari’s LUJ1D algorithm [26], and many others [27–33]. A recent survey of JDC can be found in [34]. The second special form of CP model is defined when all the factors in the CP decomposition are constrained to be nonnegative, commonly known as nonnegative tensor factorization (NTF). NTF can be regarded as the extension of nonnegative matrix factorization (NMF) [35] to higher orders. In many applications, the physical properties are inherently nonnegative, such as chemistry [1] and fluorescence spectroscopy [36, 37]. In those applications, the results are only meaningful if the nonnegativity constraint is satisfied. Various methods for computing NTF and also NMF can be found in [38, 39].

So far, the CP model with both the symmetry and nonnegativity constraints has not received much attention. Coloigner et al. proposed a family of algorithms based on line search and trust region strategies [40]. Wang et al. developed an alternating minimization scheme [41]. Those methods appear to depend on initialization, and therefore in practice require a multi-initialization procedure, leading to an increase of numerical complexity. In this paper, we propose to fit the CP model of a three-way array by imposing both the semi-nonnegativity and the semi-symmetry constraints. More precisely, we impose a nonnegativity constraint on the two symmetric modes of the INDSCAL model, which leads to the semi-nonnegative INDSCAL model or equivalently the CP decomposition of semi-nonnegative semi-symmetric three-way arrays. Such a model is often encountered in ICA problems where a nonnegative mixing matrix is frequently considered. For example, in magnetic resonance spectroscopy (MRS), the columns of the mixing matrix represent the positive concentrations of the source metabolites. Then, the three-way array built by stacking the matrix slices of a cumulant array is both nonnegative and symmetric in two modes. In such a case, the semi-nonnegative INDSCAL problem is equivalent to the JDC problem subject to a nonnegativity constraint on the joint transformation matrix. We propose two new algorithms to solve the semi-nonnegative INDSCAL problem, called ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$. Firstly, we rewrite the constrained optimization problem as an unconstrained one. Actually, the nonnegativity constraint is ensured by means of a square change of variable. Secondly, we propose two Jacobi-like approaches using LU and QR matrix factorizations, respectively, which consist in formulating high-dimensional optimization problems into several sequential polynomial and rational subproblems. By using both LU and QR matrix factorizations, we aim at studying the influence of the used matrix factorization. Numerical experiments highlight the advantages of the proposed methods especially ${\text{JD}}_{\text{LU}}^{+}$, in the case of dealing with high-variance model error and with degeneracies such as bottlenecks. A BSS application on MRS signals confirms the validity and improvement of the proposed methods. A part of this work has been recently presented at the 8th IEEE Sensor Array and Multichannel Signal Processing Workshop [42].

The rest of the paper is organized as follows. After the presentation of some notations, the ‘Multilinear algebra prerequisites and problem statement’ section introduces some basic definitions of the multilinear algebra then gives the semi-nonnegative INDSCAL problem formulation. In the ‘Methods’ section, we describe the proposed algorithms in detail and also provide an analysis of the numerical complexities. The ‘Simulation results’ section shows the computer simulation results. Finally, we conclude the paper.

## Multilinear algebra prerequisites and problem statement

### Notations

The following notations are used throughout this paper.
and
denote the set of real-valued (*N*_{1} × *N*_{2} × ⋯ × *N*_{
i
}) arrays and the set of nonnegative real-valued (*N*_{1} × *N*_{2} × ⋯ × *N*_{
i
}) arrays, respectively. Vectors, matrices, and HO arrays are denoted by bold lowercase letters (a, b, ⋯), bold uppercase letters (A, B, ⋯) and bold calligraphic letters (
,
, ⋯), respectively. The (*i*,*j*)-th entry of a matrix A is symbolized by *A*_{i,j}. Sometimes, the MATLAB^{®}; column/row notation is adopted to indicate submatrices of a given matrix or subarrays of a HO array. Also, a_{
i
} denotes the *i*-th column vector of matrix A. ⊡ denotes the Hadamard product (element-wise product), and ${\mathit{A}}^{\u22a12}=\mathit{A}\u22a1\mathit{A}$. ⊙ denotes the Khatri-Rao product. A^{
♯
} denotes the pseudo inverse of A. The superscripts ^{-1}, ^{T}, and ^{-T} stand for the inverse, the transpose, and the inverse after transpose operators, respectively. The (*N* × *N*) identity matrix is denoted by I_{
N
}. **0**_{
N
} stands for *N*-dimensional vectors of zeros. |*a*| denotes the absolute value of *a*. ∥A∥_{
F
} and $det\left(\mathit{A}\right)$ stand for the Frobenius norm and determinant of matrix A, respectively. diag(A) returns a matrix comprising only the diagonal elements of A. Diag(b) is the diagonal matrix whose diagonal elements are given by the vector b. off(A) vanishes the diagonal components of the input matrix A. vec(A) reshapes a matrix A into a column vector by stacking its columns vertically.

### Definitions and problem formulation

Now we introduce some basic definitions in multilinear algebra which are necessary for the problem formulation.

#### Definition 1.

The outer product $\mathcal{C}={\mathit{u}}^{\left(1\right)}\circ {\mathit{u}}^{\left(2\right)}\circ {\mathit{u}}^{\left(3\right)}$ of three vectors is a three-way array of whose elements are defined by ${\mathcal{C}}_{{i}_{1},{i}_{2},{i}_{3}}={u}_{{i}_{1}}^{\left(1\right)}{u}_{{i}_{2}}^{\left(2\right)}{u}_{{i}_{3}}^{\left(3\right)}$.

#### Definition 2.

Each three-way array expressed as the outer product of three vectors is a rank-1 three-way array.

More generally, the rank of a three-way array is defined as follows:

#### Definition 3.

The rank of an array , denoted by $\text{rk}\left(\mathcal{C}\right)$, is the minimal number of rank-1 arrays belonging to that yield in a linear combination.

Despite the similarity between the definition of the tensor rank and its matrix counterpart, the rank of a three-way array may exceed its dimensions [4].

#### Definition 4.

A three-way array slice is a two-dimensional section (fragment) of a three-way array, obtained by fixing one of the three indices [38].

For example, the *k*-th frontal slice of a three-way array
can be denoted by ${\mathcal{C}}_{:,:,k}$ using MATLAB notation, and sometimes it is also denoted by C^{(k)}.

The low-rank INDSCAL model of a three-way array is defined as follows:

#### Definition 5.

*P*, corresponding to the number of rank-1 terms, the INDSCAL model of a three-way array can be expressed as:

where the three-way array represents the model residual.

The notation $\mathcal{C}=\phantom{\rule{0.3em}{0ex}}\left[\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\right[\mathit{A},\mathit{A},\mathit{D}\left]\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\right]+\mathcal{V}$ refers to the INDSCAL decomposition (1) of with the associated loading matrices and $\mathit{D}=\left[{\mathit{d}}_{1},\cdots \phantom{\rule{0.3em}{0ex}},{\mathit{d}}_{P}\right]\in {\mathbb{R}}^{K\times P}$. If and only if the residual is a null tensor, we have an exact INDSCAL decomposition.

An exact INDSCAL decomposition is considered to be essentially unique when it is only subject to scale and permutation indeterminacies. It means that an INDSCAL decomposition is insensitive to a scaling of the three vectors a_{
p
}, a_{
p
}, and d_{
p
} provided that the product of the three scale numbers is equal to 1, and an arbitrary permutation of the rank-1 terms. A necessary and sufficient uniqueness condition for the INDSCAL model was established by Afsari [43].

where D^{(k)} is a diagonal matrix whose diagonal contains the elements of the *k*-th row of D, and ${\mathit{V}}^{\left(k\right)}={\mathcal{V}}_{:,:,k}$.

In this paper, we propose to fit the INDSCAL model of three-way arrays while imposing nonnegativity constraints on both equal loading matrices A. It will be referred to as the semi-nonnegative INDSCAL model, as follows:

#### Problem 1.

*Given*
*and an integer P, find a semi-nonnegative INDSCAL model of* $\mathcal{C}=\phantom{\rule{0.3em}{0ex}}\left[\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\right[\mathit{A},\mathit{A},\mathit{D}\left]\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\right]$*, subject to the (N × P) matrix* A *having nonnegative components*.

The semi-nonnegative INDSCAL problem is equivalent to the JDC problem subject to the nonnegativity constraint on the joint transformation matrix. In this paper, we mainly focus on the case of square nonnegative joint transformation matrix, for which *N* = *P*. The case of *N* > *P* will be discussed briefly in the next section. Therefore, the problem that we tackle in this paper is defined as follows:

#### Problem 2.

*Given a three-way array*

*with K symmetric frontal slices*

*, find a (N × N) joint transformation matrix*A

*and K diagonal matrices*D

^{(k)}

*of dimension (N × N) such that:*

*by minimizing the residual term* V^{(k)} *in a least-squares sense, subject to* A *having nonnegative components*.

### JDC cost functions

^{(k)}by $\text{Diag}\left\{{(\mathit{A}\odot \mathit{A})}^{\mathsf{\text{T}}}\phantom{\rule{2.77626pt}{0ex}}\text{vec}\left({\mathit{C}}^{\left(k\right)}\right)\right\}$ in (4). Then, the extra parameter D can be eliminated and the minimization of (4) is equivalent to minimizing the following indirect least square (IDLS) criterion [45, 46]:

Frequently, the minimization of criterion (7) is performed on a matrix $\mathit{Z}\stackrel{\text{def}}{=}{\mathit{A}}^{-\mathsf{\text{1}}}$ instead of A for simplicity, and Z is called the joint diagonalizer. To use this criterion, the matrix A (or Z) should be properly constrained in order to avoid the trivial zero solution and/or degenerate solutions [34].

Besides the criterions (4) and (7), Afsari [26] presented a new cost function, which is invariant to column scaling of A. Pham proposed an information theoretic criterion [48], which requires each matrix C^{(k)} to be positive definite. Tichavský and Yeredor gave a special weighted least square criterion [49].

## Methods

### Problem reformulation

Existing semi-nonnegative INDSCAL algorithms are based on the minimization of the cost function (4) [40, 41]. They are able to achieve a better estimation of A than ACDC when the data satisfies the semi-nonnegative INDSCAL model at the cost of a higher computational complexity. We propose to use criterion (7) based on elementary factorizations of A due to the fast convergence property of this kind of procedures. Generally, it is quite difficult to impose the nonnegativity constraint on A while computing its inverse A^{-1} by minimizing (7). Let us consider the structure of $\mathcal{C}=\phantom{\rule{0.3em}{0ex}}\left[\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\right[\mathit{A},\mathit{A},\mathit{D}\left]\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\right]$ with the following assumptions:

② does not contain zero entries.

^{(k,-1)}to denote ${\left({\mathit{C}}^{\left(k\right)}\right)}^{-1}$ for simplicity. Eq. 8 shows that C

^{(k,-1)}also preserves the jointly diagonalizable structure. Furthermore, instead of A

^{-1}, A serves as the joint diagonalizer. Then, A can be estimated by minimizing the following modified criterion based on (7):

where . Then, problem 2 can be reformulated as follows:

#### Problem 3.

*Given*

*, find the square nonnegative loading matrix*$\mathit{A}={\mathit{B}}^{\u22a12}$

*such that*B

*minimizes the following cost function:*

### LU and QR parameterizations of *B*

In order to minimize (11), one may consider a gradient-like approach. However, the performance of this kind of method is sensitive to the initial guess and to the search step size. In addition, the calculation of gradient of (11) with respect to B is computationally expensive due to the existence of the Hadamard product. Other algorithms, using Jacobi-like procedures [25, 26, 31], parameterize A as a product of several special elementary matrices and estimate each elementary matrix successively. We propose to follow such a minimization scheme.

Now let us recall the following definitions and lemmas:

#### Definition 6.

A unit upper (or lower) triangular matrix is an upper (or lower, respectively) triangular matrix whose main diagonal elements are equal to 1.

#### Definition 7.

An elementary upper (or lower) triangular matrix with parameters {*i*,*j*,*u*_{i,j}} and *i*<*j* is a unit upper (or lower, respectively) triangular matrix whose non-diagonal elements are zeros except the (*i*,*j*)-th entry, which is equal to *u*_{i,j}.

^{(i,j)}(

*u*

_{i,j}) with 1 ≤

*i*<

*j*≤

*N*denotes an elementary upper triangular matrix:

Similarly, L^{(i,j)}(*ℓ*_{i,j}) with 1 ≤ *j* < *i* ≤ *N* corresponds to an elementary lower triangular matrix.

#### Definition 8.

A Givens rotation matrix with parameters {*i*,*j*,*θ*_{i,j}} and *i* < *j* is equal to an identity matrix except for the (*i*,*i*)-th, (*j*,*j*)-th, (*i*,*j*)-th, and (*j*,*i*)-th entries, which are equal to $\text{cos}\left({\theta}_{i,j}\right)$, $\text{cos}\left({\theta}_{i,j}\right)$, $-\text{sin}\left({\theta}_{i,j}\right)$, and $\text{sin}\left({\theta}_{i,j}\right)$, respectively.

^{(i,j)}(

*θ*

_{i,j}) with 1 ≤

*i*<

*j*≤

*N*indicates the corresponding Givens rotation matrix:

#### Lemma 1.

*Any (N × N) unit lower triangular matrix*L

*whose (i,j)-th component is ℓ*

_{i,j}

*(i > j) can be factorized as the following product of N (N*-1)/2

*elementary lower triangular matrices [*51,

*Chapter 3*

*]:*

*where the two sets of indices*${J}_{1}$

*and*${I}_{1}\left(j\right)$

*are defined by*${J}_{1}=\{1,2,\dots ,N\}$

*and*${I}_{1}\left(j\right)=\{j+1,j+2,\dots ,N\}$

*for the sake of convenience. Similarly, any (N × N) unit upper triangular matrix*U

*whose (i,j)-th component is equal to u*

_{i,j}

*(i < j) can be factorized as a product of elementary upper triangular matrices as follows:*

*where* ${I}_{2}$ *and* ${J}_{2}\left(i\right)$ *are two sets of indices, defined by* ${I}_{2}\phantom{\rule{0.3em}{0ex}}=\phantom{\rule{0.3em}{0ex}}\{N-1,N-2,\dots ,1\}$ *and* ${J}_{2}\left(i\right)\phantom{\rule{0.3em}{0ex}}=\{N,N-1,\dots ,i+1\}$.

#### Lemma 2.

*Any (N × N) orthonormal matrix*Q

*can be factorized as the following product of N (N*-1)/2

*Givens rotation matrices [*52, Chapter 14

*]:*

*where*
${I}_{2}$
*and*
${J}_{2}\left(i\right)$
*are defined in Lemma 1.*

As a consequence, the minimization of (11) with respect to B is converted to the estimate of *N* (*N*-1) parameters: *ℓ*_{i,j} and *u*_{i,j} for the LU decomposition (17), or *θ*_{i,j} and *u*_{i,j} for the QR decomposition (18). Instead of simultaneously computing the *N*(*N*-1) parameters, we propose two Jacobi-like procedures which perform *N*(*N*-1) sequential optimizations. This yields two new algorithms: i) the first algorithm based on (17), named ${\text{JD}}_{\text{LU}}^{+}$, estimates each *ℓ*_{i,j} and *u*_{i,j} successively, and ii) the second one based on (18), called ${\text{JD}}_{\text{QR}}^{+}$, estimates each *θ*_{i,j} and *u*_{i,j} sequentially.

Now, the difficulty is how to estimate four kinds of parameter, namely L^{(i,j)}(*ℓ*_{i,j}) and U^{(i,j)}(*u*_{i,j}) for ${\text{JD}}_{\text{LU}}^{+}$, and Q^{(i,j)}(*θ*_{i,j}) and U^{(i,j)}(*u*_{i,j}) for ${\text{JD}}_{\text{QR}}^{+}$. Two points should be noted here: i) L^{(i,j)}(*ℓ*_{i,j}) and U^{(i,j)}(*u*_{i,j}) belong to the same category of matrices; therefore, they can be estimated by the same algorithmic procedure just with an emphasis on the relation between the *i* and *j* indices (*i* < *j* for U^{(i,j)}(*u*_{i,j}) and *j* < *i* for L^{(i,j)}(*ℓ*_{i,j})); ii) for both ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ algorithms, the procedure of estimating U^{(i,j)}(*u*_{i,j}) is identical. Consequently, the principal problem is reduced to estimating two kinds of parameters, namely U^{(i,j)}(*u*_{i,j}) and Q^{(i,j)}(*θ*_{i,j}).

### Minimization with respect to the elementary upper triangular matrix *U*^{(i,j)}(*u*_{i,j})

^{(i,j)}(

*u*

_{i,j}) with 1 ≤

*i*<

*j*≤

*N*. Let $\stackrel{~}{\mathit{A}}$ and $\stackrel{~}{\mathit{B}}$ denote the current estimate of A and B before estimating the parameter

*u*

_{i,j}, respectively. Let ${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}$ and ${\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}$ stand for $\stackrel{~}{\mathit{A}}$ and $\stackrel{~}{\mathit{B}}$ updated by U

^{(i,j)}(

*u*

_{i,j}), respectively. Furthermore, the update of $\stackrel{~}{\mathit{B}}$ is defined as follows:

*u*

_{i,j}, a typical way is to minimize the criterion (11) with respect to

*u*

_{i,j}by replacing matrix $\stackrel{~}{\mathit{B}}$ by ${\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}$. For the sake of convenience, we denote

*J*(

*u*

_{i,j}) instead of $J\phantom{\rule{0.3em}{0ex}}\left({\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}\right)$. Then,

*J*(

*u*

_{i,j}) can be expressed as follows:

The expression of the Hadamard square of the update ${\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}$ is shown in the following proposition:

#### Proposition 1.

${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}={\left({\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}\right)}^{\u22a12}={\left(\stackrel{~}{\mathit{B}}{\mathit{U}}^{(i,j)}\left({u}_{i,j}\right)\right)}^{\u22a12}$

*can be expressed as a function of u*

_{i,j}

*as follows:*

where ${\stackrel{~}{\mathit{b}}}_{i}$ and ${\stackrel{~}{\mathit{b}}}_{j}$ denote the *i*-th and *j*-th columns of $\stackrel{~}{\mathit{B}}$, respectively, and e_{
j
} is the *j*-th column of the identity matrix I_{
N
}.

where ${\stackrel{~}{\mathit{C}}}^{\left(k\right)}={\stackrel{~}{\mathit{A}}}^{\mathsf{\text{T}}}{\mathit{C}}^{(k,-1)}\stackrel{~}{\mathit{A}}$, ${\stackrel{~}{\mathit{c}}}^{(k,1)}=2\phantom{\rule{0.3em}{0ex}}{\stackrel{~}{\mathit{A}}}^{\mathsf{\text{T}}}{\mathit{C}}^{(k,-1)}\left({\stackrel{~}{\mathit{b}}}_{i}\u22a1{\stackrel{~}{\mathit{b}}}_{j}\right)$, ${\stackrel{~}{\mathit{c}}}^{(k,2)}=2\phantom{\rule{0.3em}{0ex}}{\left({\stackrel{~}{\mathit{b}}}_{i}\u22a1{\stackrel{~}{\mathit{b}}}_{j}\right)}^{\mathsf{\text{T}}}\times {\mathit{C}}^{(k,-1)}\stackrel{~}{\mathit{A}}$, and ${\stackrel{~}{c}}^{(k,3)}=4{\left({\stackrel{~}{\mathit{b}}}_{i}\u22a1{\stackrel{~}{\mathit{b}}}_{j}\right)}^{\mathsf{\text{T}}}{\mathit{C}}^{(k,-1)}\left({\stackrel{~}{\mathit{b}}}_{i}\u22a1{\stackrel{~}{\mathit{b}}}_{j}\right)$ are a (*N* × *N*) constant matrix, a (*N* × 1) constant column vector, a (1 × *N*) constant row vector, and a constant scalar, respectively. The term ① in (22) transforms the *j*-th column and the *j*-th row of ${\stackrel{~}{\mathit{C}}}^{\left(k\right)}$. The term ② in (22) is a zero matrix except its *j*-th column containing non-zero elements, while the term ③ contains non-zero entries only on its *j*-th row. The term ④ is a zero matrix except its (*j*,*j*)-th component being non-zero. In addition, ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}=\u2460+\u2461+\u2462+\u2463$ is a (*N* × *N*) symmetric matrix. Hence, (22) shows that only the *j*-th column and *j*-th row of ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$ involve the parameter *u*_{i,j}, while the other elements remain constant. Therefore, the minimization of the cost function (20) is equivalent to minimizing the sum of the squares of the *j*-th columns of ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$ except their (*j*,*j*)-th elements with *k* ∈ {1,⋯,*K*}. The required elements of ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$ can be expressed by the following proposition.

#### Proposition 2.

*The elements of the j-th column except the (j,j)-th entry of*${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$

*is a second-degree polynomial function in u*

_{i,j}

*as follows, for every value n different of j:*

where ${\stackrel{~}{C}}_{n,i}^{\left(k\right)}$ and ${\stackrel{~}{C}}_{n,j}^{\left(k\right)}$ *are the (n,i)-th and (n,j)-th components of matrix* ${\stackrel{~}{\mathit{C}}}^{\left(k\right)}$*, respectively, and* ${\stackrel{~}{c}}_{n}^{(k,1)}$ *is the n-th element of vector* ${\stackrel{~}{\mathit{c}}}^{(k,1)}$.

*j*-th column except the (

*j*,

*j*)-th entry of the term ① in (22) can be expressed by ${\stackrel{~}{C}}_{n,i}^{\left(k\right)}{u}_{i,j}^{2}+{\stackrel{~}{C}}_{n,j}^{\left(k\right)}$ with 1 ≤

*n*≤

*N*and

*n*≠

*j*, and those elements of the term ② in (22) are equal to ${\stackrel{~}{c}}_{n}^{(k,1)}{u}_{i,j}$ with 1 ≤

*n*≤

*N*and

*n*≠

*j*. The sum of these elements directly leads to (23). The terms ③ and ④ do not need to be considered, since they do not affect the off-diagonal elements in the

*j*-th column. Proposition 2 shows that the minimization of the cost function (20) can be expressed in the following compact matrix form:

where ${\mathit{Q}}_{E}=\sum _{k=1}^{K}{\left({\mathit{E}}^{\left(k\right)}\right)}^{\mathsf{\text{T}}}{\mathit{E}}^{\left(k\right)}$ is a (3 × 3) symmetric coefficient matrix. E^{(k)} is a ((*N* - 1) × 3) matrix defined as follows: the first column contains the *i*-th column of ${\stackrel{~}{\mathit{C}}}^{\left(k\right)}$ without the *j*-th element, the second column contains vector ${\stackrel{~}{\mathit{c}}}^{(k,1)}$ without the *j*-th entry, and the third column contains the *j*-th column of ${\stackrel{~}{\mathit{C}}}^{\left(k\right)}$ without the *j*-th component. ${\mathit{u}}_{i,j}={\left[{u}_{i,j}^{2},{u}_{i,j},1\right]}^{\mathsf{\text{T}}}$ is a three-dimensional parameter vector.

Equation (24) shows that *J*(*u*_{i,j}) is a fourth-degree polynomial function. The global minimum *u*_{i,j} can be obtained by computing the roots of its derivative and selecting the one yielding the smallest value of (24). Once the optimal *u*_{i,j} is computed, ${\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}$ is updated by (19) and the joint diagonalizer ${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}$ is updated by computing ${\left({\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}\right)}^{\u22a12}$. Then, the same procedure is repeated to compute the next *u*_{i,j} with another (*i*,*j*) index.

^{(i,j)}(

*ℓ*

_{i,j}) with 1 ≤

*j*<

*i*≤

*N*can be computed in the same way. Proposition 2 is also valid for the parameter

*ℓ*

_{i,j}when 1 ≤

*j*<

*i*≤

*N*. The detailed derivation is omitted here. The processing of all the

*N*(

*N*- 1) parameters

*u*

_{i,j}and

*ℓ*

_{i,j}is called a LU sweep. In addition, for estimating L

^{(i,j)}(

*ℓ*

_{i,j}), the (

*i*,

*j*) index obeys the following order:

^{(i,j)}(

*u*

_{i,j}), the (

*i*,

*j*) index varies according to the following sequence:

The proposed ${\text{JD}}_{\text{LU}}^{+}$ algorithm is comprised of several LU sweeps.

### Minimization with respect to the Givens rotation matrix *Q*^{(i,j)}(*θ*_{i,j})

^{(i,j)}(

*θ*

_{i,j}) with 1 ≤

*i*<

*j*≤

*N*. By abuse of notation, in this section, we continue to use $\stackrel{~}{\mathit{A}}$ and $\stackrel{~}{\mathit{B}}$ to denote the current estimate of A and B, respectively, before estimating the parameter

*θ*

_{i,j}. Also, let ${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}$ and ${\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}$ stand for $\stackrel{~}{\mathit{A}}$ and $\stackrel{~}{\mathit{B}}$ updated by Q

^{(i,j)}(

*θ*

_{i,j}), respectively. The update of $\stackrel{~}{\mathit{B}}$ is defined as follows:

*θ*

_{i,j}, we can minimize the criterion (11) with respect to

*θ*

_{i,j}by replacing matrix $\stackrel{~}{\mathit{B}}$ by ${\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}$. We denote

*J*(

*θ*

_{i,j}) instead of $J\left({\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}\right)$ for convenience purpose. Then,

*J*(

*θ*

_{i,j}) can be expressed as follows:

The Hadamard square of the update ${\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}$ now can be rewritten as shown in the following proposition.

#### Proposition 3.

${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}={\left({\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}\right)}^{\u22a12}={\left(\stackrel{~}{\mathit{B}}{\mathit{Q}}^{(i,j)}\left({\theta}_{i,j}\right)\right)}^{\u22a12}$

*can be written as a function of θ*

_{i,j}

*as follows:*

*where* ${\stackrel{~}{\mathit{b}}}_{i}$ *and* ${\stackrel{~}{\mathit{b}}}_{j}$ *denote the i-th and j-th columns of* $\stackrel{~}{\mathit{B}}$*, respectively, and* e_{
i
}*and* e_{
j
}*are the i-th and j-th columns of the identity matrix* I_{
N
}*, respectively*.

where ${\stackrel{~}{\mathit{C}}}^{\left(k\right)}={\stackrel{~}{\mathit{A}}}^{\mathsf{\text{T}}}{\mathit{C}}^{(k,-1)}\stackrel{~}{\mathit{A}}$, ${\stackrel{~}{\mathit{c}}}^{(k,1)}={\stackrel{~}{\mathit{A}}}^{\mathsf{\text{T}}}{\mathit{C}}^{(k,-1)}\left({\stackrel{~}{\mathit{b}}}_{i}\u22a1{\stackrel{~}{\mathit{b}}}_{j}\right)$, ${\stackrel{~}{\mathit{c}}}^{(k,2)}={\left({\stackrel{~}{\mathit{b}}}_{i}\u22a1{\stackrel{~}{\mathit{b}}}_{j}\right)}^{\mathsf{\text{T}}}{\mathit{C}}^{(k,-1)}\stackrel{~}{\mathit{A}}$, and ${\stackrel{~}{c}}^{(k,3)}={\left({\stackrel{~}{\mathit{b}}}_{i}\u22a1{\stackrel{~}{\mathit{b}}}_{j}\right)}^{\mathsf{\text{T}}}{\mathit{C}}^{(k,-1)}\left({\stackrel{~}{\mathit{b}}}_{i}\u22a1{\stackrel{~}{\mathit{b}}}_{j}\right)$ are a (*N* × *N*) constant matrix, a (*N* × 1) constant column vector, a (1 × *N*) constant row vector, and a constant scalar, respectively. The term ① in (30) transforms the *i*-th and *j*-th columns and the *i*-th and *j*-th rows of ${\stackrel{~}{\mathit{C}}}^{\left(k\right)}$. The term ② in (30) is a zero matrix except its *i*-th and *j*-th columns containing non-zero elements, while the term ③ contains non-zero entries only on its *i*-th and *j*-th rows. The term ④ is a zero matrix except its (*i*,*i*)-th, (*j*,*j*)-th, (*i*,*j*)-th, and (*j*,*i*)-th components being non-zero. ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}=\u2460+\u2461+\u2462+\u2463$ is a (*N* × *N*) symmetric matrix. Hence, (30) shows that only the *i*-th and *j*-th columns and the *i*-th and *j*-th rows of ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$ involve the parameter *θ*_{i,j}, while the other components remain constant. It is noteworthy that the (*i*,*j*)-th and (*j*,*i*)-th components are twice affected by the transformation. Considering the symmetry of ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$, we propose to minimize the sum of the squares of the (*i*,*j*)-th entries of the *K* matrices ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$, instead of minimizing all the off-diagonal entries. Although minimizing this quantity is not equivalent to minimizing the global cost function (28), such a simplified minimization scheme is commonly adopted in many algorithms, such as [20, 31]. We denote this local minimization by $\stackrel{~}{J}\left({\theta}_{i,j}\right)$. The (*i*,*j*)-th component of ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$ is expressed in the following proposition.

#### Proposition 4.

*The (i,j)-th entry of*${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$

*can be expressed as a function of θ*

_{i,j}

*as follows:*

*where* ${\stackrel{~}{C}}_{i,i}^{\left(k\right)}$*,* ${\stackrel{~}{C}}_{j,j}^{\left(k\right)}$*,* ${\stackrel{~}{C}}_{i,j}^{\left(k\right)}$*, and* ${\stackrel{~}{C}}_{j,i}^{\left(k\right)}$ *are the (i,i)-th, (j,j)-th, (i,j)-th, and (j,i)-th components of matrix* ${\stackrel{~}{\mathit{C}}}^{\left(k\right)}$*, respectively*. ${\stackrel{~}{c}}_{i}^{(k,q)}$ and ${\stackrel{~}{c}}_{j}^{(k,q)}$ are the *i*-th and *j*-th elements of vector ${\stackrel{~}{\mathit{c}}}^{(k,q)}$ with *q* ∈ {1,2}, respectively.

It is straightforward to show that the (*i*,*j*)-th entry of the term ① in (30) can be expressed by $\stackrel{2}{\text{sin}}\left({\theta}_{i,j}\right)\stackrel{2}{\text{cos}}\left({\theta}_{i,j}\right)\left({\stackrel{~}{C}}_{i,i}^{\left(k\right)}+{\stackrel{~}{C}}_{j,j}^{\left(k\right)}\right)+\stackrel{4}{\text{sin}}\left({\theta}_{i,j}\right){C}_{j,i}^{\left(k\right)}+\stackrel{4}{\text{cos}}\left({\theta}_{i,j}\right){\stackrel{~}{C}}_{i,j}^{\left(k\right)}$, the (*i*,*j*)-th element of the term ② is $\text{sin}\left(2{\theta}_{i,j}\right)\left(\phantom{\rule{0.3em}{0ex}}\stackrel{2}{\text{cos}}\left({\theta}_{i,j}\right){\stackrel{~}{c}}_{i}^{(k,1)}\phantom{\rule{0.3em}{0ex}}+\phantom{\rule{0.3em}{0ex}}\stackrel{2}{\text{sin}}\left({\theta}_{i,j}\right){\stackrel{~}{c}}_{j}^{(k,1)}\phantom{\rule{0.3em}{0ex}}\right)$, the (*i*,*j*)-th component of the term ③ is equal to $-\text{sin}\left(2{\theta}_{i,j}\right)\left(\phantom{\rule{0.3em}{0ex}}\stackrel{2}{\text{sin}}\left({\theta}_{i,j}\right)\right.{\stackrel{~}{c}}_{i}^{(k,2)}\phantom{\rule{0.3em}{0ex}}+\left.\stackrel{2}{\text{cos}}\left({\theta}_{i,j}\right){\stackrel{~}{c}}_{j}^{(k,2)}\right)$, and that of the term ④ is $-\stackrel{2}{\text{sin}}\left(2{\theta}_{i,j}\right){\stackrel{~}{c}}^{(k,3)}$. Then, Proposition 4 can be proved.

By substituting (32) into (31), we obtain an alternative expression of the (*i*,*j*)-th entry of ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$ which is described in the following proposition. Then, the minimization of $\stackrel{~}{J}\left({\theta}_{i,j}\right)$ transforms to $\stackrel{~}{J}\left({t}_{i,j}\right)$.

#### Proposition 5.

*The (i,j)-th entry of*${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$

*can be expressed by a rational function of t*

_{i,j}

*as follows:*

*where* ${f}_{4}^{\left(k\right)}={\stackrel{~}{C}}_{j,i}^{\left(k\right)}$*,* ${f}_{3}^{\left(k\right)}=-2{\stackrel{~}{c}}_{i}^{(k,1)}$*,* ${f}_{2}^{\left(k\right)}={\stackrel{~}{C}}_{i,i}^{\left(k\right)}+{\stackrel{~}{C}}_{j,j}^{\left(k\right)}+2{\stackrel{~}{c}}_{j}^{(k,2)}-4{\stackrel{~}{c}}^{(k,3)}$*,* ${f}_{1}^{\left(k\right)}=2{\stackrel{~}{c}}_{i}^{(k,2)}-{\stackrel{~}{c}}_{j}^{(k,1)}$*, and* ${f}_{0}^{\left(k\right)}={\stackrel{~}{C}}_{j,j}^{\left(k\right)}$.

*i*,

*j*)-th entries of the

*K*matrices ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$, is a rational function in

*t*

_{i,j}, namely $\stackrel{~}{J}\left({t}_{i,j}\right)$, where the degrees of the numerator and the denominator are 8 and 8, respectively. $\stackrel{~}{J}\left({t}_{i,j}\right)$ can be expressed in the following compact matrix form:

_{i,j}is a five-dimensional parameter vector defined as follows:

The global minimum *t*_{i,j} can be obtained by computing the roots of its derivative and selecting the one yielding the smallest value of $\stackrel{~}{J}\left({t}_{i,j}\right)$. Once *t*_{i,j} is obtained, *θ*_{i,j} can be computed from the inverse tangent function ${\theta}_{i,j}=\text{arctan}\left({t}_{i,j}\right)$. It is noteworthy that the found *θ*_{i,j} cannot guarantee to decrease the actual cost function (28). If *θ*_{i,j} leads to an increase of (28), we reset *θ*_{i,j}= 0. Otherwise, ${\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}$ is updated as described in (27) and the joint diagonalizer ${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}$ is updated by computing ${\left({\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}\right)}^{\u22a12}$. The same procedure will be repeated to compute *θ*_{i,j} with the next (*i*,*j*) index. The order of the (*i*,*j*) indices is defined in Eq. 26. The processing of all the *N*(*N* - 1)/2 parameters *θ*_{i,j} and also the other *N*(*N* - 1)/2 parameters *u*_{i,j} is called a QR sweep. Several QR sweeps yield the proposed ${\text{JD}}_{\text{QR}}^{+}$ algorithm.

Both of the ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ algorithms can be stopped when the value of cost function (11) or its relative change between two successive sweeps fall below a fixed small positive threshold. Such a stopping criterion is guaranteed to be met since the cost function is non-increasing in each Jacobi-like sweep.

### Practical issues

where the index sets ${I}_{1}\left(j\right)$, ${J}_{1}$, ${I}_{2}$, and ${J}_{2}\left(i\right)$ are defined in Lemma 1. By inserting (36) and (37) into the cost function (9), the ways of estimating the two sets of parameters {*ℓ*_{i,j},*u*_{i,j}} and {*θ*_{i,j},*u*_{i,j}} are identical to those of Afsari’s LUJ1D and QRJ1D methods [26], respectively. Therefore, in practice, in order to give an automatically SNR-adaptive method, for ${\text{JD}}_{\text{LU}}^{+}$, in each Jacobi-like iteration, we suggest to compute *u*_{i,j} by LUJ1D first. If all the elements in the *j*-th column of $\stackrel{~}{\mathit{A}}{\mathit{U}}^{(i,j)}\left({u}_{i,j}\right)$ have the same sign *ε*, the update ${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}=\epsilon \stackrel{~}{\mathit{A}}{\mathit{U}}^{(i,j)}\left({u}_{i,j}\right)$ is adopted. Otherwise, *u*_{i,j} is computed by minimizing (20) and ${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}$ is updated by computing (21). Each *ℓ*_{i,j} is computed similarly. Furthermore, the proposed ${\text{JD}}_{\text{QR}}^{+}$ and QRJ1D are combined in the same manner.

*k*∈ {1,⋯,

*K*}) are not balanced in their norms, the computation of the parameter could be inaccurate. In order to cope with this effect, we apply Afsari’s row balancing scheme every few sweeps. Such a scheme updates each ${\stackrel{~}{\mathit{C}}}^{(k,\text{new}))}$ by ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}\phantom{\rule{0.3em}{0ex}}=\phantom{\rule{0.3em}{0ex}}\mathit{\Lambda}{\stackrel{~}{\mathit{C}}}^{\left(k\right)}\phantom{\rule{0.3em}{0ex}}\mathit{\Lambda}$ and ${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}$ by ${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}\phantom{\rule{0.3em}{0ex}}=\phantom{\rule{0.3em}{0ex}}\stackrel{~}{\mathit{A}}\mathit{\Lambda}$ using a diagonal matrix , whose diagonal elements are defined as follows:

where ${\stackrel{~}{\mathit{C}}}_{n,:}^{\left(k\right)}$ denotes the *n*-th row of ${\stackrel{~}{\mathit{C}}}^{\left(k\right)}$.

*N*>

*P*is encountered, the invertibility assumption of the frontal slices C

^{(k)}does not hold. In this situation, we can compress A by means of a nonnegative matrix such that the resulting matrix $\stackrel{\u0304}{\mathit{A}}={\mathit{W}}_{\phantom{\rule{0.3em}{0ex}}+}\mathit{A}$ is a nonnegative square matrix. Then, the ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ algorithms can be used to compute the compressed loading matrix $\stackrel{\u0304}{\mathit{A}}$. W

_{+}can be computed by using the nonnegative compression algorithm (NN-COMP) that we proposed in [53]. More precisely, given a realization of an observation vector, we obtain the square root of the covariance matrix, denoted by . The classical prewhitening matrix is computed by where

*♯*denotes the pseudo inverse operator [47]. Then, the NN-COMP algorithm computes a linear transformation matrix such that W

_{+}= Ψ W has nonnegative components. Once $\stackrel{\u0304}{\mathit{A}}$ is estimated, the original matrix A is obtained as follows:

It should be noted that generally A does not need to be computed in such an ICA problem, since the sources can be estimated directly by means of $\stackrel{\u0304}{\mathit{A}}$.

### Numerical complexity

The numerical complexities of ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ are analyzed in terms of the number of floating point operations (flops). A flop is defined as a multiplication followed by an addition. In practice, only the number of multiplications, required to identify the loading matrix from a three-way array , is considered, which does not affect the order of magnitude of the numerical complexity.

^{(k,-1)}(

*k*∈ {1,⋯,

*K*}) of the frontal slices of cost

*N*

^{3}

*K*flops, the initialization of ${\stackrel{~}{\mathit{C}}}_{\text{ini}}^{\left(k\right)}={\stackrel{~}{\mathit{A}}}_{\text{ini}}^{\mathsf{\text{T}}}{\mathit{C}}^{(k,-1)}{\stackrel{~}{\mathit{A}}}_{\text{ini}}$ requires 2

*N*

^{3}

*K*flops, and at each sweep, the calculation of parameters

*u*

_{i,j}needs

*N*(

*N*-1)(5

*N*

^{2}+12

*N*-8)

*K*/2 flops. In addition, in the case of the ${\text{JD}}_{\text{LU}}^{+}$ algorithm, the calculation cost of ${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}$, ${\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}$, and ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$, with

*k*∈ {1,⋯,

*K*}, is

*N*(

*N*-1)(4

*N*+ (4

*N*+ 1)

*K*) flops, and the numerical complexity of computing the parameters

*ℓ*

_{i,j}is equal to that of

*u*

_{i,j}. Regarding the ${\text{JD}}_{\text{QR}}^{+}$ algorithm, for each sweep, the complexity of calculating the parameters

*θ*

_{i,j}is equal to

*N*(

*N*-1)(5

*N*

^{2}+ 3

*N*+ 29)

*K*/2 flops, and the estimation of ${\stackrel{~}{\mathit{A}}}^{\left(\text{new}\right)}$, ${\stackrel{~}{\mathit{B}}}^{\left(\text{new}\right)}$, and ${\stackrel{~}{\mathit{C}}}^{(k,\text{new})}$, with

*k*∈ {1,⋯,

*K*}, costs

*N*(

*N*- 1)(5

*N*+ (12

*N*+ 20)

*K*/2) flops. In practice, the proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ techniques are combined with LUJ1D and QRJ1D [26], respectively, leading to the magnitude of global numerical complexities of ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ being between $\mathcal{O}\left({N}^{3}K\right)$ and $\mathcal{O}\left({N}^{4}K\right)$. A recent nonnegative JDC method called ${\text{ACDC}}_{\text{LU}}^{+}$[41] is also based on a square change of variable and LU matrix factorization. It minimizes the cost function (4) with respect to A and D alternately, leading to a higher numerical complexity. By means of the reformulation of the cost function, the proposed methods avoid the estimation of D, therefore achieving a lower complexity compared to ${\text{ACDC}}_{\text{LU}}^{+}$. The explicit expressions of the overall complexity of ${\text{JD}}_{\text{LU}}^{+}$, ${\text{JD}}_{\text{QR}}^{+}$, and ${\text{ACDC}}_{\text{LU}}^{+}$[41], as well as those of four classical JDC algorithms, namely ACDC [23], FFDIAG [25], LUJ1D [26], and QRJ1D [26], are listed in Table 1. One can notice that numerical complexities of the proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ methods are at most one order of magnitude higher than those of the four JDC algorithms and still lower than that of ${\text{ACDC}}_{\text{LU}}^{+}$. Moreover, ${\text{JD}}_{\text{LU}}^{+}$ is less computationally expensive than ${\text{JD}}_{\text{QR}}^{+}$.

**Numerical complexities of seven JDC algorithms in terms of flops**

Numerical complexity | |
---|---|

ACDC | (13/3 |

FFDIAG | (2 |

LUJ1D | (4 |

QRJ1D | (6 |

${\text{ACDC}}_{\text{LU}}^{+}$ | $\left(\right(15{N}^{2}+4N\left)\mathit{\text{KN}}\right(N-1)+4/3{N}^{2}K+{N}^{3}+{N}^{2}){N}_{s}^{1}$ |

$+\left(\right(33{N}^{2}+7N\left)\mathit{\text{KN}}\right(N-1)+4/3{N}^{2}K+{N}^{3}+{N}^{2}){N}_{s}^{2}$ | |

${\text{JD}}_{\text{LU}}^{+}$ | $3{N}^{3}K+(4\mathit{\text{NK}}+N-2K)N(N-1){N}_{s}^{1}$ |

$+\left(\right(5{N}^{2}+16N-7)K+4N)N(N-1){N}_{s}^{2}$ | |

${\text{JD}}_{\text{QR}}^{+}$ | $3{N}^{3}K+(6\mathit{\text{NK}}+2.5N+1.5K)N(N-1){N}_{s}^{1}$ |

$+\left(\right(5{N}^{2}+15.5N+21)K+7N)N(N-1){N}_{s}^{2}$ |

## Simulation results

This section is twofold. In the first part, the performance of the proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ algorithms is evaluated with simulated semi-nonnegative semi-symmetric three-way arrays
. Several experiments are designed to study the convergence property, the influence of SNR, the impact of the third dimension *K* of
, the effect of the coherence of the loading matrix D, and the influence of the condition number of the diagonal matrices D^{(k)}. We also evaluate the proposed methods for estimating a non-square matrix A. The proposed algorithms are compared with four classical nonorthogonal JDC methods, namely ACDC [23], FFDIAG [25], LUJ1D [26], QRJ1D [26], and the nonnegative JDC method ${\text{ACDC}}_{\text{LU}}^{+}$[41]. In the second part, the source separation ability of the proposed algorithms is studied through a BSS application. In this context, the ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ are used to jointly diagonalize several matrix slices of the fourth-order cumulant array [40] of the observations and compared with several classical ICA [47, 54, 55] and NMF [56] methods.

### Simulated semi-nonnegative INDSCAL model

*N*is generated randomly according to the semi-nonnegative INDSCAL model (3). When used without further specification, all the algorithms are manipulated under the following conditions:

- i)Model generation: The loading matrix is randomly drawn from a uniform distribution on the interval [ 0,1]. The loading matrix is drawn from a Gaussian distribution with a mean of 1 and a deviation of 0.5. The pure array is perturbed by a residual INDSCAL noise array . The loading matrices of are drawn from a zero-mean unit-variance Gaussian distribution. The resulting noisy three-way array can be written as follows:${\mathcal{C}}_{N}=\frac{\mathcal{C}}{\parallel \mathcal{C}{\parallel}_{F}}+{\sigma}_{N}\frac{\mathcal{V}}{\parallel \mathcal{V}{\parallel}_{F}}$(40)
where

*σ*_{ N }is a scalar controlling the noise level. Then, the SNR is defined by $\text{SNR}=-20\underset{10}{\text{log}}\left({\sigma}_{N}\right)$. - ii)
Initialization: In each Monte Carlo trial, all the algorithms are initialized by a same random matrix whose components obey the uniform distribution over [ 0,1].

- iii)
Afsari’s row balancing scheme: The LUJ1D, QRJ1D, ${\text{JD}}_{\text{LU}}^{+}$, and ${\text{JD}}_{\text{QR}}^{+}$ algorithms perform the row balancing scheme once per run of five sweeps.

- iv)
Stopping criterion: All the algorithms stop either when the relative error of the corresponding criterion between two successive sweeps is less than 10

^{-5}or when the number of sweeps exceeds 200. A sweep of ACDC includes a full AC phase and a DCphase. - v)Performance measurement: The performance is measured by means of the error between the true loading matrix A and the estimate $\stackrel{~}{\mathit{A}}$, the numerical complexity, and the CPU time. We define the following scale-invariant and permutation-invariant distance [40]:$\alpha (\mathit{A},\stackrel{~}{\mathit{A}})=\frac{1}{N}\sum _{n=1}^{N}\phantom{\rule{1em}{0ex}}\underset{(n,{n}^{\prime})\in {I}_{n}^{2}}{min}d({\mathit{a}}_{n},{\stackrel{~}{\mathit{a}}}_{{n}^{\prime}})$(41)where a
_{ n }and ${\stackrel{~}{\mathit{a}}}_{{n}^{\prime}}$ are the*n*-th column of A and the*n*^{′}-th column of $\stackrel{~}{\mathit{A}}$, respectively. ${I}_{n}^{2}$ is defined recursively by ${I}_{1}^{2}=\{1,\cdots \phantom{\rule{0.3em}{0ex}},N\}\times \{1,\cdots \phantom{\rule{0.3em}{0ex}},N\}$, and ${I}_{n+1}^{2}={I}_{n}^{2}-{J}_{n}^{2}$ where ${J}_{n}^{2}={\text{argmin}}_{(n,{n}^{\prime})\in {I}_{n}^{2}}d({\mathit{a}}_{n},{\stackrel{~}{\mathit{a}}}_{{n}^{\prime}})$. In addition, $d({\mathit{a}}_{n},{\stackrel{~}{\mathit{a}}}_{{n}^{\prime}})$ is defined as the pseudo-distance between two vectors [13]:$d({\mathit{a}}_{n},{\stackrel{~}{\mathit{a}}}_{{n}^{\prime}})=1-\frac{{\u2225{\mathit{a}}_{n}^{\mathsf{\text{T}}}\phantom{\rule{0.3em}{0ex}}{\stackrel{~}{\mathit{a}}}_{{n}^{\prime}}\u2225}^{2}}{\parallel {\mathit{a}}_{n}{\parallel}^{2}\parallel {\stackrel{~}{\mathit{a}}}_{{n}^{\prime}}{\parallel}^{2}}$(42)The criterion (41) is an upper bound of the optimal permutation-invariant criterion. It avoids the burdensome computation of all the permutations. A small value of (41) means a good performance in the sense that $\stackrel{~}{\mathit{A}}$ is close to A.

- vi)
Test environment: The simulations are carried out in Matlab v7.14 on Mac OS X and run on Intel Quad-Core CPU 2.8 GHz with 32 GB memory. Moreover, we repeat all the experiments with 500 Monte Carlo trials.

#### Convergence

*N*= 5 and

*K*= 15. The performance is assessed under three SNR conditions: SNR = - 5, 10, and 25 dB, respectively. Figure 1 shows the convergence curves measured in terms of the cost function as a function of sweeps. It shows that FFDIAG, LUJ1D, and QRJ1D exhibit fast convergence behavior. They converge in less than 20 sweeps. ${\text{ACDC}}_{\text{LU}}^{+}$ decreases the cost function (4) quasi-linearly. ACDC and ${\text{ACDC}}_{\text{LU}}^{+}$ do not converge in a maximum of 200 sweeps. The proposed ${\text{JD}}_{\text{LU}}^{+}$ algorithm converges in about 100 sweeps when SNR =25 dB and SNR =10 dB, and it converges in about 40 sweeps when SNR =-5 dB. Regarding ${\text{JD}}_{\text{QR}}^{+}$, it reduces the cost function (11) to the values relatively higher than those achieved by ${\text{JD}}_{\text{LU}}^{+}$ and converges in about 50 sweeps whatever the SNR is. It seems that FFDIAG, LUJ1D, and QRJ1D achieve the fastest convergence rate. It should be noted that while an algorithm may converge to a point in which the value of the cost function is close to zero, such a point could be a local minimum far from the desired matrix A as shown in Figure 2. The top picture in Figure 2 shows the convergence curves measured in terms of the estimating error $\alpha (\mathit{A},\stackrel{~}{\mathit{A}})$ as a function of sweeps when SNR=25 dB. It shows that the solutions of FFDIAG, LUJ1D, and QRJ1D are still far from optimum. ACDC and ${\text{ACDC}}_{\text{LU}}^{+}$ give better estimations of A than the previous three methods. The best results are achieved by the proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ methods. The middle picture in Figure 2 displays the convergence curves when SNR=10 dB. It can be observed that ACDC converges to a local minimum which is not the global one and that the performance of the proposed methods is still better than that of the five other algorithms. For a low SNR =-5 dB, as shown in the bottom picture in Figure 2, both the methods based on alternating optimization, namely ACDC and ${\text{ACDC}}_{\text{LU}}^{+}$, converge to local minima which are less desirable. The proposed algorithms are always able to converge to better results than the classical methods. The average numerical complexities and CPU time of all the algorithms over Monte Carlo trials are shown in Table 2. It is observed that FFDIAG, LUJ1D, and QRJ1D require a small amount of calculations, whereas ${\text{ACDC}}_{\text{LU}}^{+}$ requires a large amount of calculations. The proposed ${\text{JD}}_{\text{LU}}^{+}$ just costs a bit more flops and CPU time than ACDC, but it is still much more efficient. Concerning the ${\text{JD}}_{\text{QR}}^{+}$ algorithm, it is more costly than ${\text{JD}}_{\text{LU}}^{+}$, with a comparable performance. We can then conclude that ${\text{JD}}_{\text{LU}}^{+}$ offers the best performance/complexity compromise in these experiments.

**Average numerical complexities (in flops) and computation time (in seconds) of the convergence experiment**

SNR = 25 dB | SNR = 10 dB | SNR =-5 dB | ||||
---|---|---|---|---|---|---|

Complexity | Time | Complexity | Time | Complexity | Time | |

ACDC | 2.1708 × 10 | 1.1357 | 2.0338 × 10 | 1.0535 | 1.6800 × 10 | 0.8761 |

FFDIAG | 1.1001 × 10 | 0.0331 | 1.0878 × 10 | 0.0327 | 9.4380 × 10 | 0.0287 |

LUJ1D | 2.8903 × 10 | 0.0660 | 2.3126 × 10 | 0.0526 | 1.4199 × 10 | 0.0325 |

QRJ1D | 2.2158 × 10 | 0.0383 | 2.4445 × 10 | 0.0421 | 2.6989 × 10 | 0.0468 |

${\text{ACDC}}_{\text{LU}}^{+}$ | 2.4462 × 10 | 2.6735 | 2.7498 × 10 | 2.8034 | 2.9646 × 10 | 2.9119 |

${\text{JD}}_{\text{LU}}^{+}$ | 2.8487 × 10 | 0.8107 | 4.9684 × 10 | 1.1098 | 7.1434 × 10 | 1.2938 |

${\text{JD}}_{\text{QR}}^{+}$ | 3.0766 × 10 | 1.0455 | 5.0554 × 10 | 1.1932 | 8.2185 × 10 | 1.3026 |

#### Effect of SNR

*N*= 5 and

*K*= 15. We repeat the experiments with SNR ranging from -30 to 50 dB with a step of 2 dB. The top picture in Figure 3 depicts the average curves of $\alpha (\mathit{A},\stackrel{~}{\mathit{A}})$ of the seven algorithms as a function of SNR. The obtained results show that the performance of all the methods increases as SNR grows. For the unconstrained methods, generally, ACDC performs better than FFDIAG, LUJ1D, and QRJ1D. The nonnegativity constraint obviously helps ${\text{ACDC}}_{\text{LU}}^{+}$, ${\text{JD}}_{\text{LU}}^{+}$, and ${\text{JD}}_{\text{QR}}^{+}$ to improve the results for lower SNR values. The performance of ACDC and ${\text{ACDC}}_{\text{LU}}^{+}$ remains stable for higher SNR values due to the small number of available sweeps and the lack of good initializations. Generally, the proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ algorithms outperform the others when SNR is between -20 and 30 dB and perform similar to FFDIAG, LUJ1D, and QRJ1D when SNR is above 45 dB. The average numerical complexity and CPU time at each SNR level of all the methods in this experiment are shown in the bottom of Figure 3. It shows that the proposed methods achieve better estimations of A and cost less flops and CPU time than ${\text{ACDC}}_{\text{LU}}^{+}$. The ${\text{JD}}_{\text{LU}}^{+}$ gives the best performance/complexity trade-off for all the considered SNR values.

#### Effect of dimension K

*K*of the three-way array corresponds to the number of covariance matrices at different lags, or the number of matrix slices derived from a cumulant array. In this section, we study the influence of

*K*on the performance of the seven algorithms. The first and second dimensions of ${\mathcal{C}}_{N}$ are set to

*N*= 5. The SNR value is fixed to 10 dB. We repeat the experiment with

*K*ranging from 3 to 55. The top picture in Figure 4 shows the average curves of $\alpha (\mathit{A},\stackrel{~}{\mathit{A}})$ of all the algorithms as a function of

*K*. For the five existing methods, ACDC, ${\text{ACDC}}_{\text{LU}}^{+}$, FFDIAG, LUJ1D, and QRJ1D, their performance is quite stable with respect to

*K*. The performance of the proposed methods progresses as

*K*increases and then practically stabilizes for high values of

*K*. It indicates that after some point (e.g.,

*K*≥ 20), the additional information brought by an increase of

*K*does not further improve the results. The proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ algorithms maintain competitive advantages through all the

*K*values. The two images in the bottom of Figure 4 present the average numerical complexity and CPU time of all the algorithms in this experiment, respectively. It shows that the numerical complexity of ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ is between that of ACDC and ${\text{ACDC}}_{\text{LU}}^{+}$. The ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ methods seem to be the most effective algorithms compared to the other methods.

#### Effect of coherence of D

_{ n }and d

_{ m }denote the

*n*-th and

*m*-th columns of D, respectively. The angle

*ψ*

_{n,m}between d

_{ n }and d

_{ m }can be derived by using the following Euclidean dot product formula ${\mathit{d}}_{n}^{\mathsf{\text{T}}}\phantom{\rule{0.3em}{0ex}}{\mathit{d}}_{m}=\parallel {\mathit{d}}_{n}\parallel \parallel {\mathit{d}}_{m}\parallel \text{cos}\left({\psi}_{n,m}\right)$. Then, the coherence

*ρ*of D is defined as the maximum absolute cosine of angle

*ψ*

_{n,m}between the columns of D as follows:

*ρ*is also known as the modulus of uniqueness of JDC [43]. By its definition (43),

*ρ*falls in the range of [ 0,1]. The JDC problem is considered to be ill-conditioned when

*ρ*is close to 1. Such an ill-conditioned problem can be met in ICA when A has nearly collinear column vectors. For example, in order to perform ICA, provided that all the sources are non-Gaussian, which is often the case in practice, we can build a three-way array by stacking the matrix slices of the fourth-order cumulant array of the observation data. Then, the loading matrix D can be expressed as follows:

*N*×

*N*) diagonal matrix with ${\mathcal{C}}_{n,n,n,n,\left\{\mathit{s}\right\}}$ being the fourth-order cumulant of the

*n*-th source,

*n*∈ {1,⋯,

*N*}, and where ⊙ denotes the Khatri-Rao product. It can be observed that the coherence of the columns of A will influence the coherence of the matrix D. In the following test, the dimensions of the three-way array ${\mathcal{C}}_{N}$ are set to

*N*= 5 and

*K*= 15. The SNR value is fixed to 10 dB. In order to control

*ρ*, firstly, we randomly generate an orthogonal matrix so that

*ρ*= 0 by orthogonalizing a (15×5) random matrix. Secondly, we rotate its five columns such that all the internal angles between any columns are equal to a predefined value

*ψ*. Therefore,

*ρ*is only controlled by the angle

*ψ*and equals to $\left|\text{cos}\right(\psi \left)\right|$. We repeat the experiment with the angle

*ψ*ranging from 0 to

*π*/2 with a step of

*π*/60. A small

*ψ*value means a large

*ρ*value. The top picture in Figure 5 displays the average curves of $\alpha (\mathit{A},\stackrel{~}{\mathit{A}})$ of all the algorithms as a function of

*ψ*. It shows that the nonnegativity constrained methods ${\text{ACDC}}_{\text{LU}}^{+}$, ${\text{JD}}_{\text{LU}}^{+}$, and ${\text{JD}}_{\text{QR}}^{+}$, outperform the unconstrained ones ACDC, FFDIAG, LUJ1D, and QRJ1D. The proposed algorithms are more efficient, particularly when the coherence level is high. The average numerical complexity and CPU time displayed in the bottom of Figure 5 indicate that the ${\text{JD}}_{\text{LU}}^{+}$ algorithm provides the best performance/complexity compromise, while the ${\text{JD}}_{\text{QR}}^{+}$ algorithm is also competitive with regard to ${\text{ACDC}}_{\text{LU}}^{+}$.

####
Effect of condition number of D
^{(k)}

^{(k)}could contain some diagonal elements which, despite being non-zero, are many orders of magnitude lower than some other elements, leading to an ill-conditioned matrix C

^{(k)}. For the proposed methods, the inverse of such a matrix C

^{(k)}would contain numerical errors. In this experiment, we study the performance of the seven algorithms as a function of the condition number of one of the diagonal matrices D

^{(k)}. The dimensions of the three-way array ${\mathcal{C}}_{N}$ are set to

*N*= 5 and

*K*= 15. The SNR value is set to 10 dB. We vary the condition number of the first diagonal matrix D

^{(1)}from 1 to 1,000 by fixing the ratio of its largest diagonal element to its smallest diagonal element. The top picture in Figure 6 displays the average curves of the estimating error $\alpha (\mathit{A},\stackrel{~}{\mathit{A}})$ of the seven algorithms as a function of the condition number of D

^{(1)}. The results reveal that a highly ill-conditioned diagonal matrix D

^{(1)}has a clear negative effect on the estimation accuracy of all the algorithms. The nonnegativity constrained methods ${\text{ACDC}}_{\text{LU}}^{+}$, ${\text{JD}}_{\text{LU}}^{+}$, and ${\text{JD}}_{\text{QR}}^{+}$ outperform the classical algorithms ACDC, FFDIAG, LUJ1D, and QRJ1D whatever the condition number is. The proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ algorithms maintain advantages when the condition number is less than 100. Regarding the cases of larger condition numbers, ${\text{ACDC}}_{\text{LU}}^{+}$ is more superior since it does not need to invert the highly ill-conditioned matrix. It is worthy pointing out that in practice, we can choose these sufficiently well-conditioned matrices C

^{(k)}for the proposed methods, whose condition numbers are below a predefined threshold. In addition, a weighted cost function for which weights would depend on the condition number of each matrix can be considered. On the other hand, the performance of the classical methods can also be improved by choosing a particular subset of available matrices [57] and by properly weighting the cost functions [49]. In order to give a fair comparison, all the algorithms operate on the same set of matrices in all the experiments of this paper. In addition, the average numerical complexity and CPU time at each condition number of all the methods in this experiment are shown in the bottom of Figure 6. It shows that the proposed methods give the best performance/complexity trade-off compared to ${\text{ACDC}}_{\text{LU}}^{+}$ whatever the condition number is.

#### Test with a non-square matrix A

*N*>

*P*is met in ICA, we propose to compress it by a nonnegative compression matrix [53], such that the resulting matrix $\stackrel{\u0304}{\mathit{A}}={\mathit{W}}_{\phantom{\rule{0.3em}{0ex}}+}\mathit{A}$ is a (

*P*×

*P*) nonnegative square matrix. Then, the proposed methods can be applied to estimate $\stackrel{\u0304}{\mathit{A}}$. Similar to the classical prewhitening, the nonnegative compression step could introduce numerical errors. In this experiment, we compare our methods to ACDC and ${\text{ACDC}}_{\text{LU}}^{+}$ through a simulated ICA model. The latter algorithms can directly estimate a non-square matrix A from the fourth-order cumulant matrix slices. The ICA model is established as follows:

*N*× 1) observation vector, $\mathit{s}\left[f\right]={\left[{s}_{1}\left[f\right],{s}_{2}\left[f\right],{s}_{3}\left[f\right]\right]}^{\mathsf{\text{T}}}$ is the (3 × 1) zero-mean unit-variance source vector whose elements are independently drawn from a uniform distribution over $\left[-\sqrt{3},\sqrt{3}\right]$, $\mathit{\nu}={\left[{\nu}_{1}\left[f\right],\cdots \phantom{\rule{0.3em}{0ex}},{\nu}_{N}\left[f\right]\right]}^{\mathsf{\text{T}}}$ is the (

*N*× 1) zero-mean unit-variance Gaussian noise vector, and A is the (

*N*× 3) nonnegative mixing matrix whose components are independently drawn from a uniform distribution over [ 0,1]. In this context, the SNR is defined by:

*f*] } is compressed by means of a matrix computed using the method proposed in [53], leading to a three-dimensional vector $\left\{\stackrel{\u0304}{\mathit{x}}\right[f\left]\phantom{\rule{0.3em}{0ex}}\right\}$. We compute the fourth-order cumulant array of $\left\{\stackrel{\u0304}{\mathit{x}}\right[f\left]\phantom{\rule{0.3em}{0ex}}\right\}$ and choose the first three matrix slices in order to build a three-way array. Hence, ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ decompose a (3×3×3) array. Once the compressed mixing matrix $\stackrel{\u0304}{\mathit{A}}$ is estimated, the original mixing matrix is obtained by Eq. 39. Regarding ACDC and ${\text{ACDC}}_{\text{LU}}^{+}$, the fourth-order cumulant array of {x[

*f*] } is directly computed without compression. We apply ACDC and ${\text{ACDC}}_{\text{LU}}^{+}$ on two three-way arrays with different third dimensions. The first array of dimension (

*N*×

*N*× 3) is built by choosing the first three matrix slices from the fourth-order cumulant array, while the second array of dimension (

*N*×

*N*×

*N*) is built using the first

*N*matrix slices. We study the impact of the number of observations

*N*on the performance of the JDC algorithms, by varying

*N*from 4 to 24. The SNR value is fixed to 5 dB. The number of samples used to estimate the cumulants is set to 10

^{3}. Figure 7 shows the average curves of the estimating error $\alpha (\mathit{A},\stackrel{~}{\mathit{A}})$ of all the algorithms as a function of

*N*. As it can be seen, when

*N*≤ 15, the larger the value of

*N*, the more accurate estimation of A is achieved. When

*N*> 15, the further increase of

*N*does not bring significant improvement in terms of the estimation accuracy. ACDC and ${\text{ACDC}}_{\text{LU}}^{+}$ give better results when the array with a larger third dimension is considered. Their results on (

*N*×

*N*×

*N*) arrays outperform the proposed methods when

*N*= 4. ${\text{ACDC}}_{\text{LU}}^{+}$ also gives the best estimation on (

*N*×

*N*×

*N*) arrays with

*N*= 5. It suggests that the numerical errors introduced by the compression step limit the performance of the proposed methods when only a small number of observation is available. Such a negative effect can be partially compensated by using a large number of observations, since the proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ methods maintain the highest performance in terms of estimation accuracy when

*N*≥6. The performance ACDC and ${\text{ACDC}}_{\text{LU}}^{+}$ can be further improved by using a (

*N*×

*N*×

*N*

^{2}) array, which contains all the

*N*

^{2}fourth-order cumulant matrix slices. However, it leads to a higher numerical complexity especially for a large value of

*N*. Regarding the proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ methods, their performance can also be improved by using all the nine matrix slices of the fourth-order cumulant array of the compressed observation vector. Nevertheless, the experimental result has already shown that by using only a small number of matrix slices, ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ can maintain lower numerical complexities than ACDC and ${\text{ACDC}}_{\text{LU}}^{+}$, while achieving better estimation results, when a large value of

*N*is considered. Therefore, despite the negative influence of the nonnegative compression, the proposed methods still offer a good performance/complexity compromise to estimate a non-square matrix A.

### BSS application on MRS data

In this section, we aim to illustrate the potential capability of the proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ algorithms for solving a real-life BSS problem by an application carried on simulated MRS data.

*s*

_{1}[

*f*] } and {

*s*

_{2}[

*f*] }, namely the Choline (Cho) and Myo-inositol (Ins) (see Figure 8b), are generated by Lorentzian and Gaussian functions [59]. Each of the sources contains 10

^{3}samples. The observation vector x[

*f*] is generated according to (45). The components of the (

*N*×2) mixing matrix A are randomly drawn from a uniform distribution. The additive noise ν[

*f*] is modeled as a zero-mean unit-variance Gaussian vector. The ICA methods based on the proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ algorithms, namely ${\text{JD}}_{\text{LU}}^{+}$-ICA and ${\text{JD}}_{\text{QR}}^{+}$-ICA, consist of four steps: i) compressing {x[

*f*] } by means of a matrix [53], ii) estimating the fourth-order cumulant array of the compressed observations and stacking all the cumulant matrix slices in a three-way array, iii) decomposing the resulting three-way array by means of ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$, respectively, and iv) reconstructing the sources. The ${\text{JD}}_{\text{LU}}^{+}$-ICA and ${\text{JD}}_{\text{QR}}^{+}$-ICA are compared to four state-of-the-art BSS algorithms, namely two efficient ICA methods CoM

_{2}[54] and SOBI [47], the nonnegative ICA (NNICA) method with a line search along the geodesic [55], and the NMF method [56] based on alternating nonnegativity least squares. The performance is assessed by means of the error $\alpha \phantom{\rule{0.3em}{0ex}}\left({\left\{\mathit{s}\right[f\left]\phantom{\rule{0.3em}{0ex}}\right\}}^{\mathsf{\text{T}}},{\left\{\stackrel{~}{\mathit{s}}\right[f\left]\phantom{\rule{0.3em}{0ex}}\right\}}^{\mathsf{\text{T}}}\right)$ between the true source s[

*f*] and its estimate $\stackrel{~}{\mathit{s}}\left[f\right]$, the numerical complexity, and the CPU time. To find out the detailed analysis of the numerical complexity of the classical ICA algorithms, the reader can refer to the book chapter [60]. Figure 8 shows an example of the separation results of all the methods with

*N*= 32 observations and a SNR of 10 dB. Regarding CoM

_{2}, SOBI, NNICA, and NMF, there are some obvious disturbances presented in the estimated metabolites. As far as ${\text{JD}}_{\text{LU}}^{+}$-ICA and ${\text{JD}}_{\text{QR}}^{+}$-ICA are concerned, the estimated source metabolites are quasi-perfect. Furthermore, the comprehensive performance of all the methods will be studied by the following experiments with 200 independent Monte Carlo trials.

*N*is evaluated. The SNR is fixed to 10 dB. The six methods are compared with

*N*ranging from 4 to 116 with a step of 4. The average curves of error $\alpha \phantom{\rule{0.3em}{0ex}}\left({\left\{\mathit{s}\right[f\left]\phantom{\rule{0.3em}{0ex}}\right\}}^{\mathsf{\text{T}}},{\left\{\stackrel{~}{\mathit{s}}\right[f\left]\phantom{\rule{0.3em}{0ex}}\right\}}^{\mathsf{\text{T}}}\right)$ as a function of

*N*are shown in the left image of Figure 9. It can be seen that the estimating errors of all the methods improve as

*N*increases. It suggests that in noisy BSS contexts, using more sensors often yields better results. The proposed ${\text{JD}}_{\text{LU}}^{+}$-ICA and ${\text{JD}}_{\text{QR}}^{+}$-ICA methods maintain the competitive advantages. The average curves of the numerical complexities of this experiment are shown in the bottom left picture of Figure 9. We can notice that the numerical complexities of all the methods increase with

*N*. The complexities of ${\text{JD}}_{\text{LU}}^{+}$-ICA and ${\text{JD}}_{\text{QR}}^{+}$-ICA seem identical in the logarithmic scaled plot, which is because theoretically their complexities are mainly dominated by the computation of the nonnegative compression step and of the cumulants. Indeed, ${\text{JD}}_{\text{LU}}^{+}$-ICA is more computationally efficient than ${\text{JD}}_{\text{QR}}^{+}$-ICA in the step of CP decomposition of the cumulant array. This can be verified by the average CPU time of those methods, shown in the bottom right image of Figure 9. We can observe that ${\text{JD}}_{\text{LU}}^{+}$-ICA is slower than CoM

_{2}, but it is faster than NNICA, SOBI, and NMF.

*N*is set to 32. SNR is varied from 0 to 50 dB with a step of 2 dB. The average curves of the estimating error $\alpha \phantom{\rule{0.3em}{0ex}}\left({\left\{\mathit{s}\right[f\left]\phantom{\rule{0.3em}{0ex}}\right\}}^{\mathsf{\text{T}}},{\left\{\stackrel{~}{\mathit{s}}\right[f\left]\phantom{\rule{0.3em}{0ex}}\right\}}^{\mathsf{\text{T}}}\right)$ as well as those of the numerical complexities and CPU time as a function of SNR of all the six methods are shown in Figure 10. The proposed ${\text{JD}}_{\text{LU}}^{+}$-ICA and ${\text{JD}}_{\text{QR}}^{+}$-ICA methods provide the best estimation results with moderate computational complexities and CPU time. Generally speaking, the ${\text{JD}}_{\text{LU}}^{+}$-ICA algorithm offers the best performance/complexity trade-off in this BSS experimental context.

## Conclusions

We have proposed two methods, called ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$, in order to achieve the CP decomposition of semi-nonnegative semi-symmetric three-way arrays. The nonnegativity constraint is imposed on the two symmetric modes of three-way arrays by means of a square change of variable, giving rise to an unconstrained joint diagonalization by congruence problem. Therefore, the nonnegative loading matrix can be estimated by computing the joint diagonalizer. We consider the elementary LU and QR parameterizations of the Hadamard square root of the nonnegative joint diagonalizer, leading to two Jacobilike optimization procedures. In each Jacobi-like iteration, the optimization is formulated into a minimization of a polynomial or rational function with respect to only one parameter. In addition, the numerical complexity for each algorithm has been analyzed.

The performance of the proposed ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$ algorithms is evaluated with simulated semi-nonnegative semi-symmetric three-way arrays. Four classical nonorthogonal JDC methods without nonnegativity constraints including ACDC [23], FFDIAG [25], LUJ1D [26], and QRJ1D [26] and one nonnegative JDC method ${\text{ACDC}}_{\text{LU}}^{+}$[41] are tested as reference methods. The performance is assessed in terms of the matrix estimation accuracy, the numerical complexity, and the CPU time. The convergence property, the influence of SNR, the impact of dimension, the effect of coherence, and the influence of condition number are extensively studied by Monte Carlo experiments. The obtained results show that the proposed algorithms offer better estimation accuracy by means of exploiting the nonnegativity *a priori*. The ${\text{JD}}_{\text{LU}}^{+}$ algorithm provides the best performance/complexity compromise.

The proposed algorithms are suitable tools for solving the ICA problems where a nonnegative mixing matrix is considered, such as in MRS. In this case, the three-way array built by stacking the matrix slices of a HO cumulant array fulfills the semi-nonnegative semi-symmetric structure. We proposed two ICA methods, namely ${\text{JD}}_{\text{LU}}^{+}$-ICA and ${\text{JD}}_{\text{QR}}^{+}$-ICA, based on CP decomposition of the fourth-order cumulant array using ${\text{JD}}_{\text{LU}}^{+}$ and ${\text{JD}}_{\text{QR}}^{+}$, respectively. The source separation ability of the proposed algorithms is verified through a BSS application carried out on simulated MRS data. The ${\text{JD}}_{\text{LU}}^{+}$-ICA and ${\text{JD}}_{\text{QR}}^{+}$-ICA are compared to one NMF method [56], one nonnegative ICA method [55], and two classical ICA methods, namely CoM _{2}[54] and SOBI [47]. The performance is comprehensively studied as a function of the number of observations and of SNR. The experimental results demonstrate the improvement of the proposed methods in terms of the source estimation accuracy and also show that exploiting the two *a priori* of the data, namely the nonnegativity of the mixing matrix and the statistical independence of the sources, allows us to achieve better estimation results. The ${\text{JD}}_{\text{LU}}^{+}$-ICA algorithm provides the best performance/complexity trade-off.

## Declarations

## Authors’ Affiliations

## References

- Smilde A, Bro R, Geladi P:
*Multi-way Analysis: Applications in the Chemical Sciences*. Wiley, West Sussex; 2004.View ArticleGoogle Scholar - de Almeida ALF, Favier G, Ximenes LR: Space-time-frequency (STF) MIMO communication systems with blind receiver based on a generalized PARATUCK2 model.
*IEEE Trans. Signal Process*2013, 61(8):1895-1909.MathSciNetView ArticleGoogle Scholar - De Vos M, Vergult A, De Lathauwer L, De Clercq W, Van Huffel S, Dupont P, Palmini A, Van Paesschen W: Canonical decomposition of ictal scalp EEG reliably detects the seizure onset zone.
*Neuroimage*2007, 37(3):844-854. 10.1016/j.neuroimage.2007.04.041View ArticleGoogle Scholar - Comon P, Luciani X, de Almeida ALF: Tensor decompositions, alternating least squares and other tales.
*J. Chemometr*2009, 23: 393-405. 10.1002/cem.1236View ArticleGoogle Scholar - Tucker LR: Some mathematical notes on three-mode factor analysis.
*Psychometrika*1966, 31(3):279-311. 10.1007/BF02289464MathSciNetView ArticleGoogle Scholar - De Lathauwer L, De Moor B, Vandewalle J: A multilinear singular value decomposition.
*SIAM J. Matrix Anal. Appl*2000, 21(4):1253-1278. 10.1137/S0895479896305696MathSciNetView ArticleMATHGoogle Scholar - Kruskal JB: Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics.
*Lin. Algebra Appl*1977, 18(2):98-138.MathSciNetView ArticleMATHGoogle Scholar - Hitchcock FL: The expression of a tensor or a polyadic as a sum of products.
*J. Math. Phys*1927, 6(1):164-189.MathSciNetView ArticleMATHGoogle Scholar - Kroonenberg PM:
*Applied Multiway Data Analysis*. Wiley, Hoboken; 2008.View ArticleMATHGoogle Scholar - Sidiropoulos ND, Bro R, Giannakis GB: Parallel factor analysis in sensor array processing.
*IEEE Trans. Signal Process*2000, 48(8):2377-2388. 10.1109/78.852018View ArticleGoogle Scholar - de Almeida ALF, Favier G, Motab JCM: PARAFAC-based unified tensor modeling for wireless communication systems with application to blind multiuser equalization.
*Signal Process*2007, 87(2):337-351. 10.1016/j.sigpro.2005.12.014View ArticleMATHGoogle Scholar - de Almeida ALF, Favier G: Double Khatri-Rao space-time-frequency coding using semi-blind PARAFAC based receiver.
*IEEE Signal Process. Lett*2013, 20(5):471-474.View ArticleGoogle Scholar - Albera L, Ferréol A, Comon P, Chevalier P: Blind identification of overcomplete mixtures of sources (BIOME).
*Lin. Algebra Appl*2004, 391: 3-30.View ArticleMATHMathSciNetGoogle Scholar - Röemer F, Haardt M: Tensor-based channel estimation and iterative refinements for two-way relaying with multiple antennas and spatial reuse.
*IEEE Trans. Signal Process*2010, 58(11):5720-5735.MathSciNetView ArticleGoogle Scholar - Harshman RA, Lundy ME: PARAFAC: parallel factor analysis.
*Comput. Stat. Data Anal*1994, 18(1):39-72. 10.1016/0167-9473(94)90132-5View ArticleMATHGoogle Scholar - Uschmajew A: Local convergence of the alternating least squares algorithm for canonical tensor approximation.
*SIAM. J. Matrix Anal. Appl*2012, 33(2):639-652. 10.1137/110843587MathSciNetView ArticleMATHGoogle Scholar - Rajih M, Comon P, Harshman RA: Enhanced line search: a novel method to accelerate PARAFAC.
*SIAM J. Matrix Anal. Appl*2008, 30(3):1128-1147. 10.1137/06065577MathSciNetView ArticleMATHGoogle Scholar - Acar E, Dunlavy DM, Kolda TG: A scalable optimization approach for fitting canonical tensor decompositions.
*J. Chemometr*2011, 25(2):67-86. 10.1002/cem.1335View ArticleGoogle Scholar - Röemer F, Haardt M: A semi-algebraic framework for approximate CP decompositions via simultaneous matrix diagonalizations (SECSI).
*Signal Process*2013, 93(9):2722-2738. 10.1016/j.sigpro.2013.02.016View ArticleGoogle Scholar - Luciani X, Albera L: Canonical polyadic decomposition based on joint eigenvalue decomposition.
*Chemometr. Intell. Lab*2014, 132: 152-167.View ArticleGoogle Scholar - Carroll JD, Chang J-J: Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart-Young decomposition.
*Psychometrika*1970, 35(3):283-319. 10.1007/BF02310791View ArticleMATHGoogle Scholar - Husson F, Pagés J: INDSCAL model: geometrical interpretation and methodology.
*Comput. Stat. Data Anal*2006, 50(2):358-378. 10.1016/j.csda.2004.08.005View ArticleMATHMathSciNetGoogle Scholar - Yeredor A: Non-orthogonal joint diagonalization in the least-squares sense with application in blind source separation.
*IEEE Trans. Signal Process*2002, 50(7):1545-1553. 10.1109/TSP.2002.1011195MathSciNetView ArticleGoogle Scholar - Cardoso JF, Souloumiac A: Jacobi angles for simultaneous diagonalization.
*SIAM J. Matrix Anal. Appl*1996, 17: 161-164. 10.1137/S0895479893259546MathSciNetView ArticleMATHGoogle Scholar - Ziehe A, Laskov P, Nolte G, Muller K-R: A fast algorithm for joint diagonalization with non-orthogonal transformations and its application to blind source separation.
*J. Mach. Learn. Res*2004, 5: 777-800.MathSciNetMATHGoogle Scholar - Afsari B: Simple LU and QR based non-orthogonal matrix joint diagonalization. In
*ICA 2006, Springer LNCS 3889*. Charleston, SC, USA; 5–8 March 2006.Google Scholar - Van der Veen AJ: Joint diagonalization via subspace fitting techniques. In
*Proc. ICASSP ‘01*. Salt Lake, City, UT; 7–11 May 2001:2773-2776.Google Scholar - Yeredor A: On using exact joint diagonalization for noniterative approximate joint diagonalization.
*IEEE Signal Process. Lett*2005, 12(9):645-648.View ArticleGoogle Scholar - Vollgraf R, Obermayer K: Quadratic optimization for simultaneous matrix diagonalization.
*IEEE Trans. Signal Process*2006, 54(9):3270-3278.View ArticleGoogle Scholar - Li XL, Zhang XD: Nonorthogonal joint diagonalization free of degenerate solution.
*IEEE Trans. Signal Process*2007, 55(5):1803-1814.MathSciNetView ArticleGoogle Scholar - Souloumiac A: Nonorthogonal joint diagonalization by combining Givens and hyperbolic rotations.
*IEEE Trans. Signal Process*2009, 57(6):2222-2231.MathSciNetView ArticleGoogle Scholar - Xu XF, Feng DZ, Zheng WX: A fast algorithm for nonunitary joint diagonalization and its application to blind source separation.
*IEEE Trans. Signal Process*2011, 59(7):3457-3463.MathSciNetView ArticleGoogle Scholar - Chabriel G, Barrère J: A direct algorithm for nonorthogonal approximate joint diagonalization.
*IEEE Trans. Signal Process*2012, 60(1):39-47.MathSciNetView ArticleGoogle Scholar - Chabriel G, Kleinsteuber M, Moreau E, Shen H, Tichavský P, Yeredor A: Joint matrices decompositions and blind source separation: A survey of methods, identification, and applications.
*IEEE Signal Process. Mag*2014, 31(3):34-43.View ArticleGoogle Scholar - Lee DD, Seung HS: Learning the parts of objects by non-negative matrix factorization.
*Nature*1999, 401(6755):788-791. 10.1038/44565View ArticleGoogle Scholar - Zhang Q, Wang H, Plemmons RJ, Pauca VP: Tensor methods for hyperspectral data analysis: a space object material identification study.
*J. Opt. Soc. Am. A. Opt. Image Sci. Vis*2008, 25(12):3001-3012. 10.1364/JOSAA.25.003001View ArticleGoogle Scholar - Royer J-P, Thirion-Moreau N, Comon P: Computing the polyadic decomposition of nonnegative third order tensors.
*Signal Process*2011, 91(9):2159-2171. 10.1016/j.sigpro.2011.03.006View ArticleMATHGoogle Scholar - Cichocki A, Zdunek R, Phan AH, Amari S:
*Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation*. Wiley, West Sussex; 2009.View ArticleGoogle Scholar - Zhou GX, Cichocki A, Zhao Q, Xie SL: Nonnegative matrix and tensor factorizations : an algorithmic perspective.
*IEEE Signal Process. Mag*2014, 31(3):54-65.View ArticleGoogle Scholar - Coloigner J, Karfoul A, Albera L, Comon P: Line search and trust region strategies for canonical decomposition of semi-nonnegative semi-symmetric 3rd order tensors.
*Lin. Algebra Appl*2014, 450(1):334-374.MathSciNetView ArticleMATHGoogle Scholar - Wang L, Albera L, Kachenoura A, Shu HZ, Senhadji L: Nonnegative joint diagonalization by congruence based on LU matrix factorization.
*IEEE Signal Process. Lett*2013, 20(8):807-810.View ArticleGoogle Scholar - Wang L, Albera L, Kachenoura A, Shu HZ, Senhadji L: CP decomposition of semi-nonnegative semi-symmetric tensors based on QR matrix factorization. In
*SAM’14, Proceedings of the Eighth IEEE Sensor Array and Multichannel Signal Processing Workshop*. A Coruna, Spain; 22–25 June 2014:449-452.Google Scholar - Afsari B: Sensitivity analysis for the problem of matrix joint diagonalization.
*SIAM J. Matrix Anal. Appl*2008, 30(3):1148-1171. 10.1137/060655997MathSciNetView ArticleMATHGoogle Scholar - Wax M, Sheinvald J: A least-squares approach to joint diagonalization.
*IEEE Signal Process. Lett*1997, 4(2):52-53.View ArticleGoogle Scholar - Dégerine S, Kane E: A comparative study of approximate joint diagonalization algorithms for blind source separation in presence of additive noise.
*IEEE Trans. Signal Process*2007, 55(6):3022-3031.MathSciNetView ArticleGoogle Scholar - Fadaili EM, Thirion-Moreau N, Moreau E: Nonorthogonal joint diagonalization/zero diagonalization for source separation based on time-frequency distributions.
*IEEE Trans. Signal Process*2007, 55(5):1673-1687.MathSciNetView ArticleGoogle Scholar - Belouchrani A, Abed-Meraim K, Cardoso JF, Moulines E: A blind source separation technique using second-order statistics.
*IEEE Trans. Signal Process*1997, 45(2):434-444. 10.1109/78.554307View ArticleGoogle Scholar - Pham DT: Joint approximate diagonalization of positive definite Hermitian matrices.
*SIAM J. Matrix Anal. Appl*2001, 22: 1837-1848.View ArticleMATHMathSciNetGoogle Scholar - Tichavský P, Yeredor A: Fast approximate joint diagonalization incorporating weight matrices.
*IEEE Trans. Signal Process*2009, 57(3):878-891.MathSciNetView ArticleGoogle Scholar - Chu M, Diele F, Plemmons R, Ragni S: Optimality computation and interpretation of nonnegative matrix factorizations. Technical report, Wake Forest University 2004Google Scholar
- Meyer CD:
*Matrix Analysis and Applied Linear Algebra*. SIAM, Philadelphia; 2000.View ArticleGoogle Scholar - Vaidyanathan PP:
*Multirate Systems and Filter Banks*. PTR Prentice Hall, United States; 1993.MATHGoogle Scholar - Wang L, Kachenoura A, Albera L, Karfoul A, Shu HZ, Senhadji L: Nonnegative compression for semi-nonnegative independent component analysis. In
*SAM’14, Proceedings of the Eighth IEEE Sensor Array and Multichannel Signal Processing Workshop*. A Coruna, Spain; 22–25 June 2014:81-84.Google Scholar - Comon P: Independent component analysis, a new concept?
*Signal Process*1994, 36(3):287-314. 10.1016/0165-1684(94)90029-9View ArticleMATHGoogle Scholar - Plumbley MD: Algorithms for nonnegative independent component analysis.
*IEEE Trans. Neural Netw*2003, 14(3):534-543. 10.1109/TNN.2003.810616View ArticleGoogle Scholar - Kim H, Park H: Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method.
*SIAM J. Matrix Anal. Appl*2008, 30(2):713-730. 10.1137/07069239XMathSciNetView ArticleMATHGoogle Scholar - De Lathauwer: Algebraic methods after prewhitening. In
*Handbook of Blind Source Separation, ed. by P Comon, C Jutten,*. Elsevier, Oxford; 2010:155-177. Chap. 5View ArticleGoogle Scholar - Befroy DE, Shulman GI: Magnetic resonance spectroscopy studies of human metabolism.
*Diabetes*2011, 60(5):1361-1369. 10.2337/db09-0916View ArticleGoogle Scholar - Moussaoui S: Séparation de sources non-négatives: application au traitement des signaux de spectroscopie. PhD thesis, Université Henri Poincaré, (2005)Google Scholar
- Albera L, Comon P, Parra LC, Karfoul A, Kachenoura A, Senhadji L: Biomedical applications. In
*Handbook of Blind Source Separation ed. by P Comon, C Jutten,*. Elsevier, Oxford; 2010:737-777. Chap. 18View ArticleGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.