Overview of constrained PARAFAC models

Abstract

In this paper, we present an overview of constrained parallel factor (PARAFAC) models where the constraints model linear dependencies among columns of the factor matrices of the tensor decomposition or, alternatively, the pattern of interactions between different modes of the tensor which are captured by the equivalent core tensor. Some tensor prerequisites with a particular emphasis on mode combination using Kronecker products of canonical vectors that makes easier matricization operations, are first introduced. This Kronecker product‐based approach is also formulated in terms of an index notation, which provides an original and concise formalism for both matricizing tensors and writing tensor models. Then, after a brief reminder of PARAFAC and Tucker models, two families of constrained tensor models, the co‐called PARALIND/CONFAC and PARATUCK models, are described in a unified framework, for N th‐order tensors. New tensor models, called nested Tucker models and block PARALIND/CONFAC models, are also introduced. A link between PARATUCK models and constrained PARAFAC models is then established. Finally, new uniqueness properties of PARATUCK models are deduced from sufficient conditions for essential uniqueness of their associated constrained PARAFAC models.

1 Review

1.1 Introduction

Tensor calculus was introduced in differential geometry, at the end of the nineteenth century, and then tensor analysis was developed in the context of Einstein’s theory of general relativity, with the introduction of index notation, the so‐called Einstein summation convention, at the beginning of the twentieth century, which allows to simplify and shorten physics equations involving tensors. Index notation is also useful for simplifying multivariate statistical calculations, particularly those involving cumulant tensors[1]. Generally speaking, tensors are used in physics and differential geometry for characterizing the properties of a physical system, representing fundamental laws of physics, and defining geometrical objects whose components are functions. When these functions are defined over a continuum of points of a mathematical space, the tensor forms what is called a tensor field, a generalization of vector field used to solve problems involving curved surfaces or spaces, as it is the case of curved space‐time in general relativity. From a mathematical point of view, two other approaches are possible for defining tensors, in terms of tensor products of vector spaces, or multilinear maps. Symmetric tensors can also be linked with homogeneous polynomials[2].

After the first tensor developments by mathematicians and physicists, the need of analyzing collections of data matrices that can be seen as three‐way data arrays gave rise to three‐way models for data analysis, with the pioneering works of Tucker in psychometrics[3], and Harshman in phonetics[4], who proposed what is now referred to as the Tucker and parallel factor (PARAFAC) decompositions, respectively. The PARAFAC decomposition was independently proposed by Carroll and Chang[5] under the name canonical decomposition (CANDECOMP) and then called CANDECOMP/PARAFAC (CP) in[6]. For a history of the development of multi‐way models in the context of data analysis, see[7]. Since the 1990s, multi‐way analysis has known a growing success in chemistry and especially in chemometrics (see Bro’s thesis[8] and the book by Smilde et al.[9] for a description of various chemical applications of three‐way models, with a pedagogical presentation of these models and of various algorithms for estimating their parameters). At the same period, tensor tools were developed for signal processing applications, more particularly for solving the so‐called blind source separation (BSS) problem using cumulant tensors (see[10]‐[12] and De Lathauwer’s thesis[13] where the concept of high‐order singular value decomposition (HOSVD) is introduced, a tensor tool generalizing the standard matrix SVD to arrays of order higher than two). A recent overview of BSS approaches and applications can be found in the handbook co‐edited by Comon and Jutten[14].

Nowadays, (high‐order) tensors, also called multi‐way arrays in the data analysis community, play an important role in many fields of application for representing and analyzing multidimensional data, as in psychometrics, chemometrics, food industry, environmental sciences, signal/image processing, computer vision, neuroscience, information sciences, data mining, pattern recognition, among many others. Then, they are simply considered as multidimensional arrays of numbers, constituting a generalization of vectors and matrices that are first‐ and second‐order tensors, respectively, to orders higher than two. Tensor decompositions, also called tensor models, are very useful for analyzing multidimensional data under the form of signals, images, speech, music sequences, or texts and also for designing new systems as it is the case of wireless communication systems since the publication of the seminal paper by Sidiropoulos et al.[15]. Besides the references already cited, overviews of tensor tools, models, algorithms, and applications can be found in[16]‐[19].

Tensor models incorporating constraints (sparsity; non‐negativity; smoothness; symmetry; column orthonormality of factor matrices; Hankel, Toeplitz, and Vandermonde structured matrix factors; allocation constraints...) have been the object of intensive works, during the last years. Such constraints can be inherent to the problem under study or the result of a system design. An overview of constraints on components of tensor models most often encountered in multi‐way data analysis can be found in[7]. Incorporation of constraints in tensor models may facilitate physical interpretability of matrix factors. Moreover, imposing constraints may allow to relax uniqueness conditions and to develop specialized parameter estimation algorithms with improved performance both in terms of accuracy and computational cost, as it is the case of CP models with a column‐wise orthonormal factor matrix[20]. One can classify the constraints into three main categories: i) sparsity/non‐negativity, ii) structural, and iii) linear dependencies/mode interactions. It is worth noting that the three categories of constraints involve specific parameter estimation algorithms, the first two generally inducing an improvement of uniqueness property of the tensor decomposition, while the third category implies a reduction of uniqueness, named partial uniqueness. We briefly review the main results concerning the first two types of constraints, Section 1.3 of this paper being dedicated to the third category.

Sparse and non‐negative tensor models have recently been the subject of many works in various fields of applications like computer vision[21, 22], image compression[23], hyperspectral imaging[24], music genre classification[25] and audio source separation[26], multi‐channel EEG (electroencephalography) and network traffic analysis[27], fluorescence analysis[28], data denoising and image classification[29], among many others. Two non‐negative tensor models have been more particularly studied in the literature, the so‐called non‐negative tensor factorization (NTF), i.e., PARAFAC models with non‐negativity constraints on the matrix factors, and non‐negative Tucker decomposition (NTD), i.e., Tucker models with non‐negativity constraints on the core tensor and/or the matrix factors. The crucial importance of NTF/NTD for multi‐way data analysis applications results from the very large volume of real‐world data to be analyzed under constraints of sparseness and non‐negativity of factors to be estimated, when only non‐negative parameters are physically interpretable. Many NTF/NTD algorithms are now available. Most of them can be viewed as high‐order extensions of non‐negative matrix factorization (NMF) methods, in the sense that they are based on an alternating minimization of cost functions incorporating sparsity measures (also named distances or divergences) with application of NMF methods to matricized or vectorized forms of the tensor to be decomposed (see for instance[16, 23, 28, 30] for NTF and[29, 31] for NTD). An overview of NMF and NTF/NTD algorithms can be found in[16].

The second category of constraints concerns the case where the core tensor and/or some matrix factors of the tensor model have a special structure. For instance, we recently proposed a nonlinear CDMA scheme for multiuser SIMO communication systems that is based on a constrained block‐Tucker2 model whose core tensor, composed of the information symbols to be transmitted and their powers up to a certain degree, is characterized by matrix slices having a Vandermonde or a Hankel structure[32, 33]. We also developed Volterra‐PARAFAC models for nonlinear system modeling and identification. These models are obtained by expanding high‐order Volterra kernels, viewed as symmetric tensors, by means of symmetric or doubly symmetric PARAFAC decompositions[34, 35]. Block structured nonlinear systems like Wiener, Hammerstein, and parallel‐cascade Wiener systems can be identified from their associated Volterra kernels that admit symmetric PARAFAC decompositions with Toeplitz factors[36, 37]. Symmetric PARAFAC models with Hankel factors and symmetric block PARAFAC models with block Hankel factors are encountered for blind identification of multiple‐input multiple‐output (MIMO) linear channels using fourth‐order cumulant tensors, in the cases of memoryless and convolutive channels, respectively[38, 39]. In the presence of structural constraints, specific estimation algorithms can be derived as it is the case for symmetric CP decompositions[40], CP decompositions with Toeplitz factors (in[41], an iterative solution was proposed, whereas in[42], a non‐iterative algorithm was developed), Vandermonde factors[43], circulant factors[44], banded and/or structured matrix factors[45, 46], and also for Hankel and Vandermonde structured core tensors[33].

The rest of this paper is organized as follows: In Section 1.2, we present some tensor prerequisites with a particular emphasis on mode combination using Kronecker products of canonical vectors that makes easier the matricization operations, especially to derive matrix representations of tensor models. This Kronecker product‐based approach is also formulated in terms of an index notation, which provides an original and concise formalism for both matricizing tensors and writing tensor models. Then, we present the two most common tensor models, the so‐called Tucker and PARAFAC models, in a general framework, i.e., for N th‐order tensors. In Section 1.3, two families of constrained tensor models, the co‐called PARALIND/CONFAC and PARATUCK models, are described in a unified way, with a generalization to N th‐order tensors. New tensor models, called nested Tucker models and block PARALIND/CONFAC models, are also introduced. A link between PARATUCK models and constrained PARAFAC models is also established. In Section 1.4, uniqueness properties of PARATUCK models are deduced using this link. The paper is concluded in Section 2.

Notations and definitions. and denote the fields of real and complex numbers, respectively. Scalars, column vectors, matrices, and high‐order tensors are denoted by lowercase, boldface lowercase, boldface uppercase, and calligraphic uppercase letters, e.g., a, a, A, and, respectively. The vector A_i. (resp. A_.j) represents the i th row (resp. j th column) of A.

I _N ,${\mathbf{1}}_N^T$, and${\mathbf{e}}_n^{\left(N\right)}$ stand for the identity matrix of order N, the all‐ones row vector of dimensions 1×N, and the n th canonical vector of the Euclidean space${\mathbb{R}}^N$, respectively.

A^T, A^H, A^†, tr (A), and r _A denote the transpose, the conjugate (Hermitian) transpose, the Moore‐Penrose pseudo‐inverse, the trace, and the rank of A, respectively. D _i (A)=diag(A_i.) represents the diagonal matrix having the elements of the i th row of A on its diagonal. The operator bdiag(.) forms a block diagonal matrix from its matrix arguments, while the operator vec(.) transforms a matrix into a column vector by stacking the columns of its matrix argument one on top of the other one. In case of a tensor, the vec(.) operation is defined in (6).

The outer product (also called tensor product), and the matrix Kronecker, Khatri‐Rao (column‐wise Kronecker), and Hadamard (element‐wise) products are denoted by ∘, ⊗, ◇, and ⊙, respectively.

Let us consider the set$\mathbb{S}=\{n_1,\dots ,n_N\}$ obtained by permuting the elements of the set {1,…,N}. For${\mathbf{A}}^{\left(n\right)}\in {\mathbb{C}}^{I_n\times R_n}$ and${\mathbf{u}}^{\left(n\right)}\in {\mathbb{C}}^{I_n\times 1}$, n=1,…,N, we define

$$\begin{array}{ll}\underset{n\in \mathbb{S}}{\otimes }{\mathbf{A}}^{\left(n\right)}& ={\mathbf{A}}^{\left(n_1\right)}\otimes {\mathbf{A}}^{\left(n_2\right)}\otimes \cdots \otimes {\mathbf{A}}^{\left(n_N\right)}\in {\mathbb{C}}^{I_{n_1}\cdots I_{n_N}\times R_{n_1}\cdots R_{n_N}};\end{array}$$

(1)

$$\begin{array}{ll}\underset{n\in \mathbb{S}}{◇}{\mathbf{A}}^{\left(n\right)}& ={\mathbf{A}}^{\left(n_1\right)}◇{\mathbf{A}}^{\left(n_2\right)}◇\cdots ◇{\mathbf{A}}^{\left(n_N\right)}\in {\mathbb{C}}^{I_{n_1}\cdots I_{n_N}\times R},\\ \text{when}{R}_n=R,\forall n=1,\dots ,N;\\ \underset{n\in \mathbb{S}}{\odot }{\mathbf{A}}^{\left(n\right)}& ={\mathbf{A}}^{\left(n_1\right)}\odot {\mathbf{A}}^{\left(n_2\right)}\odot \cdots \odot {\mathbf{A}}^{\left(n_N\right)}\in {\mathbb{C}}^{I\times R},\\ \text{when}{I}_n=I,\text{and}{R}_n=R,\forall n=1,\dots ,N;\\ \underset{n\in \mathbb{S}}{\circ }{\mathbf{u}}^{\left(n\right)}& ={\mathbf{u}}^{\left(n_1\right)}\circ {\mathbf{u}}^{\left(n_2\right)}\circ \cdots \circ {\mathbf{u}}^{\left(n_N\right)}\in {\mathbb{C}}^{I_{n_1}\times \cdots \times I_{n_N}}.\end{array}$$

(2)

The outer product of N non‐zero vectors defines a rank‐one tensor of order N.

By convention, the order of dimensions is directly related to the order of variation of the associated indices. For instance, in (1) and (2), the product$I_{n_1}{I}_{n_2}\cdots I_{n_N}$ of dimensions means that n₁ is the index varying the most slowly while n _N is the index varying the most fastly in the Kronecker products computation.

For$\mathbb{S}=\{1,\dots ,N\}$, we have the following identities:

$$\begin{array}{ll}{\left(\underset{n\in \mathbb{S}}{\circ }{\mathbf{u}}^{\left(n\right)}\right)}_{i_1,\dots ,i_N}& ={\left(\stackrel{N}{\underset{n=1}{\circ }}{\mathbf{u}}^{\left(n\right)}\right)}_{i_1,\dots ,i_N}=\prod _{n=1}^N{u}_{i_n}^{\left(n\right)},\\ {\left(\underset{n\in \mathbb{S}}{\otimes }{\mathbf{u}}^{\left(n\right)}\right)}_i& ={\left(\stackrel{N}{\underset{n=1}{\otimes }}{\mathbf{u}}^{\left(n\right)}\right)}_i=\prod _{n=1}^N{u}_{i_n}^{\left(n\right)}\\ \text{with}i& =i_N+\sum _{n=1}^{N-1}(i_n-1)\prod _{j=n+1}^N{I}_j.\end{array}$$

(3)

In particular, for$\mathbf{u}\in {\mathbb{C}}^{I\times 1},\mathbf{v}\in {\mathbb{C}}^{J\times 1},\mathbf{w}\in {\mathbb{C}}^{K\times 1}$

$$\begin{array}{ll}\mathcal{X}& =\mathbf{u}\circ \mathbf{v}\circ \mathbf{w}\in {\mathbb{C}}^{I\times J\times K}\iff x_{\mathit{\text{ijk}}}=u_i{v}_j{w}_k,\\ \mathbf{x}& =\mathbf{u}\otimes \mathbf{v}\otimes \mathbf{w}\in {\mathbb{C}}^{\mathit{\text{IJK}}\times 1}\iff x_{k+(j-1)K+(i-1)\mathit{\text{JK}}}=u_i{v}_j{w}_k.\end{array}$$

Some useful matrix formulae are recalled in Appendix 1.

1.2 Tensor prerequisites

In this paper, a tensor is simply viewed as a multidimensional array of measurements. Depending that these measurements are real‐ or complex‐valued, we have a real‐ or complex‐valued tensor, respectively. The order N of a tensor refers to the number of indices that characterize its elements$x_{i_1,\dots ,i_N}$, each index i _n (i _n =1,…,I _N ,for n=1,…,N) being associated with a dimension, also called a way, or a mode, and I _n denoting the mode‐n dimension.

An N th‐order complex‐valued tensor$\mathcal{X}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$, also called an N‐way array, of dimensions I₁×⋯×I _N , can be written as

$$\mathcal{X}=\sum _{i_1=1}^{I_1}\cdots \sum _{i_N=1}^{I_N}{x}_{i_1,\dots ,i_N}\stackrel{N}{\underset{n=1}{\circ }}{\mathbf{e}}_{i_n}^{\left(I_n\right)}.$$

(4)

The coefficients$x_{i_1,\dots ,i_N}$ represent the coordinates of in the canonical basis$\left\{\stackrel{N}{\underset{n=1}{\circ }}{\mathbf{e}}_{i_n}^{\left(I_n\right)},i_n=1,\dots ,I_n;n=1,\dots ,N\right\}$ of the space${\mathbb{C}}^{I_1\times \cdots \times I_N}$.

The identity tensor of order N and dimensions I×⋯×I, denoted by${\mathcal{I}}_{N,I}$ or simply, is a diagonal hypercubic tensor whose elements${\delta }_{i_1,\dots ,i_N}$ are defined by means of the generalized Kronecker delta, i.e.,${\delta }_{i_1,\dots ,i_N}=\left\{\begin{array}{c}1\text{if}{i}_1=\cdots =i_N\\ 0\text{otherwise}\end{array}\right.$, and I _n =I,∀n=1,…,N. It can be written as

$${\mathcal{I}}_{N,I}=\sum _{i=1}^I\underset{N\text{terms}}{\underbrace{{\mathbf{e}}_i^{\left(I\right)}\circ \cdots \circ {\mathbf{e}}_i^{\left(I\right)}}}.$$

Different reduced order tensors can be obtained by slicing the tensor$\mathcal{X}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$ along one mode or p modes, i.e., by fixing one index i _n or a set of p indices$\{i_{n_1},\dots ,i_{n_p}\}$, which gives a tensor of order N-1 or N-p, respectively. For instance, by slicing along its mode‐n, we get the$i_n^{\text{th}}$ mode‐n slice of, denoted by${\mathcal{X}}_{\dots i_n\dots }$, that can be written as

$$\begin{array}{ll}{\mathcal{X}}_{\dots i_n\dots }& =\sum _{i_1=1}^{I_1}\cdots \sum _{i_{n-1}=1}^{I_{n-1}}\sum _{i_{n+1}=1}^{I_{n+1}}\cdots \sum _{i_N=1}^{I_N}{x}_{i_1,\dots ,i_n,\dots ,i_N}\\ \times {\mathbf{e}}_{i_{n+1}}^{\left(I_{n+1}\right)}\circ \cdots \circ {\mathbf{e}}_{i_N}^{\left(I_N\right)}\circ {\mathbf{e}}_{i_1}^{\left(I_1\right)}\circ \cdots \\ \cdots \circ {\mathbf{e}}_{i_{n-1}}^{\left(I_{n-1}\right)}\in {\mathbb{C}}^{I_{n+1}\times \cdots \times I_N\times I_1\times \cdots \times I_{n-1}}.\end{array}$$

For instance, by slicing the third‐order tensor$\mathcal{X}\in {\mathbb{C}}^{I\times J\times K}$ along each mode, we get three types of matrix slices, respectively called horizontal, lateral, and frontal slices:

$$\begin{array}{l}{\mathbf{X}}_{\mathrm{i..}}\in {\mathbb{C}}^{J\times K},{\mathbf{X}}_{\mathrm{.j.}}\in {\mathbb{C}}^{K\times I}\text{and}{\mathbf{X}}_{\mathrm{..k}}\in {\mathbb{C}}^{I\times J},\\ \text{with}i=1,\dots ,I;j=1,\dots ,J;k=1,\dots ,\mathrm{K.}\end{array}$$

1.2.1 Tensor Hadamard product

Consider$\mathcal{A}\in {\mathbb{C}}^{R_1\times \cdots \times R_N\times I_1\times \cdots \times I_{P_1}}$ and$\mathcal{B}\in {\mathbb{C}}^{R_1\times \cdots \times R_N\times I_{P_1+1}\times \cdots \times I_P}$, where$\{i_1,\dots ,i_{P_1}\}$ and$\{i_{P_1+1},\dots ,i_P\}$ are two disjoint ordered subsets of the set of indices {i₁,…,i _P } and$\mathbb{R}=\{r_1,\dots ,r_N\}$.

We define the Hadamard product of with along their common modes, as the tensor$\mathcal{C}\in {\mathbb{C}}^{R_1\times \cdots \times R_N\times I_1\times \cdots \times I_P}$ such that

$$\mathcal{C}=\mathcal{A}\underset{\mathbb{R}}{\odot }\mathcal{B}\iff c_{r_1,\dots ,r_N,i_1,\dots ,i_P}=a_{r_1,\dots ,r_N,i_1,\dots ,i_{P_1}}{b}_{r_1,\dots ,r_N,i_{P_1+1},\dots ,i_P}$$

For instance, given two third‐order tensors$\mathcal{A}\in {\mathbb{C}}^{R_1\times R_2\times I_1}$ and$\mathcal{B}\in {\mathbb{C}}^{R_1\times R_2\times I_2}$, the Hadamard product$\mathcal{A}\underset{\{r_1,r_2\}}{\odot }\mathcal{B}$ gives a fourth‐order tensor$\mathcal{C}\in {\mathbb{C}}^{R_1\times R_2\times I_1\times I_2}$ such that

$$c_{r_1,r_2,i_1,i_2}=a_{r_1,r_2,i_1}{b}_{r_1,r_2,i_2}.$$

Such a tensor Hadamard product can be calculated by means of the matrix Hadamard product of matrix unfoldings of extended tensors, as defined in (21) and (22) (see also (94) to (96) in Appendix 2). For the example above, we have

$${\mathbf{C}}_{R_1{R}_2\times I_1{I}_2}={\mathbf{A}}_{R_1{R}_2\times I_1}\left({\mathbf{I}}_{I_1}\otimes {\mathbf{1}}_{I_2}^T\right)\odot {\mathbf{B}}_{R_1{R}_2\times I_2}\left({\mathbf{1}}_{I_1}^T\otimes {\mathbf{I}}_{I_2}\right)$$

1.2.1.0 Example

For${\mathbf{A}}_{R\times I_1}=\left[\begin{array}{cc}{a}_1& a_2\\ a_3& a_4\end{array}\right]$,${\mathbf{B}}_{R\times I_2}=\left[\begin{array}{cc}{b}_1& b_2\\ b_3& b_4\end{array}\right]$, and the tensor such as$c_{r,i_1,i_2}=a_{r,i_1}{b}_{r,i_2}$, a mode‐1 flat matrix unfolding of is given by

$$\begin{array}{ll}{\mathbf{C}}_{R\times I_1{I}_2}& ={\mathbf{A}}_{R\times I_1}\left({\mathbf{I}}_2\otimes {\mathbf{1}}_2^T\right)\odot {\mathbf{B}}_{R\times I_2}\left({\mathbf{1}}_2^T\otimes {\mathbf{I}}_2\right)\\ =\left[\begin{array}{cccc}{a}_1& a_1& a_2& a_2\\ a_3& a_3& a_4& a_4\end{array}\right]\odot \left[\begin{array}{cccc}{b}_1& b_2& b_1& b_2\\ b_3& b_4& b_3& b_4\end{array}\right]\\ =\left[\begin{array}{cccc}{a}_1{b}_1& a_1{b}_2& a_2{b}_1& a_2{b}_2\\ a_3{b}_3& a_3{b}_4& a_4{b}_3& a_4{b}_4\end{array}\right]\end{array}$$

1.2.2 Mode combination

Different contraction operations can be defined depending on the way according to which the modes are combined. Let us partition the set {1,…,N} in N₁ ordered subsets${\mathbb{S}}_{n_1}$, constituted of p(n₁) elements with$\sum _{n_1=1}^{N_1}p\left(n_1\right)=N$. Each subset${\mathbb{S}}_{n_1}$ is associated with a combined mode of dimension$J_{n_1}=\underset{n\in {\mathbb{S}}_{n_1}}{\prod I_n}$. These mode combinations allow to rewrite the N th‐order tensor$\mathcal{X}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$ under the form of an$N_1^{\text{th}}$‐order tensor$\mathcal{Y}\in {\mathcal{C}}^{J_1\times \cdots \times J_{N_1}}$ as follows

$$\mathcal{Y}=\sum _{j_1=1}^{J_1}\cdots \sum _{j_{N_1}=1}^{J_{N_1}}{x}_{j_1,\dots ,j_{N_1}}\stackrel{N_1}{\underset{n_1=1}{\circ }}{\mathbf{e}}_{j_{n_1}}^{\left(J_{n_1}\right)}\text{with}{\mathbf{e}}_{j_{n_1}}^{\left(J_{n_1}\right)}=\underset{n\in {\mathbb{S}}_{n_1}}{\otimes }{\mathbf{e}}_{i_n}^{\left(I_n\right)}.$$

(5)

Two particular mode combinations corresponding to the vectorization and matricization operations are now detailed.

1.2.3 Vectorization

The vectorization of$\mathcal{X}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$ is associated with a combination of the N modes into a unique mode of dimension$J=\prod _{n=1}^N{I}_n$, which amounts to replace the outer product in (4) by the Kronecker product

$$\text{vec}\left(\mathcal{X}\right)=\sum _{i_1=1}^{I_1}\cdots \sum _{i_N=1}^{I_N}{x}_{i_1,\dots ,i_N}\stackrel{N}{\underset{n=1}{\otimes }}{\mathbf{e}}_{i_n}^{\left(I_n\right)}\in {\mathbb{C}}^{I_1\cdots I_N\times 1}$$

(6)

the element$x_{i_1,\dots ,i_N}$ of being the i th entry of$\text{vec}\left(\mathcal{X}\right)$ with i defined as in (3).

The vectorization can also be carried out after a permutation of indices π(i _n ),n=1,…,N.

1.2.4 Matricization or unfolding

There are different ways of matricizing the tensor according to the partitioning of the set {1,…,N} into two ordered subsets${\mathbb{S}}_1$ and${\mathbb{S}}_2$, constituted of p and N-p indices, respectively. A general formula for the matricization, for$p\in \left[1,N-1\right]$, is

$${\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}=\sum _{i_1=1}^{I_1}\cdots \sum _{i_N=1}^{I_N}{x}_{i_1,\dots ,i_N}\left(\underset{n\in {\mathbb{S}}_1}{\otimes }{\mathbf{e}}_{i_n}^{\left(I_n\right)}\right){\left(\underset{n\in {\mathbb{S}}_2}{\otimes }{\mathbf{e}}_{i_n}^{\left(I_n\right)}\right)}^T\in {\mathbb{C}}^{J_1\times J_2}$$

(7)

with$J_{n_1}=\underset{n\in {\mathbb{S}}_{n_1}}{\prod I_n}$, for n₁=1 and 2. From (7), we can deduce the following expression of the element$x_{i_1,\dots ,i_N}$ in terms of the matrix unfolding${\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}$

$$x_{i_1,\dots ,i_N}={\left(\underset{n\in {\mathbb{S}}_1}{\otimes }{\mathbf{e}}_{i_n}^{\left(I_n\right)}\right)}^T{\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}\left(\underset{n\in {\mathbb{S}}_2}{\otimes }{\mathbf{e}}_{i_n}^{\left(I_n\right)}\right).$$

(8)

1.2.5 Particular case: mode‐ n matrix unfoldings X _n

A flat mode‐n matrix unfolding of the tensor corresponds to an unfolding under the form${\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}$ with${\mathbb{S}}_1=\left\{n\right\}$ and${\mathbb{S}}_2=\{n+1,\dots ,N,1,\dots ,n-1\}$, which gives

$$\begin{array}{l}{\mathbf{X}}_{I_n\times I_{n+1}\cdots I_N{I}_1\cdots I_{n-1}}={\mathbf{X}}_n\\ =\sum _{i_1=1}^{I_1}\cdots \sum _{i_N=1}^{I_N}{x}_{i_1,\dots ,i_N}{\mathbf{e}}_{i_n}^{\left(I_n\right)}{\left(\underset{n\in {\mathbb{S}}_2}{\otimes }{\mathbf{e}}_{i_n}^{\left(I_n\right)}\right)}^T\in {\mathbb{C}}^{I_n\times I_{n+1}\cdots I_N{I}_1\cdots I_{n-1}}.\end{array}$$

(9)

We can also define a tall mode‐n matrix unfolding of, by choosing${\mathbb{S}}_1=\{n+1,\dots ,N,1,\dots ,n-1\}$ and${\mathbb{S}}_2=\left\{n\right\}$. Then, we have${\mathbf{X}}_{I_{n+1}\cdots I_N{I}_1\cdots I_{n-1}\times I_n}={\mathbf{X}}_n^T\in {\mathbb{C}}^{I_{n+1}\cdots I_N{I}_1\cdots I_{n-1}\times I_n}$.

The column vectors of a flat mode‐n matrix unfolding X _n are the mode‐n vectors of, and the rank of X _n , i.e., the dimension of the mode‐n linear space spanned by the mode‐n vectors, is called mode‐n rank of, denoted by${\text{rank}}_n\left(\mathcal{X}\right)$.

In the case of a third‐order tensor$\mathcal{X}\in {\mathbb{C}}^{I\times J\times K}$, there are six different flat unfoldings, denoted X_{I×J K}, X_{I×K J}, X_{J×K I}, X_{J×I K}, X_{K×I J}, and X_{K×J I}. For instance, we have

$${\mathbf{X}}_{I\times \mathit{\text{JK}}}={\mathbf{X}}_{\left\{1\right\};\{2,3\}}=\sum _{i=1}^I\sum _{j=1}^J\sum _{k=1}^K{x}_{i,j,k}{\mathbf{e}}_i^{\left(I\right)}{\left({\mathbf{e}}_j^{\left(J\right)}\otimes {\mathbf{e}}_k^{\left(K\right)}\right)}^T.$$

(10)

Using the properties (84), (85), and (87) of the Kronecker product gives

$$\begin{array}{ll}{\mathbf{X}}_{I\times \mathit{\text{JK}}}& =\sum _{j=1}^J{\left({\mathbf{e}}_j^{\left(J\right)}\right)}^T\otimes \sum _{i=1}^I\sum _{k=1}^K{x}_{i,j,k}{\mathbf{e}}_i^{\left(I\right)}{\left({\mathbf{e}}_k^{\left(K\right)}\right)}^T\\ =\sum _{j=1}^J{\left({\mathbf{e}}_j^{\left(J\right)}\right)}^T\otimes {\left({\mathbf{X}}_{\mathrm{.j.}}\right)}^T=\left[\begin{array}{ccc}{\mathbf{X}}_{.1.}^T& \cdots & {\mathbf{X}}_{\mathrm{.J.}}^T\end{array}\right]\in {\mathbb{C}}^{I\times \mathit{\text{JK}}}.\end{array}$$

Similarly, there are six tall matrix unfoldings, denoted X_{J K×I}, X_{K J×I}, X_{K I×J}, X_{I K×J}, X_{I J×K}, X_{J I×K}, like for instance

$${\mathbf{X}}_{\mathit{\text{JK}}\times I}=\sum _{i=1}^I\sum _{j=1}^J\sum _{k=1}^K{x}_{i,j,k}\left({\mathbf{e}}_j^{\left(J\right)}\otimes {\mathbf{e}}_k^{\left(K\right)}\right){{\mathbf{e}}_i^{\left(I\right)}}^T={\mathbf{X}}_{I\times \mathit{\text{JK}}}^T\in {\mathbb{C}}^{\mathit{\text{JK}}\times I}.$$

(11)

Applying (8) to (10) gives

$$x_{i,j,k}={\left({\mathbf{e}}_i^{\left(I\right)}\right)}^T{\mathbf{X}}_{I\times \mathit{\text{JK}}}\left({\mathbf{e}}_j^{\left(J\right)}\otimes {\mathbf{e}}_k^{\left(K\right)}\right)={\left[{\mathbf{X}}_{I\times \mathit{\text{JK}}}\right]}_{i,(j-1)K+k}.$$

1.2.6 Mode‐ n product of a tensor with a matrix or a vector

The mode‐n product of$\mathcal{X}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$ with$\mathbf{A}\in {\mathbb{C}}^{J_n\times I_n}$ along the n th mode, denoted by$\mathcal{X}{\times }_n\mathbf{A}$, gives the tensor of order N and dimensions I₁×⋯×I_n-1×J _n ×I_n+1×⋯×I _N , such as[47]

$$y_{i_1,\dots ,i_{n-1},j_n,i_{n+1},\dots ,i_N}=\sum _{i_n=1}^{I_n}{a}_{j_n,i_n}{x}_{i_1,\dots ,i_{n-1},i_n,i_{n+1},\dots ,i_N}$$

(12)

which can be expressed in terms of mode‐n matrix unfoldings of and

$${\mathbf{Y}}_n={\mathbf{AX}}_n.$$

This operation can be interpreted as the linear map from the mode‐n space of to the mode‐n space of, associated with the matrix A.

The mode‐n product of$\mathcal{X}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$ with the row vector${\mathbf{u}}^T\in {\mathbb{C}}^{1\times I_n}$ along the n th mode, denoted by$\mathcal{X}{\times }_n{\mathbf{u}}^T$, gives a tensor of order N-1 and dimensions I₁×⋯×I_n-1×I_n+1×⋯×I _N , such as

$$y_{i_1,\dots ,i_{n-1},i_{n+1},\dots ,i_N}=\sum _{i_n=1}^{I_n}{u}_{i_n}{x}_{i_1,\dots ,i_{n-1},i_n,i_{n+1},\dots ,i_N}$$

that can be written in vectorized form as${\text{vec}}^T\left(\mathcal{Y}\right)={\mathbf{u}}^T{\mathbf{X}}_n\in {\mathbb{C}}^{1\times I_{n+1}\cdots I_N{I}_1\cdots I_{n-1}}$.

When multiplying a N th‐order tensor by row vectors along p different modes, we get a tensor of order N-p. For instance, for a third‐order tensor$\mathcal{X}\in {\mathbb{C}}^{I\times J\times K}$, we have

$${\mathbf{x}}_{\mathrm{ij.}}=\mathcal{X}{\times }_1{{\mathbf{e}}_i^{\left(I\right)}}^T{\times }_2{{\mathbf{e}}_j^{\left(J\right)}}^T,x_{\mathit{\text{ijk}}}=\mathcal{X}{\times }_1{{\mathbf{e}}_i^{\left(I\right)}}^T{\times }_2{{\mathbf{e}}_j^{\left(J\right)}}^T{\times }_3{{\mathbf{e}}_k^{\left(K\right)}}^T.$$

Considering an ordered subset$\mathbb{S}=\{m_1,\dots ,m_P\}$ of the set {1,…,N}, a series of mode‐ m _p products of$\mathcal{X}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$ with${\mathbf{A}}^{\left(m_p\right)}\in {\mathbb{C}}^{J_{m_p}\times I_{m_p}}$, p∈{1,…,P}, P≤N, will be concisely noted as

$$\mathcal{X}{\times }_{m_1}{\mathbf{A}}^{\left(m_1\right)}\cdots {\times }_{m_P}{\mathbf{A}}^{\left(m_P\right)}=\mathcal{X}{\times }_{m=m_1}^{m_P}{\mathbf{A}}^{\left(m\right)}.$$

1.2.6.0 Properties

• For any permutation π(.) of P distinct indices m _p ∈{1,…,N} such as q _p =π(m _p ), p∈{1,…,P}, with P≤N, we have

  $$\mathcal{X}{\times }_{q=q_1}^{q_P}{\mathbf{A}}^{\left(q\right)}=\mathcal{X}{\times }_{m=m_1}^{m_P}{\mathbf{A}}^{\left(m\right)}$$

  which means that the order of the mode‐ m _p products is irrelevant when the indices m _p are all distinct.

• For two products of$\mathcal{X}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$ along the same mode‐n, with$\mathbf{A}\in {\mathbb{C}}^{J_n\times I_n}$ and$\mathbf{B}\in {\mathbb{C}}^{K_n\times J_n}$, we have[13]

  $$\begin{array}{l}\mathcal{Y}=\mathcal{X}{\times }_n\mathbf{A}{\times }_n\mathbf{B}\\ =\mathcal{X}{\times }_n\left(\mathbf{B}\mathbf{A}\right)\in {\mathbb{C}}^{I_1\times \cdots \times I_{n-1}\times K_n\times I_{n+1}\times \cdots \times I_N}.\end{array}$$

  (13)

1.2.7 Kronecker product‐based approach using index notation

In this subsection, we propose to reformulate our Kronecker product‐based approach for tensor matricization in terms of an index notation introduced in[48]. Using this index notation, for$\mathbf{u}\in {\mathbb{C}}^{I\times 1}$,${\mathbf{v}}^T\in {\mathbb{C}}^{1\times J}$, and$\mathbf{X}\in {\mathbb{C}}^{I\times J}$, we can write

$$\begin{array}{ll}\mathbf{u}& =\sum _{i=1}^I{u}_i{\mathbf{e}}_i^{\left(I\right)}=u_i{\mathbf{e}}_i\\ {\mathbf{v}}^T& =\sum _{j=1}^J{v}_j{\left({\mathbf{e}}_j^{\left(J\right)}\right)}^T=v_j{\mathbf{e}}^j\\ \mathbf{X}& =\sum _{i=1}^I\sum _{j=1}^j{x}_{\mathit{\text{ij}}}\left({\mathbf{e}}_i^{\left(I\right)}\otimes {\left({\mathbf{e}}_j^{\left(J\right)}\right)}^T\right)=x_{\mathit{\text{ij}}}{\mathbf{e}}_i^j\\ {\mathbf{X}}^T& =x_{\mathit{\text{ij}}}{\mathbf{e}}_j^i\\ \text{vec}\left(\mathbf{X}\right)& =x_{\mathit{\text{ij}}}{\mathbf{e}}_{\mathit{\text{ji}}}\end{array}$$

As with Einstein summation convention, the index notation allows to drop summation signs. If an index i∈[ 1,I] is repeated in an expression (or more generally in a term of an equation), it means that this expression (or this term) must be summed over that index from 1 to I. However, it is worth noting the two differences between the index notation used in this paper and Einstein summation convention: (i) each index can be repeated more than twice in any expression and (i i) the index notation can be used with ordered sets of indices. We have to notice that the index notation can be interpreted in terms of two separate combinations of indices, one associated with the column (superscript) indices and the other one with the row (subscript) indices, with the following rules:

• the ordering of the column indices is independent of that of the row indices;

• the ordering of the column and row indices cannot be changed.

Considering the set$\mathbb{S}=\{n_1,\dots ,n_N\}$ obtained by permuting the elements of {1,…,N} and defining the ordered set of indices$\mathbb{I}=\{i_{n_1},\dots ,i_{n_N}\}$ associated with, we denote by${\mathbf{e}}_{\mathbb{I}}$ and${\mathbf{e}}^{\mathbb{I}}$ the Kronecker products$\underset{n\in \mathbb{S}}{\otimes }{\mathbf{e}}_{i_n}$ and$\underset{n\in \mathbb{S}}{\otimes }{\mathbf{e}}_{i_n}^T$, respectively. So, we have

$$\underset{n\in \mathbb{S}}{◇}{\mathbf{u}}^{\left(n\right)}=\left(\prod _{n\in \mathbb{S}}{u}_{i_n}^{\left(n\right)}\right){\mathbf{e}}_{\mathbb{I}}$$

(14)

Partitioning two ordered sets of indices and into two subsets (${\mathbb{I}}_1,{\mathbb{I}}_2$) and (${\mathbb{J}}_1,{\mathbb{J}}_2$), respectively, the rules enounced previously imply the following identities:

$$\begin{array}{ll}{\mathbf{e}}_{\mathbb{I}}^{\mathbb{J}}& ={\mathbf{e}}_{\mathbb{I}}\otimes {\mathbf{e}}^{\mathbb{J}}={\mathbf{e}}^{\mathbb{J}}\otimes {\mathbf{e}}_{\mathbb{I}}={\mathbf{e}}_{{\mathbb{I}}_1{\mathbb{I}}_2}^{{\mathbb{J}}_1{\mathbb{J}}_2}\\ ={\mathbf{e}}_{{\mathbb{I}}_1}\otimes {\mathbf{e}}_{{\mathbb{I}}_2}\otimes {\mathbf{e}}^{{\mathbb{J}}_1}\otimes {\mathbf{e}}^{{\mathbb{J}}_2}={\mathbf{e}}_{{\mathbb{I}}_1}\otimes {\mathbf{e}}^{{\mathbb{J}}_1}\otimes {\mathbf{e}}_{{\mathbb{I}}_2}\otimes {\mathbf{e}}^{{\mathbb{J}}_2}\\ ={\mathbf{e}}_{{\mathbb{I}}_1}\otimes {\mathbf{e}}^{{\mathbb{J}}_1}\otimes {\mathbf{e}}^{{\mathbb{J}}_2}\otimes {\mathbf{e}}_{{\mathbb{I}}_2}\\ ={\mathbf{e}}^{{\mathbb{J}}_1}\otimes {\mathbf{e}}^{{\mathbb{J}}_2}\otimes {\mathbf{e}}_{{\mathbb{I}}_1}\otimes {\mathbf{e}}_{{\mathbb{I}}_2}={\mathbf{e}}^{{\mathbb{J}}_1}\otimes {\mathbf{e}}_{{\mathbb{I}}_1}\otimes {\mathbf{e}}^{{\mathbb{J}}_2}\otimes {\mathbf{e}}_{{\mathbb{I}}_2}\\ ={\mathbf{e}}^{{\mathbb{J}}_1}\otimes {\mathbf{e}}_{{\mathbb{I}}_1}\otimes {\mathbf{e}}_{{\mathbb{I}}_2}\otimes {\mathbf{e}}^{{\mathbb{J}}_2}\end{array}$$

These identities directly result from the property that the Kronecker product of a column vector with a row vector is independent of the order of the vectors (u⊗v^T=v^T⊗u), which implies that in a sequence of Kronecker products of column and row vectors, a column vector can be permuted with a row vector without altering the final result, if the proper ordering of the column vectors and of the row vectors is not changed in the sequence (${\mathbf{u}}_1\otimes {\mathbf{u}}_2\otimes {\mathbf{v}}^T={\mathbf{u}}_1\otimes {\mathbf{v}}^T\otimes {\mathbf{u}}_2={\mathbf{v}}^T\otimes {\mathbf{u}}_1\otimes {\mathbf{u}}_2\ne {\mathbf{v}}^T\otimes {\mathbf{u}}_2\otimes {\mathbf{u}}_1$ if u₁≠u₂).

Using the index notation, the horizontal, lateral, and frontal slices of a third‐order tensor$\mathcal{X}\in {\mathbb{C}}^{I\times J\times K}$ can be written as

$${\mathbf{X}}_{\mathrm{i..}}=x_{\mathit{\text{ijk}}}{\mathbf{e}}_j^k;{\mathbf{X}}_{\mathrm{.j.}}=x_{\mathit{\text{ijk}}}{\mathbf{e}}_k^i;{\mathbf{X}}_{\mathrm{..k}}=x_{\mathit{\text{ijk}}}{\mathbf{e}}_i^j.$$

The Kronecker products of vectors ($\mathbf{u}\in {\mathbb{C}}^{I\times 1},\mathbf{v}\in {\mathbb{C}}^{J\times 1}$) and matrices ($\mathbf{A}\in {\mathbb{C}}^{I\times J},\mathbf{B}\in {\mathbb{C}}^{K\times L}$) can be concisely written as

$$\begin{array}{ll}\mathbf{u}\otimes \mathbf{v}& =\left(u_i{\mathbf{e}}_i\right)\otimes \left(v_j{\mathbf{e}}_j\right)=u_i{v}_j{\mathbf{e}}_{\mathit{\text{ij}}}\\ {\mathbf{u}}^T\otimes {\mathbf{v}}^T& =u_i{v}_j{\mathbf{e}}^{\mathit{\text{ij}}}\\ \mathbf{u}\otimes {\mathbf{v}}^T& =u_i{v}_j{\mathbf{e}}_i^j\\ \mathbf{A}\otimes \mathbf{B}& =\left(a_{\mathit{\text{ij}}}{\mathbf{e}}_i^j\right)\otimes \left(b_{\mathit{\text{kl}}}{\mathbf{e}}_k^l\right)=a_{\mathit{\text{ij}}}{b}_{\mathit{\text{kl}}}{\mathbf{e}}_{\mathit{\text{ik}}}^{\mathit{\text{jl}}}\\ {\mathbf{A}}^T\otimes {\mathbf{B}}^T& =a_{\mathit{\text{ij}}}{b}_{\mathit{\text{kl}}}{\mathbf{e}}_{\mathit{\text{jl}}}^{\mathit{\text{ik}}}\end{array}$$

For$\mathbf{U}=\left[{\mathbf{u}}^{\left(1\right)}\cdots {\mathbf{u}}^{\left(N\right)}\right]\in {\mathbb{C}}^{I\times N}$ and$\mathbf{V}=\left[{\mathbf{v}}_1\cdots {\mathbf{v}}_N\right]\in {\mathbb{C}}^{J\times N}$, we have

$$\mathbf{U}{\mathbf{V}}^T=\sum _{n=1}^N{\mathbf{u}}^{\left(n\right)}{\left({\mathbf{v}}^{\left(n\right)}\right)}^T=u_i^{\left(n\right)}{v}_j^{\left(n\right)}{\mathbf{e}}_i^j$$

(15)

where the summation over n is to be done after the matricization u⁽ⁿ⁾(v⁽ⁿ⁾)^T.

Using the index notation, the Khatri‐Rao product can be written as follows:

$$\begin{array}{cc}\mathbf{A}◇\mathbf{B}& =a_{\mathit{\text{ik}}}{b}_{\mathit{\text{jk}}}{\mathbf{e}}_{\mathit{\text{ij}}}^k\\ {(\mathbf{A}◇\mathbf{B})}^T& =a_{\mathit{\text{ik}}}{b}_{\mathit{\text{jk}}}{\mathbf{e}}_k^{\mathit{\text{ij}}}\end{array}$$

(16)

The Kronecker and Khatri‐Rao products defined in (1) and (2), with$a_{i_n,r_n}^{\left(n\right)}$ as entry of A⁽ⁿ⁾, can then be defined as

$$\begin{array}{ll}\underset{n\in \mathbb{S}}{\otimes }{\mathbf{A}}^{\left(n\right)}& =\left(\prod _{n\in \mathbb{S}}{a}_{i_n,r_n}^{\left(n\right)}\right){\mathbf{e}}_{i_{n_1},\dots ,i_{n_N}}^{r_{n_1},\dots ,r_{n_N}}=\left(\prod _{n\in \mathbb{S}}{a}_{i_n,r_n}^{\left(n\right)}\right){\mathbf{e}}_{\mathbb{I}}^{\mathbb{R}}\end{array}$$

(17)

$$\begin{array}{ll}\underset{n\in \mathbb{S}}{◇}{\mathbf{A}}^{\left(n\right)}& =\left(\prod _{n\in \mathbb{S}}{a}_{i_n,r}^{\left(n\right)}\right){\mathbf{e}}_{i_{n_1},\dots ,i_{n_N}}^r=\left(\prod _{n\in \mathbb{S}}{a}_{i_n,r}^{\left(n\right)}\right){\mathbf{e}}_{\mathbb{I}}^r\end{array}$$

(18)

where$\mathbb{R}=\{r_{n_1},\dots ,r_{n_N}\}$.

Applying these results, the unfoldings (7), (10), and (11) and the formula (8) can be rewritten respectively as

$$\begin{array}{ll}{\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}& =x_{i_1,\dots ,i_N}{\mathbf{e}}_{{\mathbb{I}}_1}^{{\mathbb{I}}_2}\end{array}$$

(19)

$$\begin{array}{ll}{\mathbf{X}}_{I\times \mathit{\text{JK}}}& =x_{i,j,k}{\mathbf{e}}_i^{\mathit{\text{jk}}}\\ {\mathbf{X}}_{\mathit{\text{JK}}\times I}& =x_{i,j,k}{\mathbf{e}}_{\mathit{\text{jk}}}^i\\ x_{i_1,\dots ,i_N}& ={\mathbf{e}}^{{\mathbb{I}}_1}{\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}{\mathbf{e}}_{{\mathbb{I}}_2}\end{array}$$

(20)

where${\mathbb{I}}_1$ and${\mathbb{I}}_2$ represent the sets of indices i _n associated with the sets${\mathbb{S}}_1$ and${\mathbb{S}}_2$ of index n, respectively.

We can also use the index notation for deriving matrix unfoldings of tensor extensions of a matrix$\mathbf{B}\in {\mathbb{C}}^{I\times J}$. For instance, if we define the tensor$\mathcal{A}\in {\mathbb{C}}^{I\times J\times K}$ such as a_i,j,k=b_i,j for k=1,…,K, mode‐1 flat unfoldings of are given by

$$\begin{array}{ll}{\mathbf{A}}_{I\times \mathit{\text{JK}}}& =\sum _{i,j,k}{a}_{i,j,k}{\mathbf{e}}_i\otimes {\mathbf{e}}^j\otimes {\mathbf{e}}^k=a_{i,j,k}{\mathbf{e}}_i^{\mathit{\text{jk}}}\\ =\sum _{i,j}{b}_{i,j}{\mathbf{e}}_i^j\otimes \sum _{k=1}^K{\mathbf{e}}^k=\mathbf{B}\otimes {\mathbf{1}}_K^T=\mathbf{B}\left({\mathbf{I}}_J\otimes {\mathbf{1}}_K^T\right)\end{array}$$

(21)

$$\begin{array}{ll}{\mathbf{A}}_{I\times \mathit{\text{KJ}}}& =a_{i,j,k}{\mathbf{e}}_i^{\mathit{\text{kj}}}=\sum _{k=1}^K{\mathbf{e}}^k\otimes b_{i,j}{\mathbf{e}}_i^j\\ ={\mathbf{1}}_K^T\otimes \mathbf{B}=\mathbf{B}\left({\mathbf{1}}_K^T\otimes {\mathbf{I}}_J\right)\end{array}$$

(22)

These two formulae will be used later for establishing the link between PARATUCK‐(2,4) models and constrained PARAFAC‐4 models.

1.2.8 Basic tensor models

We now present the two most common tensor models, i.e., the Tucker[3] and PARAFAC[4] models. In[7], these models are introduced in a constructive way, in the context of a three‐way data analysis. The Tucker models are presented as extensions of the matrix singular value decomposition (SVD) to three‐way arrays, which gave rise to the generalization as HOSVD[13, 49], whereas the PARAFAC model is introduced by emphasizing Cattell’s principle of parallel proportional profiles[50] that underlies this model, so explaining the acronym PARAFAC. In the following, we adopt a more general presentation for multi‐way arrays, i.e., tensors of any order N.

1.2.8.0 Tucker models

For a N th‐order tensor$\mathcal{X}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$, a Tucker model is defined in an element‐wise form as

$$x_{i_1,\dots ,i_N}=\sum _{r_1=1}^{R_1}\cdots \sum _{r_N=1}^{R_N}{g}_{r_1,\dots ,r_N}\prod _{n=1}^N{a}_{i_n,r_n}^{\left(n\right)}$$

(23)

with i _n =1,…,I _n for n=1,…,N, where$g_{r_1,\dots ,r_N}$ is an element of the core tensor$\mathcal{G}\in {\mathbb{C}}^{R_1\times \cdots \times R_N}$ and$a_{i_n,r_n}^{\left(n\right)}$ is an element of the matrix factor${\mathbf{A}}^{\left(n\right)}\in {\mathbb{C}}^{I_n\times R_n}$.

Using the index notation and defining the set of indices$\mathbb{R}=\{r_1,\dots ,r_N\}$, the Tucker model can also be written simply as

$$x_{i_1,\dots ,i_N}=g_{r_1,\dots ,r_N}\prod _{\mathbb{R}}{a}_{i_n,r_n}^{\left(n\right)}$$

(24)

Taking the definition (4) into account and noting that$\sum _{i_n=1}^{I_n}{a}_{i_n,r_n}^{\left(n\right)}{\mathbf{e}}_{i_n}^{\left(I_n\right)}={\mathbf{A}}_{.r_n}^{\left(n\right)}$, this model can be written as a weighted sum of$\prod _{n=1}^N{R}_n$ outer products, i.e., rank‐one tensors

$$\begin{array}{ll}\mathcal{X}& =\sum _{r_1=1}^{R_1}\cdots \sum _{r_N=1}^{R_N}{g}_{r_1,\dots ,r_N}\stackrel{N}{\underset{n=1}{\circ }}{\mathbf{A}}_{.r_n}^{\left(n\right)}\\ =g_{r_1,\dots ,r_N}\underset{\mathbb{R}}{\circ }{\mathbf{A}}_{.r_n}^{\left(n\right)}\left(\text{with the index notation}\right)\end{array}$$

(25)

Using the definition (12) allows to write (23) in terms of mode‐n products as

$$\begin{array}{l}\mathcal{X}=\mathcal{G}{\times }_1{\mathbf{A}}^{\left(1\right)}{\times }_2{\mathbf{A}}^{\left(2\right)}{\times }_3\cdots {\times }_N{\mathbf{A}}^{\left(N\right)}\\ =\mathcal{G}{\times }_{n=1}^N{\mathbf{A}}^{\left(n\right)}.\end{array}$$

(26)

This expression evidences that the Tucker model can be viewed as the transformation of the core tensor resulting from its multiplication by the factor matrix A⁽ⁿ⁾ along its mode‐n, which corresponds to a linear map applied to the mode‐n space of, for n=1,…,N, i.e., a multilinear map applied to. From a transformation point of view, and can be interpreted as the input tensor and the transformed tensor, or output tensor, respectively.

Matrix representations of the Tucker model. A matrix representation of a Tucker model is directly linked with a matricization of tensor like (7), corresponding to the combination of two sets of modes${\mathbb{S}}_1$ and${\mathbb{S}}_2$. These combinations must be applied both to the tensor and its core tensor.

The matrix representation (7) of the Tucker model (23) is given by

$${\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}=\left(\underset{n\in {\mathbb{S}}_1}{\otimes }{\mathbf{A}}^{\left(n\right)}\right){\mathbf{G}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}{\left(\underset{n\in {\mathbb{S}}_2}{\otimes }{\mathbf{A}}^{\left(n\right)}\right)}^T$$

(27)

with${\mathbf{G}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}\in {\mathbb{C}}^{J_1\times J_2}$, and$J_{n_1}=\underset{n\in {\mathbb{S}}_{n_1}}{\prod R_n}$, for n₁=1 and 2.

Proof

See Appendix 3.

For the flat mode‐n unfolding, defined in (9), the formula (27) gives

$${\mathbf{X}}_n={\mathbf{A}}^{\left(n\right)}{\mathbf{G}}_n{\left({\mathbf{A}}^{(n+1)}\otimes \cdots \otimes {\mathbf{A}}^{\left(N\right)}\otimes {\mathbf{A}}^{\left(1\right)}\otimes \cdots \otimes {\mathbf{A}}^{(n-1)}\right)}^T.$$

(28)

Applying the vec formula (92) to the right‐hand side of (28), we obtain the vectorized form of associated with its mode‐n unfolding X _n

$$\begin{array}{l}\text{vec}\left(\mathcal{X}\right)=\text{vec}\left({\mathbf{X}}_n\right)\\ =\left({\mathbf{A}}^{(n+1)}\otimes \cdots \otimes {\mathbf{A}}^{\left(N\right)}\otimes {\mathbf{A}}^{\left(1\right)}\otimes \cdots \otimes {\mathbf{A}}^{\left(n\right)}\right)\text{vec}\left({\mathbf{G}}_n\right).\end{array}$$

1.2.8.0 Tucker‐(N ₁ ,N) models

A Tucker‐ (N₁,N) model for a N th‐order tensor$\mathcal{X}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$, with N≥N₁, corresponds to the case where N-N₁ factor matrices are equal to identity matrices. For instance, assuming that${\mathbf{A}}^{\left(n\right)}={\mathbf{I}}_{I_n}$, which implies R _n =I _n , for n=N₁+1,…,N, (23) and (26) become

$$\begin{array}{ll}{x}_{i_1,\dots ,i_N}& =\sum _{r_1=1}^{R_1}\cdots \sum _{r_{N_1}=1}^{R_{N_1}}{g}_{r_1,\dots ,r_{N_1},i_{N_1+1},\dots ,i_N}\prod _{n=1}^{N_1}{a}_{i_n,r_n}^{\left(n\right)}\\ \mathcal{X}& =\mathcal{G}{\times }_1{\mathbf{A}}^{\left(1\right)}{\times }_2\cdots {\times }_{N_1}{\mathbf{A}}^{\left(N_1\right)}{\times }_{N_1+1}{\mathbf{I}}_{I_{N_1+1}}\cdots {\times }_N{\mathbf{I}}_{I_N}\end{array}$$

(29)

$$\begin{array}{l}=\mathcal{G}{\times }_{n=1}^{N_1}{\mathbf{A}}^{\left(n\right)}.\end{array}$$

(30)

One such model that is currently used in applications is the Tucker‐(2,3) model, usually denoted Tucker2, for third‐order tensors$\mathcal{X}\in {\mathbb{C}}^{I\times J\times K}$. Assuming${\mathbf{A}}^{\left(1\right)}=\mathbf{A}\in {\mathbb{C}}^{I\times P},{\mathbf{A}}^{\left(2\right)}=\mathbf{B}\in {\mathbb{C}}^{J\times Q}$, and${\mathbf{A}}^{\left(3\right)}={\mathbf{I}}_K$, such a model is defined by the following equations:

$$\begin{array}{ll}{x}_{\mathit{\text{ijk}}}& =\sum _{p=1}^P\sum _{q=1}^Q{g}_{\mathit{\text{pqk}}}{a}_{\mathit{\text{ip}}}{b}_{\mathit{\text{jq}}}\end{array}$$

(31)

$$\begin{array}{ll}\mathcal{X}& =\mathcal{G}{\times }_1\mathbf{A}{\times }_2\mathbf{B}\end{array}$$

(32)

with the core tensor$\mathcal{G}\in {\mathbb{C}}^{P\times Q\times K}$.

1.2.8.0 PARAFAC models

A PARAFAC model for a N th‐order tensor corresponds to the particular case of a Tucker model with an identity core tensor of order N and dimensions R×⋯×R

$$\mathcal{G}={\mathcal{I}}_{N,R}=\mathcal{I}\iff g_{r_1,\dots ,r_N}={\delta }_{r_1,\dots ,r_N}$$

(23) to (26) then become, respectively

$$\begin{array}{ll}{x}_{i_1,\dots ,i_N}& =\sum _{r=1}^R\prod _{n=1}^N{a}_{i_n,r}^{\left(n\right)}\end{array}$$

(33)

$$\begin{array}{l}=\prod _{n=1}^N{a}_{i_n,r}^{\left(n\right)}\left(\text{with the index notation}\right)\end{array}$$

(34)

$$\begin{array}{ll}\mathcal{X}& =\sum _{r=1}^R\left(\stackrel{N}{\underset{n=1}{\circ }}{\mathbf{A}}_{\mathrm{.r}}^{\left(n\right)}\right)\\ \mathcal{X}& ={\mathcal{I}}_{N,R}{\times }_{n=1}^N{\mathbf{A}}^{\left(n\right)}\end{array}$$

(35)

with the factor matrices${\mathbf{A}}^{\left(n\right)}\in {\mathbb{C}}^{I_n\times R},n=1,\dots ,N$.

1.2.8.0 Remarks

• The expression (33) as a sum of polyads is called a polyadic form of by Hitchcock[51].

• The PARAFAC model (33, 34 and 35) amounts to decomposing the tensor into a sum of R components, each component being a rank‐one tensor. When R is minimal in (33), it is called the rank of[52]. This rank is related to the mode‐n ranks by the following inequalities${\text{rank}}_n\left(\mathcal{X}\right)\le R,\forall n=1,\dots ,N$. Furthermore, contrary to the matrices for which the rank is always at most equal to the smallest of the dimensions, for higher‐order tensors, the rank can exceed any mode‐n dimension I _n .

• There exists different definitions of rank for tensors, like typical and generic ranks, or also symmetric rank for a symmetric tensor (see[53, 54] for more details).

• In telecommunication applications, the structure parameters (rank, mode dimensions, and core tensor dimensions) of a PARAFAC or Tucker model are design parameters that are chosen in function of the performance desired for the communication system. However, in most of the applications, as for instance in multi‐way data analysis, the structure parameters are generally unknown and must be determined a priori. Several techniques have been proposed for determining these parameters (see[55]‐[58] and references therein).

• The PARAFAC model is also sometimes defined by the following equation

  $$x_{i_1,\dots ,i_N}=\sum _{r=1}^R{g}_r\prod _{n=1}^N{a}_{i_n,r}^{\left(n\right)}\text{with}{g}_r>0.$$

  (36)

• In this case, the identity tensor${\mathcal{I}}_{N,R}$ in (35) is replaced by the diagonal tensor$\mathcal{G}\in {\mathbb{C}}^{R\times \cdots \times R}$ whose diagonal elements are equal to scaling factors g _r , i.e.

  $$g_{r_1,\dots ,r_N}=\left\{\begin{array}{l}{g}_r\text{if}{r}_1=\cdots =r_N=r\\ 0\text{otherwise}\end{array}\right.$$

  and all the column vectors${\mathbf{A}}_{\mathrm{.r}}^{\left(n\right)}$ are normalized, i.e., with a unit norm, for 1≤n≤N.

• It is important to notice that the PARAFAC model (33) is multilinear (more precisely N‐linear) in its parameters in the sense that it is linear with respect to each matrix factor. This multilinearity property is exploited for parameter estimation using the standard alternating least squares (ALS) algorithm[4, 5] that consists in alternately estimating each matrix factor by minimizing a least squares error criterion conditionally to the knowledge of the other matrix factors that are fixed with their previously estimated values.

Matrix representations of the PARAFAC model. The matrix representation (7) of the PARAFAC model (33)‐(35) is given by

$${\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}=\left(\underset{n\in {\mathbb{S}}_1}{◇}{\mathbf{A}}^{\left(n\right)}\right){\left(\underset{n\in {\mathbb{S}}_2}{◇}{\mathbf{A}}^{\left(n\right)}\right)}^T.$$

(37)

Proof.

See Appendix 4.

1.2.8.0 Remarks

• From (37), we can deduce that a mode combination results in a Khatri‐Rao product of the corresponding factor matrices. Consequently, the tensor contraction (5) associated with the PARAFAC‐N model (35) gives a PARAFAC‐ N₁ model whose factor matrices are equal to$\underset{n\in {\mathbb{S}}_{n_1}}{◇}{\mathbf{A}}^{\left(n\right)}\in {\mathbb{C}}^{J_{n_1}\times R}$, n₁=1,…,N₁, with$J_{n_1}=\underset{n\in {\mathbb{S}}_{n_1}}{\prod I_n}$.

• For the PARAFAC model, the flat mode‐n unfolding, defined in (9), is given by

  $${\mathbf{X}}_n={\mathbf{A}}^{\left(n\right)}{\left({\mathbf{A}}^{(n+1)}◇\cdots ◇{\mathbf{A}}^{\left(N\right)}◇{\mathbf{A}}^{\left(1\right)}◇\cdots ◇{\mathbf{A}}^{(n-1)}\right)}^T,$$

  (38)

• and the associated vectorized form is obtained in applying the vec formula (93) to the right‐hand side of the above equation, with I _R =diag(1 _R )

  $$\text{vec}\left(\mathcal{X}\right)=\text{vec}\left({\mathbf{X}}_n\right)=\left({\mathbf{A}}^{(n+1)}◇\cdots ◇{\mathbf{A}}^{\left(N\right)}◇{\mathbf{A}}^{\left(1\right)}◇\cdots ◇{\mathbf{A}}^{\left(n\right)}\right){\mathbf{1}}_R$$

  (39)

• In the case of the normalized PARAFAC model (36), (37) and (39) become, respectively,

  $$\begin{array}{ll}{\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}& =\left(\underset{n\in {\mathbb{S}}_1}{◇}{\mathbf{A}}^{\left(n\right)}\right)\text{diag}\left(\mathbf{g}\right){\left(\underset{n\in {\mathbb{S}}_2}{◇}{\mathbf{A}}^{\left(n\right)}\right)}^T\\ \text{vec}\left(\mathcal{X}\right)& =\text{vec}\left({\mathbf{X}}_n\right)\\ =\left({\mathbf{A}}^{(n+1)}◇\cdots ◇{\mathbf{A}}^{\left(N\right)}◇{\mathbf{A}}^{\left(1\right)}◇\cdots ◇{\mathbf{A}}^{\left(n\right)}\right)\mathbf{g}\end{array}$$

• where$\mathbf{g}={\left[g_1\cdots g_R\right]}^T\in {\mathbb{C}}^{R\times 1}$.

• For the PARAFAC model of a third‐order tensor$\mathcal{X}\in {\mathbb{C}}^{I\times J\times K}$ with factor matrices (A,B,C), the formula (37) gives for${\mathbb{S}}_1=\{i,j\}$ and${\mathbb{S}}_2=\left\{k\right\}$

  $${\mathbf{X}}_{\mathit{\text{IJ}}\times K}=\left[\begin{array}{c}{\mathbf{X}}_{1\mathrm{..}}\\ ⋮\\ {\mathbf{X}}_{\mathrm{I..}}\end{array}\right]=(\mathbf{A}◇\mathbf{B}){\mathbf{C}}^T\in {\mathbb{C}}^{\mathit{\text{IJ}}\times K}.$$

• Noting that$\mathbf{A}◇\mathbf{B}=\left[\begin{array}{c}\mathbf{B}{D}_1\left(\mathbf{A}\right)\\ ⋮\\ \mathbf{B}{D}_I\left(\mathbf{A}\right)\end{array}\right]$, we deduce the following expression for mode‐1 matrix slices

  $${\mathbf{X}}_{\mathrm{i..}}=\mathbf{B}{D}_i\left(\mathbf{A}\right){\mathbf{C}}^T.$$

• Similarly, we have

  $$\begin{array}{ll}{\mathbf{X}}_{\mathit{\text{JK}}\times I}& =(\mathbf{B}◇\mathbf{C}){\mathbf{A}}^T,{\mathbf{X}}_{\mathit{\text{KI}}\times J}=(\mathbf{C}◇\mathbf{A}){\mathbf{B}}^T,\\ {\mathbf{X}}_{\mathrm{.j.}}& =\mathbf{C}{D}_j\left(\mathbf{B}\right){\mathbf{A}}^T,{\mathbf{X}}_{\mathrm{..k}}=\mathbf{A}{D}_k\left(\mathbf{C}\right){\mathbf{B}}^T.\end{array}$$

• For the PARAFAC model of a fourth‐order tensor$\mathcal{X}\in {\mathbb{C}}^{I\times J\times K\times L}$ with factor matrices (A,B,C,D), we obtain

  $$\begin{array}{ll}{\mathbf{X}}_{\mathit{\text{IJK}}\times L}& =(\mathbf{A}◇\mathbf{B}◇\mathbf{C}){\mathbf{D}}^T\\ =\left[\begin{array}{c}(\mathbf{B}◇\mathbf{C})D_1\left(\mathbf{A}\right)\\ ⋮\\ (\mathbf{B}◇\mathbf{C})D_I\left(\mathbf{A}\right)\end{array}\right]{\mathbf{D}}^T\\ =\left[\begin{array}{c}\mathbf{C}{D}_1\left(\mathbf{B}\right)D_1\left(\mathbf{A}\right)\\ ⋮\\ \mathbf{C}{D}_J\left(\mathbf{B}\right)D_I\left(\mathbf{A}\right)\end{array}\right]{\mathbf{D}}^T\in {\mathbb{C}}^{\mathit{\text{IJK}}\times L}\\ {\mathbf{X}}_{\mathrm{ij..}}& =\mathbf{C}{D}_j\left(\mathbf{B}\right)D_i\left(\mathbf{A}\right){\mathbf{D}}^T\in {\mathbb{C}}^{K\times L}\end{array}$$

  (40)

  Other matrix slices can be deduced from (40) by simple permutations of the matrix factors.

In the next section, we introduce two constrained PARAFAC models, the so‐called PARALIND and CONFAC models, and then PARATUCK models.

1.3 Constrained PARAFAC models

The introduction of constraints in tensor models can result from the system itself that is under study or from a system design. In the first case, the constraints are often interpreted as interactions or linear dependencies between the PARAFAC factors. Examples of such dependencies are encountered in psychometric and chemometric applications that gave origin, respectively, to the PARATUCK‐2 model[59] and the parallel profiles with linear dependencies (PARALIND) model[60, 61], introduced in[47] under the name canonical decomposition with linear constraints (CANDELINC), for the multiway case. A first application of the PARATUCK‐2 model in signal processing was made in[62] for blind joint identification and equalization of Wiener‐Hammerstein communication channels. The PARALIND model was applied for identifiability and propagation parameter estimation purposes in a context of array signal processing[63, 64].

In the second case, the constraints are used as design parameters. For instance, in a telecommunications context, we proposed two constrained tensor models: the CONFAC (constrained factor) model[65] and the PARATUCK‐ (N₁,N) model[66, 67]. The PARATUCK‐2 model was also applied for designing space‐time spreading‐multiplexing MIMO systems[68]. For these telecommunication applications of constrained tensor models, the constraints are used for resource allocation. We are now going to describe these various constrained PARAFAC models.

1.3.1 PARALIND models

Let us define the core tensor of the Tucker model (26) as follows:

$$\mathcal{G}={\mathcal{I}}_{N,R}{\times }_{n=1}^N{\mathit{\Phi }}^{\left(n\right)}$$

(41)

where${\mathit{\Phi }}^{\left(n\right)}\in {\mathbb{R}}^{R_n\times R},n=1,\dots ,N$, with$R\ge \underset{n}{\text{max}}\left(R_n\right)$, are constraint matrices. In this case, will be called the ‘interaction tensor’, or ‘constraint tensor’.

The PARALIND model is obtained by substituting (41) into (26) and applying the property (13), which gives

$$\mathcal{X}=\mathcal{G}{\times }_{n=1}^N{\mathbf{A}}^{\left(n\right)}={\mathcal{I}}_{N,R}{\times }_{n=1}^N\left({\mathbf{A}}^{\left(n\right)}{\mathit{\Phi }}^{\left(n\right)}\right).$$

(42)

This equation leads to two different interpretations of the PARALIND model, as a constrained Tucker model whose core tensor admits a PARAFAC decomposition with factor matrices Φ⁽ⁿ⁾, called ‘interaction matrices,’ and as a constrained PARAFAC model with constrained factor matrices${\bar{\mathbf{A}}}^{\left(n\right)}={\mathbf{A}}^{\left(n\right)}{\mathit{\Phi }}^{\left(n\right)}$.

The interaction matrix Φ⁽ⁿ⁾ allows taking into account linear dependencies between the columns of A⁽ⁿ⁾, implying a rank deficiency for this factor matrix. When the columns of Φ⁽ⁿ⁾ are formed with 0’s and 1’s, the dependencies simply consist in a repetition or an addition of certain columns of A⁽ⁿ⁾. In this particular case, the diagonal element${\xi }_{r,r}^{\left(n\right)}\ge 1$ of the matrix${\mathit{\Xi }}^{\left(n\right)}={{\mathit{\Phi }}^{\left(n\right)}}^T{\mathit{\Phi }}^{\left(n\right)}\in {\mathbb{R}}^{R\times R}$ represents the number of columns of A⁽ⁿ⁾ that are added to form the r th column of the constrained factor A⁽ⁿ⁾Φ⁽ⁿ⁾. The choice${\mathit{\Phi }}^{\left(n\right)}={\mathbf{I}}_R$ means that there is no such dependency among the columns of A⁽ⁿ⁾.

Note that (42) can be written element‐wise as

$$\begin{array}{ll}{x}_{i_1,\dots ,i_N}& =\sum _{r_1=1}^{R_1}\dots \sum _{r_N=1}^{R_N}{g}_{r_1,\dots ,r_N}\prod _{n=1}^N{a}_{i_n,r_n}^{\left(n\right)}\\ =\sum _{r=1}^R\prod _{n=1}^N{ā}_{i_n,r}^{\left(n\right)}\text{with}{ā}_{i_n,r}^{\left(n\right)}=\sum _{r_n=1}^{R_n}{a}_{i_n,r_n}^{\left(n\right)}{\varphi }_{r_n,r}^{\left(n\right)}.\\ \text{with}{g}_{r_1,\dots ,r_N}& =\sum _{r=1}^R\prod _{n=1}^N{\varphi }_{r_n,r}^{\left(n\right)}\end{array}$$

(43)

This constrained PARAFAC model constitutes an N‐way form of the three‐way PARALIND model, used for chemometric applications in[60, 61].

1.3.2 CONFAC models

When the constraint matrices${\mathit{\Phi }}^{\left(n\right)}\in {\mathbb{R}}^{R_n\times R}$ are full row rank and their columns are chosen as canonical vectors of the Euclidean space${\mathbb{R}}^{R_n}$, for n=1,…,N, the constrained PARAFAC model (42) constitutes a generalization to N th‐order of the third‐order CONFAC model, introduced in[65] for designing MIMO communication systems with resource allocation. This CONFAC model was used in[69] for solving the problem of blind identification of underdetermined mixtures based on cumulant generating function of the observations. In a telecommunications context where represents the tensor of received signals, such a constraint matrix Φ⁽ⁿ⁾ can be interpreted as an ‘allocation matrix’ allowing to allocate resources, like data streams, codes, and transmit antennas, to the R components of the signal to be transmitted. In this case, the core tensor will be called the ‘allocation tensor.’ By assumption, each column of the allocation matrix Φ⁽ⁿ⁾ is a canonical vector of${\mathbb{R}}^{R_n}$, which means that there is only one value of r _n such that${\varphi }_{r_n,r}^{\left(n\right)}=1$, and this value of r _n corresponds to the n th resource allocated to the r th component.

Each element$x_{i_1,\dots ,i_N}$ of the received signal tensor is equal to the sum of R components, each component r resulting from the combination of N resources, each resource being associated with a column of the matrix factor A⁽ⁿ⁾, n=1,…,N. This combination, determined by the allocation matrices, is defined by a set of N indices {r₁,…,r _N } such that$\prod _{n=1}^N{\varphi }_{r_n,r}^{\left(n\right)}=1$. As for any$r\in \left[1,R\right]$, there is one and only one N‐uplet (r₁,…,r _N ) such as$\prod _{n=1}^N{\varphi }_{r_n,r}^{\left(n\right)}=1$, we can deduce that each component r of$x_{i_1,\dots ,i_N}$ in (43) is the result of one and only one combination of the N resources under the form of the product$\prod _{n=1}^N{a}_{i_n,r_n}^{\left(n\right)}$. For the CONFAC model, we have

$$\sum _{r_n=1}^{R_n}{D}_{r_n}\left({\mathit{\Phi }}^{\left(n\right)}\right)={\mathbf{I}}_R,\forall n=1,\dots ,N$$

meaning that each resource r _n is allocated at least once, and the diagonal element of Ξ⁽ⁿ⁾=Φ^(n)TΦ⁽ⁿ⁾ is such as${\xi }_{r,r}^{\left(n\right)}=1,\forall n=1,\dots ,N$, because only one resource r _n is allocated to each component r. Moreover, we have to notice that the assumption$R\ge \underset{n}{\text{max}}\left(R_n\right)$ implies that each resource can be allocated several times, i.e., to several components. Defining the interaction matrices

$${\mathit{\Gamma }}^{\left(n\right)}={\mathit{\Phi }}^{\left(n\right)}{\mathit{\Phi }}^{{\left(n\right)}^T}\in {\mathbb{R}}^{R_n\times R_n},{\mathit{\Gamma }}^{(n_1,n_2)}={\mathit{\Phi }}^{\left(n_1\right)}{\mathit{\Phi }}^{{\left(n_2\right)}^T}\in {\mathbb{R}}^{R_{n_1}\times R_{n_2}}$$

the diagonal element${\gamma }_{r_n,r_n}^{\left(n\right)}\in \left[1,R-R_n+1\right]$ represents the number of times that the$r_n^{\text{th}}$ column of A⁽ⁿ⁾ is repeated, i.e., the number of times that the$r_n^{\text{th}}$ resource is allocated to the R components, whereas${\gamma }_{r_{n_1},r_{n_2}}^{(n_1,n_2)}$ determines the number of interactions between the$r_{n_1}^{\text{th}}$ column of${\mathbf{A}}^{\left(n_1\right)}$ and the$r_{n_2}^{\text{th}}$ column of${\mathbf{A}}^{\left(n_2\right)}$, i.e., the number of times that the$r_{n_1}^{\text{th}}$ and$r_{n_2}^{\text{th}}$ resources are combined in the R components. If we choose R _n =R and${\mathit{\Phi }}^{\left(n\right)}={\mathbf{I}}_R,\forall n=1,\dots ,N$, the PARALIND/CONFAC model (42) becomes identical to the PARAFAC one (35).

The matrix representation (7) of the PARALIND/CONFAC model can be deduced from (37) in replacing A⁽ⁿ⁾ by A⁽ⁿ⁾Φ⁽ⁿ⁾

$${\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}=\left(\underset{n\in {\mathbb{S}}_1}{◇}{\mathbf{A}}^{\left(n\right)}{\mathit{\Phi }}^{\left(n\right)}\right){\left(\underset{n\in {\mathbb{S}}_2}{◇}{\mathbf{A}}^{\left(n\right)}{\mathit{\Phi }}^{\left(n\right)}\right)}^T.$$

Using the identity, (86) gives

$${\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}=\left(\underset{n\in {\mathbb{S}}_1}{\otimes }{A}^{\left(n\right)}\right)\left(\underset{n\in {\mathbb{S}}_1}{◇}{\mathit{\Phi }}^{\left(n\right)}\right){\left(\underset{n\in {\mathbb{S}}_2}{◇}{\mathit{\Phi }}^{\left(n\right)}\right)}^T{\left(\underset{n\in {\mathbb{S}}_2}{\otimes }{\mathbf{A}}^{\left(n\right)}\right)}^T,$$

(44)

or, equivalently,

$${\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}=\left(\underset{n\in {\mathbb{S}}_1}{\otimes }{\mathbf{A}}^{\left(n\right)}\right){\mathbf{G}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}{\left(\underset{n\in {\mathbb{S}}_2}{\otimes }{\mathbf{A}}^{\left(n\right)}\right)}^T,$$

where the matrix representation${\mathbf{G}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}$ of the constraint/allocation tensor, defined by means of its PARAFAC model (41), can also be deduced from (37) as

$${\mathbf{G}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}=\left(\underset{n\in {\mathbb{S}}_1}{◇}{\mathit{\Phi }}^{\left(n\right)}\right){\left(\underset{n\in {\mathbb{S}}_2}{◇}{\mathit{\Phi }}^{\left(n\right)}\right)}^T.$$

1.3.3 Nested Tucker models

The PARALIND/CONFAC models can be viewed as particular cases of a new family of tensor models that we shall call nested Tucker models, defined by means of the following recursive equation:

$$\begin{array}{ll}{\mathcal{X}}^{\left(p\right)}& ={\mathcal{X}}^{(p-1)}{\times }_{n=1}^N{\mathbf{A}}^{(p,n)}\text{for}p=1,\dots ,P\\ =\mathcal{G}{\times }_{n=1}^N\prod _{q=P}^1{\mathbf{A}}^{(q,n)}\end{array}$$

with the factor matrices${\mathbf{A}}^{(p,n)}\in {\mathbb{C}}^{R^{(p,n)}\times R^{(p-1,n)}}$ for p=1,…,P, such as R^(0,n)=R _n and R^(P,n)=I _n , for n=1,…,N, the core tensor${\mathcal{X}}^{\left(0\right)}=\mathcal{G}\in {\mathbb{C}}^{R_1\times \cdots \times R_N}$, and${\mathcal{X}}^{\left(P\right)}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$. This equation can be interpreted as P successive linear transformations applied to each mode‐n space of the core tensor. So, P nested Tucker models can then be interpreted as a Tucker model for which the factor matrices are products of P matrices. When$\mathcal{G}={\mathcal{I}}_{N,R}$, which implies R^(0,n)=R _n =R for n=1,…,N, we obtain nested PARAFAC models. The PARALIND/CONFAC models correspond to two nested PARAFAC models (P=2), with A^(1,n)=Φ⁽ⁿ⁾, A^(2,n)=A⁽ⁿ⁾, R^(0,n)=R, R^(1,n)=R _n , and R^(2,n)=I _n , for n=1,…,N.

By considering nested PARAFAC models with P=3,${\mathbf{A}}^{(1,n)}={\mathit{\Phi }}^{\left(n\right)}\in {\mathbb{C}}^{K_n\times R}$,${\mathbf{A}}^{(2,n)}={\mathbf{A}}^{\left(n\right)}\in {\mathbb{C}}^{J_n\times K_n}$, and${\mathbf{A}}^{(3,n)}={\mathit{\Psi }}^{\left(n\right)}\in {\mathbb{C}}^{I_n\times J_n}$, for n=1,…,N, we deduce doubly PARALIND/CONFAC models described by the following equation:

$$\mathcal{X}={\mathcal{I}}_{N,R}{\times }_{n=1}^N\left({\mathit{\Psi }}^{\left(n\right)}{\mathbf{A}}^{\left(n\right)}{\mathit{\Phi }}^{\left(n\right)}\right).$$

Such a model can be viewed as a doubly constrained PARAFAC model, with factor matrices Ψ⁽ⁿ⁾A⁽ⁿ⁾Φ⁽ⁿ⁾, the constraint matrix Ψ⁽ⁿ⁾, assumed to be full column rank, allowing to take into account linear dependencies between the rows of A⁽ⁿ⁾. A third‐order nested Tucker model is visualized in Figure1.

Figure 1

Visualization of a third‐order nested Tucker model.

1.3.4 Block PARALIND/CONFAC models

In some applications, the data tensor$\mathcal{X}\in {\mathbb{C}}^{I_1\times \cdots \times I_N}$ is written as a sum of P sub‐tensors${\mathcal{X}}^{\left(p\right)}$, each sub‐tensor admitting a tensor model with a possibly different structure. So, we can define a block‐PARALIND/CONFAC model as

$$\begin{array}{ll}\mathcal{X}& =\sum _{p=1}^P{\mathcal{X}}^{\left(p\right)},\end{array}$$

(45)

$$\begin{array}{ll}{\mathcal{X}}^{\left(p\right)}& ={\mathcal{G}}^{\left(p\right)}{\times }_{n=1}^N{\mathbf{A}}^{(p,n)},\\ {\mathcal{G}}^{\left(p\right)}& ={\mathcal{I}}_{N,R^{\left(p\right)}}{\times }_{n=1}^N{\mathit{\Phi }}^{(p,n)},\end{array}$$

(46)

where${\mathbf{A}}^{(p,n)}\in {\mathbb{C}}^{I_n\times R^{(p,n)}}$,${\mathit{\Phi }}^{(p,n)}\in {\mathbb{C}}^{R^{(p,n)}\times R^{\left(p\right)}}$, and${\mathcal{G}}^{\left(p\right)}\in {\mathbb{C}}^{R^{(p,1)}\times \cdots \times R^{(p,N)}}$ are the mode‐n factor matrix, the mode‐n constraint/allocation matrix, and the core tensor of the PARALIND/CONFAC model of the p th sub‐tensor, respectively. The matrix representation (44) then becomes

$$\begin{array}{l}{\mathbf{X}}_{{\mathbb{S}}_1;{\mathbb{S}}_2}=\sum _{p=1}^P\left(\underset{n\in {\mathbb{S}}_{}}{\otimes }\right)\end{array}$$