A regularized matrix factorization approach to induce structured sparse-low-rank solutions in the EEG inverse problem

Abstract

We consider the estimation of the Brain Electrical Sources (BES) matrix from noisy electroencephalographic (EEG) measurements, commonly named as the EEG inverse problem. We propose a new method to induce neurophysiological meaningful solutions, which takes into account the smoothness, structured sparsity, and low rank of the BES matrix. The method is based on the factorization of the BES matrix as a product of a sparse coding matrix and a dense latent source matrix. The structured sparse-low-rank structure is enforced by minimizing a regularized functional that includes the ℓ₂₁-norm of the coding matrix and the squared Frobenius norm of the latent source matrix. We develop an alternating optimization algorithm to solve the resulting nonsmooth-nonconvex minimization problem. We analyze the convergence of the optimization procedure, and we compare, under different synthetic scenarios, the performance of our method with respect to the Group Lasso and Trace Norm regularizers when they are applied directly to the target matrix.

1 Introduction

The solution of the electroencephalographic (EEG) inverse problem to obtain functional brain images is of high value for neurological research and medical diagnosis. It involves the estimation of the Brain Electrical Sources (BES) distribution from noisy EEG measurements, whose relation is modeled according to the linear model

$$\begin{array}{l}\mathbf{Y}=\mathbf{AS}+\mathbf{E},\end{array}$$

(1)

where $\mathbf{Y}\in {\mathbb{R}}^{M\times T}$ and $\mathbf{A}\in {\mathbb{R}}^{M\times N}$ are known and represent, respectively, the EEG measurements matrix and the forward operator (a.k.a lead field matrix), $\mathbf{S}\in {\mathbb{R}}^{N\times T}$ denotes the BES matrix, and $\mathbf{E}\in {\mathbb{R}}^{M\times T}$ is a noise matrix. M denotes the number of EEG electrodes, N is the number of brain electrical sources, and T is the number of time instants.

This estimation problem is very challenging: N≫M, and the existence of silent BES (BES that produce nonmeasurable fields on the scalp surface) implies that the EEG inverse problem has infinite solutions: a silent BES can always be added to a solution of the inverse problem without affecting the EEG measurements. For all these reasons, the EEG inverse problem is an undetermined ill-posed problem [1–4].

A classical approach to solve an ill-posed problem is to use regularization theory, which involves the replacement of the original ill-posed problem with a ‘nearby’ well-posed problem whose solution approximates the required solution [5]. Solutions developed by this theory are stated in terms of a regularization function, which helps us to select, among the infinite solutions, the one that best fulfills a prescribed constrain (e.g., smoothness, sparsity, and low rank). To define the constrain, we can use mathematical restrictions (minimum norm estimates) or anatomical, physiological, and functional prior information. Some examples of useful neurophysiological information are [1, 6]: the irrotational character of the brain current sources, the (smooth) dynamic of the neural signals, the clusters formed by neighboring or functional related BES, and the smoothness and focality of the electromagnetic fields generated and propagated within the volume conductor media (brain cortex, skull, and scalp).

Several regularization functions have been proposed in the EEG community: Hämäläinen and Ilmoniemi in [7] proposed a squared Frobenius norm penalty ($\parallel \mathbf{S}{\parallel }_{\mathrm{F}}^2$), which they named Minimum Norm Estimate (MNE). This regularization function usually induces solutions that spread over a considerable part of the brain. Uutela et al. in [8] proposed an ℓ₁-norm penalty (∥S∥₁). They named their approach Minimum Current Estimate (MCE). This penalty function promotes solutions that tend to be scattered around the true sources. Mixed ℓ₁ℓ₂-norm penalties have also been proposed in the framework of the time basis, time frequency dictionaries, and spatial basis decomposition. These mixed norm approaches induce structured sparse solutions and depend on decomposing the BES signals as linear combinations of multiple basis functions, e.g., Ou et al. in [9] proposed the use of temporal basis functions obtained with singular value decomposition (SVD), Gramfort et al. in [10, 11] proposed the use of time-frequency Gabor dictionaries, and Haufe et al. in [12] proposed the use of spatial basis Gaussian functions. For a more detailed overview on inverse methods for EEG, see [2, 3, 13] and references therein. For a more detailed overview on regularization functions applied to structured sparsity problems, see [14–16] and references therein.

All of these regularizers try to induce neurophysiological meaningful solutions, which take into account the smoothness and structured sparsity of the BES matrix: during a particular cognitive task, only the BES related with the brain area involved in such a task will be active, and their corresponding time evolution will vary smoothly, that is, the BES matrix will have few nonzero rows, and in addition, the columns will vary smoothly. In this paper, we propose a regularizer that takes into account not only the smoothness and structured sparsity of the BES matrix but also its low rank, capturing this way the linear relation between the active sources and their corresponding neighbors. In order to do so, we propose a new method based on matrix factorization and regularization, with the aim of recovering the latent structure of the BES matrix. In the factorization, the first matrix, which acts as a coding matrix, is penalized using the ℓ₂₁-norm, and the second one, which acts as a dense, full rank latent source matrix, is penalized using the squared Frobenius norm.

In our approach, the resulting optimization problem is nonsmooth and nonconvex. A standard approach to deal with the nonsmoothness introduced by the nonsmooth regularizers mentioned above is to reformulate the regularization problem as a second-order cone programming (SOCP) problem [12] and use interior point-based solvers. However, interior point-based methods can not handle large scale problems, which is the case of large EEG inverse problems involving thousands of brain sources. Another approach is to try to solve the nonsmooth problem directly, using general nonsmooth optimization methods, for instance, the subgradient method [17]. This method can be used if a subgradient of the objective function can be computed efficiently [14]. However, its convergence rate is, in practice, slow ($O(1/\sqrt{k})$), where k is the iteration counter. In this paper, in order to tackle the nonsmoothness of the optimization problem, we depart from these optimization methods and use instead efficient first-order nonsmooth optimization methods [5, 18, 19]: forward-backward splitting methods. These methods are also called proximal splitting because the nonsmooth function is involved via its proximity operator. Forward-backward splitting methods were first introduced in the EEG inverse problem by Gramfort et al. [10, 11, 13], where they used them to solve nonsmooth optimization problems resulting from the use of mixed ℓ₁ℓ₂-norm penalties functions. These methods have drawn, increasing attention in the EEG, machine learning, and signal processing community, especially because of their convergence rates and their ability to deal with large problems [19–21].

On the other hand, in order to handle the nonconvexity of the optimization problem, we use an iterative alternating minimization approach: minimizing over the coding matrix while maintaining fixed the latent source matrix and viceversa. Both of these optimization problems are convex: the first one can be solved using proximal splitting methods, while the second one can be solve directly in terms of a matrix inversion.

The rest of the paper is organized as follows. In Section 2, we give an overview of the EEG inverse problem. In Section 3, we present the mathematical background related with the proximal splitting methods. The resulting nonsmooth and nonconvex optimization problem is formally described in Section 4. In Section 5, we propose an alternating minimization algorithm, and its convergence analysis is presented in Section 6. Section 7 is devoted to the numerical evaluation of the algorithm and its comparison with the Group Lasso and Trace Norm regularizers, which consider partially the characteristics of the matrix S: its structured sparsity by using the ℓ₂₁-norm and its low rank by using the ℓ_∗-norm, respectively. The advantages of considering both characteristics in a single method, like in the proposed one, become clear in comparison with the independent use of the Group Lasso and Trace Norm regularizers. Finally, conclusions are presented in Section 8.

2 EEG inverse problem background

The EEG signals represent the electrical activity of one or several assemblies of neurons [22]. The area of a neuron assembly is small compared to the distance to the observation point (the EEG sensors). Therefore, the electromagnetic fields produced by an active neuron assembly at the sensor level are very similar to the field produced by a current dipole [23]. This simplified model is known as the equivalent current dipole (ECD). These ECDs are also known by other names such as BES and current sources. Due to the uniform spatial organization of their dendrites (perpendicular to the brain cortex), the pyramidal neurons are the only neurons that can generate a net current dipole over a piece of cortical surface, whose field is detectable on the scalp [3]. According to [24], it is necessary to add the field of ∼10⁴ pyramidal neurons in order to produce a voltage that is detectable on the scalp. These voltages can be recorded by using different types of electrodes [22], such as disposable (gel-less, and pre-gelled types), reusable disc electrodes (gold, silver, stainless steel, or tin), headbands and electrode caps, saline-based electrodes, and needle electrodes.

Under the quasi-static approximation of Maxwell’s equations, we can express the general model for the observed EEG signals y(t) at time t as linear functions of the BES s(t) [9]:

$$\mathbf{y}\left(t\right)=\mathbf{As}\left(t\right)+\mathbf{e}\left(t\right),$$

(2)

where $\mathbf{y}\left(t\right)\in {\mathbb{R}}^{M\times 1}$ is the EEG measurements vector, $\mathbf{s}\left(t\right)\in {\mathbb{R}}^{N\times 1}$ is the BES vector, $\mathbf{e}\left(t\right)\in {\mathbb{R}}^{M\times 1}$ is the noise vector, and $\mathbf{A}\in {\mathbb{R}}^{M\times N}$ is the lead field matrix. In a typical experimental setup, the number of electrodes (M) is ∼10², and the number of BES (N) is ∼10³, 10⁴. We can express the former model for all time instants {t₁,t₂,…,t _T } (corresponding to some observation time window) by using the matrix formulation (1), where $\mathbf{Y}=\left[\mathbf{y}\left(t_1\right),\mathbf{y}\left(t_2\right),\dots ,\mathbf{y}\left(t_T\right)\right]\in {\mathbb{R}}^{M\times T}$, $\mathbf{S}=\left[\mathbf{s}\left(t_1\right),\mathbf{s}\left(t_2\right),\dots ,\mathbf{s}\left(t_T\right)\right]\in {\mathbb{R}}^{N\times T}$, and $\mathbf{E}=\left[\mathbf{e}\left(t_1\right),\mathbf{e}\left(t_2\right),\dots ,\mathbf{e}\left(t_T\right)\right]\in {\mathbb{R}}^{M\times T}$. The i th row of the matrix Y represents the electrical activity recorded by the i th EEG electrode during the observation time window. In the BES matrix S, each row represents the time evolution of one brain electrical source, and each column represents the activity of all the corresponding sources in a particular time instant. Finally, the forward operator A summarizes the geometric and electric properties of the conducting media (brain, skull, and scalp) and establishes the link between the current sources and EEG sensors (A _{i j} tells us how the j th BES influences the measure obtained by the i th electrode). Following this notation, the EEG inverse problem can be stated as follows: Given a set of EEG signals (Y) and a forward model (A), estimate the current sources within the brain (S) that produce these signals.

3 Mathematical background

3.1 Proximity operator

The proximity operator [19, 25] corresponding to a convex function f is a mapping from ${\mathbb{R}}^n$ to itself and is defined as follows:

$$\begin{array}{l}{\text{prox}}_f\left(z\right)=\underset{x\in {\mathbb{R}}^n}{argmin}\left\{f\left(x\right)+\frac{1}{2}\parallel x-z{\parallel }^2\right\},\end{array}$$

(3)

where ∥·∥ denotes the Euclidean norm. Note that the proximity operator is well defined, because the above minimum exists and is unique (the objective function if strongly convex).

3.2 Subdifferential-proximity operator relationship

If f is a convex function on ${\mathbb{R}}^n$ and $y\in {\mathbb{R}}^n$, then [26]

$$\begin{array}{l}x\in \mathrm{\partial f}\left(y\right)\iff y={\text{prox}}_f(x+y),\end{array}$$

(4)

where ∂ f(y) denotes the subdifferential of f at y.

3.3 Principles of proximal splitting methods

Proximal splitting methods are specifically tailored to solve an optimization problem of the form

$$\begin{array}{l}\underset{\mathbf{S}}{\text{minimize}}f\left(\mathbf{S}\right)+r\left(\mathbf{S}\right),\end{array}$$

(5)

where f(S) is a smooth convex function, and r(S) is also a convex function, but nonsmooth. From convex analysis [17], we know that S is a minimizer of (5) if and only if 0∈∂(f+r)(S). This implies the following [18]:

$$\begin{array}{rcl}0\in \partial (f+r)\left(\mathbf{S}\right)& \iff & 0\in \left\{\mathrm{\partial f}\right(\mathbf{S})+\mathrm{\partial r}(\mathbf{S}\left)\right\}\\ \iff & -\nabla f\left(\mathbf{S}\right)\in \mathrm{\partial r}\left(\mathbf{S}\right)\\ \iff & -\gamma \nabla f\left(\mathbf{S}\right)\in \mathrm{\gamma \partial r}\left(\mathbf{S}\right)\\ \iff & (\mathbf{S}-\gamma \nabla f(\mathbf{S}\left)\right)-\mathbf{S}\in \mathrm{\partial \gamma r}\left(\mathbf{S}\right)\end{array}$$

Using (4) in the former expression, we get

$$\begin{array}{l}\mathbf{S}={\text{prox}}_{\mathrm{\gamma r}}(\mathbf{S}-\gamma \nabla f(\mathbf{S}\left)\right)\end{array}$$

(6)

Equation 6 suggests that we can solve (5) using a fixed point iteration:

$$\begin{array}{l}{\mathbf{S}}_{k+1}={\text{prox}}_{\mathrm{\gamma r}}({\mathbf{S}}_k-\gamma \nabla f({\mathbf{S}}_k\left)\right)\end{array}$$

(7)

In optimization, (7) is known as forward-backward splitting process [19]. It consists of two steps: first, it performs a forward gradient descend step ${\mathbf{S}}_k^{\ast }={\mathbf{S}}_k-\gamma \nabla f\left({\mathbf{S}}_k\right)$ and then it performs a backward step ${\mathbf{S}}_{k+1}={\text{prox}}_{\mathrm{\gamma r}}\left({\mathbf{S}}_k^{\ast }\right)$.

From (7), we can see the importance of the proximity operator (associated to γ r(S)) with respect to the forward-backward splitting methods, since their main step is to calculate it. If we would have a closed-form expression for such proximity operator or if we could approximate it efficiently (with the approximation errors decreasing at appropriate rates [27]), then we could efficiently solve (7). Furthermore, when f has a Lipschitz continuous gradient, there are fast algorithms to solve (7). For instance, the Iterative Soft Thresholding Algorithm (ISTA) has a convergence rate of O(1/k), and the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) has a convergence rate of O(1/k²) [5].

4 Problem formulation

The regularized EEG inverse problem can be stated as follows:

$$\begin{array}{l}\widehat{\mathbf{S}}=\underset{\mathbf{S}}{\text{argmin}}\left\{\frac{1}{2}\left|\right|\mathbf{\text{AS}}-\mathbf{Y}|{|}_{\mathrm{F}}^2+\mathrm{\lambda \Omega }(\mathbf{S}),\lambda >0\right\},\end{array}$$

(8)

where $\frac{1}{2}\left|\right|\mathbf{AS}-\mathbf{Y}|{|}_{\mathrm{F}}^2$ is the square loss function (∥ ∥_F denotes the Frobenius norm), and λ Ω(S) is a nonsmooth penalty term that is used to encode the prior knowledge about the structure of the target matrix S.

In order to induce structured sparse-low-rank solutions, we propose to reformulate (1) using a matrix factorization approach, which involves expressing the matrix S as the product of two matrices, S=B C, obtaining the following nonlinear estimation model:

$$\begin{array}{l}\mathbf{Y}=\mathbf{\text{ABC}}+\mathbf{E},\end{array}$$

(9)

where B and C are penalized using the ℓ₂₁-norm and the squared Frobenius norm, respectively. The resulting optimization problem can be stated as follows:

$$\begin{array}{l}\begin{array}{ll}\hfill \widehat{\mathbf{B}},\widehat{\mathbf{C}}=\underset{\mathbf{B},C}{\text{argmin}}& \left\{\frac{1}{2}\parallel \mathbf{\text{A(BC)}}-\mathbf{Y}{\parallel }_{\mathrm{F}}^2+\lambda \left(\sum _{i=1}^N\parallel \mathbf{B}(i,:){\parallel }_2\right.\right.\\ +\left.\left.\frac{\rho }{2}\sum _{i=1}^K\parallel \mathbf{C}(i,:){\parallel }_2^2\right)\right\}\\ \hfill =\underset{\mathbf{B},C}{\text{argmin}}& \left\{\frac{1}{2}\parallel \mathbf{\text{A(BC)}}-\mathbf{Y}{\parallel }_{\mathrm{F}}^2+\lambda \left(\parallel \mathbf{B}{\parallel }_{2,1}+\frac{\rho }{2}\parallel \mathbf{C}{\parallel }_{\mathrm{F}}^2\right)\right\},\end{array}\end{array}$$

(10)

where λ>0, ρ>0, $\mathbf{B}\in {\mathbb{R}}^{N\times K}$, $\mathbf{C}\in {\mathbb{R}}^{K\times T}$, and B(i,:), C(i,:) denote the i th row of B and C, respectively. K≪{N,T}, λ, and ρ are parameters of the model that must be adjusted.

In this formulation, which we denote as matrix factorization approach, the ℓ₂₁-norm and the squared Frobenius norm induce structured sparsity and smoothness in the rows of B and C, respectively, and therefore also in the rows of S. Finally, the parameter K encloses the low rank of S:

$$\begin{array}{ll}\hfill \text{rank}\left(\mathbf{B}\right)& \le \text{min}\left\{N,K\right\}\Rightarrow \text{rank}\left(\mathbf{B}\right)\le K\\ \hfill \text{rank}\left(\mathbf{C}\right)& \le \text{min}\left\{K,T\right\}\Rightarrow \text{rank}\left(\mathbf{C}\right)\le K\\ \hfill \text{rank}\left(\mathbf{BC}\right)& \le \text{min}\left\{\text{rank}\left(\mathbf{B}\right),\text{rank}\left(\mathbf{C}\right)\right\}\le K\\ \hfill \Rightarrow \text{rank}\left(\mathbf{S}\right)& \le K\end{array}$$

Hence, the proposed regularization framework takes into account all the prior knowledge about the structure of the target matrix S.

5 Optimization algorithm

5.1 Matrix factorization approach

In this section, we address the issue of implementing the learning method (10) numerically. We propose the following reparameterization of (10):

$$\begin{array}{ll}\mathbf{B}=\sqrt{\mathrm{\lambda \rho }}\tilde{\mathbf{B}},\mathbf{C}=\frac{1}{\sqrt{\mathrm{\lambda \rho }}}\tilde{\mathbf{C}}& \Rightarrow \mathbf{BC}=\left(\sqrt{\mathrm{\lambda \rho }}\tilde{\mathbf{B}}\right)\left(\frac{1}{\sqrt{\mathrm{\lambda \rho }}}\tilde{\mathbf{C}}\right)\\ \Rightarrow \mathbf{BC}=\tilde{\mathbf{B}}\tilde{\mathbf{C}}\end{array}$$

(11)

Using (11) in the objective function of (10), we get

$$\begin{array}{l}\Rightarrow \frac{1}{2}\parallel \mathbf{A}\left(\tilde{\mathbf{B}}\tilde{\mathbf{C}}\right)-\mathbf{Y}{\parallel }_F^2+\lambda \left(\parallel \sqrt{\mathrm{\lambda \rho }}\tilde{\mathbf{B}}{\parallel }_{2,1}+\frac{\rho }{2}\parallel \frac{1}{\sqrt{\mathrm{\lambda \rho }}}\tilde{\mathbf{C}}{\parallel }_F^2\right)\\ \Rightarrow \frac{1}{2}\parallel \mathbf{A}\left(\tilde{\mathbf{B}}\tilde{\mathbf{C}}\right)-\mathbf{Y}{\parallel }_F^2+\lambda \sqrt{\mathrm{\lambda \rho }}\parallel \tilde{\mathbf{B}}{\parallel }_{2,1}+\frac{\mathrm{\lambda \rho }}{2{\left(\sqrt{\mathrm{\lambda \rho }}\right)}^2}\parallel \tilde{\mathbf{C}}{\parallel }_F^2\\ \Rightarrow \frac{1}{2}\parallel \mathbf{A}\left(\tilde{\mathbf{B}}\tilde{\mathbf{C}}\right)-\mathbf{Y}{\parallel }_F^2+\tilde{\lambda }\parallel \tilde{\mathbf{B}}{\parallel }_{2,1}+\frac{1}{2}\parallel \tilde{\mathbf{C}}{\parallel }_F^2\end{array}$$

where $\tilde{\lambda }=\lambda \sqrt{\mathrm{\lambda \rho }}$, and therefore, we get an optimization problem with only one regularization parameter:

$$\begin{array}{cc}\widehat{\mathbf{\text{B}}},\widehat{\mathbf{\text{C}}}& =\underset{\mathbf{\text{B,C}}}{\text{argmin}}\left\{\frac{1}{2}\parallel \mathbf{\text{A(BC)}}-\mathbf{Y}{\parallel }_F^2+\lambda \parallel \mathbf{B}{\parallel }_{2,1}+\frac{1}{2}\parallel \mathbf{C}{\parallel }_F^2,\lambda >0\right\}\end{array}$$

(12)

The optimization problem (12) is a simultaneous minimization over matrices B and C. For a fixed C, the minimum over B can be obtained using FISTA. On the other hand, for a fixed B, the minimum over C can be solved directly in terms of a matrix inversion. These observations suggest an alternating minimization algorithm [15, 28]:

In order to obtain the initialization matrix C ₀ , we use an approach based on the singular value decomposition of Y. Without loss of generality, let us work with (9) in the noiseless case:

$$\begin{array}{l}\mathbf{Y}=\mathbf{\text{ABC}}\end{array}$$

(15)

From (15), we can see that {Y ₁ ,Y ₂ ,…,Y _M }⊂RowSpace(C), where Y _i denotes the i th row of Y.

Now, let us obtain a rank-K approximation of Y by using a truncated SVD (truncated at the singular value σ _K ):

$$\begin{array}{l}\mathbf{Y}\approx {\mathbf{U}}_{M\times K}{\mathbf{\Sigma }}_{K\times K}{\mathbf{V}}_{K\times T}^{\top }\end{array}$$

(16)

From the SVD theory [29], we know that {Y₁,Y ₂ ,…,Y _M }⊂Row Space(V^⊤); therefore, we can choose C ₀ =V^⊤. Then, given C ₀ , we can start iterating using (13) and (14).

5.1.1 Minimization over B(fixed C)

The minimization over B can be stated as follows:

$$\begin{array}{l}{\mathbf{B}}_{\mathbf{t}}=\underset{\mathbf{B}}{\text{argmin}}\left\{F_{\mathbf{B}}\left(\mathbf{B}\right)+\lambda \parallel \mathbf{B}{\parallel }_{2,1},\lambda >0\right\},\end{array}$$

(17)

where $F_{\mathbf{B}}\left(\mathbf{B}\right)=\frac{1}{2}\parallel {\mathbf{\text{A(BC}}}_{\mathbf{\text{t-1}}})-\mathbf{Y}{\parallel }_{\mathrm{F}}^2+\frac{1}{2}\parallel {\mathbf{C}}_{\mathbf{\text{t-1}}}{\parallel }_{\mathrm{F}}^2$. This is a composite convex optimization problem involving the sum of a smooth function (F _B (B)) and a nonsmooth function (λ∥B∥_2,1). As we have seen in Section 3, this kind of problem can be efficiently handled using proximal splitting methods (e.g., FISTA). In order to apply FISTA to solve (17), we first need to compute the following:

• 1. The gradient of the smooth function F _B (B)

  $$\begin{array}{l}\nabla F_{\mathbf{B}}\left(\mathbf{B}\right)=\frac{\partial F_{\mathbf{B}}\left(\mathbf{B}\right)}{\partial \mathbf{B}}={\mathbf{A}}^{\top }\left(\mathbf{A}\right(\mathbf{B}{\mathbf{C}}_{t-1})-\mathbf{Y}){\mathbf{C}}_{t-1}^{\top }\end{array}$$

  where A^⊤ denotes the transpose of the matrix A.

• 2. An upper bound of the Lipschitz constant (L) of ∇F _B (B) (it can also be estimated using a backtracking search routine [5])

  $$\begin{array}{ll}\parallel \nabla F_{\mathbf{B}}\left({\mathbf{B}}_1\right)-\nabla F_{\mathbf{B}}\left({\mathbf{B}}_2\right){\parallel }_2^2& =\parallel {\mathbf{A}}^{\top }\mathbf{A}{\mathbf{B}}_1{\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }\\ -{\mathbf{A}}^{\top }\mathbf{A}{\mathbf{B}}_2{\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }{\parallel }_2^2\\ =\parallel {\mathbf{A}}^{\top }\mathbf{A}({\mathbf{B}}_1-{\mathbf{B}}_2){\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }{\parallel }_2^2\\ =\sum _{j=1}^K\parallel \left({\mathbf{A}}^{\top }\mathbf{A}\left({\mathbf{B}}_1-{\mathbf{B}}_2\right)\right){\left({\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }\right)}_j{\parallel }_2^2\end{array}$$

  where ${\left({\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }\right)}_j$ denotes the j-th column of the matrix ${\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }$. Taking into account that $\parallel \mathbf{Q}x{\parallel }_2\le \parallel |\mathbf{Q}\parallel {|}_2\parallel x{\parallel }_2,\forall x\in {\mathbb{R}}^N,\forall \mathbf{Q}\in {\mathbb{R}}^{M\times N}$[29], where ∥|·∥|₂ denotes the spectral norm, we get:

  $$\begin{array}{ll}\hfill \parallel \nabla F_{\mathbf{B}}\left({\mathbf{B}}_1\right)-\nabla F_{\mathbf{B}}\left({\mathbf{B}}_2\right){\parallel }_2^2& =\sum _{j=1}^K\parallel \left({\mathbf{A}}^{\top }\mathbf{A}\left({\mathbf{B}}_1-{\mathbf{B}}_2\right)\right)\\ \times {\left({\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }\right)}_j{\parallel }_2^2\\ \le \sum _{j=1}^K\parallel |{\mathbf{A}}^{\top }\mathbf{A}\left({\mathbf{B}}_1-{\mathbf{B}}_2\right)\parallel {|}_2^2\\ \times \parallel {\left({\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }\right)}_j{\parallel }_2^2\\ \le \parallel |{\mathbf{A}}^{\top }\mathbf{A}\left({\mathbf{B}}_1-{\mathbf{B}}_2\right)\parallel {|}_2^2\\ \times \sum _{j=1}^K\parallel {\left({\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }\right)}_j{\parallel }_2^2\\ \le \parallel |{\mathbf{A}}^{\top }\mathbf{A}\left({\mathbf{B}}_1-{\mathbf{B}}_2\right)\parallel {|}_2^2\\ \times \parallel {\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }{\parallel }_2^2\end{array}$$

  (18)

  From (18), taking into account that the spectral norm is submultiplicative ($\parallel |\mathbf{\text{PQ}}\parallel {|}_2\le \parallel |\mathbf{P}\parallel {|}_2\parallel |\mathbf{Q}\parallel {|}_2,\forall \mathbf{P}\in {\mathbb{R}}^{M\times N},\forall \mathbf{Q}\in {\mathbb{R}}^{N\times T}$), it follows that:

  $$\begin{array}{rcl}\parallel \nabla F_{\mathbf{B}}\left({\mathbf{B}}_1\right)& -& \nabla F_{\mathbf{B}}\left({\mathbf{B}}_2\right){\parallel }_2^2\le \parallel |{\mathbf{A}}^{\top }\mathbf{A}\parallel {|}_2^2\\ \parallel |{\mathbf{B}}_1& -& {\mathbf{B}}_2\parallel {|}_2^2\parallel |{\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }{\parallel }_2^2\end{array}$$

  and, using the fact that $\parallel |\mathbf{P}\parallel {|}_2^2\le \parallel \mathbf{P}{\parallel }_2^2,\forall \mathbf{P}\in {\mathbb{R}}^{M\times N}$, we obtain:

  $$\begin{array}{ll}\parallel \nabla F_{\mathbf{B}}\left({\mathbf{B}}_1\right)-\nabla F_{\mathbf{B}}\left({\mathbf{B}}_2\right){\parallel }_2^2& \le \parallel |{\mathbf{A}}^{\top }\mathbf{A}\parallel {|}_2^2\parallel {\mathbf{B}}_1-{\mathbf{B}}_2{\parallel }_2^2\\ \times \parallel {\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }{\parallel }_2^2\\ \le L\parallel {\mathbf{B}}_1-{\mathbf{B}}_2{\parallel }_2^2\end{array}$$

  (19)

  where $L=\parallel |{\mathbf{A}}^{\top }\mathbf{A}\parallel {|}_2\parallel {\mathbf{C}}_{t-1}{\mathbf{C}}_{t-1}^{\top }{\parallel }_2$.

• 3. Proximal operator associated to the nonsmooth function λ∥·∥_2,1

  $$\begin{array}{ll}{\text{prox}}_{\lambda \parallel ·{\parallel }_{2,1}}\left(\mathbf{B}\right)& =\underset{\mathbf{X}}{\text{argmin}}\left\{\lambda \parallel \mathbf{X}{\parallel }_{2,1}+\frac{1}{2}\parallel \mathbf{X}-\mathbf{B}{\parallel }_2^2\right\}\\ ={\left[{\left({\text{prox}}_{\lambda \parallel ·{\parallel }_{2,1}}\left(\mathbf{B}\right)\right)}_{i,:}\right]}_{i=1}^{i=N}\\ ={\left[\frac{{\mathbf{B}}_{i,:}}{\parallel {\mathbf{B}}_{i,:}{\parallel }_2}{(\parallel {\mathbf{B}}_{i,:}{\parallel }_2-\lambda )}_{+}\right]}_{i=1}^{i=N}\end{array}$$

  (20)

  where (·)₊= max(·,0), and by convention $\frac{0}{0}=0$.

5.1.2 Minimization over C(fixed B)

The minimization over C can be stated as follows:

$${\mathbf{C}}_{\mathbf{t}}=\underset{\mathbf{C}}{\text{argmin}}\left\{F_{\mathbf{C}}\left(\mathbf{C}\right)\right\}$$

(21)

where $F_{\mathbf{C}}\left(\mathbf{C}\right)=\frac{1}{2}\parallel {\mathbf{\text{A(B}}}_{\mathbf{t}}\mathbf{\text{C)}}-\mathbf{Y}{\parallel }_{\mathrm{F}}^2+\lambda \parallel {\mathbf{B}}_{\mathbf{t}}{\parallel }_{2,1}+\frac{1}{2}\parallel \mathbf{C}{\parallel }_{\mathrm{F}}^2$ is a smooth function of C. In what follows, we show how the minimum over C can be solved directly in terms of a matrix inversion:

$$\begin{array}{ll}\hfill \nabla F_{\mathbf{C}}\left(\mathbf{C}\right)=\frac{\partial F_{\mathbf{C}}\left(\mathbf{C}\right)}{\partial \mathbf{C}}& ={\mathbf{B}}_{\mathbf{t}}^{\top }{\mathbf{A}}^{\top }\left(\mathbf{A}\right({\mathbf{B}}_{\mathbf{t}}\mathbf{C})-\mathbf{Y})+\mathbf{C}\\ \hfill \nabla F_{\mathbf{C}}\left(\mathbf{C}\right)=0& \Rightarrow {\mathbf{B}}_{\mathbf{t}}^{\top }{\mathbf{A}}^{\top }\left(\mathbf{A}\right({\mathbf{B}}_{\mathbf{t}}{\mathbf{C}}_t)-\mathbf{Y})+{\mathbf{C}}_{\mathbf{t}}=0\\ \Rightarrow {\mathbf{C}}_{\mathbf{t}}={\left[{\mathbf{B}}_{\mathbf{t}}^{\top }{\mathbf{A}}^{\top }\mathbf{A}{\mathbf{B}}_{\mathbf{t}}+{\mathbf{I}}_K\right]}^{-1}{\mathbf{B}}_{\mathbf{t}}^{\top }{\mathbf{A}}^{\top }\mathbf{Y}\end{array}$$

(22)

The matrix $\left[{\mathbf{B}}_{\mathbf{t}}^{\top }{\mathbf{A}}^{\top }\mathbf{A}{\mathbf{B}}_{\mathbf{t}}+{\mathbf{I}}_K\right]\in {\mathbb{R}}^{K\times K}$, and K is supposed to be small; therefore, calculating its corresponding inverse matrix is quite cheap.

6 Convergence analysis

We are going to analyze the convergence behavior of Algorithm 1 by using the global convergence theory of iterative algorithms developed by Zangwill [30]. Note that in this theory, the term ‘global convergence’ do not imply convergence to a global optimum for all initial points. The property of global convergence expresses, in a sense, the certainty that the algorithm converges to the solution set. Formally, an iterative algorithm ξ, on the set X, is said to be globally convergent provided, for any starting point x₀∈X, the sequence {x _n } generated by ξ has a limit point [31].

In order to use the global convergence theory of iterative algorithms, we need a formal definition of iterative algorithm, as well as the definition of a set-valued mapping (a.k.a point-to-set mapping) [30]:

Definition 6.1

Set-valued mapping. Given two sets, X and Y, a set-valued mapping defined on X, with range in the power set of Y, $\mathcal{P}\left(Y\right)$, is a map, Φ, which assigns to each x∈X a subset $\Phi \left(x\right)\in \mathcal{P}\left(Y\right)$,

$$\Phi :X\to \mathcal{P}\left(Y\right)$$

Definition 6.2

Iterative algorithm. Let X be a set and x₀∈X a given point. Then, an iterative algorithm ξ, with initial point x₀, is a set-valued mapping

$$\xi :X\to \mathcal{P}\left(X\right)$$

which generates a sequence ${\left\{x_n\right\}}_{n=1}^{\infty }$ via the rule x_n+1∈ξ(x _n ), n=0,1,…

Now that we know the main building blocks of the global convergence theory of iterative algorithms, we are in a position to state the convergence theorem related to Algorithm 1:

Theorem 6.1

Let Φ denotes the iterative Algorithm 1, and suppose that given $\mathbf{Y}\in {\mathbb{R}}^{M\times T}$, $\mathbf{A}\in {\mathbb{R}}^{M\times N}$, ${\mathbf{B}}_{\mathbf{0}}\in {\mathbb{R}}^{N\times K}$, ${\mathbf{C}}_{\mathbf{0}}\in {\mathbb{R}}^{K\times T}$, K, and λ, the sequence ${\left\{{\mathbf{B}}_{\mathbf{t}},{\mathbf{C}}_{\mathbf{t}}\right\}}_{t=1}^{\infty }$ is generated and satisfies {B _t+1 ,C _t+1 }∈Φ(B _t ,C _t ). Also, let Ω _B and Ω _C denote the solution sets of (13) and (14), respectively:

$$\begin{array}{ll}{\Omega }_B& =\left\{\mathbf{B}\in {\mathbb{R}}^{N\times K}\left|0\in \partial \right.\left(\frac{1}{2}\left|\right|{\mathbf{\text{A(BC}}}_{\mathbf{\text{t-1}}})-\mathbf{Y}|{|}_F^2+\lambda |\left|\mathbf{B}\right|{|}_{2,1}\right.\right.\\ +\left.\left.\frac{1}{2}\left|\right|{\mathbf{C}}_{\mathbf{\text{t-1}}}|{|}_F^2\right)\right\}\\ {\Omega }_C& =\left\{\mathbf{C}\in {\mathbb{R}}^{K\times T}\left|\nabla \right.\left(\frac{1}{2}\left|\right|{\mathbf{\text{A(B}}}_{\mathbf{t}}\mathbf{\text{C)}}-\mathbf{Y}|{|}_F^2+\lambda |\left|{\mathbf{B}}_{\mathbf{t}}\right|{|}_{2,1}\frac{1}{2}\left|\right|\mathbf{C}|{|}_F^2\right)=0\right\}\end{array}$$

Then, the limit of any convergent subsequence of ${\left\{{\mathbf{B}}_{\mathbf{t}},{\mathbf{C}}_{\mathbf{t}}\right\}}_{t=1}^{\infty }$ is in Ω _B and Ω _C .

This convergence theorem is a direct application of Zangwill’s global convergence theorem [30]. Before going in this assertion, let us show some definitions and theorems used in the proof.

Definition 6.3

Compact set. A set X is said to be compact if any sequence (or subsequence) contains a convergent subsequence whose limit is in X. More explicitly, given a subsequence ${\left\{x_n\right\}}_{n\in \hat{N}}$ in X, there exists a $\widehat{N_1}\subset \hat{N}$ such that

$$x_n\to x_{\infty },n\in \widehat{N_1}$$

with x _∞ ∈X (we write convergence of subsequences as x _n →x _∞ , which is equivalent to $\underset{n\to \infty }{\text{lim}}{x}_n=x_{\infty }$).

Definition 6.4

Composite map. Let Π _A :X→Y and Π _B :Y→Z be two set-valued mappings. The composite map Π _C =Π _B ∘Π _A which takes points x∈X to sets Π _C (x)⊂Z is defined by

$${\Pi }_C\left(x\right):=\bigcup _{y\in {\Pi }_A\left(x\right)}{\Pi }_B\left(y\right)$$

Definition 6.5

Closed map. A set-valued mapping $\Pi :X\to \mathcal{P}\left(Y\right)$ is closed at x₀∈X provided

• 1. x _n →x ₀ a s n→∞, x _n ∈X

• 2. y _n →y ₀ a s n→∞, y _n , y ₀∈Y

• 3. y _n ∈Π(x _n )

implies y₀∈Π(x₀). The map Π is called closed on S⊂X provided is closed at each x∈S.

Theorem 6.2

Composition of closed maps. Let Π _A :X→Y and Π _B :Y→Z be two set-valued mappings. Suppose

• 1. Π _A is closed at x ₀

• 2. Π _B is closed on Π _A (x ₀)

• 3. if x _n →x ₀ and y _n ∈Π _A (x _n ), then there exists y ₀∈Y, such that for some sequence $\left\{y_{n_j}\right\}$, $y_{n_j}\to y_0$ as j→∞.

Then, the composite map Π _C =Π _B ∘Π _A is closed at x₀.

Lemma 6.1

[32] Given a real-valued function defined on X×Y, define the set-valued mapping $\Psi :X\to \mathcal{P}\left(Y\right)$ by

$$\Psi \left(x\right)=\underset{y\in Y}{\text{argmin}}h(x,y)$$

then, Ψ is closed at x if Ψ(x) is nonempty.

Theorem 6.3

Weierstrass theorem. If f is a real continuous function on a compact set $S\subset {\mathbb{R}}^n$, then the problem

$$\underset{x\in {\mathbb{R}}^n}{\text{argmin}}\left\{f\left(x\right),x\in S\right\}$$

has an optimal solution x^∗∈S.

Theorem 6.4

[30] Zangwill’s global convergence theorem. Let the set-valued mapping $M_x\left(x\right):X\to \mathcal{P}\left(X\right)$ determine an algorithm that given a point x₀ generates a sequence ${\left\{x_n\right\}}_{n=0}^{\infty }$ through the iteration x_n+1∈M _x (x _n ). Also, let a solution set Γ be given. Suppose

• 1. All point x _n are in a compact set S⊂X.

• 2. There is a continuous function $\alpha :X\to \mathbb{R}$ such that

  • (a) if x∉Γ, then α(x ^′)<α(x)∀x ^′∈M _x (x).

  • (b) if x∈Γ, then α(x ^′)≤α(x)∀x ^′∈M _x (x).

• 3. The map M _x (x) is closed at x if x∉Γ.

Then, the limit of any convergent subsequence of ${\left\{x_n\right\}}_{n=0}^{\infty }$ is in Γ. That is, accumulation points x^∗ of the sequence x _n lie in Γ. Furthermore, α(x _n ) converges to α^∗, and α(x^∗)= α^∗ for all accumulation points x^∗.

Proof

Theorem 6.1. The iterative algorithm Φ can be decomposed into two well-defined iterative algorithms Φ _B and Φ _C :

$$\begin{array}{ll}{\Phi }_B\left({\mathbf{C}}_{\mathbf{\text{t-1}}}\right)={\mathbf{B}}_{\mathbf{t}}& =\underset{\mathbf{B}}{\text{argmin}}\left\{\frac{1}{2}\left|\right|{\mathbf{\text{A(BC}}}_{\mathbf{\text{t-1}}})-\mathbf{Y}|{|}_{\mathrm{F}}^2+\lambda |\left|\mathbf{B}\right|{|}_{2,1}\right.\\ +\left.\frac{1}{2}\left|\right|{\mathbf{C}}_{\mathbf{\text{t-1}}}|{|}_{\mathrm{F}}^2\right\}\end{array}$$

(23)

$$\begin{array}{ll}{\Phi }_C\left({\mathbf{B}}_{\mathbf{t}}\right)={\mathbf{C}}_{\mathbf{t}}& =\underset{\mathbf{C}}{\text{argmin}}\left\{\frac{1}{2}\left|\right|{\mathbf{\text{A(B}}}_{\mathbf{t}}\mathbf{\text{C)}}-\mathbf{Y}|{|}_{\mathrm{F}}^2\right.\\ +\left.\lambda \left|\right|{\mathbf{B}}_{\mathbf{t}}|{|}_{2,1}+\frac{1}{2}|\left|\mathbf{C}\right|{|}_{\mathrm{F}}^2\right\}\end{array}$$

(24)

As we can see from (23) and (24), at iteration t, the result of Φ _B becomes the input of Φ _C , and at iteration t+1, the result of Φ _C becomes the input of Φ _B ; therefore, we can express Φ as the composition of Φ _C and Φ _B , that is, Φ(C _t-1 )= Φ _C (Φ _B (C _t-1 )):

$$\begin{array}{ll}{\Phi }_C\left({\Phi }_B\right({\mathbf{C}}_{\mathbf{\text{t-1}}}\left)\right)={\mathbf{C}}_{\mathbf{t}}& =\underset{\mathbf{C}}{\text{argmin}}\left\{\frac{1}{2}\left|\right|{\mathbf{\text{A(B}}}_{\mathbf{t}}\mathbf{\text{C)}}-\mathbf{Y}|{|}_{\mathrm{F}}^2\right.\\ +\left.\lambda \left|\right|{\mathbf{B}}_{\mathbf{t}}|{|}_{2,1}+\frac{1}{2}|\left|\mathbf{C}\right|{|}_{\mathrm{F}}^2\right\}\end{array}$$

(25)

$$\begin{array}{ll}\mathit{\text{subject}}\mathit{\text{to}}{\mathbf{B}}_{\mathbf{t}}& =\underset{\mathbf{B}}{\text{argmin}}\left\{\frac{1}{2}\left|\right|{\mathbf{\text{A(BC}}}_{\mathbf{\text{t-1}}})-\mathbf{Y}|{|}_{\mathrm{F}}^2\right.\\ +\left.\lambda \left|\right|\mathbf{B}|{|}_{2,1}+\frac{1}{2}|\left|{\mathbf{C}}_{\mathbf{\text{t-1}}}\right|{|}_{\mathrm{F}}^2\right\}\end{array}$$

Let Γ be the solution set of Φ

$$\Gamma =\left\{\mathbf{C}\in {\mathbb{R}}^{K\times T}\left|\right.\frac{\mathrm{\partial Z}(\mathbf{C},t)}{\partial \mathbf{C}}=0\right\},$$

where $Z(\mathbf{C},t)=\frac{1}{2}\left|\right|{\mathbf{\text{A(B}}}_{\mathbf{t}}\mathbf{\text{C)}}-\mathbf{Y}|{|}_{\mathrm{F}}^2+\lambda |\left|{\mathbf{B}}_{\mathbf{t}}\right|{|}_{2,1}+\frac{1}{2}\left|\right|\mathbf{C}|{|}_{\mathrm{F}}^2$.

To prove this theorem by using Zangwill’s global convergence theorem, we need to prove that all its corresponding assumptions are fulfilled. In order to prove assumption 1, let us analyze the sequences ${\left\{{\mathbf{B}}_{\mathbf{t}}\right\}}_{t=1}^{\infty }$ and ${\left\{{\mathbf{C}}_{\mathbf{t}}\right\}}_{t=1}^{\infty }$. The sequence ${\left\{{\mathbf{B}}_{\mathbf{t}}\right\}}_{t=1}^{\infty }$ is generated by using FISTA, which is a convergent algorithm (B _t →B _∞ ) that guarantees that B _t ∈Ω _B [5, 18]. Hence, using Definition 6.3, we can see that the sequence ${\left\{{\mathbf{B}}_{\mathbf{t}}\right\}}_{t=1}^{\infty }$ generated by (23) lies in a compact set. On the other hand, the sequence ${\left\{{\mathbf{C}}_{\mathbf{t}}\right\}}_{t=1}^{\infty }$ is generated by (22), which guarantees that C _t ∈Ω _C . This sequence always converges to a point inside Ω _C , which implies that Ω _C also lies in a compact set. This concludes the proof of assumption 1.

To prove assumption 2, let us use Z(C,t) as the function α(·); thus, in order to verify the fulfillment of assumption 2, we need to prove that

• (a) if C _t ∉Γ, then Z(C _t+1 ,t+1)<Z(C _t ,t) ∀C _t+1 ∈Φ(C _t )

• (b) if C _t ∈Γ, then Z(C _t+1 ,t+1)≤Z(C _t ,t) ∀C _t+1 ∈Φ(C _t )

From (25), we can see that the sequence ${\left\{{\mathbf{C}}_{\mathbf{t}}\right\}}_{t=1}^{\infty }$ will always lie in Γ (because C _t is generated by (22)); therefore, we only need to prove (b).

Let C _t+1 be the solution of (25) at iteration t+1; this implies

$$\begin{array}{l}\begin{array}{ll}\frac{1}{2}\left|\right|{\mathbf{\text{A(B}}}_{\mathbf{\text{t+1}}}{\mathbf{C}}_{\mathbf{\text{t+1}}})-\mathbf{Y}|{|}_{\mathrm{F}}^2& +\lambda \left|\right|{\mathbf{B}}_{\mathbf{\text{t+1}}}|{|}_{2,1}+\frac{1}{2}|\left|{\mathbf{C}}_{\mathbf{\text{t+1}}}\right|{|}_{\mathrm{F}}^2\\ \le \frac{1}{2}\left|\right|{\mathbf{\text{A(B}}}_{\mathbf{\text{t+1}}}\mathbf{\text{C)}}-\mathbf{Y}|{|}_{\mathrm{F}}^2+\lambda |\left|{\mathbf{B}}_{\mathbf{\text{t+1}}}\right|{|}_{2,1}\\ +\frac{1}{2}\left|\right|\mathbf{C}|{|}_{\mathrm{F}}^2,\forall \mathbf{C}\in {\mathbb{R}}^{K\times T}\\ \le \frac{1}{2}\left|\right|{\mathbf{\text{A(B}}}_{\mathbf{\text{t+1}}}{\mathbf{C}}_{\mathbf{t}})-\mathbf{Y}|{|}_{\mathrm{F}}^2+\lambda |\left|{\mathbf{B}}_{\mathbf{\text{t+1}}}\right|{|}_{2,1}\\ +\frac{1}{2}\left|\right|{\mathbf{C}}_{\mathbf{t}}|{|}_{\mathrm{F}}^2\end{array}\end{array}$$

(26)

On the other hand, if B _t+1 is the solution of (23) at iteration t+1, this implies

$$\begin{array}{ll}\frac{1}{2}\left|\right|{\mathbf{\text{A(B}}}_{\mathbf{\text{t+1}}}{\mathbf{C}}_{\mathbf{t}})-\mathbf{Y}|{|}_{\mathrm{F}}^2& +\lambda \left|\right|{\mathbf{B}}_{\mathbf{\text{t+1}}}|{|}_{2,1}+\frac{1}{2}|\left|{\mathbf{C}}_{\mathbf{t}}\right|{|}_{\mathrm{F}}^2\\ \le \frac{1}{2}\left|\right|{\mathbf{\text{A(BC}}}_{\mathbf{t}})-\mathbf{Y}|{|}_{\mathrm{F}}^2+\lambda |\left|\mathbf{B}\right|{|}_{2,1}\\ +\frac{1}{2}\left|\right|{\mathbf{C}}_{\mathbf{t}}|{|}_{\mathrm{F}}^2,\forall \mathbf{B}\in {\mathbb{R}}^{N\times K}\\ \le \frac{1}{2}\left|\right|{\mathbf{\text{A(B}}}_{\mathbf{t}}{\mathbf{C}}_{\mathbf{t}})-\mathbf{Y}|{|}_{\mathrm{F}}^2+\lambda |\left|{\mathbf{B}}_{\mathbf{t}}\right|{|}_{2,1}\\ +\frac{1}{2}\left|\right|{\mathbf{C}}_{\mathbf{t}}|{|}_{\mathrm{F}}^2\end{array}$$

(27)

and from (26) and (27), we can prove assumption 2(b):

$$\begin{array}{l}\begin{array}{ll}\hfill \frac{1}{2}\left|\right|{\mathbf{\text{A(B}}}_{\mathbf{\text{t+1}}}{\mathbf{C}}_{\mathbf{\text{t+1}}})-\mathbf{Y}|{|}_{\mathrm{F}}^2& +\lambda \left|\right|{\mathbf{B}}_{\mathbf{\text{t+1}}}|{|}_{2,1}+\frac{1}{2}|\left|{\mathbf{C}}_{\mathbf{\text{t+1}}}\right|{|}_{\mathrm{F}}^2\\ \le \frac{1}{2}\left|\right|{\mathbf{\text{A(B}}}_{\mathbf{t}}{\mathbf{C}}_{\mathbf{t}})-\mathbf{Y}|{|}_{\mathrm{F}}^2+\lambda |\left|{\mathbf{B}}_{\mathbf{t}}\right|{|}_{2,1}\\ +\frac{1}{2}\left|\right|{\mathbf{C}}_{\mathbf{t}}|{|}_{\mathrm{F}}^2\\ \hfill Z({\mathbf{C}}_{\mathbf{\text{t+1}}},t+1)& \le Z({\mathbf{C}}_{\mathbf{t}},t)\end{array}\end{array}$$

In order to prove assumption 3, we need to prove that Φ is closed at C if C∉Γ. To do so, we are going to use Theorem 6.2; therefore, we need to prove that Φ _B and Φ _C are both closed maps: from (23) and (24), we can see that their corresponding objective functions are both continuous $\forall \mathbf{B}\in {\mathbb{R}}^{N\times K}$ and $\forall \mathbf{C}\in {\mathbb{R}}^{K\times T}$, respectively; hence, by using Weierstrass Theorem and Lemma 6.1, we can conclude that Φ _B and Φ _C are both closed maps for any C _t-1 and B _t , respectively, and by using Theorem 6.2, we can conclude that Φ is closed on any C _t-1 .

Finally, from all the previous proofs and Zangwill’s global convergence theorem, it follows that the limit of any convergent subsequence of ${\left\{{\mathbf{B}}_{\mathbf{t}},{\mathbf{C}}_{\mathbf{t}}\right\}}_{t=1}^{\infty }$ is in Ω _B and Ω _C .

7 Numerical experiments

In this section, we evaluate the performance of the matrix factorization approach and compare it with the Group Lasso regularizer:

$$\begin{array}{c}\widehat{\mathbf{S}}=\underset{\mathbf{S}}{\text{argmin}}\left\{\frac{1}{2}\parallel \mathbf{\text{AS}}-\mathbf{Y}{\parallel }_F^2+\lambda \sum _{i=1}^N\parallel \mathbf{S}(i,:){\parallel }_2,\lambda >0\right\}\end{array}$$

(28)

and the Trace Norm regularizer:

$$\begin{array}{cc}\widehat{\mathbf{S}}& =\underset{\mathbf{S}}{\text{argmin}}\left\{\frac{1}{2}\parallel \mathbf{\text{AS}}-\mathbf{Y}{\parallel }_F^2+\lambda \sum _{i=1}^q{\sigma }_i\left(\mathbf{S}\right),\lambda >0\right\}\end{array}$$

(29)

where q=min{N,T} and σ _i (S) denotes the i th singular value of S. Both problems (28) and (29) were solved using the FISTA implementation of the SPArse Modeling Software (SPAMS) [33, 34].

In order to have a reproducible comparison of the different regularization approaches, we generated two synthetic scenarios:

• M=128 EEG electrodes, T=161 time instants, N=413 current sources within the brain, but only 12 of them are active: 4 main active sources with their corresponding 2 nearest neighbor sources are also active. The other 401 sources are not active (zero electrical activity). Therefore, in this scenario, the synthetic matrix S is a structured sparse matrix with only 12 nonzero rows (the rows associated to the active sources).

• M=128 EEG electrodes, T=161 time instants, N=2,052 current sources within the brain, but only 40 of them are active: 4 main active sources with their corresponding 9 nearest neighbor sources are also active. The other 2012 sources are not active (zero electrical activity). Therefore, in this scenario, the synthetic matrix S is a structured sparse matrix with only 40 nonzero rows (the rows associated to the active sources).

In both scenarios, the simulated electrical activity (simulated waveforms) associated to the four Main Active Sources (MAS) was obtained from a face perception-evoked potential study [35, 36]. To obtain the simulated electrical activity associated to each one of the active neighbor sources, we simply set it as a scaled version of the electrical activity of its corresponding nearest MAS (with a scaled factor equal to 0.5). Hence, there is a linear relation between the four MAS and their corresponding nearest neighbor sources; therefore, in both scenarios, the rank of the synthetic matrix S is equal to 4.

As forward model (A), we used a three-shell concentric spherical head model. In this model, the inner sphere represents the brain, the intermediate layer represents the skull, and the outer layer represents the scalp [37]. To obtain the values of each one of the components of the matrix A, we need to solve the EEG forward problem [38]: Given the electrical activity of the current sources within the brain and a model for the geometry of the conducting media (brain, skull and scalp, with its corresponding electric properties), compute the resulting EEG signals. This problem was solved by using the SPM software [39]. Taking into account the comments mentioned in Section 2, the N simulated current sources were positioned on a mesh located on the brain cortex, with an orientation fixed perpendicular to it.

Finally, the simulated EEG signals were generated according to (1), where E is a Gaussian noise G(0,σ²I) whose variance was set to satisfy a $\text{SNR}=20\underset{10}{log}\left(\frac{\left|\right|\mathbf{\text{AS}}|{|}_{\mathrm{F}}}{\left|\right|\mathbf{E}|{|}_{\mathrm{F}}}\right)=10\text{dB}$. Summarizing, our synthetic problems can be stated as follows: Given matrices $\mathbf{Y}\in {\mathbb{R}}^{128\times 161}$ and $\mathbf{A}\in {\mathbb{R}}^{128\times N}$, recover the synthetic BES matrix $\mathbf{S}\in {\mathbb{R}}^{N\times 161}$. According to this, in both scenarios, we want to estimate a BES matrix which is structured sparse and low rank, with its rank equal to the number of MAS simulated. The activity of the four MAS, the synthetic EEG measurements as well as the sparsity pattern of the synthetic BES matrix are shown in Figures 1 and 2 (Ground Truth).

Figure 1

Simulation results: waveforms of the MAS, EEG estimated, and sparsity pattern of the estimated BES matrix. Experiment setup: 413 sources, 128 EEG electrodes, 161 time instants, 4 main active sources with their corresponding 2 nearest neighbor sources also active.

Figure 2

Simulation results: waveforms of the MAS, EEG estimated, and sparsity pattern of the estimated BES matrix. Experiment setup: 2,052 sources, 128 EEG electrodes, 161 time instants, 4 main active sources with their corresponding 9 nearest neighbor sources also active.

We have used cross-validation to select the regularization parameter λ associated to the Group Lasso and Trace Norm regularizers, as well as the parameters λ and K in the case of the Matrix Factorization approach (K∈ [ 1,2,3,…,10], λ∈ [ 10⁻³,10⁻²,10⁻¹,…,10³]): the rows of Y are randomly partitioned into three groups of approximately equal size. Each union of two groups forms a train set (TrS), while the remaining group forms a test set (TS). This procedure is carried out three times, each time selecting a different test group. Inverse reconstructions are carried out based on the training sets, obtaining different regression matrices ${\widehat{\mathbf{S}}}_i$. We then evaluate the root mean square error (RMSE) using the test sets and the regression matrices ${\widehat{\mathbf{S}}}_i$:

$$\begin{array}{c}\mathit{\text{RMSE}}:\frac{1}{3}\sum _{i=1}^3\left(\frac{1}{\sqrt{M_{{\mathit{\text{TS}}}_i}\times T}}\parallel {\mathbf{A}}_{{\mathit{\text{TS}}}_i}{\widehat{\mathbf{S}}}_i-{\mathbf{Y}}_{{\mathit{\text{TS}}}_i}{\parallel }_F\right),\end{array}$$

where ${\mathbf{Y}}_{{\text{TS}}_i}\in {\mathbb{R}}^{M_{{\text{TS}}_i}\times T}$, and ${\mathbf{A}}_{{\text{TS}}_i}\in {\mathbb{R}}^{M_{{\text{TS}}_i}\times N}$ (TS _i denotes the index set of the rows that belongs to the i th test set). Once the estimated matrix $\widehat{\mathbf{S}}$ has been found, we apply a threshold to remove spurious sources with almost zero activity. We have set this threshold equal to the 1% of the mean energy of all the sources.

7.1 Performance evaluation

In order to evaluate the performance of the regularizers, we compare the waveform and localization of the four MAS present in the synthetic BES matrix against the four MAS estimated by each one of the regularizers. We also compare the sparsity pattern of the estimated BES matrix $\widehat{\mathbf{S}}$ against the sparsity pattern of the synthetic BES matrix S, as well as the synthetic and predicted EEG measurements.

As we can see from Figures 1 and 2, the Group Lasso and Trace Norm regularizers do not reveal the correct number of linear independent sources, while the Matrix Factorization does: it finds out four linear independent sources in both scenarios. To select such four linear independent MAS, we find a basis for the $\text{Column Space}\left({\tilde{\mathbf{S}}}^{\top }\right)$ (using a QR factorization), where $\tilde{\mathbf{S}}$ is a matrix whose rows are a sorted version of the rows of S (sorted in a descending order of their corresponding energy value). To get the four linear independent MAS estimated by the Group Lasso and Trace Norm regularizers, we followed the same procedure described before and retained the first four components of the basis of the $\text{Column Space}\left({\tilde{\mathbf{S}}}^{\top }\right)$.According to Figures 1 and 2, the Matrix Factorization approach is able to estimate a BES matrix with the correct rank and whose sparsity pattern follows closely the sparsity pattern of the true BES matrix, that is, both matrices have a similar structure, which implies that the proposed approach is able to induce the desired solution: A row-structured sparse matrix, whose nonzero rows encode the linear relation between the active sources and their corresponding nearest neighbor sources. Using the estimated BES matrix, the Matrix Factorization approach is also able to predict a smooth version of the noisy EEG, and the waveforms of the estimated MAS follow closely the waveforms of the true MAS.As we can see from Figures 1 and 2, Group Lasso is able to estimate a BES matrix with a similar row-sparsity pattern to the true BES matrix, but it does not take into account the linear relation between the nonzero rows, which can be seen from the rank of the estimated BES matrix. The waveforms of the estimated MAS are very similar to the true MAS, but they are not so smooth as the ones estimated by the Matrix Factorization approach.As we can see from Figures 1 and 2, the Trace Norm regularizer takes into account the linear relation of the active sources by inducing solutions which are low rank, but, on the other hand, it does not take into account the structured sparsity pattern of the BES matrix. All of this implies that the Trace Norm tends to induce low rank dense solutions, which are not biologically plausible.According to Figures 3 and 4, the position of the MAS obtained from the BES matrix estimated by the Matrix Factorization approach, the Group Lasso, and Trace Norm regularizers follows closely the position of the true MAS. Nevertheless, it is worth highlighting that before selecting the MAS, we first need an accurate estimation of their number, and the Group Lasso and Trace Norm regularizers were not able to get a precise estimate of it, only the Matrix Factorization were able to.

Figure 3

Localization of the MAS, N=413 sources. From left to right: Ground Truth, Matrix Factorization, Group Lasso, and Trace Norm.

Figure 4

Localization of the MAS, N=2,052 sources. From left to right: Ground Truth, Matrix Factorization, Group Lasso, and Trace Norm.

From these results, we can see that the proposed Matrix Factorization approach outperforms both the Group Lasso and Trace Norm regularizers. The main reason for this is because it combines their two main features: it combines the structured sparsity (from Group Lasso) and the low rank (from Trace Norm) into one unified framework, which implies that it is able to induce structured sparse-low-rank solutions which are biologically plausible: few active sources, with linear relations between them.

8 Conclusions

We have presented a novel approach to solve the EEG inverse problem, which is based on matrix factorization and regularization. Our method combines the ideas behind the Group Lasso (structured sparsity) and Trace Norm (low rank) regularizers into one unified framework. We have also developed and analyzed the convergence of an alternating minimization algorithm to solve the resulting nonsmooth-nonconvex regularization problem. Finally, using simulation studies, we have compared our method with the Group Lasso and Trace Norm regularizers when they are applied directly to the target matrix, and we have shown the gain in performance obtained by our method, hence proving the effectiveness and efficiency of the proposed algorithm.