# Identification of MIMO systems with sparse transfer function coefficients

- Wanzhi Qiu
^{1}Email author, - Syed Khusro Saleem
^{1}and - Efstratios Skafidas
^{1}

**2012**:104

https://doi.org/10.1186/1687-6180-2012-104

© Qiu et al; licensee Springer. 2012

**Received: **9 January 2012

**Accepted: **9 May 2012

**Published: **9 May 2012

## Abstract

We study the problem of estimating transfer functions of multivariable (multiple-input multiple-output--MIMO) systems with sparse coefficients. We note that subspace identification methods are powerful and convenient tools in dealing with MIMO systems since they neither require nonlinear optimization nor impose any canonical form on the systems. However, subspace-based methods are inefficient for systems with sparse transfer function coefficients since they work on state space models. We propose a two-step algorithm where the first step identifies the system order using the subspace principle in a state space format, while the second step estimates coefficients of the transfer functions via L1-norm convex optimization. The proposed algorithm retains good features of subspace methods with improved noise-robustness for sparse systems.

### Keywords

system identification MIMO system sparse representation L1-norm optimization## 1. Introduction

The problem of identifying multiple-input multiple-output (MIMO) systems arises naturally in spatial division multiple access architectures for wireless communications. Subspace system identification methods refer to the category of methods which obtain state space models from subspaces of certain matrices constructed from the input-output data [1]. Being based on reliable numerical algorithms such as the singular value decomposition (SVD), subspace methods do not require nonlinear optimization and, thereby, are computationally efficient and stable without suffering from convergence problems. They are particularly suitable for identifying MIMO systems since there is no need to impose on the system a canonical form and, therefore, they are free from the various inconveniences encountered in classical parametric methods.

Another good feature of subspace methods is they incorporate a reliable order estimation process. Although this is largely ignored by other identification methods, system order estimation should be an integral part of a system identification algorithm. In fact, correctly identifying the system order is essential to ensure that subsequent parameter estimation will yield a well-defined set of model coefficient estimates. While it is obvious that an underestimated order will result in large modeling errors, it is equally dangerous to have an over-parameterized model as a result of selecting an improperly large order. Over-parameterization not only creates a larger set of parameters to be estimated, but also leads to poorly defined (high variance) coefficient estimates and surplus unvalidated content in the resulting model [2]. For parameterized linear models, a usual way of estimating the order is to conduct a series of tests on different orders, and select the best one based on the goodness-of-fit using certain criterion such as the Akaike's information criterion [3]. Subspace-based order identification determines the order as the number of principle eigenvalues (or singular values) of a certain input-output data matrix. This mechanism has been proven to be a simple and reliable way of order estimation and the same principle has been applied to detect the number of emitter sources in array signal processing [4], to determine the number of principal components in signal and image analysis [5, 6] and to estimate the system order in blind system identification [7, 8].

It has become well known that many systems in real applications actually have sparse representations, i.e., with a large portion of their transfer function coefficients equal to zero [9]. For example, communication channels exhibit great sparseness. In particular, in high-definition television, there are few echoes but the channel response spans many hundreds of data symbols [10–12]. In broadband wireless communications, a hilly terrain delay profile consists of a sparsely distributed multipath [13]. Underwater acoustic channels also exhibit sparseness [14].

Despite of their good features, subspace methods are not suited for systems with sparse transfer function coefficients since they deal with state space models. One fact is that for a given transfer function matrix, there exist infinite number of state space representations related by a similarity transform. As a result, when the system to be identified has sparse transfer function coefficients, the state space model produced by a subspace method is almost certain to be non-sparse due to the inherent arbitrary similarity transform associated with the model.

Work on sparse system identification has been reported in [15–17], however, they consider only single-input single-out (SISO) systems. In addition, these methods assume the system order is known *a priori*, which is not necessarily true in practice. This article studies the problem of the identification of MIMO systems with sparse transfer functions coefficients. In particular, we leverage on reliable system order identification of subspace methods to build input-output relationship in terms of transfer function coefficients, and exploit L1-norm optimization proven to be able to produce robust sparse solutions [9] to estimate these coefficients. The resulting method consists of a systematic way of order identification and efficient coefficient estimation for sparse systems.

The rest of this article is organized as follows. Section 2 gives an overview of subspace identification methods. Section 3 proposes the LRL1 algorithm. Section 4 presents simulation results and Section 5 draws conclusions.

## 2. Subspace identification

*M*-input {

*u*

_{ m }(

*k*),

*m*= 1,...,

*M*}

*L*-output {

*y*

_{ l }(

*k*),

*l*= 1,...,

*L*} system described in the state space form:

where ${\mathbf{u}}_{k}\stackrel{\text{def}}{=}{\left[{u}_{1}\left(k\right),\dots ,{u}_{M}\left(k\right)\right]}^{T}$, ${\mathbf{y}}_{k}\stackrel{\text{def}}{=}{\left[{y}_{1}\left(k\right),\dots ,{y}_{L}\left(k\right)\right]}^{T},$ and ${\mathbf{x}}_{k}\stackrel{\text{def}}{=}{\left[{x}_{1}\left(k\right),\dots ,{x}_{N}\left(k\right)\right]}^{T}$ are the input, output, and state vectors, respectively, with *T* denoting matrix transpose. **A** ∈ *R*^{N×N}, **B** ∈ *R*^{N×M}, **C** ∈ *R*^{L×N}, and **D** ∈ *R*^{L×M}are the system, input, output, and direct feed-through matrices, respectively. Noise in (1) has been omitted for brevity purposes and will be added in the simulation in Section 4. Given *S* measurements (i.e., *k* = 0,..., *S* - 1) of the input and output, subspace identification methods estimate the system order *N* and matrices (**A, B, C, D**), and derive system transfer functions via **H**(z) = **D** + **C**(z **I**_{
NN
}- **A**)^{-1}**B**, with **I**_{
NN
} denoting the identity matrix of size *N* × *N*. The procedure is as follows.

*i*and define the input and output block Hankel matrices as:

**U**

_{0/i-1}has

*j*(=

*S*-

*i*+ 2) columns with the first one formed by

**u**

_{ k }(

*k*= 0,...,

*i*- 1). Similarly, we can define the

*j*-column block Hankel matrix

**U**

_{i/2i-1}using

**u**

_{ k }(

*k*=

*i*,..., 2

*i*- 1) to form the first column. For convenience, the following short-hand notations are adopted:

*i*is a user-specified parameter and is required to be larger than the system order, i.e.,

*i*>

*N*. In general, a subspace identification algorithm consists of three steps. The first step performs projections of the row spaces of the data matrices and estimates the order of the system. In particular, the following oblique projections are first calculated:

**Y**

_{ f }along the row space of

**U**

_{ f }on the row space of

**W**

_{ p }and can be calculated according to

with $\left[{\mathbf{Y}}_{f}/{\mathbf{U}}_{f}^{T}\right]$ denoting the projection of the row space of **Y**_{
f
} on the row space of the orthogonal compliment of the row space of **U**_{
f
}[1].

*N*) is determined as the number of principal singular vales in

**Σ**, leading to the identification of the principal subspaces

**U**

_{1}and

**V**

_{1},

**W**

_{1}and

**W**

_{2}are weighting matrices and a specific choice of them leads to different algorithms [1]. For example,

result in the popular N4SID and MOESP algorithms, respectively, with † denoting Moore-Penrose pseudo-inverse.

**A**,

**B**,

**C**, and

**D**(up to within a similarity transformation) by

Note that the sizes of matrices involved in the subspace methods are dependant on *i*. It will be shown in Section 4 that while the order can be estimated reliably across a broad range of *i*'s as long as the condition *i* > *N* is satisfied, the quality of subsequent coefficient estimates depends on *i* in a non-monotonous fashion.

## 3. The proposed LRL1

**A**,

**B**,

**C**,

**D**). In order to exploit the sparseness in the system, system (1) is represented using transfer function coefficients. In particular, for the

*l*th output the following linear regression (LR) relationship holds:

*a*

_{ n },

*b*

_{ mn, l }} are the system transfer function coefficients. Equation (10) can be re-written into a vector form:

*m*th input to the

*l*th output is presented by

**β**to be obtained by L1-norm minimization with guaranteed convergence since, unlike the cases in CS, this is strict convex optimization. The minimization generally takes the following form [19]:

Where *λ* balances the sparsity of the solution with the fidelity to the data and should be inversely proportional to the noise level. Efficient algorithms for solving (16) have been developed [9, 19].

_{1}= diag(

*λ*

_{1},...,

*λ*

_{ N }

*)*contains the principal singular-values and Σ

_{2}= diag(

*λ*

_{N+1},...,

*λ*

_{ iL }

*)*contains noise singular-values. When noise is absent, Σ

_{2}is an all-zero matrix. Otherwise, its diagonal entries will be non-zero positive numbers. Although it is impossible here to accurately estimate the noise variance, these diagonal entries do reflect the noise level and can be represented by

In our proposed method, we will set the parameter in (16) to $\lambda =\delta /\stackrel{\u0304}{\sigma}$ where *δ* is a positive constant. The proposed LRL1 algorithm can now be summarized as follows:

*Step 1*: Subspace identification of system order using (5) and (6), and estimation of noise level using (17). This step is a systematic order selection procedure with reliable performance.

*Step 2*: Estimation of system transfer function coefficients using (16). This step is simultaneous optimization over all outputs with robust sparse solutions.

## 4. Simulation

We evaluate the performances of the proposed LRL1 method against that of the subspace method N4SID [1]. The first MIMO system under consideration is generated by modifying the SISO system in [[1], p. 155]. In particular, the system order and eigenvalues (i.e., the vector [1 **a**^{
T
} ] ^{
T
}**)** are kept unchanged and one more input and one more output are added to the original system. Then, some elements in each of the transfer function vector **b**_{
m,l
}(see (13)) are set to zero to create the desired sparsity. The resulting 2-input 2-output system of order *N* = 4 has the following transfer function coefficients:

*System 1:*

*S*= 200 and the inputs are sequences of Gaussian variables with zero mean and unit variance. A white noise vector

**v**

_{ k }is added to the output vector

**y**

_{ k }(see (1b)) and the SNR as defined below is kept constant for each

*k*,

*T*=

*N*+

*M*(

*N*+ 1)

*L*parameters in

**β**is measured as

where **β**(*r*) denotes the estimates in the *r* th run. In the simulation, *δ* = 0.5 is fixed for all the scenarios and the convex programming package from [20] is used in solving (16).

*N*) by examining the

*iL*diagonal entries of

**Σ**in (6), i.e., the singular values (

*λ*

_{1},...,

*λ*

_{ iL }) of the system, where

*i*is the number of block rows of the Hankel data matrices (see (2)) and

*L*= 2 is the number of outputs. For SNR = 30 and

*i*changing from 4 to 10, we obtain

*iL*singular values for each particular choice of

*i*. Figure 1 shows (

*λ*

_{1},...,

*λ*

_{ iL }, 0,..., 0) where zero padding is made for cases with

*i*< 10.

Checking for the number of principal (dominant) singular values clearly indicates that the system order is *N* = 4. This result verifies that the identification of system order is reliable regardless of the value of *i* as long as it is chosen to be larger than *N*. This feature is very attractive in practice since it is relatively easy to select an *i* which is larger than the largest possible value the system order might have.

*i*changes from 5 to 10 at SNR = 30. It can be seen here that the performance of the subspace method depends on

*i*in a non-monotonous fashion. That is, a larger

*i*does not necessarily leads to better coefficient estimates. This is due to the fact that only finite datasets are available; increasing the number of rows of the data matrices will lead to a reduction in the number of columns. In the subsequent simulations,

*i*= 9 is used.

Further test is conducted by modifying *System 1* to create an even sparser system. In particular, some of the transfer function coefficients of *System 1* are set to zero, which results in

*System 2:*

## 5. Conclusion

A noise-robust algorithm for the identification of MIMO systems has been presented. The proposed method leverages reliable system order identification of subspace principle and exploits L1-norm optimization to achieve high effectiveness for identifying systems with sparse transfer function coefficients. While retaining good features of original subspace methods such as the convenience for multivariable systems, the proposed method is shown to be able to significantly improve estimation accuracy for sparse systems.

## Declarations

## Authors’ Affiliations

## References

- Vanoverschee P, DeMoor B:
*Subspace Identification for Linear Systems: Theory-Implementation-Applications*. 1996. SpringerView ArticleGoogle Scholar - Young P:
*Recursive Estimation and Time-series Analysis: An Introduction*. 1984. SpringerView ArticleGoogle Scholar - Akaike H: A new look at the statistical model identification.
*IEEE Trans Autom Control*1974, 19(6):716-723.MathSciNetView ArticleGoogle Scholar - Ouyang S, Hua Y: Bi-iterative least square method for subspace tracking.
*IEEE Trans Signal Process*2005, 53(8):2984-2996.MathSciNetView ArticleGoogle Scholar - Qiu W, Skafidas E: Robust estimation of GCD with sparse coefficients.
*Signal Process*2010, 90(3):972-976.View ArticleGoogle Scholar - Andrews H, Patterson C: Singular value decompositions and digital image processing.
*IEEE Trans Acoust Speech Signal Process*2003, 24(1):26-53.View ArticleGoogle Scholar - Meraim KA, Qiu W, Hua Y: Blind system identification.
*Proc IEEE*1997, 85(8):1310-1322.View ArticleGoogle Scholar - Qiu W, Saleem SK, Pham M: Blind identification of multichannel systems driven by impulsive signals.
*Digital Signal Process*2010, 20(3):736-742.View ArticleGoogle Scholar - Candès E, Wakin M: An introduction to compressive sampling.
*IEEE Signal Process Mag*2008, 25(2):21-30.View ArticleGoogle Scholar - Cotter SF, Rao BD: Sparse channel estimation via matching pursuit with application to equalization.
*IEEE Trans Commun*2002, 50(3):374-377.View ArticleGoogle Scholar - Schreiber WF: Advanced television systems for terrestrial broadcasting: some problems and some proposed solutions.
*Proc IEEE*1995, 83: 958-981.View ArticleGoogle Scholar - Fevrier IJ, Gelfand SB, Fitz MP: Reduced complexity decision feedback equalization for multipath channels with large delay spreads.
*IEEE Trans Commun*1999, 47: 927-937.View ArticleGoogle Scholar - Ariyavisitakul S, Sollenberger NR, Greenstein LJ: Tap selectable decision-feedback equalization.
*IEEE Trans Commun*1997, 45: 1497-1500.View ArticleGoogle Scholar - Kocic M, Brady D, Stojanovic M: Sparse equalization for real-time digital underwater acoustic communications. In
*Proc OCEANS95*. San Diego, CA; 1995:1417-1422.Google Scholar - Chen Y, Gu Y, Hero AO III: Sparse LMS for system identification. In
*Proceedings of ICASSP*. Taipei, Taiwan; 2009:3125-3128.Google Scholar - Sharp M, Scaglione A: Application of sparse signal recovery to pilot-assisted channel estimation. In
*Proc of Intl Conf on Acoustics, Speech and Signal Proc*. Las Vegas, NV; 2008.Google Scholar - Sharp M, Scaglione A: Estimation of sparse multipath channels. In
*Proceedings of IEEE Military Communications Conference*. San Diego, CA; 2008:1-7.Google Scholar - Dayal BS, MacGregor JF: Multi-output process identification.
*J Process Control Elsevier Sci*1997, 7(4):269-282.View ArticleGoogle Scholar - Candès E, Tao T: The Dantzig selector: statistical estimation when
*p*is much larger than n.*Ann Stat*2005, 35: 2313-2351.View ArticleGoogle Scholar - Grant M, Boyd S: CVX: Matlab software for disciplined convex programming (web page and software).[http://stanford.edu/~boyd/cvx]

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.