Multiple descriptions for packetized predictive control over erasure channels

We consider a networked control system with random delays and erasures on a data-rate limited forward channel between the controller and the plant. The feedback channel from the plant to the controller is assumed noiseless. We combine two techniques to enhance the reliability of the system. First, we use packetized predictive control, where a quantized control vector with future predicted control signals is transmitted to the plant at each time instant. Second, we utilize multiple descriptions to further aid in the robustness towards packet erasures. In particular, we transmit M redundant packets, which are constructed such that when receiving any 1 ≤ J ≤ M packets, the current control signal as well as J −1 future control signals can be reliably reconstructed at the plant side. For the particular case of LTI systems and when the packets are not received out-of-order, we prove stability by showing that the system can be cast as a Markov jump linear system with M+1 states. We further show by simulations that, by use of multiple descriptions, the system state variance can be significantly reduced without increasing the total data rate.


I. INTRODUCTION
Networked control systems (NCSs) generalize the notion of control systems by allowing the control loops to be closed through networks.While NCSs open the door for new control applications e.g., remote and distributed control, it also brings forth a wealth of new challenges that go beyond conventional control techniques.Specifically, networks are generally unreliable, e.g., they could be data-rate or SNR limited and be suscept to random bit-errors or data-dropouts due to fading on wireless channels or router congestions on packet-switched networks.Moreover, the network could introduce (random) delays or even break down, which would leave the plant in open loop.Several issues of NCSs are discussed in detail in the survey papers [1]- [3] see also [4], [5].
In this paper, we are interested in quantized packetized predictive control (PPC) over data-rate limited (digital) channels [6]- [9].In PPC, a control vector with the current and N − 1 future predicted plant inputs are constructed at the controller side to compensate for random delays and packet dropouts in the forward channel (i.e., the channel between the controller and the plant).We will assume that the Jan Østergaard is with the Department of Electronic Systems, Aalborg University, Denmark; e-mail: janoe@ieee.org.
Daniel Quevedo is with the School of Electrical Engineering & Computer Science, The University of Newcastle, NSW 2308, Australia; e-mail: dquevedo@ieee.org.
This research was partially supported by the Danish Research Council for Technology and Production Sciences, grant no.274-07-0383 and partially supported under Australian Research Council's Discovery Projects funding scheme (project number DP0988601).feedback channel (i.e., the channel between the plant and the controller) is noiseless.The main contribution of the paper is to show that PPC can be combined with multiple descriptions (MDs) and thereby increase the reliability of the system in the case of channel failures (i.e., packet delays and dropouts).MDs is a joint source-channel coding technique, which has received increasingly attention in the information theoretic and signal processing communities [10].The main idea of MDs is to encode the source signal (e.g., control vector) into multiple descriptions (packets), which individually provide a desired approximation of the control vector.Moreover, when the descriptions are combined, they refine each other and, thus, jointly provide a better approximation of the control vector.In this work, we construct M partially redundant descriptions (or packets) based on each control vector.The packets are constructed so that the current control signal and J −1 future control signals can be obtained by combining any subset of J ∈ {1, . . ., M} packets.Thus, the more packets are received at the plant, the more future plant predictions becomes available.Notice that on reception of at least one packet out of the M packets, the current control signal can be completely recovered at the plant input side.
While our idea is applicable to general PPC systems, we will focus on the case of discrete-time noisy LTI systems and i.i.d.packet dropouts.We first describe the control law when packets are received out-of-order.In this case, however, we are not able to assess the stability of the system.Then we consider the case where packets are received in order, i.e., out-of-order packets are discarded.In this case, due to the MDs, the control values may, at times, be applied out-of-order.We show that the system can be cast as an M + 1 state Markov jump linear system (MJLS).Thus, the stability can addressed via linear matrix inequalities.Through a simulation study, we then address the issue of bitrates due to quantization, and show that for a fixed data rate, the use of MDs significantly increases the system reliability in the presence of i.i.d.packet dropouts.

II. PACKETIZED CONTROL OVER ERASURE CHANNELS
In this section, we provide a summary of existing results on quantized PPC and relate them to the present situation.

A. System Model
We consider the following discrete-time stochastic linear time invariant (LTI) possibly unstable dynamical plant model with state x t ∈ R z , z ≥ 1 and scalar input u t ∈ R: In ( 1), w t ∈ R z , z ≥ 1, is an unmeasured disturbance, modelled as an arbitrarily distributed (and with possibly unbounded support) zero-mean stochastic process with bounded variance.Throughout this work, we will assume that the pair (A, B 1 ) is stabilizable.The initial state x 0 is arbitrarily distributed with bounded variance.

B. Cost Function
At each time instant t and for a given plant state x t , the following cost function is minimized: where N ≥ 1 is the horizon length.The design variables P 0, Q 0 and λ > 0 allow one to trade-off control performance versus control effort.
The cost function in (2) examines a prediction of the plant model over a finite horizon of length N .The predicted state trajectories at time t are independent of the buffer contents at the decoder (i.e., they are independent of what has been received at the plant input side), the dropout probabilities and the external disturbances w, and are generated by T represent the associated predicted plant inputs.Thus, the current control vector ūt = [u t (1), . . ., u t (N )] T contains the control signal u t (1) for the current time instant t as well as N − 1 future predictive control signals for time up to t + N − 1.
Following the ideas underlying PPCs, see, e.g., [6], at each time instant t, and for current state x t , the controller sends the entire optimizing sequence, ūt , to the actuator node.Depending upon future packet dropout scenarios, a subsequence of ūt will be applied at the plant input, or not.Following the receding horizon paradigm, at the next time instant, x t+1 is used to carry out another optimization, yielding ūt+1 , etc.

C. Network Effects
We consider an unreliable packet-switched network, which due to transmission errors and congestion, introduces packet dropouts and packet delays.We assume the availability of M separate and independent channels.We model transmission effects via the discrete processes {d i t,t } ∞ t =0 , where 0 ≤ t ≤ t and i = 1, . . ., M, defined via: t is in the buffer at time t ≥ t, 0, else.These processes are generally not i.i.d., since if a packet has been received at time instant t it is still received at time instances t + n, n ≥ 1.However, for t = t, the outcomes d i t,t , i = 1, . . ., M, t ≥ 0, are considered mutually independent.In other words, we assume that the packet dropout probabilities are independent across channels as well as across time.

D. Quantization Constraints
We consider a bit-rate limited digital network between controller output and plant input and all data to be transmitted needs therefore to be quantized.This introduces a quantization constraint into the problem of minimizing J(ū , x t ).A closed form solution to this problem was derived in [11].Furthermore, in [8] the problem was cast into the framework of entropy-constrained (subtractively) dithered (lattice) quantization (ECDQ) [12].
For future reference, we present below some important results of [8], [11] that we will be needing in the sequel: Let Q diag(Q, . . ., Q, P ) and let Φ, Υ be defined by Theorem 2.1 (Quantized Predictive Control [11]): Consider any quantized set U ⊂ R N , the matrices Q, Φ, and Υ given in (3), and define: where Ψ ∈ R N ×N is obtained from the factorization Ψ T Ψ = Φ T QΦ + λI, where λ is as in (2).
Then the constrained optimizer ũt = arg min where Q(•) is a (nearest neighbour) vector quantizer with alphabet ΨU.When using ECDQs, a dither vector ζ t is added to the input prior to quantization and then subtracted again at the decoder to obtain the reconstruction. 1Specifically, let Q Λ denote an ECDQ with underlying lattice Λ.Then the discrete output ξ t of the ECDQ is given by ξ Furthermore, the reconstruction ξt at the decoder is then obtained by subtracting the dither, i.e., ξt = ξ t − ζ t .Interestingly, this quantization operation may be exactly modelled by an additive noise channel, i.e., ξt = ξ t + n t , where the noise n t is independent of ξ t , see [12] for details.Using this, it was furthermore shown in [8] that the quantized (and reconstructed) control variable u t can be written as where n t and ξ t are mutually independent and ξ t = Γx t .Throughout the paper, we will use u t to denote the quantized (and reconstructed) control vector, which has been found by using an ECDQ on ξ t .Note that u t is a continuous variable whereas ũt = Ψ −1 ξ t is the corresponding discrete variable, which is entropy coded and thereby converted into a bitstream (to be transmitted over the network).

III. MULTIPLE DESCRIPTIONS
To aid in robustness towards packet loss we will be using multiple descriptions (MDs).In particular, we will construct M packets ci t , i = 1, . . ., M, at time t based on the single control vector ũt .These M packets, are created in such a way that they are individually useful for the plant.Moreover, the more of the M packets that are received, the better information about future predictive control signals will become available to the plant.
For the NCS studied, we apply a sequence of erasure codes on the control vector ũt in order to obtain M packets.First, we recall that ũt contains quantized elements, which belong to a discrete alphabet, i.e., each symbol u t (i) can be identified with a bit string b t (i) of L i bits.Let us for the moment assume that L i = L = 24 bits and that we wish to construct M = 3 packets ci t , i = 1, . . ., 3.Then, we create the first packet c1 t in the following way: Let t (3)).Similarly, we construct the second packet as t (3)).Finally, we construct the third packet as c3 t (3)).The M = N = 3 packets are transmitted over separate channels. 2It may be noticed that upon reception of any single packet, the first control signal u t (1) for the current time t may be recoverd.Upon reception of any two packets, the first control signal u t (1) as well as the future predicted control signal u t (2) for time t + 1 may be recovered.Finally, upon reception of all three packets, all N control signals ũt = (u t (1), u t (2), u t (3)) are recovered.
The size of a packet (i.e., the packet bit-rate R i ) is on average bits and there are M packets.When not using MDs, the coding rate is R = L 1 + • • • + L M .Thus, the total coding overhead compared to when not using multiple descriptions is We refer the reader to the simulation study for further details on the operational coding rates. 2 The procedure may be extended in a trivial manner to any M ≤ N .indicates that the packet is in the buffer and "0" indicates that it is not."X" indicates that the control value does not depend on the given packet.In all other cases, we set ût = 0.

IV. SYSTEM ANALYSIS WITH TIME-DELAYS AND DROPOUTS
We will in this section consider the general case where the network introduces time-delays and therefore packets may be received out-of-order.

A. Buffering and Reconstruction of Control Signals
At time t, the buffer at the plant input side contains all received packets, which are not older than t − N + 1.These will be used for obtaining the current control signal ût giving preference to newer data.For example, assume the buffer is initially empty.Then, for the case of M = N = 3, if we at time t receive c2 t , then clearly we obtain ût = u t (1).If we then at time t+1 receive c1 t+1 and the delayed packet c3 t then we should form ût+1 = u t+1 (1) from c1 t+1 and, thus, simply ignore c3 t .However, if we now at time t + 2, only receive the very late c1 t , then we recover ût+2 = u t (3).Thus, we use the older packets to obtain the control signal.This process is clarified in Table I for M = N = 3 and given for general M = N ≥ 1 below: 3Let Θ k,M t,t ∈ {0, 1} be an indicator function, which is "1" if at least k out of M packets of time stamp t are in the buffer at time t and "0" otherwise.In particular, With this, the control signal ût to be used at time t is given by: where it follows that ût = 0 in the event that Θ 1,M t−N +1,t = 0, which is the case if no packets have arrived in N = M consecutive time instances.

B. State Evolution with Packets Out-of-Order
Let Ξ t be the augmented state variable given by then . . .
However, since u t = Ψ −1 Γx t + Ψ −1 n t from (6), it follows that Ξ t+1 in (9) depends linearly upon N past state values i.e., In other words, due to the fact that packets can be out-of-order, the system does not immediately admit a first-order MJLS-structure.Conventional MJLS stability results do therefore not apply.We suspect that the stability of this system with out-oforder packets could be assessed with the stability techniques introduced in [14] and [8].However, such an analysis is left for future works.

V. SYSTEM ANALYSIS WITHOUT TIME-DELAYS
In this section, we consider the case where the network does not introduce delays.Thus, packets are not received out-of-order, i.e., at given time t, we receive only a subset of the current M packets.
Remark 1: We would like to emphasize that one is still allowed to apply out-of-order control signals.To see this, assume that we at time t − 2 receive three packets and apply u t−2 (1).Then at time t − 1 we receive only one packet and apply u t−1 (1).If then at time t, we do not receive any packets, then we apply the older u t−2 (3) control signal.

A. State Evolution with Packets in Order
Let fk denote the buffer vector at the plant input side holding past control values and let S be the one-step shiftforward matrix.Then where ΘM k is a diagonal matrix given by Let Ξ t be the augmented state variable given by Then

B. Markov Jump Linear System
We will now show that ( 13) can be rewritten in a form, which satisfies the conditions of an MJLS with M +1 states.Let where i t ∈ {0, . . ., M}, ∀t ≥ 0, e T 1 is the all-ones vector, and where I it is a time-varying diagonal M ×M matrix with ones on the first i t entries on the diagonal and zero elsewhere, i.e., I it = diag([1, . . ., 1, 0, . . ., 0]), e T 1 I it e 1 = i t .Moreover, let Then it is easy to show that where ν t = [w t , n t ] T .Notice that the sequence {i t } is i.i.d., since i t ∈ {0, . . ., M} corresponds to the number of received packets at time t and the dropouts are independent across channels and over time.Thus, the probability p it|i t of entering state i t when the previous state was i t is independent of i t and given by p it|i t = p it , where p it = Prob{receiving i packets at time t}.Moreover, p it = p i since p it does not depend on t.If i t = k, ∀t for some fixed k ≥ 0, then the system ( 16) is clearly linear.On the other hand, since the dropouts are i.i.d., it follows that the state sequence {i t } is ergodic and the process {Ξ t+1 , Ā(i t )} is first-order Markovian.Thus, ( 16) is an MJLS with M + 1 states [15].From an analysis point of view, this is very convenient since it allows one to assess the stability of an MJLS using e.g., linear matrix inequalities.In particular, we include the following well-known results: Corollary 5.1 (Sufficient Condition for MSS): The system ( 16) is MSS if there exists Γ 0 such that Γ − M i=0 p i ĀT (i)Γ Ā(i) 0. Proof: Follows immediately from [15,Corollary 3.26] since the state sequence {i t } is i.i.d., and the number of jump-states is finite.

A. Quantization
We use a (non-dithered) scalar uniform quantizer with step-size Δ > 0. We quantize the N -dimensional vector ξ t = Γx t independently along each dimension to obtain ξ t = ( ξ t (1) Δ , ξ t (2) Δ , ξ t (3) Δ ), where • Δ denotes rounding towards nearest iΔ, i ∈ Z.After quantizing ξ t to obtain the vector ξ t we further apply Ψ −1 to obtain the quantized control vector ũt .Since Ψ −1 is generally a full real matrix, it follows that each elements of ũt is a function of the complete quantized vector ξ t .Moreover, the alphabet of ũt , say U, i.e., the discrete set U = Ψ −1 (ΔZ × • • • × ΔZ) obtained by using Ψ −1 ∈ R N ×N on ξ t , may be large.We therefore discretize Ψ −1 to obtain Ψ−1 = Ψ −1 1 30 , which is a discrete approximation of Ψ −1 that greatly reduces the size of U without introducing significant distortion, i.e., the average state variance J 1 given by is only slightly increased.

B. Discrete Entropies
We now assess the average empirical discrete entropy H({u t (i)}), where and where Pj denotes the empirical probability that u t (i) belongs to the j th quantization cell of the i th codebook U i , i.e., Prob{u t (i) = U i (j)}, where U i denotes the discrete alphabet of u t (i).We estimate Pj from the histogram of the complete sequence {u t (i)} T t=1 .It is useful to work with entropies, since the average operational bit-rate R op obtained when using an entropy coder designed on the true statistics of {u t (i)} T t=1 , can be sandwiched in the following way: where c is a function that is upper-bounded by 1 and tends to zero as the size of the optimum codebook (of the entropy coder) becomes large [17], [18].As p increases, the variance of {ξ t } T t=1 will also increase.However, if the variance of the input to the quantizer increases, the resulting bit-rate also increases (if the step-size is kept fixed).It is well-known that under high-resolution assumptions, and when there is no feedback, the bit-rate of a uniform scalar quantizer when quantizing a stationary sequence, say {ϕ t }, decreases logarithmically with its stepsize Δ, i.e., R ≈ h({ϕ t })−log 2 (Δ), where h({ϕ t }) denotes the differential entropy rate of {ϕ t } [17], [18].Such a simple property has not been rigorously proven for quantized closedloop control.However, in this particular case, a similar behavior can be observed.Specifically, to seek that the bitrate is held approximately fixed as the packet-loss rate is increased, we adapt Δ accordingly by using where Δ 0 = 2 is chosen as the initial step-size at zero packetloss probability.

C. Construction of Packets
The packets are constructed by first entropy coding u t (1) to obtain the bit sequence b t (1) = E(u t (1)) using a scalar entropy coder E(•) designed on the statistics of u t (1).Then, the bit-stream b t (1) is repeated in both packets.Next, the remaining control vectors u t (2) and u t (3) are jointly entropy coded, conditional upon u t (1).It should be noted that u t (i), i > 1 are generally correlated with u t (1) and to achieve efficient encoding, one should use conditional codebooks.With this, the bit-stream b t (2) is obtained as b t (2) = E((u t (2), u t (3))|u t (1)) using a conditional vector entropy coder E(•|•).

D. Data Rates
The operational average data rate R op (1) when encoding {u t (1)} using an optimal entropy coder E(•) designed on the source statistics of {u t (1)} is shown in Table II.For comparison, the average entropy H({u t (1)}) is also shown.In practice, the exact source statistics might not be available and an optimal entropy coder cannot be constructed.Towards that end, we have designed a suboptimal entropy coder Ẽ on i.i.d., Gaussian samples having a variance identical to u t (1).Thus, only knowledge of the variance is required.The operational data rate R sub (1) of the suboptimal entropy coder is also shown in Table II.
As mentioned above, the remaining two quantized control variables (u t (2), u t (3)) should theoretically be losslessly encoded using conditional vector entropy coding.However, generally it is complicated to form optimal conditional codebooks.For simplicity, we will encode u t (2) independently, and will therefore incur a rate penalty.It turns out that u t (3) is deterministically related to u t (1) and u t (2) in the simulations (this would generally not be the case for different prediction horizons or plant models).Thus, knowing the pair (u t (1), u t (2)), one can immediately obtain u t (3).The resulting total (average) operational data rate R T for the M = 2 packets system is therefore given by

E. Comparison to M = 1 Packet System
In the next simulation, we assess the performance J 1 (18) for a fixed bit-rate (entropy), as a function of the packetloss probability p for a network which does not introduce delays.We construct an MD system with M = 2 packets and N = 3.The two packets are constructed so that receiving any single packet, allows one to reconstruct the first control signal u t (1).Moreover, if both packets are received, the entire 3dimensional control sequence ũt can be obtained.The step size for the MD system is given by (21).For comparison, we also construct a baseline system without MDs, i.e., with M = 1 packet, cf.[8].The baseline system also uses N = 3, and to obtain a fair comparison, we scale the step-size of the baseline system appropriately, in order to increase the rate so that it matches that of the MD system. 4 In both simulations T = 10 6 and p is varied in the interval 0.01 to 0.2 in increments of 0.01.The packet dropouts are i.i.d.At p = 0, the resulting state variance is J 1 = 47.46, which is slightly better than the J 1 = 47.93 obtained with the M = 2 system.Since there are no erasures at p = 0, and due to the high bit-rates, the performance is dominated by the variance of the external disturbance.At lower rates, the differences in performances would be significantly greater (and favoring the M = 1 system).As p increases, it quickly becomes advantageous to use the MD system as is apparent from Fig. 1, where log(J 1 ) of the baseline system is compared to that of the M = 2 system described in the previous section.A sufficient condition for stability of the baseline system is given in [8, Lemma 6], from which it can be shown that at least for p < 0.054, the baseline system is stable.It is clear from Fig. 1 that for this particular setup, a substantial improvement can be obtained by using MDs, i.e., PPC with M = 2 significantly outperforms PPC with M = 1.
Finally, we split b t (3) into three equal lenght bitstreams b

Fig. 1 :
Fig. 1: Performance log(J 1 ) (18) as a function of the packet-loss probability for the cases with and without MDs.

TABLE I :
Control value ût at time t from available buffer contents."1"

TABLE II :
Average discrete entropy H({ut(1)}).Also shown are the average operational data rates Rop(1) and R sub (1) obtained using optimal E and suboptimal Ẽ entropy coding, respectively.