In general, designing a multi-terminal VQ is more difficult than designing a WZ-VQ, due to the mutual dependence among the encoders. However, one could use WZ-VQs to realize a multi-terminal VQ by using source-splitting [12]. It is known that in the quadratic-Gaussian case, source-splitting can be used to realize any rate-pair in the achievable rate-region by only using ideal WZ-VQs which correspond to the corner-points of the achievable rate-region [[12], Section V-C], [15]. While the same optimality properties cannot be claimed for source-splitting by linear transforms, the aforementioned observation still provides us the motivation to take a similar approach in practically realizing the DTCs which can operate at arbitrary rates, by using only WZ quantizers.
A block diagram of the SP-DTC system is shown in Figure 1. Let the total number of bits available for encoding two jointly Gaussian vectors and be B bits. The terminal 1 performs source splitting by providing an approximation of X1 at the rate bits/vector as decoder side-information for WZ coding of terminal 2, where . In a TC framework, the goal is to provide the best (in MMSE sense) linear approximation of X1 as the decoder side-information. Therefore, is the -bit approximation of a linear projection , where is a M1 × M1 unitary matrix. Given the side-information at the decoder, the terminal 2 quantizes a linear projection of X2 using bits/vector, where T2 is a M2 × M2 unitary matrix. Let , be the quantized value of U2, where N2≤ M2. Then, given the quantized linear projections of both X1 and X2 available at the decoder, the terminal 1 quantizes the a linear projection of X1 using bits/vector, where is a M1 × M1 unitary matrix. Let be the quantized value of , where . In the receiver, each source vector is reconstructed by a WZ decoder. The MMSE optimal reconstructions for X1 and X2 are, respectively given by and , where . The total transmission rate for source X1 is thus bits/vector. The rates used by terminals 1 and 2 in bits/sample are given by R1 = B1/M1 and R2 = B2/M2, respectively.
Let , and . Also, let the bit-rates allocated to quantizing these transform coefficients be , , and respectively, and define Given a total of B bits for encoding both X1 and X2, the design of a SP-DTC involves determining the values of the transforms , , and , and a bit allocation among the transform coefficients such that the total MSE
(6)
is minimized. By writing the quantization MSEs of and U2,jas , and , respectively, i = 1, . . ., M1, j = 1, . . ., M2, the total MSE can be expressed as
(7)
The bit allocation problem can now be stated as follows:
Given a total bit-budget of B bits
subject to
where . The explicit solution of this minimization problem is unfortunately intractable due to the inter-dependence of the three transform codes involved.
However, an explicit solution can be found for a variant of this problem obtained by fixing , and B2, so that the number of bits allocated to each transform code is fixed and it is only required to optimize the bit allocation among the quantizers within each transform code. For simplicity, we refer to this problem as the constrained bit-allocation problem. In the following, an explicit solution to this problem is derived. Based on the result, we then present a tree-search algorithm to solve the unconstrained problem (8). Under both RD-WZQ and SWC-HRSQ models, the optimal transforms for Gaussian sources are CKLTs. Therefore, we refer to the solution to problem (8) as the SP-DKLT.
3.1 Solution to the constrained bit-allocation problem
3.1.1 RD optimal quantization
Let , , and B2 be fixed in Figure 1 and let the coefficient quantization be represented by the RD-WZQ model (for components of , this reduces to the non-distributed RD-optimal quantization). From Theorem 1, it follows that the MMSE optimal transform (in the sense of providing the best linear approximation as decoder side information for terminal 2) is the KLT of X1. Let the eigenvalues of be . Then, using (2), the optimal bit allocation for can be given by
(9)
for some . Let the quantized value of be . We note that for m = 1, . . ., M1 and for . In RD theoretic sense, the quantized value (up to a scaling factor) of the mean-zero Gaussian variable can be given by , , where is a mean-zero Gaussian random-variable independent of such that
(10)
see [[16], Section 10.3.2]. Therefore, we can write
(11)
where denotes the matrix consisting of first columns of . The covariance matrix of "quantization noise" vector is given by . According to (11), and X1 are jointly Gaussian, and it follows that
(12)
Furthermore, X2 and are jointly Gaussian with the conditional covariance matrix
(13)
Next consider TC X2 given as decoder side-information. From Theorem 1, it follows that the is minimized by choosing T2 as the CKLT of X2 given and by applying RD optimal WZ quantization to each element of given decoder side information based on a bit-allocation specified by the eigenvalues of , . The optimal bit allocation for U2,mis given by
(14)
for some N2≤ M2. The resulting MSE is given by
(15)
As before, the quantized value of U2 up to a scaling factor, can be represented by [[8], Theorem 3],
(16)
where denotes the M2 × N2 matrix consisting of first N2 columns of K2 and the quantization noise is a mean zero iid Gaussian vector independent of X2. The covariance matrix of Z2 is given by , where , m = 1, . . ., N2 are given by
(17)
The covariance matrix of Y2 is
(18)
Therefore X1 and are jointly Gaussian with the cross-covariance matrix
(19)
where
and
Finally, consider quantizing X1, given as the decoder side-information. As before, is minimized by choosing as the CKLT of X2 given V, and RD optimal WZ quantization of each element of given V, based on a bit-allocation specified by the eigenvalues of , . The bit rate allocated to quantizing given by
(20)
for some and the resulting MSE is
(21)
Given a rate tuple , the MMSE achievable with a SP-DKLT code for two jointly Gaussian vectors is given by
(22)
3.1.2 High-resolution scalar quantization and SW coding
Due to Theorem 2, the expressions for bit-allocations given by (9), (14) and (20) applies to SWC-HRSQ model as well. However, the resulting MSE and hence the quantization noise variances are different to those of the RD-WZQ model. More specifically, since the decoder side-information in a SP-DTC depends on quantization noise of the other terminals, the optimal transforms T2 and and the associated bit allocations obtained with the SWC-HRSQ model are different to those obtained with the RD-WZQ model. In order to make the problem tractable, we assume that the quantization noise of SWC-HRSQ model also follows (11) and (16) (for a discussion on the validity of this assumption, see [17]). This assumption essentially allows us to compute the conditional covariance matrices and as in the previous case, and then apply (9), (14) and (20) to find the optimal transforms and the bit-allocations. However, due to (27), the quantization noise variance in (10) in this case is given by
(23)
A similar expression exists for the quantization noise variance in (17).
3.2 A tree-search solution to the unconstrained bit-allocation problem
We note that the optimal solution to the unconstrained problem defined in (8) corresponds to the MMSE solution of the constrained problem over the set of rate-tuples . This set is shown in Figure 2. One approach to locating the MMSE solution is to search over an appropriately discretized grid of points inside
. As we will see, even though an exhaustive search on a fine grid can be prohibitively complex, a much simpler constrained tree-search algorithm exists which can be used to locate the required solution with a very high probability.
The proposed algorithm is a generalization of a class of bit-allocation algorithms in which a small fraction ΔB of the total bit-budget B is allocated to the "most deserving" quantizer among a set of quantizers in an incremental fashion, until the entire bit-budget is exhausted [[4], Section 8.4]. Unfortunately, this type of a greedy search cannot guarantee that the final solution is overall optimal and can yield poor results in our problem where the bit allocation among three sets of dependent quantizers must be achieved. On the other hand, if the increment ΔB is chosen small enough, a near-optimal solution can be found by resorting to a tree-search. Even though a full tree-search is intractable, a simple algorithm referred to as the (M, L)-algorithm[18] exists for detecting the minimum cost path in the tree with a high probability. We use this insight to formulate a tree-search algorithm for solving the unconstrained bit allocation problem, in which a set of constrained bit allocation problems are solved in each iteration.
In order to describe the proposed tree-search algorithm in detail, let ΔB be the incremental amount of bits to be allocated in each step of the search, where 0 < ΔB ≪ B. The algorithm is initialized by setting , i.e., the origin in Figure 2. Now if we are to allocate ΔB bits to only one of the three transform codes , or T2, then there are three possible choices for the rate-tuple , namely (ΔB, 0, 0), (0, ΔB, 0), and (0, 0, ΔB). For each of these choices, we can explicitly solve the constrained bit allocation problem as described in the previous section and find the MMSE solution. Each of these candidate solutions can be viewed as a node in a tree as shown in Figure 3. The root node of the tree corresponds to a SP-DTC of rate 0, and a node in the first level of nodes obtained in the first iteration of the algorithm corresponds to a SP-DTC of rate of ΔB bits per source pair (X1, X2). In the second iteration of the algorithm, we allocate ΔB more bits to each of the three candidate SP-DTCs (but to one of the SP-DTCs at a time) in the first level of nodes. Note that, for each SP-DTC we can allocate ΔB bits in three different ways, i.e., ΔB bits can be added to either , , or B2. This requires the solution of three constrained bit allocations problems for each of the 3 nodes in the first-level. As a result, the tree will be extended to a second level of 32 nodes, in which each node corresponds to a SP-DTC of 2ΔB bits per source-pair, as shown in Figure 3. We can repeat this procedure, allocating ΔB bits to each of terminal node of the tree in a given iteration, until all B bits are exhausted. After the final iteration, the tree would consist of L = ⌈B/ΔB⌉ levels with 3Lnodes in the last level (terminal nodes). Each terminal node corresponds to a candidate SP-DTC of rate B, and rate-tuples of these SP-DTCs lie on the plane in Figure 2. The MMSE terminal node of the tree is the optimal solution to the unconstrained bit allocation problem, provided that the latter solution is on the search-grid. If ΔB is chosen small enough, then we can ensure that the optimal solution is nearly on the search-grid. Suppose that, in each iteration, the algorithm saves the MSE of the solution to the constrained bit-allocation problem associated with each node. In theory, the optimal solution can be found by an exhaustive tree-search, using the MSE of a node [given by (22)] as the path-cost. In order to practically implement the tree-search, we use the (M, L)-algorithm, in which the parameter M can be chosen to reduce the complexity at the expense of decreased accuracy (i.e., the probability of detecting the lowest cost path in the tree). In the (M, L) algorithm [[18], p. 216] for a tree of depth L, one only retains the M best (lowest MSE) nodes in each iteration. When M = 1 we have a completely greedy search. On the other hand when M
n
= 3nin the n th iteration, we have a full-tree search which has a complexity that grows exponentially with the iteration number. When M (1 ≤ M ≤ 3L) is a prescribed constant, the complexity is linear in M, independent of the number of iterations. In obtaining the simulation results presented in Section 4, M = 27 and L = 135 (ΔB = 0.2) were found to be sufficient to obtain near optimal results. For example, it was observed that even for M = 81 and L = 405, nearly the same result was obtained.