- Research
- Open Access
Distributed transform coding via source-splitting
- Pradeepa Yahampath^{1}Email author
https://doi.org/10.1186/1687-6180-2012-78
© Yahampath; licensee Springer. 2012
- Received: 21 October 2011
- Accepted: 5 April 2012
- Published: 5 April 2012
Abstract
Transform coding (TC) is one of the best known practical methods for quantizing high-dimensional vectors. In this article, a practical approach to distributed TC of jointly Gaussian vectors is presented. This approach, referred to as source-split distributed transform coding (SP-DTC), can be used to easily implement two terminal transform codes for any given rate-pair. The main idea is to apply source-splitting using orthogonal-transforms, so that only Wyner-Ziv (WZ) quantizers are required for compression of transform coefficients. This approach however requires optimizing the bit allocation among dependent sets of WZ quantizers. In order to solve this problem, a low-complexity tree-search algorithm based on analytical models for transform coefficient quantization is developed. A rate-distortion (RD) analysis of SP-DTCs for jointly Gaussian sources is presented, which indicates that these codes can significantly outperform the practical alternative of independent TC of each source, whenever there is a strong correlation between the sources. For practical implementation of SP-DTCs, the idea of using conditional entropy constrained (CEC) quantizers followed by Slepian-Wolf coding is explored. Experimental results obtained with SP-DTC designs based on both CEC scalar quantizers and CEC trellis-coded quantizers demonstrate that actual implementations of SP-DTCs can achieve RD performance close to the analytically predicted limits.
Keywords
- distributed transform coding
- Wyner-Ziv quantization
- multi-terminal quantization
- Karhunen-Loéve transform (KLT)
- optimal bit-allocation
1 Introduction
Many new applications such as multi-camera imaging systems rely on networks of distributed wireless sensors to acquire signals in the form of high-dimensional vectors [1]. In such situations, an encoder in each sensor quantizes a vector of observation variables (without exchanging any information with other sensors) and transmits its output to a central processor which jointly decodes all the sources. The strong statistical dependencies among the signals observed by different sensors can be exploited in the decoder to reduce the transmission bit-rate of each sensor. This problem, in general, is referred to as distributed (or multiterminal) vector quantization (VQ). The design of a distributed VQ for a large number of source variables is a difficult task. A practically simpler, yet very effective approach to quantizing a large number of correlated variables by using a bank of single variable quantizers is transform coding (TC) [2–4]. Clearly, TC can be used for distributed VQ when separately observed vectors have both inter-vector and intra-vector statistical dependencies, a situation typical in applications such as camera networks. Most of the previous work [5–7] studies Wyner-Ziv (WZ) transform coding (WZ-TC), which is a special case of more general multiterminal transform coding (MT-TC) [8]. In WZ-TC, a single source is quantized given that the decoder has access to side information about the source.
Information-theoretic studies of distributed transform coding (DTC) can be found in [5, 8]. In [8], the optimal linear transform for Gaussian WZ-TC under the mean square-error (MSE) criterion is shown to be the conditional Karhunen-Loéve transform (CKLT), which is a natural extension of the result in [2]. This result is based on the assumption that each transform coefficient is compressed by a rate-distortion (RD) optimal WZ quantizer and hence describes the optimal performance theoretically attainable (OPTA) in Gaussian WZ-TC. However, the optimal solution to the more general MT-TC problem remains unsolved, even for the Gaussian case. In [8], an iterative descent algorithm for determining the OPTA of Gaussian MT-TC problem is given. It is shown that, while this algorithm [referred to as the distributed KLT (DKLT)] always converges to a solution, the final solution is not necessarily the global optimum. In any case, the practical implementation of distributed quantizers implied by the DKLT remains an open problem. In [5] WZ-TC based on high-rate scalar quantization and ideal Slepian-Wolf (SW) coding [9] is studied. In particular, it is shown that, for jointly Gaussian vectors, CKLT followed by uniform scalar quantization is asymptotically optimal, a natural extension of the result in [10] for entropy-coded quantization at high-rate. More importantly, the bit-allocations and quantizer-step sizes found in [5] can be used for practical design of WZ-transform codes as long as high-rate approximations hold. However, we note that, even when scalar quantizers are used, achieving good performance with this approach still requires the use of a subsequent block-based SW coding method (e.g., Turbo codes or LDPC codes). Other previous studies on WZ-TC can be found in [6, 7]. However, they rely on WZ scalar quantization of transform coefficients. Such methods are therefore most suitable for applications requiring low coding delay as their performance is strictly inferior to block-based quantization.
In contrast to WZ-TC, we consider in this article the practical design of two-terminal transform codes for jointly Gaussian vectors in which arbitrary transmission rates can be assigned to each terminal. Our approach is based on the idea of source-splitting[11, 12] to convert the two-terminal TC problem into two WZ-TC problems. Since transform codes quantize linear projections, we perform source splitting in terms of optimal linear approximations, i.e., a linear approximation of one source is provided as decoder side-information for the other source. The proposed source-split DTC (SP-DTC) approach only requires the design of two-sets of WZ quantizers sequentially, and avoids having to iteratively optimize two sets of WZ quantizers to each other as in [8]. However, this approach requires the solution of a bit allocation problem involving dependent WZ quantizers. To solve this problem for Gaussian sources, we propose an efficient tree-search algorithm, which can be used to find the a good SP-DTC under different models for quantization of transform coefficients. When used with the RD-optimal WZ quantization model [8], this algorithm can potentially locate the optimal SP-DTC for Gaussian sources. In practice, with constraints imposed on tree-search complexity, the algorithm yields a near-optimal solution. We refer to the optimal solution to the Gaussian problem as the source-split DKLT (SP-DKLT). Using this algorithm, we numerically compute the rate-region achievable with a SP-DKLT code for two examples of jointly Gaussian vector sources. This study shows that, when there is sufficient inter-source correlation, optimal SP-DKLT codes can achieve substantially better performance than independent transform codes for the two sources. However, we find that the rates achievable with SP-DKLT codes are strictly inside the optimal achievable rate-region predicted by the DKLT algorithm of [8]. In order to approach the performance predicted by the optimal SP-DKLT in practice, block WZ quantization of transform coefficients is required. For implementation of block WZ quantizers, we consider the use of trellis-coded quantization (TCQ) followed by SW coding. This two stage approach is known to achieve the RD function of Gaussian WZ coding [13]. In order practically implement this approach, we introduce the idea of designing conditional entropy constrained TCQ (CEC-TCQ) based on analytically found bit-allocations. We present experimental results to demonstrate that practical implementations of SP-DTCs for Gaussian sources can closely approach the performance limits indicated by the optimal SP-DKLT. On the other hand when SW coded high-rate scalar quantization model [5] is assumed for encoding transform coefficients, the tree-search algorithm proposed in this article can also be used to find asymptotically good SP-DTC codes for scalar quantization based implementations. These codes can be readily implemented using CEC scalar quantizers (CEC-SQ) as demonstrated by experimental studies presented in this article. In our experimental study, we also investigate the design of good SP-DTCs based on widely used discrete cosine transform (DCT).
This article is organized as follows. Section 2 presents a review of WZ-TC of Gaussian vectors and motivates the particular approach introduced in this article. Section 3 presents the idea of SP-DTC and develops the tree-search algorithm for finding the optimal transforms and the bit-allocation for SP-DKLT codes. Section 4 computes the achievable rate region of SP-DKLT codes for two example Gaussian source models, and presents experimental results obtained by designing SP-DTCs based on both KLT and DCT. Finally, some concluding remarks are given in Section 5.
Notation: As usual, bold letters denote vectors and matrices, upper case denotes random variables, and lower case denotes realizations. Σ_{ X }denotes the auto-covariance matrix of the vector X. Σ_{ XY }and Σ_{ X|Y }, respectively denote the joint covariance matrix of (X, Y) and the conditional covariance matrix of X given Y. The eigenvalues λ_{1} , . . ., λ_{ M } of a M × M covariance matrix are always indexed such that λ_{1}≥ λ_{2} . . . ≥ λ_{ M } , and the corresponding KLT matrix has the structure $\mathbf{T}=\left({\mathbf{u}}_{1}^{T},\dots ,{\mathbf{u}}_{M}^{T}\right)$, where u_{ m }is the eigenvector associated λ_{ m } .
2 WZ-TC of Gaussian vectors
Consider encoding of a Gaussian vector $\mathbf{X}\in {\mathbb{R}}^{{M}_{1}}$ using B bits per vector, given that the decoder has access to a jointly Gaussian vector $\mathbf{Y}\in {\mathbb{R}}^{{M}_{2}}$. Assume that both vectors have zero mean, and let the auto-covariance matrix of X be Σ_{ X }= E{XX^{ T }}. In WZ-TC, a linear transform is first applied to X and each component of the transform coefficient vector U = T^{ T }X is separately compressed by a WZ quantizer, considering Y as decoder side-information, where T is a M_{1} × M_{1} unitary matrix. Let Û be the quantized value of U. The decoder then estimates the source vector based on Û and Y. We wish to find the optimal transform and the allocation of B bits among M_{1} transform coefficients, which minimize the quantization MSE $E\left\{{\u2225\mathbf{X}-\widehat{\mathbf{X}}\u2225}^{2}\right\}$, where $\widehat{\mathbf{X}}=E\left\{\mathbf{X}|\widehat{\mathbf{U}},\mathbf{Y}\right\}$ is the optimal estimate (at the decoder) of the source vector. The solution of this problem requires an analytical model for coefficient quantization. To this end, [8] considers RD optimal WZ quantization (RD-WZQ) model, the solution based on which is appropriate for practical block quantization techniques such as TCQ. On the other hand [5] considers SW-coded high-rate scalar quantization (SWCHRSQ) model.
where N ≤ M_{1} is a positive integer.
2.1 RD-WZQ model
where λ_{ m } = var(U_{ m }| Y) and var(·|·) denotes the conditional variance. The optimal solution to the WZ-TC problem under RD-WZQ model is given by the following theorem.
Proof 1 Directly follows from [[8], Section III-B].
Note that RD-WZQ model implies infinite-dimensional VQ of each coefficient and hence the above MSE is the OPTA in the Gaussian WZ-TC problem.
2.2 SWC-HRSQ model
The asymptotically (in rate) optimal solution to the WZ-TC problem under SWC-HRSQ model is given by the following theorem.
Proof 2 See Section "Proof of Theorem 2" in Appendix 1.
2.3 Sufficiency of scalar side-information
The WZ quantizers with vector-valued decoder side-information as considered in Theorem 1 are difficult to design in practice. However, the following theorem establishes that when CKLT is used and RD-WZQ model applies for quantization of coefficients, a linear transformation of the side-information vector can be used to convert the vector side-information problem into an equivalent scalar side-information problem. Furthermore, [[5], Section 6] shows that this result applies in asymptotic sense to the SWC-HRSQ model as well.
Theorem 3 Let the mean-zero vectors$\mathbf{X}\in {\mathbb{R}}^{{M}_{1}}$and$\mathbf{Y}\in {\mathbb{R}}^{{M}_{2}}$be jointly Gaussian, and let T be the CKLT of X given Y. Suppose that transform coefficients U_{ m } , m = 1, . . ., M_{1}, where U = T^{ T }X, are each compressed by an RD optimal WZ quantizer relative to decoder side-information Y. Then, the minimum MSE (MMSE) estimate ũ_{ m }( y ) = E{U_{ m }| y} of U_{ m } given Y = y is a sufficient statistic for decoder side-information for quantizing U_{ m } .
Proof 3 See Section "Proof of Theorem 3" in Appendix 1.
Wyner-Ziv transform coding is a special case of more general MT-TC where two or more terminals apply TC to their respective inputs and transmit the quantized outputs to a single decoder which exploits the inter-source correlation to jointly reconstruct all the sources. In this case, the problem is to optimally allocate a given bit budget among all the terminals such that the total MSE is minimized. However, the closed-from solution to this problem appears difficult, due to the inter-dependence of the encoders in different terminals. An iterative descent algorithm is given in [8] for solving the Gaussian MT-TC problem. Given a total bit-budget, the bit-rate of the system is incremented by a small amount in each iteration, and the optimal WZ-TC for each terminal is determined by fixing the encoders of all other terminals and considering their outputs as decoder-side information. The solution that gives the MMSE is accepted and the iterations are repeated until the total bit-budget is exhausted. While this algorithm, referred to as the DKLT algorithm, is guaranteed to converge to at least a locally optimal solution, there is no tractable way to implement the quantizers implied by the final solution since it is not practical to optimize a set of near-optimal WZ quantizers in each iteration of this algorithm. Note also that, DKLT requires joint decoding of two vector sources.
3 Source-splitting based distributed TC
In general, designing a multi-terminal VQ is more difficult than designing a WZ-VQ, due to the mutual dependence among the encoders. However, one could use WZ-VQs to realize a multi-terminal VQ by using source-splitting [12]. It is known that in the quadratic-Gaussian case, source-splitting can be used to realize any rate-pair in the achievable rate-region by only using ideal WZ-VQs which correspond to the corner-points of the achievable rate-region [[12], Section V-C], [15]. While the same optimality properties cannot be claimed for source-splitting by linear transforms, the aforementioned observation still provides us the motivation to take a similar approach in practically realizing the DTCs which can operate at arbitrary rates, by using only WZ quantizers.
The bit allocation problem can now be stated as follows:
where $\mathbf{r}={\left({r}_{1},\dots ,{r}_{2{M}_{1}+{M}_{2}}\right)}^{T}$. The explicit solution of this minimization problem is unfortunately intractable due to the inter-dependence of the three transform codes involved.
However, an explicit solution can be found for a variant of this problem obtained by fixing ${B}_{1}^{\prime}$, ${B}_{1}^{\u2033}$ and B_{2}, so that the number of bits allocated to each transform code is fixed and it is only required to optimize the bit allocation among the quantizers within each transform code. For simplicity, we refer to this problem as the constrained bit-allocation problem. In the following, an explicit solution to this problem is derived. Based on the result, we then present a tree-search algorithm to solve the unconstrained problem (8). Under both RD-WZQ and SWC-HRSQ models, the optimal transforms for Gaussian sources are CKLTs. Therefore, we refer to the solution to problem (8) as the SP-DKLT.
3.1 Solution to the constrained bit-allocation problem
3.1.1 RD optimal quantization
3.1.2 High-resolution scalar quantization and SW coding
A similar expression exists for the quantization noise variance in (17).
3.2 A tree-search solution to the unconstrained bit-allocation problem
The proposed algorithm is a generalization of a class of bit-allocation algorithms in which a small fraction ΔB of the total bit-budget B is allocated to the "most deserving" quantizer among a set of quantizers in an incremental fashion, until the entire bit-budget is exhausted [[4], Section 8.4]. Unfortunately, this type of a greedy search cannot guarantee that the final solution is overall optimal and can yield poor results in our problem where the bit allocation among three sets of dependent quantizers must be achieved. On the other hand, if the increment ΔB is chosen small enough, a near-optimal solution can be found by resorting to a tree-search. Even though a full tree-search is intractable, a simple algorithm referred to as the (M, L)-algorithm[18] exists for detecting the minimum cost path in the tree with a high probability. We use this insight to formulate a tree-search algorithm for solving the unconstrained bit allocation problem, in which a set of constrained bit allocation problems are solved in each iteration.
4 Numerical results and discussion
Source model A: Let the components of X_{1} be M_{1} consecutive samples of a first-order Gauss-Markov process with a unit-variance and the correlation coefficient |ρ| < 1, i.e., X_{1,m}= ρX_{1,(m-1)}+ Z_{ m } , m = 2, . . ., M_{1}, where Z_{ m }, m = 1, . . ., M_{1} are mean-zero iid Gaussian variables such that $E{Z}_{m}^{2}=1-{\rho}^{2}$. The auto-covariance matrix ${\sum}_{{X}_{1}}$ is a Teoplitz matrix with the first row $\left(1,\rho ,{\rho}^{2},\dots ,{\rho}^{{M}_{1}-1}\right)$. Now define the components of X_{2} to be noisy observations of the components of X_{1}, i.e., X_{2,m}= γX_{1,m}+ W_{ m } , where |γ| < 1 and W_{ m } is a mean-zero, iid Gaussian variable with $E{W}_{m}^{2}=1-{\gamma}^{2}$, m = 1, . . ., M_{1} (M_{2} = M_{1}). It follows that, ${\sum}_{{X}_{2}}$ is a Teoplitz matrix with the first row $\left(1,{\gamma}^{2}\rho ,{\gamma}^{2}{\rho}^{2},\dots ,{\gamma}^{2}{\rho}^{{M}_{1}-1}\right)$. Furthermore, the cross-covariance matrix ${\sum}_{{X}_{1}{X}_{2}}={\gamma}^{2}{\sum}_{{X}_{1}}$. Note that X_{1} and X_{2} are not statistically similar and the components of X_{1} are more correlated than those of X_{2}.
Source model B: Consider a spatial Gaussian random field in which the correlation function decays with distance d according to the squared exponential model [19]. We define the random vectors X_{1} and X_{2} to be observations picked-up by a pair of sensor arrays placed in this random filed. In this case, the auto-covariance matrix of X_{1} is given by ${\left[{\sum}_{{X}_{1}}\right]}_{ij}=exp\left\{-{\left(\alpha {d}_{ij}\right)}^{2}\right\}$, where α is a constant and d_{ ij } is the distance between X_{1,i}and X_{1,j}. The auto-covariance matrix of X_{2} also has a similar form. For simplicity assume that the sensors in each array are placed on a M × M square grid of unit spacing (i.e., M_{1} = M_{2} = M^{2}), the two arrays are on parallel planes separated by a distance r, and the two grids are aligned so that the distance between X_{1i}and X_{2i}is r for all i. With this setup, the distance between X_{1i}and X_{2j}is $\sqrt{{d}_{ij}^{2}+{r}^{2}}$, and the cross-covariance matrix is given by ${\left[{\sum}_{{X}_{1}{X}_{2}}\right]}_{ij}=\theta \phantom{\rule{0.3em}{0ex}}exp\left\{-{\left(\alpha {d}_{ij}\right)}^{2}\right\}$, where θ = exp {- (αr)^{2}}. This sensor structure ensures that X_{1} and X_{2} are statistically similar. However, ${\sum}_{{X}_{1}{X}_{2}}$ can be chosen independently (by choosing array separation r) of ${\sum}_{{X}_{1}}$ and ${\sum}_{{X}_{2}}$.
4.1 RD performance
Bit allocations found by the tree-search algorithm for 16-dimensional SP-DKLT coding of source model A (ρ = 0.9, γ = 0.9)
X_{1} Split | X_{2} Split | ||||||||
---|---|---|---|---|---|---|---|---|---|
X _{1} | X _{2} | X _{2} | X _{1} | ||||||
${\mathit{B}}_{\mathbf{1}}^{\mathbf{\prime}}$ | ${\mathit{B}}_{\mathbf{1}}^{\u2033}$ | B _{1} | B _{2} | B | ${\mathit{B}}_{\mathbf{1}}^{\prime}$ | ${\mathit{B}}_{\mathbf{1}}^{\u2033}$ | B _{1} | B _{2} | B |
0 | 14.8 | 14.8 | 96 | 110.8 | 0 | 22 | 22 | 96 | 118 |
0 | 15 | 15 | 80 | 95 | 0 | 22.4 | 22.4 | 80 | 102.4 |
0 | 15.8 | 15.8 | 64 | 79.8 | 0 | 22.6 | 22.6 | 64 | 86.6 |
0 | 19 | 19 | 48 | 67 | 0 | 24.4 | 24.4 | 48 | 72.4 |
0 | 22 | 22 | 44 | 66 | 0 | 25.6 | 25.6 | 44 | 69.6 |
3.2 | 22.8 | 26 | 40 | 66 | 0 | 26.8 | 26.8 | 40 | 66.8 |
10 | 20 | 30 | 36 | 66 | 0 | 30 | 30 | 36 | 66 |
16.6 | 16.4 | 33 | 33 | 66 | 2.8 | 30.2 | 33 | 33 | 66 |
18.6 | 15.4 | 34 | 32 | 66 | 4.2 | 29.8 | 34 | 32 | 66 |
23.2 | 12.8 | 36 | 30 | 66 | 7.6 | 28.4 | 36 | 30 | 66 |
34.2 | 5.8 | 40 | 27.2 | 67.2 | 17.4 | 22.6 | 40 | 26 | 66 |
44 | 0 | 44 | 25.8 | 69.8 | 32 | 12 | 44 | 22 | 66 |
48 | 0 | 48 | 24.4 | 72.4 | 43.8 | 4.2 | 48 | 19.4 | 68.4 |
64 | 0 | 64 | 22.6 | 86.6 | 64 | 0 | 64 | 16 | 90 |
80 | 0 | 80 | 22.2 | 102.2 | 80 | 0 | 80 | 15.2 | 95.2 |
96 | 0 | 96 | 22 | 118 | 96 | 0 | 96 | 14.8 | 110.8 |
4.2 Design examples
In this section, we focus on the practical design of SP-DKLT codes for a given pair of rates (R_{1}, R_{2}) based on both scalar and block-quantization. RD-WZQ model used in Section 3.1.1 implies infinite block-length WZ-VQ of each coefficient. A practically realizable approach to block WZ quantization is SWC-TCQ [20]. Experimental results obtained with LDPC codes of block length up to 10^{6} bits and TCQs up to 8,192 states are presented in [20] for quadratic Gaussian WZ quantization, which indicate that performance very close to the theoretical limit can be achieved with SWC-TCQ. Motivated by these results, we aim to implement SP-DTCs which can approach theoretical performance predicted in Section 3.1.1 using TCQ and SW coding for encoding transform coefficients. However, the SWC-TCQ design procedure followed in [20] is to first design a TCQ whose MSE satisfies a constraint (by choosing a sufficiently high rate) and then to estimate the output conditional entropy (which is the target rate of the SW code) of the resulting TCQ. This is sufficient for verifying the achievable rate pairs for a given MSE which is the goal of [20]. Our problem is different in that the rate of the SW code is specified by the solution to the bit-allocation problem and our goal is to design a TCQ which minimizes the MSE, subject to a constraint on the output conditional entropy. This requires an alternative formulation of the design procedure, which we refer to as CEC-TCQ. In previous work on non-distributed quantization, entropy constrained TCQ (EC-TCQ) has been investigated in [21–24]. CEC-TCQ is a modification of EC-TCQ in [21, 22] to accommodate block SW-coding of the TCQ output relative to a decoder side-information sequence. Our formulation of CEC-TCQ follows the supersetentropy formulation of EC-TCQ in [22].
where $B\in \left\{{b}_{1},\dots ,{b}_{{2}^{{R}_{\mathsf{\text{TCQ}}}}}\right\}$ is the binary-labeled output of the TCQ. Given a TCQ code-book, the probabilities P(B = b_{ i } |Ŷ = η_{ k } ) and P(Ŷ = η_{ k } |B = b_{ i } ) can be estimated using the training set [it is sufficient to estimate p_{ i, k } = P(B = b_{ i }, Ŷ = η_{ k } ), $k=1,\dots \mathcal{Y},i=1,\dots ,N$]. To complete the design, it is necessary to search for the value of β for which E{- log_{2}P (Û_{ n } |Y_{ n } )} ≈ R by repeating the codebook optimization for an appropriately chosen sequence of β values.
For block WZ-code designs, the transforms and the bit allocations are found by RD-WZQ model (Section 3.1.1) and WZ quantizers are implemented using CEC-TCQ followed by binary SW coding. More specifically, the rate found by the bit allocation algorithm for each transform coefficient is used as the conditional entropy constraint in the design of a CEC-TCQ for that coefficient. As described in Section 2.3, the CEC-TCQ designs are based on scalar-side information obtained by a linear transform of the vector side-information at the decoder, see Theorem 3. All CEC-TCQ designs are based on the 8-state trellis used in JPEG2000 [[26], Figure 3.16]. For trellis encoding and decoding, a sequence length of 256 source samples has been used. For design and testing quantizers, sample sequences of length 5 × 10^{5} have been used. Since, the main focus this paper is the design of transforms and the quantizers, we assume ideal SW coding of the binary output of each CEC-TCQ, so that our results do not depend on any particular SW coding method. In a practical implementation (e.g., [20]), near optimal performance can be obtained by employing a sufficiently long SW code (note that sequence length for SW-coding can be chosen arbitrary larger than the sequence length used for TCQ encoding). This type of coding is well suited for applications such as distributed image compression, where the coding is inherently block-based.
We also consider SP-DKLT code designs based on scalar quantization. In this case, the transforms and bit allocations are found by using the SWC-HRSQ model (Section 3.1.2). While it is possible to use the step-size predicted by SWC-HRSQ model to design uniform quantizers, we found that such quantizers in reality do not satisfy the required entropy constraint at lower rates. We instead use conditional entropy constrained scalar quantizers (CEC-SQ), designed by modifying the algorithm in [27] to accommodate a conditional entropy constraint similar to CEC-TCQ approach above.
RSNR (in dB) of KLT-based transform code designs for quantizing 16-dimensional vectors (M_{1} = M_{2} = 16) of source model B (α = 0.32, θ = 0.9) at R_{1} = R_{2} = R bits/sample
R | IKLT | SP-DKLT | |||
---|---|---|---|---|---|
(bits/sample) | EC-SQ | EC-TCQ | CEC-SQ | CEC-TCQ | |
0.5 | 10.3 | 11.7 | 12.4 | 14.0 | (Analytical) |
10.3 | 11.0 | 11.6 | 12.7 | (design) | |
1 | 16.6 | 18.0 | 19.1 | 20.6 | |
16.4 | 17.3 | 18.6 | 19.6 | ||
1.5 | 21.6 | 23.0 | 23.8 | 25.6 | |
21.7 | 22.5 | 23.3 | 24.4 | ||
2 | 25.7 | 27.1 | 28.2 | 29.8 | |
25.2 | 26.6 | 27.8 | 28.6 |
SNR (in dB) of DCT-based transform code designs for quantizing 16-dimensional vectors (M_{1} = M_{2} = 16) of source model B (α = 0.32, θ = 0.9) at R_{1} = R_{2} = R bits/sample
R | IDCT | SP-DDCT | |||
---|---|---|---|---|---|
(bits/sample) | EC-SQ | EC-TCQ | CEC-SQ | CEC-TCQ | |
0.5 | 7.9 | 9.1 | 11.7 | 12.6 | (Analytical) |
8.0 | 8.4 | 11.0 | 12.0 | (design) | |
1 | 11.9 | 13.2 | 15.9 | 17.8 | |
11.9 | 12.6 | 16.6 | 17.0 | ||
1.5 | 15.3 | 16.8 | 19.7 | 21.9 | |
15.3 | 16.1 | 20.5 | 21.3 | ||
2 | 18.9 | 20.3 | 23.6 | 25.6 | |
18.7 | 19.4 | 24.5 | 25.1 |
5 Concluding remarks
Rate-distortion analysis and experimental results demonstrate that SP-DTC is a promising practical approach to implementing distributed VQ of high-dimensional correlated vectors. The comparisons shown in Table 2, as well as similar comparisons for source model A and the source model in [[8], Example 6], indicate that these codes can substantially outperform the independent transform codes, when there is sufficient inter-vector correlation. This approach has also been demonstrated to be effective for DCT-based systems. Therefore, the proposed approach can be potentially used in applications such as stereo image compression when inter-camera communication is impractical. Our RD analysis however indicates that the achievable rate-region of SP-DKLT codes for jointly Gaussian sources is strictly inside that predicted by the DKLT of [8]. An interesting avenue of future work is to find implementable distributed transforms codes which can achieve the rate-pairs below the "time-sharing" line in Figure 6. Another issue is the extension of the proposed approach to more than two vector sources. In principle, source-splitting can be easily applied to more than two sources. However, with more than two vector sources, the complexity of the bit-allocation will be significantly higher.
1 Appendix
1.1 Proof of Theorem 2
for m = 1, . . ., N.
1.2 Proof of Theorem 3
For jointly Gaussian and mean-zero X and Y, there exists a matrix A and a mean-zero Gaussian vector W_{1} independent of Y such that X = AY + W_{1}, where $\mathbf{A}={\sum}_{XY}{\sum}_{Y}^{-1}$, ${\sum}_{{W}_{1}}={\sum}_{X|Y}=\mathbf{T}\mathbf{\Lambda}{\mathbf{T}}^{T}$, and Λ is the diagonal matrix of eigenvalues of ∑_{X|Y}. Furthermore U = T^{ T }AY + W_{2}, where W_{2} = T^{ T }W_{1} is an uncorrelated Gaussian vector since ${\sum}_{{W}_{2}}={\mathbf{T}}^{T}{\sum}_{{W}_{1}}\mathbf{T}=\mathbf{\Lambda}$ (note that for CKLT, T^{-1} = T). Therefore ∑_{U|Y}= Λ. The MMSE estimate of U given Y is Ũ = E{U|Y} = T^{ T }E{X | Y} = T^{ T }AY. Thus, U = Ũ + W_{2} and ∑_{ U|Ũ }= Λ, and it follows that U_{ i } is independent of Ũ_{ j } if j ≠ i and var(U_{ i }| Ũ_{ i } ) = var(U_{ i } | Y), where var(·|·) denotes the conditional variance. Now, since h(U_{ i }| Y) = h(U_{ i }| Ũ_{ i } ), we conclude that Ũ_{ i } is a sufficient statistic [16] for decoder side-information Y in WZ quantization of U_{ i } .
Declarations
Authors’ Affiliations
References
- Xiong Z, Liveris AD, Cheng S: Distributed source coding for sensor networks. IEEE Signal Process Mag 2004, 21(5):80-94.View ArticleGoogle Scholar
- Huang JY, Schultheiss P: Block quantization of correlated random vectors. IEEE Trans Commun 1963, 11(9):289-296.View ArticleGoogle Scholar
- Goyal V: Theoretical foundations of transform coding. IEEE Signal Process Mag 2001, 18(5):9-21.View ArticleGoogle Scholar
- Gersho A, Gray RM: Vector Quantization and Signal Compression. Kluwer Academic Publishers, Norwell MA, USA; 1992.View ArticleGoogle Scholar
- Rebollo-Monedero D, Rane S, Aaron A, Girod B: High-rate quantization and transform coding with side information at the decoder. Signal Process 2006, 88(11):3160-3179.View ArticleGoogle Scholar
- Vosoughi A, Scaglione A: Precoding and decoding paradigms for distributed vector data compression. IEEE Trans Signal Process 2007, 55(4):1445-1459.MathSciNetView ArticleGoogle Scholar
- Chen X, Tuncel E: Low-delay prediction and transform-based Wyner-Ziv coding. IEEE Trans Signal Process 2011, 59(2):653-666.MathSciNetView ArticleGoogle Scholar
- Gastpar M, Dragotti PL, Vetterli M: The distributed Karhunen-Loéve transform. IEEE Trans Inf Theory 2006, 52: 5177-5196.MathSciNetView ArticleGoogle Scholar
- Slepian D, Wolf JK: Noiseless coding of correlated information sources. IEEE Trans Inf Theory 1973, 19(4):471-480.MathSciNetView ArticleGoogle Scholar
- Gish H, Peirce JP: Asymptotically efficient quantizing. IEEE Trans Inf Theory 1968, 14: 676-683.View ArticleGoogle Scholar
- Rimoldi B, Urbanke R: Asynchronous Slepian-Wolf coding via source-splitting. In IEEE Int Symp Inform Theory (ISIT). Ulm, Germany; 1997:271.View ArticleGoogle Scholar
- Zamir R, Shamai S, Erez U: Nested linear/lattice codes for structured multiterminal bining. IEEE Trans Inf Theory 2002, 48(6):1250-1276.MathSciNetView ArticleGoogle Scholar
- Wagner AB, Tavildar S, Viswanath P: Rate region of the quadratic Gaussian two-encoder source-coding problem. IEEE Trans Inf Theory 2008, 54(5):1938-1961.MathSciNetView ArticleGoogle Scholar
- Wyner AD, Ziv J: The rate-distortion function for source coding with side information. IEEE Trans Inf Theory 1976, 22: 1-10.MathSciNetView ArticleGoogle Scholar
- Yang Y, Stankovic V, Xiong Z, Zhao W: On multiterminal source code design. IEEE Trans Inf Theory 2008, 54(5):2278-2302.MathSciNetView ArticleGoogle Scholar
- Cover TM, Thomas JA: Elements of Information Theory. John Wiley, New York, USA; 1991.View ArticleGoogle Scholar
- Marco D, Neuhoff DL: The validity of additive noise model for uniform scalar quantizers. IEEE Trans Inf Theory 2005, 51(5):1739-1755.MathSciNetView ArticleGoogle Scholar
- Berger T: Rate Distortion Theory: A Mathematical Basis for Data Compression. Prentice-Hall, Englewood Cliffs NJ, USA; 1971.Google Scholar
- Vuran MC, Akan OB, Akylidz IF: Spatio-temporal correlation: theory and applications for wireless networks. Comput Netw 2004, 45: 245-259.View ArticleGoogle Scholar
- Yang Y, Cheng S, Xiong Z, Zhao W: Wyner-Ziv coding based on TCQ and LDPC codes. IEEE Trans Commun 2009, 57(2):376-387.View ArticleGoogle Scholar
- Fischer TR, Wang M: Entropy-cosntrianed trellis-coded quantization. IEEE Trans Inf Theory 1992, 38(2):415-426.View ArticleGoogle Scholar
- Marcellin MW: On entropy-cosntrianed trellis-coded quantization. IEEE Trans Commun 1994, 42: 14-16.View ArticleGoogle Scholar
- Marcellin MW: Transform coding of images using trellis-coded quantization. In Proc ICASSP. Albuquerque NM, USA; 1990:2241-2244.Google Scholar
- Farvardin N, Ran X, Lee CC: Adaptive DCT coding of images using entropy-cosntrianed trellis-coded quantization. In Proc ICASSP. Minneapolis MN, USA; 1993:397-400.Google Scholar
- Marcellin MW, Fischer TR: Trellis coded quantization memoryless and Gauss-Markov sources. IEEE Trans Commun 1990, 38: 82-93.MathSciNetView ArticleGoogle Scholar
- Taubman DS, Marcellin MW: JPEG 2000: Image Compression Fundamentals, Standrads and Practice. Kluwer Academic Publishers, Norvell MA, USA; 2004.Google Scholar
- Chou P, Lookabaugh T, Gray RM: Entropy-constrained vector quantization. IEEE Trans Acoust Speech Signal Process 1989, 37: 31-42.MathSciNetView ArticleGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.