Joint Source and Channel Decoding for Variable Length Encoded Turbo Codes

Joint source and channel decoding (JSCD) has been proved to be an e ﬀ ective technique which can improve decoding performance by exploiting residual source redundancy. Most previous publications on this subject focus on a traditional coding scheme in which the source variable-length coding (VLC) is serially concatenated with a channel code. In this paper, a parallel concatenated coding scheme for the VLC combined with a turbo code is presented. By merging a symbol-level VLC trellis with a convolutional trellis, we construct a symbol-level joint trellis with compound states. Also, a solution of the symbol-by-symbol a posteriori probability (APP) decoding algorithm based on this joint trellis is derived, which leads to an iterative JSCD approach in the similar way to the classical turbo decoder. The simulation results show that our joint source-channel en/decoding system achieves some gains at the cost of increasing decoding complexity, when compared to the joint iterative decoding based on the bit-level super trellis for the separate coding system.


INTRODUCTION
Variable-length coding (VLC) is an effective technique to remove source redundancy.It is in turn essential for many communication applications, including text, voice, images, and video.Unfortunately, it is very sensitive to even a single binary error, which leads to error propagation, thus channel coding is always employed after source coding.In the classical communication system, the two coding parts are usually optimized separately, which has been theoretically justified by Shannon's source-channel separation theory [1].But the separation theory holds only under asymptotic conditions, where both codes are allowed infinite length and complexity.If the practical system is heavily constrained by complexity or delay, the separate source-channel coding can be largely suboptimal.These arguments have motivated the active research areas of joint source and channel coding/decoding (JSCC/JSCD).
Several recent studies on the JSCD technology focus on iterative decoding for the VLC concatenated with a convolutional code through an interleaver.Obvious performance gains can be obtained via iterations between two soft-input soft-output (SISO) modules.Bauer and Hagenauer [2,3] proposed an iterative scheme based on a symbol-level VLC trellis and derived a symbol-based a posteriori probability (APP) algorithm by modifying the Bahl-Cocke-Jelinek-Raviv (BCJR) algorithm [4].They also studied a bit-level iterative decoding [5] based on the Balakirsky's trellis [6], which was well suited for long data packets.The extended work on the VLC serially concatenated [7,8] or parallel concatenated [9] with a single convolutional code for the first-order Markov source was considered by Kliewer et al. [7,9] Due to the outstanding decoding performance at low signal-to-noise ratio (SNR) level, turbo codes [10] have entered into service in many communication applications; naturally, the new subject of JSCD for the VLC with a turbo code attracted increasing research interests in recent years.Laković and Villasensor [11] suggested a model of iterative decoding between two SISO modules, in which the first constituent code of a turbo code was decoded based on the bit-level super trellis.Also, Guivarch et al. [12] and Jeanne et al. [13,14] proposed a method that a priori source information of Huffman codes in the form of bit transition probabilities was introduced at a bit level.Peng et al. [15] studied a feedback approach by modifying the extrinsic information.Additionally, an iterative scheme with three SISO decoding modules was presented by Jaspar and Vandendorpe [16,17].
We can observe that most of the above joint sourcechannel decoding methods focus on the classical coding system with serial concatenation.Moreover, the performance of the VLC parallel concatenated with a single convolutional code [9] is not optimistic without the protection for the VLC systematic information.Additionally, in the above cases combined with turbo codes [11][12][13][14][15][16][17], residual source redundancy is usually utilized at the decoder side with the bitlevel joint decoding, while no improvement has been done at the encoder side.The main contribution of this paper is to present a different parallel joint source-channel coding scheme by combining the VLC with a turbo code, and then suggest its JSCD method based on the proposed symbollevel joint trellis.In this paper, we do not present a rigorous proof of stability and convergence of iterative decoding; however, simulation results indicate that the proposed scheme achieves some gains at the cost of increasing decoding delay, when compared to the bit-level joint iterative decoding scheme [11].
This paper is organized as follows.Section 2 gives the basic transmission system.Section 3 presents the proposed symbol-level joint trellis, and a symbol-level APP decoding algorithm suited for the new joint trellis is illustrated in Section 4. Iterative decoding of our system is discussed in Section 5.The simulation results are presented in Section 6, and conclusions of our work are made in Section 7.

TRANSMISSION SYSTEM
The model of our transmission system is depicted in Figure 1.Different from the traditional turbo coding system with serial concatenation, the VLC is integrated with the upper recursive systematic convolutional (RSC) code of a turbo code into a single constituent code.We assume the VLC is implemented on a K-symbol source sequence U = [U 1 , U 2 , . . ., U K ], where each U k , k = 1, 2, . . ., K from a finite source alphabet U, must be mapped to a variable length codeword c(U k ).The output of the source variable length encoder can be denoted as which is composed of K variable length codewords, or denoted as a binary sequence w s = [w s1 , w s2 , . . ., w sN1 ] with the total bit length N 1 .The VLC sequence w s is then protected by the RSC1 code, which produces a parity check sequence w p = [w p1 , w p2 , . . ., w pN1 ].Utilizing a Q-bit quantizer, the symbol sequence U is converted into a bit sequence This bit sequence U is interleaved by Π, and then channel coded by another RSC2 encoder.However, only the parity check sequence v p = [v p1 , v p2 , . . ., v pN2 ] is reserved since the systematic information has already been included in w s .In our case, the RSC2 encoder with the memory length μ 2 is terminated, so the sequence length N 2 equals (K•Q + μ 2 ).Note that quantizing the source symbols directly and then coding the sequence U with the RSC2 encoder increases some redundancy, while the higher channel code rate can be achieved by puncturing the parity check sequence v p to v p .Finally, w s , w p and v p are passed through a multiplexer and then sent to the wireless channel.
We assume the coherently detected binary phase-shift keying (BPSK) modulation, and signals are transmitted over the additive white Gaussian noise (AWGN) channel.After the channel output is received, an iterative JSCD between two SISO modules is carried out in order to obtain the decoding output

REPRESENTATION OF JOINT TRELLIS
As mentioned above, in our joint turbo coding scheme, the serial concatenated VLC and an RSC code are treated as a single constituent code, it should be appropriate to deal with decoding of these two parts as a single module, which has inspired us to construct a symbol-level joint trellis.
We assume a K-symbol source sequence is variablelength encoded to an N-bit binary sequence.As suggested by Bauer and Hagenauer [3], this process can be denoted by a symbol-level VLC trellis.Figure 2 gives an example of the trellis representation, which corresponds to a foursymbol code table with the constraint of K = 5 and N = 10.In the trellis, the state index n denotes the bit length of the sequence after k source symbols have been variablelength encoded (e.g., n 3 = 7), and the state transition from n k−1 = n 1 to n k = n 2 is caused by a variable length codeword c k ∈ C, with the bit length l(c k ) = n 2 − n 1 .Especially, all available states at the symbol instant k belong to a subset R k (e.g., R 3 ).With the transform where l min is the minimum codeword length in the table C, the symbol-level VLC trellis in Figure 2 can be transformed to another VLC KV-trellis [3] with a different state index v instead of n.
We can further represent the process of the VLC and the convolutional coding by a single joint KT-trellis.At the symbol instant k, if a state in the VLC KV-trellis is denoted as v k , and a state in the convolutional trellis is denoted as S k (the value of shift registers), then a certain state in our joint KTtrellis can be written as T k = (v k , S k ), which actually consists of two substates.An example under the constraint of K = 5 and v max = 5 (the maximal v caused by N = 10 and l min = 1) is shown in Figure 3, which is derived from the KV-trellis for the code table C and a two-state RSC encoder with code polynomials G 1 = (3, 1) 8 .Similarly to Figure 2, all available states at the symbol instant k belong to a set R k (e.g., R 3 ).Especially, each transition (T k−1 , T k ) between two state nodes must correspond to a pair of variable-length input/output codewords (e.g., c(1)/0010).In Figure 3, there are two terminating states in the joint trellis since a two-state RSC code is considered.This time-varying joint trellis can be utilized for the symbol-level APP decoding.2) c( 1) c(0) }, with K = 5 and N = 10.

SYMBOL-LEVEL APP ALGORITHM FOR JOINT TRELLIS
In this section, we give a description of the modified APP algorithm suitable for the symbol-level joint trellis, especially, an independent memoryless source is considered here.Codeword sequences w s and w p from the first constituent code are compounded and BPSK-modulated into a sequence X 2N1 is channel coded and then BPSK-modulated into a codeword x k (i, t , t) with the length 2•l(c(i)), which is associated with the state transition (T k−1 = t , T k = t).In addition, if the bit length of the VLC sequence w s associated with a compound state t is denoted as n(t), we can represent the bit length of the channel sequence produced by the upper constituent code as m(t) = 2•n(t), and the maximum of m(t) should be M = 2•N 1 .Note that, the transform between the parameter m(t) and the substate v(t) can be obtained from (1).
The key point of our decoding algorithm is to calculate symbol-based APPs for each VLC codeword c k = c(i) giving the observations Y M 1 .Using Bayesian principles, we have where C = 1/ p(Y M 1 ) is a constant term.We name α k (t) as the forward recursion, β k (t) as the backward recursion, and , t , t) as the transition probability from t to t associated with the input codeword c k = c(i), respectively.The forward recursion α k (t) can be calculated from Similarly, the backward recursion can be calculated from If the memory length of the RSC1 encoder is μ 1 , the initial conditions for performing the backward recursion are Using Bayesian principles, the transition probability can be finally factorized into three terms as follows: Note that, the implementation of the above symbol-level algorithm should be performed in logarithm domain [18].

ITERATIVE DECODING FOR VARIABLE LENGTH ENCODED TURBO CODES
The basic iterative decoding model of the system is shown in Figure 4. We denote a priori information as L ai (i = 1, 2) and logarithm likelihood ratios as L i (i = 1, 2) for the two constituent decoders.The inner decoder for the RSC2 code first decodes the observations by a bit-level APP algorithm, and the bit-based extrinsic information L e2 is passed to the outer decoder after being deinterleaved by Π −1 .Whereafter, the joint symbol-level APP decoder carries out a symbolbased APP algorithm as described in Section 4, however, the feedback information from the outer decoder to the inner decoder is the systematic extrinsic information L s&e1 owing to the nonsystematic property of the VLC.After the last iteration, a symbol decision is made on L 1 and we get the symbol sequence estimation U.
The function of T −1 in Figure 4 is to transform a priori information from bit levels to symbol levels.Since the VLC is performed as a special one-to-one mapping from a source symbol U k = i, i ∈ U, to a VLC codeword c k = c(i), it is equivalent to represent a priori information for a codeword c k = c(i) by a priori information for a source symbol U k = i, as We further assume all bits u kl , l = 1, 2, . . ., Q within the quantized codeword U k are uncorrelated.Then symbol a priori probability P a (U k = i) can be calculated from a multiplication of several bit a priori probabilities P a (u kl = i l ) as where i l ∈ {0, 1} is the lth bit of the quantized symbol i. Utilizing ( 7) and ( 8), symbol a priori information L a (U k = i) can be finally written as the summation of those bit a priori information L a (u kl ) with i l = 1 as follows: Correspondingly, the function of T in Figure 4 is to convert the symbol-based a priori information L a (U k = i) to the bit-based a priori information L a (u kl ) according to We further replace all probability terms P a (U k = i) in ( 10) by L a (U k = i) according to (7).Based on the Jacobian logarithm [18], the bit-based a priori information can be approximately calculated from

SIMULATION RESULTS
In this section, simulations were performed over the AWGN channel with BPSK in order to access the performance of the proposed joint en/decoding approach.The VLC was carried out on the independent memoryless source with 4 symbols, and the corresponding Huffman codes and reversible VLCs (RVLCs) are listed in Table 1 [3].In our system, the first constituent code was selected to be G p1 = (3, 1) 8 in order to reduce the decoding complexity, and the second one was selected to be G p2 = (11, 12) 8 according to [9].However, both constituent codes were not optimal yet.The interleaver permuted the bit sequence pseudorandomly, where trellis termination of the RSC2 encoder was considered.Simulation comparisons were done between the proposed scheme and the joint iterative decoding scheme in [11], in which the VLC was serially concatenated with a turbo code and the upper constituent code of the turbo code was decoded based on the bit-level super trellis.Due to the higher complexity of the proposed symbol-based decoding, we selected both constituent codes to be G S = (11, 12) 8 for the separate coding scheme, in which the memory length of the upper RSC code was enlarged to μ 1 = 3.
In order to reduce the simulation delay, the bit stream was divided into several short packets with a fixed number of symbols (K = 100) as in [3,7,9].Each packet was interleaved and transmitted independently.Furthermore, we assumed the parameters K and N 1 were protected by a strong channel code and thus obtained at the receiver side without errors.The overall code rates for the separate coding scheme (R S ) and the proposed coding scheme (R P ) are denoted as where H(U) is the source entropy, L av is the average codeword length, and R ci (i = 1, 2) is the code rate for the ith constituent code, respectively.At the receiver side, the channel observations were decoded with the Max-LogMAP algorithm [18].Simulations were carried out for each E b /N 0 in dB, where E b denotes the average energy per information bit, and N 0 is the singlesided noise power spectral density.The symbol error rates (SERs) were accounted using the Levenstein distance [19].Simulations at the 4th and the 8th iterations were performed.Figure 5 shows the results for the Huffman codes with the overall code rate 0.330, and Figure 6 shows the results for the reversible VLCs with the overall code rate 0.308.These overall code rates were obtained by setting all R c1 to be 1/2 and Demultiplexer ( w s , w p , v p ) Joint symbol-level APP decoder for VLCs with RSC1  puncturing the parity check bits from the second constituent code, which resulted in R c2 , given in Table 2.
It can be found that the proposed JSCC/JSCD scheme outperforms the joint iterative decoding with the bit-level super trellis at low SER level at high SNR level after the 4th iteration or the 8th iteration.In the case of Huffman codes, the bit-level super trellis decoding yields better decoding performance at the low channel SNR level, however, the achieved reconstruction quality is relatively poor, thus this region of high SER is not of interest.Moreover, any joint iterative decoding system works well when residual redundancy exists in source coding, thus the JSCD system with Huffman codes does not show obvious performance gains even relative to the classical separate decoding.From Figures 5 and  6, compared to the bit-level super trellis decoding with the higher memory length μ 1 , the proposed JSCC/JSCD scheme achieves about 0.3 dB gains at an SER of 10 −4 at the 4th iteration for both VLCs.The influence of code memory and different code polynomials on the convergence behavior can be further analyzed by the extrinsic information transfer (EXIT) charts [20], whereas the computation of EXIT characteristics should be performed for both symbol-based decoding and bit-based decoding.
Owing to the short packet length used in the simulations (hundreds of symbols), the performance of channel coding is not close to the Shannon capacity, therefore, it is possible to obtain gains by joint en/decoding.With our parallel encoding structure through a quantizer, the outer decoder in the proposed JSCD scheme is decoded by a symbol-based APP algorithm which minimizes the symbol error rates [4] instead of a bit-based APP algorithm which minimizes the bit error rates (BERs).It maybe more suitable to source a priori characteristic because the VLC bit stream actually consists of several codeword units.This symbol-based decoding might lead to some improvement since the decoding performance is evaluated by the SERs.We should state that the performance advantage in the simulations lies on using suboptimal codes, and the optimal codes for the proposed system still need to be found.This could be implemented by a code search as mentioned in [9].In addition, the above gains are obtained at the cost of decoding complexity.If I VLC is the state number in the Balakirsky's VLC trellis, there will be 2 μ1 •I VLC time-invariant states through N 1 bit time instants in the bit-level super trellis.Nevertheless, the average of v max in the stationary section of the KV-trellis [3] is K•(L av − l min ), which increases with K, so that the state space of our joint trellis also increases

Figure 1 :
Figure 1: The model of the transmission system.

3 Figure 3 :
Figure 3: Joint KT-trellis representation with compound states derived from the VLC KV-trellis for C and a two-state RSC code.

1 = 1 =
[x 1 , x 2 , . . ., x 2N1 ] before being sent to a wireless channel.Let Y 2N1 [y 1 , y 2 , . . ., y 2N1 ] represent the channel observations of X 2N1 1 , and its subsequence from the bit position a to b is indicated as Y b a = [y a , y a+1 , . . ., y b ].At the symbol instant k, a VLC codeword c k = c(i) with the length l(c(i))

Figure 4 :
Figure 4: Iterative decoding model of the system.

Figure 5 :
Figure 5: Simulation results for the Huffman codes on the AWGN channel, K = 100, and the overall code rate is 0.330.

Table 1 :
Huffman codes and RVLCs used in the simulations.

Table 2 :
Code rate for the second constituent code in turbo codes.