Skip to content

Advertisement

  • Research Article
  • Open Access

Very Low Rate Scalable Speech Coding through Classified Embedded Matrix Quantization

EURASIP Journal on Advances in Signal Processing20102010:480345

https://doi.org/10.1155/2010/480345

Received: 21 June 2009

Accepted: 19 February 2010

Published: 12 April 2010

Abstract

This paper proposes a scalable speech coding scheme using the embedded matrix quantization of the LSFs in the LPC model. For an efficient quantization of the spectral parameters, two types of codebooks of different sizes are designed and used to encode unvoiced and mixed voicing segments separately. The tree-like structured codebooks of our embedded quantizer, constructed through a cell merging process, help to make a fine-grain scalable speech coder. Using an efficient adaptive dual-band approximation of the LPC excitation, where voicing transition frequency is determined based on the concept of instantaneous frequency in the frequency domain, near natural sounding synthesized speech is achieved. Assessment results, including both overall quality and intelligibility scores show that the proposed coding scheme can be a reasonable choice for speech coding in low bandwidth communication applications.

Keywords

  • Instantaneous Frequency
  • Vector Quantization
  • Training Sequence
  • Speech Quality
  • Distortion Measure

1. Introduction

Scalable speech coding refers to the coding schemes that reconstruct speech at different levels of accuracy or quality at various bit rates. The bit-stream of a scalable coder is composed of two parts: an essential part called the core unit and an optional part that includes enhancement units. The core unit provides minimal quality for the synthesized speech, while a higher quality is achieved by adding the enhancement units.

Embedded quantization, which provides the ability of successive refinement of the reconstructed symbols, can be employed in speech coders to attain the scalability property. This quantization method has found useful applications in variable-rate and progressive transmission of digital signals. The output symbol of an -bit quantizer, in an embedded quantizer, is embedded in all output symbols of the ( )-bit quantizers, where [1]. In other words, higher rate codes contain lower rate codes plus bits of refinement.

Embedded quantization was first introduced by Tzou [1] for scalar quantization. Tzou proposed a method to achieve embedded quantization by organizing the threshold levels in the form of binary trees, using the numerical optimization of Max [2]. Subsequently, embedded quantization was generalized to vector quantization (VQ). Some examples of such vector quantizers, which are based on the natural embedded property of tree-structured VQ (TSVQ), can be found in [35]. Ravelli and Daudet [6] proposed a method for embedded quantization of complex values in the polar form which is applicable to some parametric representations that produce complex coefficients. In the scalable image coding method introduced in [7] by Said and Pearlman, wavelet coefficients are quantized using scalar embedded quantizers.

Even though broadband technologies have significantly increased transmission bandwidth, heavy degradation of voice quality may occur due to the traffic-dependent variability of transmission delay in the network. A nonscalable coder operates well only when all bits, representing each frame of the signal, are recovered. Conversely, a scalable coder adjusts the need for optional bits, based on the data transmission quality, which could have significant impact on the overall performance of the reconstructed voice quality. Accordingly, only the core information is used for recovering the signal in the case of network congestion [8].

Scalable coders may also be used to optimize a multi-destination voice service in case of unequal or varying bandwidth allocations. Typically, voice servers have to produce the same data at different rates for users demanding the same voice signal [6]. This imposes an additional computational load on the server that may even result in congesting the network. A scalable coder can resolve this problem by adjusting the rate-quality balance and managing the number of optional bits allocated to each user.

A desirable feature of a coder is the ability to dynamically adjust coder properties to the instantaneous conditions of transmission channels. This feature is very useful in some applications, such as DCME (Digital Circuit Multiplication Equipment) and PCME (Packet Circuit Multiplication Equipment), in overload situations (too many concurrent active channels), "in-band" signaling, or "in-band" data transmission [9]. In case of varying channel condition that could lead to various channel error rates, a scalable coder can use a lengthier channel code, which in turn forces us to lower the source rate when bandwidth is fixed, to improve the transmission reliability. This is basically a tradeoff between voice quality and error correction capability.

Scalability has become an important issue in multimedia streaming over packet networks such as the Internet [9]. Several scalable coding algorithms have been proposed in literature. The embedded version of the G.726 (ITU-T G.727 ADPCM) [10], the MPEG-4 Code-Excited Linear Prediction (CELP) algorithm, and the MPEG-4 Harmonic Vector Excitation Coding (HVXC) are some of the standardized scalable coders [5]. The recently standardized ITU-T G.729.1 [11], an 8–32 kbps scalable speech coder for wideband telephony and voice over IP (VoIP) applications, is scalable in bit rate, bandwidth and computational complexity. Its bitstream comprises 12 embedded layers with a core layer interoperable with ITU-T G.729 [12]. The G.729.1 output bandwidth is 50–4000 Hz at 8 and 12 kbit/s and 50–7000 Hz from 14 to 32 kbit/s (per 2 kbit/s steps). A Scalable Phonetic Vocoder (SPV), capable of operating at rates 300–1100 bps, is introduced in [13]. The proposed SPV uses a Hidden Markov Model (HMM) based phonetic speech recognizer to estimate the parameters for a Mixed Excitation Linear Prediction (MELP) speech synthesizer [14]. Subsequently, it employs a scalable system to quantize the error signal between the original and phonetically-estimated MELP parameters.

In this paper, we introduce a very low bit-rate scalable speech coder by generalizing embedded quantization to matrix quantization (MQ), which is our main contribution in this paper. The MQ scheme, to which we add the embedded property, is based on the split matrix quantization (SMQ) of the line spectral frequencies (LSFs) [15]. By exploiting the SMQ, both the computational complexity and the memory requirement of the quantization are significantly reduced. Our embedded MQ coder of the LSFs leads to a fine-grain scalable scheme, as shown in the next sections.

The rest of the paper is organized as follows. Section 2 describes the method used to produce the initial codebooks for an SMQ. In Section 3, the embedded MQ of the LSFs is presented. Section 4 is devoted to the model of the linear predictive coding (LPC) excitation and determination of the excitation parameters, including band-splitting frequency, pitch period, and voicing. Performance evaluation and some experimental results using the proposed scalable coder are given in Section 5 with conclusions presented in Section 6.

2. Initial Codebook Production for SMQ

In our implementation, the LSFs are used as the spectral features in an MQ system. Each matrix is composed of four 40 ms frames, each frame extracted using a hamming window of 50% overlap with adjacent frames, that is, a frame shift of 20 ms, sampled at 8 kHz. The LSF parameters are obtained from an LPC model of order 10, based on the autocorrelation method.

One of the problems we encounter in the codebook production for the MQ is the high computational complexity that usually forces us to use short training sequence or codebooks of small sizes. Although this is an one time process for the training of each codebook, it is time consuming to tune the codebooks by changing some parameters. In this case, writing fast codes (e.g., see [16]), exploiting a computationally modest distortion measure, and suboptimal quantization methods, make the MQ scheme feasible even for processors with moderate processing power. Multistage MQ (MSMQ) [17, 18] and SMQ [15] are two possible solutions to suboptimality in MQ. The Suboptimality of these quantizers mostly arises from the fact that not all potential correlations are used. By using SMQ, we achieve both a lower computational complexity for the codebook production and a lower memory requirement, as compared to a nonsplit MQ.

The LSFs are ideal for split quantization. This is because the spectral sensitivity of these parameters is localized; that is, a change in a given LSF merely affects neighboring frequency regions of the LPC power spectrum. Hence, split quantization of the LSFs cause negligible leakage of the quantization distortion from one spectral region to another [19].

The best dimensions of submatrices resulting from splitting the spectral parameters matrix is addressed according to the empirical results given by Xydeas and Papanastasiou in [15]. It is shown that with four-frame length matrices of the spectral parameters and an LPC frame shift of 20 ms, the matrix quantizer operates effectively at 12.5 segments per second. This is comparable to the average phoneme rate and thus makes it possible to exploit most of the existing interframe correlation [15]. In addition, they found that the best SMQ performance at low rates was achieved when the spectral parameters matrix (assuming a size for each matrix of LSFs) was split into five equal dimension size submatrices , = , given by
(1)

where indicates the th LSF in the th analysis frame.

One of the most important issues in the design and operation of a quantizer is the distortion metric used in codebook generation and codeword selection from codebooks during quantization. The distortion measure we use here is the squared Frobenius norm of weighted difference between the LSFs, defined as
(2)
The operator given in (2) stands for the Hadamard matrix product that is an element-by-element multiplication [20]. The input matrix, , is considered as the th split of the matrix of the spectral parameters beginning with the th frame. The reference matrix, , in (2) can be a codeword of the th split codebook. The time weighting matrix, , is to weight frames having a higher energy more than lower energy frames, as they are subjectively more important. Elements of the th column ( ) of this matrix are identical and are proportional to the power of the ( )th speech frame, given by
(3)

where s(n) represents the speech signal, and stand for the frame shift and the frame length, respectively. According to [15], = 0.15 is a reasonable choice.

The definition of the spectral weighting matrix, , is based on the weighting proposed by Paliwal and Atal [19]. The ( )th element of this matrix is proportional to the value of the power spectrum at corresponding LSFs of the frames included in the segment to be encoded, as
(4)
As we know, quantization of unvoiced frames can be done with a lower precision, as compared to voiced frames, with a negligible loss of quality. Accordingly, we exploit two types of codebooks: one for quantization of segments containing only unvoiced frames, , , and another for segments including either all voiced frames or a combination of voiced and unvoiced frames, , . The unvoiced codebook, , is of smaller size in comparison to the mixed voicing codebook, . This selective codebook scheme leads to a classification-based quantization system that is known as classified quantizer ([3, pages 423-424]). This quantizer encodes the spectral parameters at different bit rates, depending on the voicing information, and thus leads to a variable rate coding system. In this two-codebook design, an extra bit is employed for the codebook selection to indicate which codebook is to be used to extract the proper codeword. Table 1 illustrates codebook sizes in our SMQ system. As shown, a lower resolution codebook is used for quantization of upper LSFs due to the lower sensitivity of the human auditory system (HAS) to higher frequencies. The bit allocation given in Table 1 results in an average bit rate of 550 bps for representing the spectral parameters.
Table 1

Number of bits allocated to the SMQ codebooks.

Codebook type

1st split

2nd split

3rd split

4th split

5th split

Total

Mixed voicing

10

10

10

9

8

47

Unvoiced

8

8

8

7

6

37

We designed codebooks of this split matrix quantizer, based on the LBG algorithm [21], using 1200 TIMIT files [22] as our training database. A sliding block technique is used to capture all interframe transitions in the training set. This is accomplished by using a four-frame window sliding over the training data in one-frame steps.

The centroid of the th voronoi region is obtained by finding the derivatives of the accumulated distortion with respect to each element of the th codeword of the SMQ codebooks and equating it to zero, leading to
(5)
where represents the voronoi region of the th codeword of the th split codebook that is, , and represents frame indexes for which belongs to . Therefore, only the submatrices of the training data that fall into the voronoi region of the th codeword are incorporated in the calculation of the centroid of the voronoi region. A closed form of the centroid calculation can be shown as
(6)
where
(7)

and the operator denotes an element-by-element matrix division.

To guarantee stability of the LPC synthesis filters, the LSFs must appear in ascending order. However, with the spectrally weighted LSF distance measure used for designing the split quantizer, the LSF ascending order is not guaranteed. As a solution, Paliwal and Atal [19] used the mean of the LSF vectors, within a given voronoi region, to define the centroid. Our solution to preserve stability of the LPC synthesis filters is to put all five generated codewords into a matrix and then sort each column of not yet ascended order columns of the reproduced spectral parameters matrix across all 5 codewords in ascending order. However, the resulting synthesis filters might become marginally stable due to the poles located too close to the unit circle. The problem is aggravated in fixed-point implementation, where a marginally stable filter can actually become unstable after quantization and loss of precision during processing. Thus, in order to avoid sharp spectral peaks in the spectrum that may lead to unnatural synthesized speech, bandwidth expansion through modification of the LPC vectors is employed. In this case, each LPC filter coefficient, a i, is replaced by , for , where = 0.99. This operation flattens the spectrum, especially around formant frequencies. Another advantage of the bandwidth expansion is to shorten the duration of the impulse response of the LPC filter, which limits the propagation of channel errors ([8, page 133]).

The next section introduces the method to construct the tree structured codebooks for the embedded quantizer, using the initial codebooks designed in this section.

3. Codebook Production for Embedded Matrix Quantizer

Consider the initial codebook generated using the SMQ method described in the preceding section. For notational convenience, we have dropped the superscript " " and subscripts "uv" and "vuv". The codewords of the codebook are denoted by
(8)
where is the number of codewords or the codebook size. We organize these initial codewords in a tree structure to determine the internal codewords of the constructed tree, such that each internal codeword is a good approximation to its children. Codewords emanating from an internal codeword are called children of that internal codeword. In a binary tree, each internal codeword has two children. The index length of each initial codeword determines the depth of the tree. Figure 1 illustrates a binary tree of depth three. We place initial codewords at the leaves of the tree. Hence, each terminal node on the tree corresponds to a particular initial codeword. To produce a tree structure having the embedded property, symbols at lower depths (farther from the leaves) must be the refined versions of the symbols at higher depths (closer to the leaves). One of the methods that can be used to incorporate the embedded property into the tree is cell-merging or region-merging method. A cell-merging tree is formed by merging the Voronoi regions in pairs and allocating new centroids to these larger encoding areas. Merging two regions can be interpreted as erasing the boundary between the regions on the Voronoi diagram [23].
Figure 1
Figure 1

A depth-3 tree structure for an embedded quantization scheme. Indexes of terminal nodes, corresponding to initial codewords, are indicated below the nodes.

Now the problem is to find the regions that should be merged to minimize the distortion of the internal codewords in their Voronoi regions. By merging the proper codewords, the constructed tree makes a fine-grain scalable system. A simple solution to this problem is to exhaustively evaluate all possible index assignment sequences for the leaves of the tree and find the corresponding tree for each sequence, and then keep the sequence that leads to the lowest total accumulated distortion (TAD) on the training sequence for all depths, as
(9)
where td = is the depth of the tree and is the sum of the accumulated distortions for all codewords in depth on the training sequence , defined as
(10)

where represents the Voronoi region of and the metric is the distance between and . It is worth mentioning that we have codewords at depth . In (10), the summation is over all valid , that is, , for which belongs to the voronoi region .

According to [4], the total number of index assignment sequences for the leaves of the tree that need to be evaluated in an exhaustive search to minimize (9) is given by
(11)

This number becomes quite large even for moderate values of . Hence, this simple solution cannot be used in practice due to its prohibitively high complexity.

Hence, in order to make the merging process feasible, we need to use more computationally efficient methods. A simple suboptimal solution is to merge the pairs of regions at depth that only minimize the accumulated distortion in depth . In this method, the total accumulated distortion on the designated cell-merging tree, defined in (9), may not come to its minimum. To choose proper pairs of the Voronoi regions to merge at depth , we may generate an undirected graph with nodes, labeled from 0 to , as shown in Figure 2. In this graph, each node corresponds to one particular codeword at depth and the arc between every two nodes is the value of accumulated distortion on the training sequence for the codeword resulting from merging two codewords at two ends of the arc.
Figure 2
Figure 2

The graph for codewords at depth . Arc value is determined based on the encountered distortion resulting from merging th and th codewords at depth .

The problem of finding proper regions to merge is similar to a complete bipartite matching problem ([24, page 182]). In fact, we must select a subset of the graph illustrated in Figure 2 that minimizes the accumulated distortion in depth d, while no two arcs are incident to the same node and all of the nodes are matched. Some methods to solve this problem are presented in [24] that offer a computational complexity of , where is the number of nodes in the graph. However, we used the suboptimal method proposed by Chu in [4] to reduce the merging processing time, which worked well in our implementation. In this method, we sort arc values in ascending order, select arcs with lower values, and remove arcs ending at nodes belonging to the arcs already selected. Therefore, no sharing occurs between Voronoi regions at depth , which is a necessary characteristic for the constructed tree. The select-remove procedure is continued until a complete matched graph is achieved.

In the following part of this section, we propose four types of distortion criteria to attribute to arc values in the merging process and give details of a comparative assessment.

Consider the training sequence , where is a large number. And, suppose and are the Voronoi regions of codewords and at depth , respectively. Consider as the mother of and at depth . The mother codeword is a codeword for representing both and Voronoi regions. We estimate a measure of the accumulated squared distortion for the training matrices that fall into the Voronoi region of at depth , that is, for all , according to the accumulated squared distortions of the codewords and . For the Voronoi region of , , we have
(12)
where the approximation in (12) arises from the fact that an input matrix which has or as its nearest neighbor codeword at depth may no longer have as its nearest neighbor codeword at depth ([3, page 415]). The approximation in (12) turns into equality, when the Voronoi regions of codewords and are determined through a tree search, as
(13)
Hereafter, we assume that (13) is satisfied, even if no tree search is made. We define the sum of element-by-element squared weights for the training matrices that fall into and Voronoi regions, as
(14)
We define the accumulated squared weighted distortion for the Voronoi region of codeword at depth , as
(15)
By taking the derivatives of this accumulated distortion with respect to each element of , and equating them to zero, the optimum is obtained, as
(16)
We decompose (15) into two Voronoi regions and , as
(17)
where
(18)
and stands for the sum of all elements of the operand matrix. We also have
(19)
By substituting (19) into (18) we get
(20)
where denotes an element-by-element square of the operand matrix. By replacing from (16) into (20), we get
(21)
Similarly, we can compute , as
(22)
Finally, the accumulated squared weighted distortion for the Voronoi region of the codeword at depth can be simplified to
(23)
where, in the no-weighting case, it reduces to
(24)

In (24), and are the number of training matrices that fall into the Voronoi region of and , respectively. Equation (23) in the case of no-weighting and vector codewords reduces to the Equitz's formula in [23].

Therefore, by considering the term added to the accumulated distortions of children codewords at the right side of (23) or (24), as the value of the arc between nodes corresponding to children codewords, and then selecting a complete matching subset of the graph so that the sum of its arcs is minimized, the proper codewords for merging can be determined. Generalizing Chu's distortion measure [4] to our case results in the arc value of
(25)
Equation (25) in the no-weighting case reduces to
(26)
where
(27)
In case the th codeword and the th codeword are to be merged, the accumulated weighting for the codeword (that is an average over children codewords, and , as mentioned in (16) and (27) for weighting and no-weighting conditions, respectively) is
(28)

where it turns into in the case of no-weighting.

By continuing the cell-merging procedure (allocating distortion criterion to arcs, and then selecting a matched graph) for the codewords of all depths, we construct the tree-structured codebooks corresponding to each initial codebook. One of the most effective and readily available techniques for reducing the search complexity is to rely on the tree-structured codebooks in our embedded quantizer design. Figure 3 illustrates spectral distortion (SD) versus the average number of bits per segment in both full and fast tree searches for tree-structured codebooks constructed by exploiting four types of accumulated distortion measures. Types 1, 2, 3, and 4 distortion measures correspond to distortion criteria based on (23), (24), (25), and (26), respectively.
Figure 3
Figure 3

Spectral Distortion (SD) in dB versus average number of bits per segment for four types of accumulated distortion measures.

Table 2 summarizes the bit allocation for every codebook at various rates used for the LSF embedded quantizer. An experiment over a long training sequence extracted from the TIMIT database shows that each codeword is selected from an unvoiced codebook with an average probability of 1/3. As is represented in Table 2 by lowering the rate, the amount of bits allocated to high-frequency LSFs is reduced first, due to their lower perceptual importance. By decreasing one bit, we select a codeword from a lower depth stage of the tree-structured codebook. Each step of bit reduction in Table 2 is equivalent to 12.5 bps decrease in bit rate.
Table 2

The bit allocation used for embedded quantization at different rates. UV and VUV correspond to unvoiced and mixed voicing codebooks, respectively.

Average bits per segment

No. of bits for

No. of bits for

No. of bits for

No. of bits for

No. of bits for

 

representing

representing

representing

representing

representing

 

LSF1 & LSF2

LSF3 & LSF4

LSF5 & LSF6

LSF7 & LSF8

LSF9 & LSF10

 

VUV

UV

VUV

UV

VUV

UV

VUV

UV

VUV

UV

43

10

8

10

8

10

8

9

7

8

6

42

10

8

10

8

10

8

9

7

7

5

41

10

8

10

8

10

8

8

6

7

5

40

10

8

10

8

9

7

8

6

7

5

39

10

8

9

7

9

7

8

6

7

5

38

9

7

9

7

9

7

8

6

7

5

37

9

7

9

7

9

7

8

6

6

4

36

9

7

9

7

9

7

7

5

6

4

35

9

7

9

7

8

6

7

5

6

4

34

9

7

8

6

8

6

7

5

6

4

33

8

6

8

6

8

6

7

5

6

4

32

8

6

8

6

8

6

7

5

5

3

31

8

6

8

6

8

6

6

4

5

3

30

8

6

8

6

7

5

6

4

5

3

29

8

6

7

5

7

5

6

4

5

3

28

7

5

7

5

7

5

6

4

5

3

27

7

5

7

5

7

5

6

4

4

2

The Spectral Distortion (SD) is applied to 4 minutes of speech utterances outside the training set. As depicted in Figure 3, in the case of full search, type 1 and type 3 distortion measures perform almost similarly and a little better than their unweighted versions (types 2 and 4). Indeed, full codebook search results in the same performance for these four types of measures at full resolution, because all the four types of trees have the same terminal nodes. Although the type 3 measure performs better than the type 2 measure in full search, it is outperformed by types 1 and 2 distortion measures in the fast tree search. This behavior comes from the fact that equality (13) is satisfied for the fast tree search.

It is clear from Figure 3 that the fast tree search does not necessarily find the best matched codeword. Generally speaking, it may be thought that there should be a slight difference between the spectral distortions in full search and fast tree search; nevertheless, we believe this relatively considerable difference, which we see in Figure 3, is due to the codebook structures having matrix codewords.

4. Adaptive Dual-Band Excitation

Multiband excitation (MBE) was originally proposed by Griffin and Lim and was shown to be an efficient paradigm for low rate speech coding to produce natural sounding speech [25]. The original MBE model, however, is inapplicable to speech coding at very low rates, that is, below 4 kbps, due to the large number of frequency bands it employs. On the other hand, dual-band excitation, as the simplest possible MBE model, has attracted lots of attention by the research community [26]. It has been shown that most (more than 70%) of the speech frames can be represented by only two bands [26]. Further analysis of the speech spectra revealed that the low frequency band is usually voiced, where the high-frequency band usually contains a noise-like signal (i.e., unvoiced) [26]. In our coding system, we use the dual-band MBE model proposed in [27], in which the two bands join at a variable frequency determined based on the voicing characteristics of speech signals on a frame-by-frame basis in the LPC model. For convenience, we have quoted the main idea of this two-band excitation model from [27] below.

In this dual-band model, three voicing patterns may happen in the frequency domain, including pure voiced, pure unvoiced, or a mixed pattern of voiced and unvoiced, with voiced at the lower band. The two bands join at a time-varying transition frequency at which spectral characteristics of the signal change. Figure 4 shows the block diagram of the two-band synthesizer where near zero values for transition frequency mean pure unvoiced, near 4 KHz values mean pure voiced, and mid values mean mixed patterns of voiced and unvoiced. Given a transition frequency, an artificial excitation is constructed by adding a periodic signal located at the low band, that is, below transition frequency, and a random signal at the high band, that is, above transition frequency. For the voiced part, the excitation pulse of the LPC-10 coder is used as the pulse-train generator [28]. This excitation signal improves the quality of the synthesized speech over the simple rectangular pulse train. This excitation pulse is shown in Figure 5.
Figure 4
Figure 4

Block diagram of the adaptive dual-band synthesizer. Transition frequency controls cutoff frequency of low-pass and high-pass filters.

Figure 5
Figure 5

One excitation pulse of the LPC-10 coder [ 28 ].

The transition frequency is computed from the spectrum of the LPC residual for each frame of the signal using a periodicity measure, which is based on the flatness of the instantaneous frequency (IF) contour in the frequency domain. For IF estimation in the frequency domain, which gives the pitch period when the frame is voiced, we use a spectrogram technique that employs a segment-based analysis using an appropriate window in the frequency domain [29]. Pay attention that this windowing process is different from the one we used in the time domain. The windowing in the time domain is same as the one we used in Section 2. Here, the windowing is performed in the frequency domain using a Hanning window
(29)
where represents a filtered version of the spectrum magnitude of the residual signal, N is the total number of samples in each frame of the speech signal which is 320 here, = min , , in the th spectrogram coefficient, in the number of DFT points which is 64 here, is the predefined window length which is 32 here, and , = , is a Hanning window in the frequency domain. As is evident, as long as , equals . The peak of the spectrogram, , = , gives the IF of the spectrum
(30)

where represents IF of the spectrum over frequencies from 0 to , where is the sampling frequency which is 8 kHz in our designated coder.

The transition frequency, , which specifies a change in the spectrum characteristics from periodic to random, is obtained through measuring the flatness of in a number of subbands, . This is formulated as
(31)

where is the subband index, , and the vector is the th part of , = , located in the th band, whose flatness is represented by . The bar over the vector stands for the mean of this vector.

As evident, , which is used as an indication of flatness, where 1 is for an absolutely-flat vector ( ). , is then calculated through comparing with the threshold th, as
(32)

where = min , that means the minimum value of for which .

The threshold is calculated based on the mean of the spectrum flatness within a certain band, averaged over a number of previous frames composed of voiced and unvoiced frames [27]. In this way, the spectrum is assumed to be periodic at frequencies below , and it is considered random at frequencies over , with a resolution specified by .

The fundamental frequency, , is computed using where is the mean value of the IF contour within a certain band below 1 kHz regardless of its voicing status, as illustrated in Figure 6, where a mixed speech signal and its corresponding IF curve are shown. The degree of voicing, or periodicity, is determined by the transition frequency. A low means that the periodic portion of the excitation spectrum is dominated by the random part and vice versa. For this reason, the accuracy in pitch detection during unvoiced periods, which is intrinsically ambiguous, is insignificant and noneffective in naturalness. A detailed description of this dual-band excitation method can be found in [27] by Ghaemmaghami and Deriche.
Figure 6
Figure 6

IF based analysis of a mixed-excitation speech signal: (a) absolute value of LPC residual where its mean value is removed, (b) IF contour over frequency domain, and (c) speech signal waveform. The portion of the IF contour between vertical lines is used to compute the fundamental frequency [27].

We exploit interframe correlation between adjacent frames (in each segment of four frames) to efficiently encode gain, pitch period, and transition frequency using a dimension vector quantization for each set of excitation parameters. Codebooks for these parameters are built using the LBG algorithm by a simple norm-2 distortion measure. The training vectors are produced using 1200 speech files from TIMIT. Table 3 illustrates the number of bits we assign to the codebooks of these parameters. This bit allocation scheme and the one extra bit employed for the codebook type selection lead to a rate of 350 bps ((27 + 1)/80 ms) for encoding the excitation parameters, and the total rate of 900 bps (350 + 550) in full resolution embedded quantization of spectral parameters. Reducing the number of bits for representing pitch and the transition frequency severely affects the speech quality. Since, we encode these excitation parameters using a fixed number of bits, given in Table 3, at any rate selected.
Table 3

Bits allocation for pitch, transition frequency, and gain codebooks.

Codebook type

Pitch

Transition Frequency

Gain

Total

No. of bits allocated

11

9

7

27

5. Performance Evaluation and Experiments

5.1. Spectral Dynamics of MQ Versus VQ

The dynamics of the power spectrum envelope play a significant role in the perceived distortion [30]. According to Knagenhjelm and Kleijn [30], smooth evolution of the quantized power spectrum envelope leads to a significant improvement in the performance of the LPC quantizers. To evaluate the spectral evolution, the spectral difference between adjacent frames is used which is given by
(33)
where indicates the power spectrum envelope of the th frame. Table 4 compares average spectral evolution (ASE) and average spectral distortion (ASD) of the embedded matrix quantizer (produced by type 1 distortion criterion) versus VQ for three different numbers of bits assigned to each segment of spectral parameters.
Table 4

Spectral dynamics and spectral distortion of matrix quantization versus vector quantization at the same rate.

Average number of bits per segment of four frames

43

38

33

ASE for original speech

6.57

6.57

6.57

ASE for MQ

6.21

6.15

6.11

ASE for MQ with segments junction smoothing

6.08

6.02

5.97

ASE for VQ at the same rate as MQ

6.56

6.54

6.43

ASD for MQ

1.65

1.75

2.05

ASD for MQ with segments junction smoothing

1.63

1.72

2.01

ASD for VQ at the same rate as MQ

2.50

2.68

3.02

As mentioned earlier, codewords of the designated matrix quantizer are obtained through averaging over real input matrices of the spectral parameters. These matrices have smooth spectral trajectories, thus the averaging process over the matrices results in codewords having relatively smooth spectral dynamics. This is while codewords of the VQ are obtained by averaging over a set of single frame input vectors and not a trajectory of spectral parameters like MQ. This results in better performance of the MQ over the VQ, in terms of spectral dynamics, as confirmed by experimental results given in Table 4. According to this table, the MQ yields both smoother spectral trajectories and lower average spectral distortions, as compared to the VQ at a same rate.

To improve the performance of the MQ, we use simple spectral parameter smoothing at the junction of codewords selected in consecutive segments. In this smoothing method, we replace the first column of the selected minimum distortion codeword by a weighted mean of the first column of the currently selected codeword and the last column of the previously selected codeword. Weighting used for the first column of the recent codeword is 0.75 and for the last column of the previously selected codeword is 0.25. In this smoothing method, the ascending order of the LSFs is guaranteed.

5.2. Intelligibility and Quality Assessment

We use the ITU-T P.862 PESQ standard [31] to compare the quality of synthesized speech at various bit rates. The PESQ (Perceptual Evaluation of Speech Quality) score ranges from 0.5 to 4.5, with 1 for a poor quality and 4 denoting a high quality signal. The PESQ, which is an objective measure to evaluate speech quality, correlates well with subjective test scores at mid and above mid bit rates. However, PESQ does not give a reasonable estimate of MOS at low bit rates. Therefore, we have just used PESQ for quality comparison between various bit rates and not for an estimate of MOS. The material used for the PESQ test is a 3-minute long speech signal outside the training set. Table 5 illustrates the PESQ score at different rates of the scalable coder for full and fast tree searches, where the tree-structured codebook is produced using type 1 distortion criterion. Figure 7 shows the results of the MOS subjective quality test [32] at three different rates exploiting a tree-structured codebook identical to the one used in PESQ tests using a full search for choosing codewords. The MOS test was conducted by asking 24 listeners to score 3 stimuli sentences.
Table 5

PESQ scores at different rates.

Bit rate

PESQ score

PESQ score

 

(full search)

(tree search)

900

2.512

2.331

850

2.468

2.298

800

2.447

2.293

750

2.437

2.28

700

2.38

2.24

No-quantization case

2.651

Figure 7
Figure 7

MOS score at three different rates. Scores of 2.71, 2.82, and 2.92 are achieved for 700, 800, and 900 bps, respectively.

We also conducted the MUSHRA ITU-R recommendation BS.1534-1 test [33] at the same bit rates and with the same codebooks (Figure 8). MUSHRA stands for "MUltiple Stimuli with Hidden Reference and Anchor" and is a method for subjective quality evaluation of lossyaudio compression algorithms. MUSHRA listening test is a 0–100 scale that is particularly suited to compare high quality reference sounds with lower quality test sounds. Thus, test items where the test sounds have a near-transparent quality or where the reference sounds have a low quality should not be used. For the MUSHRA test we used the MUSHRAM interface given in [34] and asked 10 subjects to help us in the experiment.
Figure 8
Figure 8

MUSHRA score at three different rates. Scores of 38, 40, 43 are achieved for 700, 800, and 900 bps, respectively.

As it is clear in Figures 7 and 8, the quality difference between these three rates is relatively small, consistent with the fine-granularity property. In some speech samples the quality difference at different rates was almost imperceptible. The results shown in these figures are achieved by doing the test over a variety of samples and taking the average over the scores.

Figure 9 illustrates spectrograms for a sample speech utterance from TIMIT, uttered by a male speaker, "Do not ask me to carry an oily rag like that," at different rates. As shown in the figure, details of the spectrograms tend to disappear at lower rates. This figure also reveals that the difference between the original and the synthesized speech spectra mainly stems from the inaccuracy of the dual-band approximation of the LPC excitation, as compared to the effect of the LSF quantization.
Figure 9
Figure 9

Spectrograms of synthesized speech signals using the proposed coder at different rates. The utterance is "Don't ask me to carry an oily rag like that" from TIMIT, uttered by a male speaker. The vertical axis ranges from 0 to 4 kHz and the horizontal axis is from 0 to 2.5 seconds.Original SpeechSynthesized speech without parameter quantizationSynthesized speech at average bit rate of 825 bpsSynthesized speech at average bit rate of 762 bpsSynthesized speech at average bit rate of 700 bps

In addition to the quality test, we conducted the diagnostic rhyme test (DRT) [35] to measure the intelligibility of the synthesized speech. Table 6 gives results of this test at three different rates.
Table 6

DRT assessment results.

Bit-rate

900

800

700

Voicing

100

100

100

Nasality

67

62

56

Sustention

78

73

70

Sibilation

87.5

87.5

85

Graveness

100

100

100

Compactness

100

87.5

87.5

Total

89

85

83

5.3. Memory Requirement of the Embedded Quantizer

In the tree-structured codebook, storage memory is needed to store the internal codewords, in addition to the memory required to store codewords of the initial codebook placed on the leaves of the tree. The total number of noninternal codewords is given by
(34)

Thus, the total amount of memory required for the embedded quantizer is slightly less than twice of the memory used for the initial codebooks. In the applications based on fast tree-structured search, there is no need to have internal codewords at the decoder. This is while the internal codewords must be available in both coder and decoder in an embedded quantization scheme ([3, page 413]).

The total memory required to store spectral parameters for the designated classified embedded SMQ is computed as
(35)
where and denote sizes of th initial split codebooks corresponding to mixed voicing and unvoiced codebooks, respectively. And, in the case of the nonsplit embedded quantizer of the same resolution, the amount of memory is given as
(36)

Hence, the embedded SMQ proposes a memory requirement that is much lower than that of a nonSMQ of the same resolution. This confirms a proper selection of the SMQ for our embedded matrix quantizer in the sense of both the computational complexity and size of the memory.

6. Conclusion

In this paper, which was a detailed version of [36], we have introduced a very low rate scalable speech coder with 80 ms coding delay, using classified embedded matrix quantization and adaptive dual-band excitation. Although the delay is relatively high with respect to many standardized coders, it is still suitable for some applications, since a delay as high as 250 ms has found to be tolerable for some practical applications according to [3739]. The transition frequency of the dual-band excitation model is determined based on the evaluation of flatness of the instantaneous frequency contour in the frequency domain. A cell-merging process is applied to the initial codebooks of the SMQ scheme to organize codewords into a tree-structure. The natural embedded property of the constructed tree codebooks helped to build a fine-grain scalable coder operating in the range of 700–900 bps at 12.5 bps steps. It is obvious that a same cell merging process can be applied to larger size initial codebooks in order to get a wider range of bit rate operation. Our intention of testing the bit range of 700–900 was just to evaluate the granularity of the designed embedded quantizer. Four types of distortion measures to assign to the arc values of the initial graph in the merging process, in both full and fast-tree searches, have been introduced and assessed comparatively. Interframe correlation between adjacent frames is exploited to efficiently encode gain, pitch, and the transition frequency using the VQ method. Better performance of the proposed embedded matrix quantizer in comparison with the VQ, at the same bit rate, has been confirmed, in terms of both spectral dynamics and spectral distortion. Speech quality assessment and DRT comparison of the synthesized speech at different rates show that the proposed scalable coding system has the property of fine-granularity.

Declarations

Acknowledgment

The authors want to express their thankfulness to Dr. Wai C. Chu and also our friend Tim Han for reviewing this paper several times and making valuable comments and suggestions.

Authors’ Affiliations

(1)
Department of Electrical & Computer Engineering, Johns Hopkins University, Baltimore, USA
(2)
Department of Electrical Engineering, Sharif University of Technology, Tehran, Iran

References

  1. Tzou K-H: Embedded Max quantization. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '86), 1986, Tokyo, Japan 505-508.View ArticleGoogle Scholar
  2. Max J: Ouantization for minimum distortion. IEEE Transactions on Information Theory 1960, 6: 7-12. 10.1109/TIT.1960.1057548MathSciNetView ArticleGoogle Scholar
  3. Gersho A, Gray RM: Vector Quantization and Signal Compression. Kluwer Academic Publishers, Dordrecht, The Netherlands; 1992.View ArticleMATHGoogle Scholar
  4. Chu WC: Embedded quantization of line spectral frequencies using a multistage tree-structured vector quantizer. IEEE Transactions on Audio, Speech and Language Processing 2006, 14(4):1205-1217.View ArticleGoogle Scholar
  5. Chu WC: A scalable MELP coder based on embedded quantization of line spectral frequencies. Proceedings of International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS '05), December 2005, Hong Kong 29-32.Google Scholar
  6. Ravelli E, Daudet L: Embedded polar quantization. IEEE Signal Processing Letters 2007, 14(10):657-660.View ArticleGoogle Scholar
  7. Said A, Pearlman WA: A new, fast, and efficient image codec based on set partitioning in hierarchical trees. IEEE Transactions on Circuits and Systems for Video Technology 1996, 6(3):243-250. 10.1109/76.499834View ArticleGoogle Scholar
  8. Chu W: Speech Coding Algorithms: Foundation and Evolution of Standardized Coders. John Wiley & Sons, New York, NY, USA; 2003.View ArticleMATHGoogle Scholar
  9. Hersent O, Petit JP, Gurle D: Beyond VoIP Protocols: Understanding Voice Technology and Networking Techniques for IP Telephony. John Wiley & Sons, New York, NY, USA; 2005.View ArticleGoogle Scholar
  10. ITU : 5−, 4−, 3−, and 2-Bits Sample Embedded Adaptive Differential Pulse Code Modulation (ADPCM)—Recommend. G.727, Geneva, Switzerland, 1990Google Scholar
  11. ITU-T Rec. G.729.1 : G.729-based embedded variable bit-rate coder: an 8-32 kbit/s scalable wideband coder bitstream interoperable with G.729. May 2006Google Scholar
  12. ITU-T Rec. G.729 : Coding of Speech at 8 kbit/s Using Conjugate Structure Algebraic Code Excited Linear Prediction (CSACELP). March 1996Google Scholar
  13. McCree A: A scalable phonetic vocoder framework using joint predictive vector quantization of melp parameters. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '06), May 2006, Toulouse, France 1: 705-709.Google Scholar
  14. Supplee LM, Cohn RP, Collura JS, McCree AV: MELP: the new federal standard at 2400 bps. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '), April 1997, Munich, Germany 2: 1591-1594.Google Scholar
  15. Xydeas CS, Papanastasiou C: Split matrix quantization of LPC parameters. IEEE Transactions on Speech and Audio Processing 1999, 7(2):113-125. 10.1109/89.748117View ArticleGoogle Scholar
  16. Getreuer P: Writing Fast MATLAB Code. 2006, http://www.math.ucla.edu/~getreuer/matopt.pdf
  17. Ozaydin S, Baykal B: Multi stage matrix quantization for very low bit rate speech coding. Proceedings of the 3rd Workshop on Signal Processing Advances in Wireless Communications, 2001 372-375.Google Scholar
  18. Özaydın S, Baykal B: Matrix quantization and mixed excitation based linear predictive speech coding at very low bit rates. Speech Communication 2003, 41(2-3):381-392. 10.1016/S0167-6393(03)00009-8View ArticleGoogle Scholar
  19. Paliwal KK, Atal BS: Efficient vector quantisation of LPC parameters at 24 bits/frame. IEEE Transactions on Speech and Audio Processing 1993, 1(1):3-14. 10.1109/89.221363View ArticleGoogle Scholar
  20. Van Trees HL: Optimum Array Processing: Part IV of Detection, Estimation, and Modulation Theory. John Wiley & Sons, New York, NY, USA; 2002.View ArticleGoogle Scholar
  21. Linde Y, Buzo A, Gray RM: An algorithm for vector quantizer design. IEEE Transactions on Communications Systems 1980, 28(1):84-95. 10.1109/TCOM.1980.1094577View ArticleGoogle Scholar
  22. DARPA TIMIT : Acoustic-Phonetic Continuous Speech Corpus. National Institute of Standards and Technology, Gaitherburg, Md, USA; 1993.Google Scholar
  23. Riskin EA, Ladner R, Wang R-Y, Atlas LE: Index assignment for progressive transmission of full-search vector quantization. IEEE Transactions on Image Processing 1994, 3(3):307-312. 10.1109/83.287025View ArticleGoogle Scholar
  24. Lawler EL: Combinatorial Optimization: Networks and Matroids. Dover, New York, NY, USA; 2001.MATHGoogle Scholar
  25. Griffin DW, Lim JS: Multiband excitation vocoder. IEEE Transactions on Acoustics, Speech, and Signal Processing 1988, 36(8):1223-1235. 10.1109/29.1651View ArticleMATHGoogle Scholar
  26. Chiu KM, Ching PC: A dual-band excitation LSP codec for very low bit rate transmission. Proceedings of the International Symposium on Speech, Image Processing, and Neural Networks (ISSIPNN '94), April 1994, Hong Kong 479-482.View ArticleGoogle Scholar
  27. Ghaemmaghami S, Deriche M: A new approach to modeling excitation in very low-rate speech coding. Proceedings of International Conference on Acoustics, Speech and Acoustics, Speech and Signal Processing (ICASSP '98), May 1998, Seattle, WA, USA 597-600.Google Scholar
  28. Tremain TE: The government standard linear predictive coding algorithm: LPC-10. Speech Technology Magazine 1982, 40-49.Google Scholar
  29. Boashash B: Estimating and interpreting the instantaneous frequency of a signal—part 1: fundamentals. Proceedings of the IEEE 1992, 80(4):520-538. 10.1109/5.135376View ArticleGoogle Scholar
  30. Knagenhjelm HP, Kleijn WB: Spectral dynamics is more important than spectral distortion. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '95), May 1995, Detroit, Mich, USA 1: 732-735.Google Scholar
  31. ITU : Perceptual Evaluation of Speech Quality (PESQ), an Objective Method for End-to-End Speech Quality Assessment of Narrow-Band Telephone Networks and Speech Codecs—ITU-T Recommendation P.862. 2001.Google Scholar
  32. ITU : Mean Opinion Score (MOS), Methods For Subjective Determination of Transmission Quality—ITU-T Recommendation P.800.1. 1996.Google Scholar
  33. ITU : MUlti Stimulus test with Hidden Reference and Anchor (MUSHRA), Method For The Subjective Assessment of Intermediate Quality Levels of Coding Systems—ITU-R BS.1534-1. January 2003.Google Scholar
  34. Vincent E: MUSHRAM: a MATLAB interface for MUSHRA listening tests. 2005, http://www.elec.qmul.ac.uk/people/emmanuelv/mushram/
  35. Deller JR, Hansen JHL, Proakis JG: Discrete-Time Processing of Speech Signals. John Wiley & Sons, New York, NY, USA; 2000.Google Scholar
  36. Jahangiri E, Ghaemmaghami S: Scalable speech coding at rates below 900 BPS. Proceedings of IEEE International Conference on Multimedia and Expo (ICME '08), June 2008, Hannover, Germany 85-88.Google Scholar
  37. Dusan S, Flanagan JL, Karve A, Balaraman M: Speech compression by polynomial approximation. IEEE Transactions on Audio, Speech and Language Processing 2007, 15(2):387-395.View ArticleGoogle Scholar
  38. Klemmer ET: Subjective evaluation of transmission delay in telephone conversations. Bell Labs Technical Journal 1967, 1141-1147.Google Scholar
  39. Brady PT: Effects of transmission delay on conversational behaviour on echo-free telephone circuits. Bell Labs Technical Journal 1971, 115-134.Google Scholar

Copyright

© E. Jahangiri and S. Ghaemmaghami. 2010

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement