Skip to main content

A Two-Channel Training Algorithm for Hidden Markov Model and Its Application to Lip Reading


Hidden Markov model (HMM) has been a popular mathematical approach for sequence classification such as speech recognition since 1980s. In this paper, a novel two-channel training strategy is proposed for discriminative training of HMM. For the proposed training strategy, a novel separable-distance function that measures the difference between a pair of training samples is adopted as the criterion function. The symbol emission matrix of an HMM is split into two channels: a static channel to maintain the validity of the HMM and a dynamic channel that is modified to maximize the separable distance. The parameters of the two-channel HMM are estimated by iterative application of expectation-maximization (EM) operations. As an example of the application of the novel approach, a hierarchical speaker-dependent visual speech recognition system is trained using the two-channel HMMs. Results of experiments on identifying a group of confusable visemes indicate that the proposed approach is able to increase the recognition accuracy by an average of 20% compared with the conventional HMMs that are trained with the Baum-Welch estimation.

Author information



Corresponding author

Correspondence to Liang Dong.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Dong, L., Foo, S.W. & Lian, Y. A Two-Channel Training Algorithm for Hidden Markov Model and Its Application to Lip Reading. EURASIP J. Adv. Signal Process. 2005, 347367 (2005).

Download citation

Keywords and phrases

  • viseme recognition
  • two-channel hidden Markov model
  • discriminative training
  • separable-distance function