Skip to content


  • Research Article
  • Open Access

Recognizing Uncertainty in Speech

EURASIP Journal on Advances in Signal Processing20102011:251753

  • Received: 1 August 2010
  • Accepted: 23 November 2010
  • Published:


We address the problem of inferring a speaker's level of certainty based on prosodic information in the speech signal, which has application in speech-based dialogue systems. We show that using phrase-level prosodic features centered around the phrases causing uncertainty, in addition to utterance-level prosodic features, improves our model's level of certainty classification. In addition, our models can be used to predict which phrase a person is uncertain about. These results rely on a novel method for eliciting utterances of varying levels of certainty that allows us to compare the utility of contextually-based feature sets. We elicit level of certainty ratings from both the speakers themselves and a panel of listeners, finding that there is often a mismatch between speakers' internal states and their perceived states, and highlighting the importance of this distinction.


  • Information Technology
  • Quantum Information
  • Internal State
  • Speech Signal
  • Full Article

Publisher note

To access the full article, please see PDF.

Authors’ Affiliations

School of Engineering and Applied Sciences, Harvard University, 33 Oxford Street, Cambridge, MA 02138, USA