Open Access

Recognizing Uncertainty in Speech

EURASIP Journal on Advances in Signal Processing20102011:251753

Received: 1 August 2010

Accepted: 23 November 2010

Published: 8 December 2010


We address the problem of inferring a speaker's level of certainty based on prosodic information in the speech signal, which has application in speech-based dialogue systems. We show that using phrase-level prosodic features centered around the phrases causing uncertainty, in addition to utterance-level prosodic features, improves our model's level of certainty classification. In addition, our models can be used to predict which phrase a person is uncertain about. These results rely on a novel method for eliciting utterances of varying levels of certainty that allows us to compare the utility of contextually-based feature sets. We elicit level of certainty ratings from both the speakers themselves and a panel of listeners, finding that there is often a mismatch between speakers' internal states and their perceived states, and highlighting the importance of this distinction.

Publisher note

To access the full article, please see PDF.

Authors’ Affiliations

School of Engineering and Applied Sciences, Harvard University


© Heather Pon-Barry and Stuart M. Shieber. 2011

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.