Open Access

Summarizing Audiovisual Contents of a Video Program

EURASIP Journal on Advances in Signal Processing20032003:102838

https://doi.org/10.1155/S1110865703211082

Received: 19 March 2002

Published: 25 February 2003

Abstract

In this paper, we focus on video programs that are intended to disseminate information and knowledge such as news, documentaries, seminars, etc, and present an audiovisual summarization system that summarizes the audio and visual contents of the given video separately, and then integrating the two summaries with a partial alignment. The audio summary is created by selecting spoken sentences that best present the main content of the audio speech while the visual summary is created by eliminating duplicates/redundancies and preserving visually rich contents in the image stream. The alignment operation aims to synchronize each spoken sentence in the audio summary with its corresponding speaker′s face and to preserve the rich content in the visual summary. A Bipartite Graph-based audiovisual alignment algorithm is developed to efficiently find the best alignment solution that satisfies these alignment requirements. With the proposed system, we strive to produce a video summary that: (1) provides a natural visual and audio content overview, and (2) maximizes the coverage for both audio and visual contents of the original video without having to sacrifice either of them.

Keywords

video summarization audiovisual summarization partial audiovisual alignment bipartite graph minimum spanning tree maximum bipartite matching

Authors’ Affiliations

(1)
NEC Laboratories America, Inc.

Copyright

© Copyright © 2003 Hindawi Publishing Corporation 2003