Video Adaptation for Heterogeneous Environments

The explosive growth of compressed video streams and repositories accessible worldwide, the recent addition of new video-related standards such as H.264/AVC, MPEG-7, and MPEG-21, and the ever-increasing prevalence of heterogeneous, video-enabled terminals such as computer, TV, mobile phones, and personal digital assistants have escalated the need for efficient and effective techniques for adapting compressed videos to better suit the different capabilities, constraints, and requirements of various transmission networks, applications, and end users. For instance, Universal Multimedia Access (UMA) advocates the provision and adaptation of the same multimedia content for different networks, terminals, and user preferences. Video adaptation is an emerging field that offers a rich body of knowledge and techniques for handling the huge variation of resource constraints (e.g., bandwidth, display capability, processing speed, and power consumption) and the large diversity of user tasks in pervasive media applications. Considerable amounts of research and development activities in industry and academia have been devoted to answering the many challenges in making better use of video content across systems and applications of various kinds. Video adaptation may apply to individual or multiple video streams and may call for different means depending on the objectives and requirements of adaptation. Transcoding, transmoding (cross-modality transcoding), scalable content representation, content abstraction and summarization are popular means for video adaptation. In addition, video content analysis and understanding, including low-level feature analysis and high-level semantics understanding, play an important role in video adaptation as essential video content can be better preserved. The aim of this special issue is to present state-of-theart developments in this flourishing and important research field. Contributions in theoretical study, architecture design, performance analysis, complexity reduction, and real-world applications are all welcome. Topics of interest include (but are not limited to):

tional and display capabilities, energy resources, features, accessibilities, and user preferences. Such heterogeneity in networks and user devices has escalated the need for efficient and effective techniques for adapting compressed videos to better suit the different capabilities, constraints, and requirements of various transmission networks, applications, and end users. For instance, universal multimedia access (UMA) advocates the provision and adaptation of the same multimedia content for different networks, terminals, and user preferences.
Video adaptation is an emerging field that offers a rich body of knowledge and techniques for handling the huge variation of resource constraints (e.g., bandwidth, display capability, processing speed, and power consumption) and the large diversity of user tasks in pervasive media applications. Video adaptation may apply to individual or multiple video streams and call for different means depending on the objectives and requirements of adaptation. Transcoding, transmoding (cross-modality transcoding), scalable content representation, and content abstraction and summarization are popular means for video adaptation. In addition, video con-tent analysis and understanding, including low-level feature analysis and high-level semantics understanding, play an important role in video adaptation as essential video content can be better preserved.
Many research and development activities in industry and academia have been devoted to answering the challenges of making better use of video content across systems and applications of various kinds. This special issue aims at providing state-of-the-art developments in this flourishing and important research field. After a thorough review process, a total of 13 papers were selected, covering the topics of video adaptation tools, architecture design, performance analysis, complexity reduction, and real-world applications.
The first three papers cover the topic of video transcoding algorithms and their applications. Video transcoding is an operation of converting a video bit stream from one format into another format. It is an efficient means of achieving fine and dynamic video adaptation. The first paper by J. Xin et al., "efficient MPEG-2 to H.264/AVC transcoding of intra coded video," proposes a low-complexity transformdomain architecture and the corresponding mode-decision algorithms for transcoding intracoded video from MPEG-2 to H.264/AVC format. Compared to the conventional pixel-domain approach, the proposed architecture reduces the transcoding complexity but incurs negligible or no loss in quality by performing direct coefficient conversion and mode decision in the transform domain. To further reduce the complexity, the paper also presents two fast mode decision algorithms, one evaluating the rate-distortion costs for a reduced set of modes decided by a simple cost function, and the other exploiting the strong temporal correlations between adjacent frames. The paper "efficient hybrid DCT-domain algorithm for video spatial downscaling," by N. Roma and L. Sousa, proposes a DCT-domain spatial downscaling transcoder for any arbitrary integer downscaling factor. The proposed algorithm reduces the computational cost while maintaining the visual quality by taking advantage of the scaling mechanism and by restricting the involved operations in order to avoid useless computations. In the paper "A multiple-window video embedding transcoder based on H.264/AVC standard," by C.-H. Li et al., a new transcoder is proposed to embed multiple foreground videos into one background video. The transcoding speed has been significantly improved by 25 times with negligible quality loss by using a combination of the slice group-based transcoding, reduced frame memory transcoding, and syntax level bypassing techniques. An improvement of up to 1.5 dB in PSNR is registered which is significant for multigeneration transcoding over the cascaded pixel domain transcoder.
Scalable video coding (SVC) is another powerful tool of adapting video content as it can provide different scaling options, such as temporal, spatial, and SNR scalability, where rate reduction by discarding enhancement layers of different scalability-type results in different kinds and/or levels of visual distortion depending on the content and bitrate. Two papers cover the topic of SVC-based video adaptation. The paper by W. Yang el al., "Scalable video coding with interlayer signal decorrelation techniques," proposes to improve the coding performance of the enhancement layers of SVC through efficient interlayer decorrelation techniques. This paper investigates, for both the open-loop and the closed-loop cases, two improved Laplacian pyramid structures for SVC that exploit the inherent redundancy of the underlying Laplacian pyramid with nonbiorthogonal filters by rendering the enhancement layer signal less correlated with the base layer. In "Content-aware scalabilitytype selection for rate adaptation of scalable video," E. Akyol et al. propose an objective function that quantifies flatness, blockiness, blurriness, and temporal jerkiness artifacts caused by rate reduction by spatial size, frame rate, and quantization parameter scaling. An adaptation method is proposed for choosing the best scaling type for each temporal segment that results in minimum visual distortion according to this objective function given the content type of temporal segments. Two subjective tests have been performed to validate the proposed procedure for content-aware selection of the best scalability type on soccer videos.
The next three papers address the problems of video adaptation in heterogeneous environments in terms of computation resources, perceptual quality, and channel bitrate. In "A complexity-aware video adaptation mechanism for live streaming systems," M.-T. Lu et al. propose a new video adaptation mechanism for live video streaming of multiple channels. This mechanism utilizes the complexity-distortion model to optimize globally through piecewise linear approximation in allocating the computational resource to each channel. A block-based complexity control method is also proposed to accurately control the computational resource of each channel on the live streaming server. The paper "An attention-information-based spatial adaptation framework for browsing videos via mobile devices," by H. Li et al., presents an attention information-based spatial adaptation framework to address the problem of limited display sizes of mobile devices. The proposed framework includes two major parts: video content generation and video adaptation system. During video compression, the attention information in a video sequence is detected using an attention model and embedded into the compressed bitstream. The attention information is then employed to generate a bitstream of attention areas in each frame to adapt to the display sizes of mobile devices. Besides, an attention-biased QP adjustment scheme based on the attention information is proposed to regulate the output bitrate. The paper "Contentaware video adaptation under low bitrate constraint," by M. H. Hsiao et al., proposes a content-aware video adaptation method to retain visual quality under low-bitrate condition. The method first analyzes regions that are visually important based on the brightness, location, motion vector, and energy features derived in the compressed domain. Based on the analysis result, bit allocation is then performed, using additional motion features, at the frame level and, using a ratedistortion model, at the object level to achieve better visual quality.
Error robustness is crucial in video transport in error prone environments such as wireless networks. Four papers address this problem using different video adaptation techniques. The paper entitled "Comparison of error protection methods for audio-video broadcast over DVB-H," by M. Hannuksela et al., provides an excellent overview of audio-video transmission within the DVB-H environment, and analyzes the effectiveness of applying an unequal error protection scheme to compressed video within the context of such systems. The paper entitled "Transcoding based error-resilient video adaptation for 3G wireless networks," by S. Eminsoy et al. presents a video transcoding system that applies a combination of error resilience tools on the input compressed video streams to provide robust communications while regulating the output rates over a 3G W-CDMA wireless network. The paper also presents a new adaptive intra-refresh algorithm, which is responsive to the detected scene-activity inherently embedded into the video content and the reported time varying channel error conditions of the wireless network. In "Cross-layer design for video transmission over wireless rician slowfading channels using an adaptive multi-resolution modulation and coding scheme" by Y. Pei and Modestino, a scalable H.263+ video source coder is combined with unequal error protection across layers by employing different channel codes together with a multiresolution modulation schemes using nonuniform MPSK signal constellations. An adaptive joint source channel coding is also proposed and the results indicate good improvements in delivered video quality for specified channel conditions. The paper, "Multiple adaptations and content-adaptive FEC using parameterized RD model for embedded wavelet video," by Y.-H. Yu et al., presents a framework for performing multiple adaptations of wavelet-coded video whereby ratedistortion information is embedded in the video bitstream. The proposed framework also enables content-adaptive FEC protection.
Chia-Wen Lin et al.

3
The final paper, "OLGA: a unified scalable framework for online gaming," by F. Morán et al. addresses an interesting application of video adaptation: online gaming. This paper describes how multiresolution representation and scalable coding can be exploited to adapt and deliver graphics content over heterogeneous networks and platforms. Some experimental results are presented to demonstrate the tradeoffs among the display quality, computational complexity, and bandwidth in rendering graphics content. The paper also discusses how content adaptation and load balancing can be achieved over a distributed content-delivery network.

ACKNOWLEDGMENTS
Guest editors are extremely grateful to all the authors who contributed their high quality papers to this special issue. They would like to thank all the reviewers who took time and consideration to assess the submitted manuscripts. They would also like to acknowledge the Editor-in-Chief and the Editorial Board members of the Journal on Advances in Signal Processing for their support of this special issue.