Memory bandwidth-scalable motion estimation for mobile video coding

Abstract

The heavy memory access of motion estimation (ME) execution consumes significant power and could limit ME execution when the available memory bandwidth (BW) is reduced because of access congestion or changes in the dynamics of the power environment of modern mobile devices. In order to adapt to the changing BW while maintaining the rate-distortion (R-D) performance, this article proposes a novel data BW-scalable algorithm for ME with mobile multimedia chips. The available BW is modeled in a R-D sense and allocated to fit the dynamic contents. The simulation result shows 70% BW savings while keeping equivalent R-D performance compared with H.264 reference software for low-motion CIF-sized video. For high-motion sequences, the result shows our algorithm can better use the available BW to save an average bit rate of up to 13% with up to 0.1-dB PSNR increase for similar BW usage.

1. Introduction

With the rapid progress of semiconductor technology, video coding is becoming popular in modern mobile devices to provide video services. In these devices, motion-compensated temporally predictive coding with motion estimation (ME) not only contributes the most to the coding efficiency of modern video encoder designs [1], but also requires large amounts of computations as well as data bandwidth (BW) [2]. This leads to severe design challenges for power-limited mobile devices. In power-limited mobile device, the available power could be changed dynamically due to low battery power or dynamic power management, such as dynamic voltage and frequency scaling [2, 3]. In such cases, the available data BW could be inconsistent with the video requirements and be lower than expected. Once this situation occurs, the video coding will be delayed or forced to drop frames. Either case leads to unwanted low video quality. This BW constrained problem is getting worse with increasing camera resolution in mobile devices.

Broadly speaking, the BW-constrained ME problem is one of the resource constraints. Other resource constrained designs [2–9] focus on lowering power consumption, with or without rate-distortion (R-D) optimization [2–5], or adjusting computational complexity with rate-control like methods [6–9]. He et al. [2] developed a new R-D analysis framework with a power constraint. Subsequently, the power-aware designs [3, 4] directly change their search algorithms without R-D optimization to predesigned ones to fit a lower power mode. Chen et al. [5] used a fast algorithm and data reuse to achieve a power-aware design. Tai et al. [6] proposed a novel computation-aware scheme to determine the target amount of computation power allocated to a frame and allocated this to each block in a computation-distortion-optimized manner. The computational complexity complexity-aware designs [7–9] used a rate-control like method to combine complexity constraints into R-D optimization. The basic assumption of these approaches is that there are limited computational resources in handheld devices but sufficient memory BW. This assumption could easily fail because of dynamic mobile environment in which videos are coded and decoded at the same time or because of the dynamic power management mentioned above.

To solve the above issue, we propose a BW-scalable ME algorithm to fit the available data BW constraint. We assume that the data BW are the limited resource and could be dynamically changed [3]. The available data BW will be sufficient in full or normal battery mode and have a higher working frequency. In low battery or power-saving mode, the available data BW will be insufficient due to the lower working frequency or lower voltage supply. With a lower than expected BW supply, ME computations could fail to meet real-time constraints or lead to significant R-D performance loss due to the macroblock (MB) skipping coding. The proposed method predicts and allocates the memory BW according to its R-D gain (RDG) and the available BW to model the bandwidth-rate-distortion (B-R-D) behavior of the existing ME algorithm. This B-R-D algorithm is a rate-control like method for MB MB-based BW allocation, which maximizes the coding efficiency under the BW constraint. The simulation results show that the proposed algorithm can better utilize the BW instead of wasting it as other designs do, and it can be scaled to the available BW.

The rest of this article is organized as follows. The review of related studies is presented in Section 2. In Section 3, we propose an analytical B-R-D optimized model. The online R-D optimized BW-scalable ME scheme is summarized in Section 4. Section 5 presents the simulation results and comparisons with traditional approaches. Finally, Section 6 concludes this article.

2. Review of related studies

To solve the computational complexity and data BW challenges of ME, various approaches have been proposed, such as parallel full search hardware design and fast ME algorithms.

Full search ME designs handle the computational complexity by using parallel processing elements for matching cost computation [10]. Furthermore, with its search center at (0, 0), it can reduce the data BW by reusing the overlapped search area, termed Level C data reuse in [11]. Such a design style is simple to use, but it will need constant data BW regardless of the video contents. Besides, to meet the Level C data reuse requirement, such a design also needs a larger search range (SR) to cover the possible best matching point due to the (0, 0) search center [12], which implies a waste of data BW compared to methods with a search center at the motion vector (MV) predictor (MVP).

On the other hand, fast ME algorithms only search a few candidates so that the computational complexity is lower. To facilitate such searching, most of the fast algorithms adopt the MVP as the search center [13]. In [14], most of best matching points are around the MVP, which can cover over 90% of the best matching points within ± 8 SR. Thus, it can have a smaller SR and could have lower data BW even with poor data reuse between consecutive searches. However, even the fast ME algorithm still assumes constant and sufficient data BW support for the required SR. Some designs with a dynamic SR [15–17] could have even lower data BW demands by changing the SR according to the content content-dependent prediction, but they still assume constant and sufficient BW support in the planning of chip design. Besides, none of the designs can adapt to dynamic data BWs. Several approaches have tried to reduce the required data BW. Designs in [18, 19] use a cache to maximize the possible data reuse for irregular search patterns. Bus BW-effective ME designs in [20, 21] lower the BW requirement by reducing the pixel representation from 8 bits to a binary pattern. However, these designs are only useful for specific search algorithms without a data BW constraint.

In summary, none of above approaches has considered data BW as a limited resource to explore the possibility of optimizing its usage in an R-D sense. The assumption that there will be constant and sufficient BW has the benefit of simplifying the design procedure, and thus, it is widely used in VLSI hardware design, but it usually wastes a lot of data BW because only a portion of the MBs in a high-motion video will need such a large amount of data. Such data BW waste is a serious problem for power-limited mobile devices because data access to DRAM is off-chip access and thus consumes significant power, which can be as much as the power consumption of the video chip [22]. As indicated in [22], the power consumption of external DRAM access could be up to 50% of the total power consumed by the video decoding chip. For encoding, this portion will be larger but is often neglected in the previous design. Besides, with a dynamically changing BW, the current approaches with constant and sufficient BW assumption would have insufficient BW for coding, could need more time to complete the coding and fail the real-time constraint or drop MB coding and quality to fulfill the timing constraint. Both situations are not acceptable to attain a high-quality visual experience.

3. Analytical B-R-D optimized modeling

For a given video coding distortion (or equivalent picture quality), D, and bit rate, R, if we decrease the available encoding BW, the coding will generate more distortion and bits, which in turn implies a higher D and R for ME operation and more data BW for video coding. Therefore, the overall BW usage of a ME module is linearly proportional to its search area. We introduce a set of BW control parameters, B = [β₁,β₂,...,β_L], to control the search area of the ME module. The model with the BW control parameters is of a more generic form and captures the available data BW under different system conditions. Consequently, the ME SR selection is then a function of these control parameters, denoted by SR(β₁,β₂,...,β_L). However, the overall BW usage of a ME module is linearly proportional to its search area. Within the BW-limited design framework, the encoder BW requirement, denoted by BW, is a function of SR, and is also a function of B, denoted by

$$BW=\Phi \left(SR\right)=BW\left({\beta }_1,{\beta }_2,...,{\beta }_L\right)$$

(1)

where Φ(·) is the SR selection model of the ME module. To optimize the BW usage, the available data BW, β _i , should dynamically be allocated among the MBs according to their motion characteristics. Thus, we execute the ME algorithm with a different SR of BW control parameters and obtain the corresponding R-D data. According to our measurements and analysis, the R-D performance model can well be approximated by the following expression, denoted by RDG(BW(β₁,β₂, ...,β_L)) as (2).

$$RDG\left(BW\right)=RDG\left(BW\left({\beta }_1,{\beta }_2,...,{\beta }_L\right)\right)$$

(2)

where

$$RDG=RD{C}_{\mathsf{\text{init}}}-RD{C}_{\mathsf{\text{BMA}}}$$

(3)

and the RDG is the difference of the Lagrange R-D cost (RDC) at the MVP (RDC_init) and the final best matching position (RDC_BMA). The Lagrange RDC function is frequently employed as a measure of ME efficiency, which is defined as

$$RD{C}_{\mathsf{\text{motion}}}\left(mv,{\lambda }_{\mathsf{\text{motion}}}\right)=\mathsf{\text{min}}\left\{SAD\left(s,c\left(mv\right)\right)+{\lambda }_{\mathsf{\text{motion}}}R\left(mv-pmv\right)\right\}$$

(4)

where mv is the MV received by the ME, and λ_motion indicates the Lagrange multiplier. The distortion term SAD(s, c(mv)) is the sum of the absolute differences between the original signal s and the coded video signal c. The rate term, λ_motionR(mv - pmv), represents the motion information and the coded bit length of the MV difference (MVD) between the MV and predicted MV. Note that Equation 2 is computationally intensive and is intended for offline analysis to obtain the B-R-D model.

Next, we optimally configure the BW control parameters to maximize the video quality (or minimize the video distortion) and minimize the video bit rate under the BW constraint. Mathematically, this can be formulated as in (5).

$$\begin{array}{c}\underset{\left\{{\beta }_1,{\beta }_2,...{\beta }_L\right\}}{\mathbf{max}}\mathbf{RDG}=\mathbf{RDG}\left(\mathbf{\text{BW}}\left({\beta }_1\mathsf{\text{, }}{\beta }_2\mathsf{\text{, }}...\mathsf{\text{, }}{\beta }_L\right)\right)\\ \mathsf{\text{s}}\mathsf{\text{.t}}\mathsf{\text{.}}\mathbf{BW}\left({\beta }_1,{\beta }_2,...{\beta }_L\right)\le \mathbf{BW}\end{array}$$

(5)

where BW is the available BW pool for video encoding. The optimum solution, denoted by RDG(BW), describes the B-R-D behavior of the video encoder. The corresponding optimum BW control parameters are denoted by {β _i *(BW)}, 1 ≤ i ≤ L.

More specifically, we develop an analytical B-R-D model to perform on-line BW optimization for real-time video coding. For the simplicity of on-line execution, the RDG formulation can be well approximated by the following expression.

$$RD{C}_{\mathsf{\text{init}}}-RD{C}_{\mathsf{\text{BMA}}}=\gamma \times BW\left({\beta }_1,{\beta }_2,...,{\beta }_L\right)$$

(6)

where γ is a positive constant. In this study, we refer to BW as the maximum required data BW for ME.

4. Online R-D optimized BW-scalable ME

Section 3 provides a theoretical analysis of the data BW-limited performance of the B-R-D optimization. However, in this section, we discuss how this theoretical limited data BW performance can be realized in practical video coding. There are four major issues that need to be addressed. First, the real BW calculation requires global knowledge of the on-chip SRAM buffer resource and reuse strategy. Second, in BW variations between video coding and decoding as discussed in this section, we assume that the available data BW for video coding are time-varying because of non-stationary video input on the real-time coding and decoding side. Third, once the optimum BW efficiency of the previous coded MB is determined, we need to develop a scheme to allocate and predict the BW interval to achieve the video smoothness constraint. This approach is computationally intensive and its corresponding parameter adjustment is only suitable for offline analysis. In real-time video encoding on mobile devices, it is desirable to develop a low-complexity scheme that is able to estimate the BW interval parameters from the frame statistics collected in the video coding. Fourth, to avoid under- or over-use of the BW pool, the target SR is further refined by the neighboring MV. In the following, we will discuss these issues.

4.1. BW budget initialization

First, the BW budget (BW_budget) is initialized for BW allocation of the overall data BW pool later in the coding process. This initialization takes the available system BW and converts it to a default system SR for the ME. Then, the BW budget is allocated with the above system SR for a GOP, as in (7).

$$B{W}_{\mathrm{budget}}=\frac{B{W}_{\mathrm{Bus}}}{\mathrm{Frame}\text{\_}\mathrm{Rate}}\times \mathrm{GOP}\text{\_}\mathrm{size}$$

(7)

where the BW_Bus denotes the bus data transmission rate (bytes/s), Frame_Rate is the number of coded frames per second, and GOP_size denotes the frame numbers in a GOP. Larger GOP size allows for more freedom in adjusting the BW. For the purposes of having a concrete example that represents common practices in video coding, the BW budget for the GOP is set 16 frames in this article.

4.2. BW evaluation in an R-D sense

To justify the BW usage from (6), the BW efficiency, G_ave, is defined as the sum of the RDG before the current coded k th MB divided by the total used BW ($B{W}_{usage}^k$), which denotes the accumulated used data BW up to the (k - 1)th MB, as in (8) and (9).

$$G_{\mathrm{ave}}=\frac{{\sum }_{i=1}^{k-1}\left(\mathrm{RD}{C}_{\mathrm{init}}^i-{RDC}_{\mathrm{BMA}}^i\right)}{B{W}_{\mathrm{usage}}^k}$$

(8)

where

$$B{W}_{\mathrm{usage}}^k=\sum _{i=1}^{k-1}B{W}_{\mathrm{usage}}^i$$

(9)

and $RD{C}_{init}^i$ denotes the RDC at the predicted MV position. $RD{C}_{BMA}^i$ denotes the RDC after the motion search of the block-matching algorithm, and $B{W}_{\mathrm{usage}}^k$ denotes the used data BW in the i th MB with a Level C data reuse scheme.

G_ave measures the BW efficiency by averaging the RDG over the used BW before the k th MB, which implies how much RDG can be achieved with a unit of data BW. Thus, the more G_ave we gain, the better BW and coding efficiency we will obtain. In the following step, we will use G_ave for BW prediction.

4.3. BW prediction and allocation with the smoothness constraint

With the BW efficiency, G_ave, we can derive the allowed BW interval with the BW prediction and allocation. The BW prediction predicts the available BW for the next coded MB with the smoothness constraint. The smoothness constraint maintains the quality and the smoothness (i.e., similar RDC) between consecutively coded MBs. With this constraint and the RDG per unit BW from (8), we can predict the forward and backward BW usage and thus, constrain the possible BW usage of the next coded MB.

First, to keep the quality and the smoothness between the current and the previous MBs, we use the RDC data from previous MBs to make further predictions (10).

$$RD{C}_{\mathrm{init}}^k-G_{\mathrm{ave}}B{W}_{BP}^k=\frac{{\sum }_{\mathbf{i}=1}^{k-1}{RDC}_{\mathrm{BMA}}^i}{k-1}$$

(10)

where BW _BP denotes the backward BW prediction, as shown in latter equation. In (10), the left-hand side is the target RDC of the current MB, and the right-hand side is the average RDC of the previous MBs. To maintain the quality and the smoothness, ideally, the target RDC of the current MB will be equal to the average past RDC s. Thus, if we have larger G_ave, (10) implies that less BW (i.e., BW _BP ) is needed to maintain a similar RDG as the previous MBs. Therefore, the backward prediction for the current k th MB can be derived, as in (11) from (10).

$$B{W}_{BP}^k=\frac{{RDC}_{\mathrm{init}}^k-\frac{{\sum }_{i=1}^{k-1}{RDC}_{\mathrm{BMA}}^i}{k-1}}{G_{\mathrm{ave}}}$$

(11)

In contrast to BW _BP , we define the forward prediction BW _FP to keep the quality and smoothness between the current and the future MBs by adopting BW information as in (12).

$${BW}_{FP}^k=\frac{{BW}_{\mathrm{budget}}-B{W}_{\mathrm{usage}}^k}{n-\left(k-1\right)}$$

(12)

where n is the overall MB numbers in a GOP. Because we have no knowledge of the future RDG, the forward prediction, BW _FP , is set to the remaining BW budget divided by the remaining MBs in the GOP that are not coded yet.

These two BW predictions link the BW usage between the past MBs and the future MBs. Their relationship can be used to allocate the available BW as follows:

if (BW _FP > BW_{B P}) { (condition 1)

BW_lower= BW _BP + 0.5 × (BW _FP - BW _BP );

BW_upper= BW _FP + 0.25 × (BW _FP - BW _BP );

}

else { (condition 2)

BW_lower = BW _FP - 0.5 × (BW _BP - BW _FP );

BW_upper = BW _FP ;

}

in which, BW_lower and BW_upper are the lower and upper bounds of the BW usage per MB, respectively. The parameters, 0.5 and 0.25, are selected empirically and are easy to implement because they are powers of 2. The parameters are obtained from a two-step process. In the first step, we execute the proposed BW-scalable ME algorithm with different configurations of parameters to obtain the corresponding BW_lower, BW_upper, and R-D data. Note that this step is computationally intensive and is intended for offline analysis to obtain BW_lower, BW_upper, and the B-R-D model only. Once the B-R-D model and the BW intervals BW_lower and BW_upper are established, we perform the second step, which optimizes the configuration of the BW control parameters to maximize the video quality under the system BW constraint. Meanwhile, the parameters, which are empirically selected in the following section, are obtained by the same method. For condition 1, as shown in Figure 1, BW _BP is smaller than BW _FP , which implies that less BW had been allocated to the previous MBs, and thus, more BW can be allocated to the next MB. As a result, we set the lower bound, BW_lower, higher than the average BW in the past MBs (equal to BW _BP + 0.5 × (BW _FP - BW _BP )), and also set the upper bound, BW_upper, higher than the average BW prediction in the future MB coding (equal to BW _FP + 0.25 × (BW _FP - BW _BP )). This larger BW allocation enables better quality. In contrast, for condition 2 in Figure 1, BW _FP is smaller than or equal to BW _BP , which implies that too much BW had been allocated to the previous MBs, and hence less BW can be allocated to the next MB. As a result, both bounds should be lower than BW _FP to keep the smoothness and quality, and we set BW_lower equal to BW _FP - 0.5 × (BW _BP - BW _FP ) and set BW_upper equal to BW _FP .

Figure 1

Illustration of the available BW interval determination.

4.4. SR decision and refinement

Finally, we employ the above available BW interval and R-D data to make an SR decision for the next MB coding. The SR decision is divided into three cases, and the corresponding SR adjustment coefficient is resolution independent, as shown in Figure 2. Case 1 is the BW limited case because the average BW usage of the previous MBs falls outside the available BW interval bounded by BW_upper and BW_lower. Thus, the current SR is decreased by 8 if it is larger than BW_upper or increased by 8 if it is smaller than BW_lower for next MB coding.

Figure 2

Illustration of the SR decision.

The average BW usage of the previous MBs falling inside the available BW interval implies sufficient BW is available for R-D optimization. This can be further divided into two cases, case 2 and case 3. If the RDC (R × D_cur) is larger than a predefined threshold (case 2), the video has a bad quality, and thus, the SR is increased by 16 for better quality in the next MB. This threshold is set empirically to 4 times, the average RDC of the previous MBs, i.e., 4(R × D_avg), for coarse-grained refinement of the quality. However, if the RDC (R × D_cur) is smaller than the predefined threshold (case 3), the video has a quite smooth quality, and thus, the SR is adjusted slightly. Thus, the SR remains unchanged if the RDG of the current MB (RDG_cur) is within the average RDG (RDG_avg) plus or minus an adaptive offset (i.e., RDC _BMA /20000 empirically for fine-grained refinement of quality). However, if the RDG_cur is smaller than RDG_{a vg}- offset, the video is of good enough quality, and thus, the SR is decreased by 4 to save BW. On the other hand, if the RDG_cur is larger than RDG_avg+ offset, the quality is low, and the SR is increased by 4 to improve the quality.

The above SR decisions are further refined to avoid BW waste by considering the SR values in the adjacent MBs, as illustrated in Figure 3a. First, we get the adjacent MVs from the neighboring blocks and the MV of previous frame on the co-located block, such as MV_UL, MV_U, MV_UR, MV_L, and MV_Cur, shown in Figure 3b. All these MVs are of sub-pel precision. Then, we compare these five MVs and choose a maximum MV (max_mv). After that, we set the available SR value using this maximum MV. The refined SR, max_avail_SR, is

Figure 3

Illustration of the SR refinement. (a) Flowchart of the SR refinement method. (b) The relationship between neighboring blocks and the current block.

$$max\text{\_}\mathrm{avail}\text{\_}SR=\left\{\begin{array}{l}S{R}_{\mathrm{lower}},max\text{\_}mv\le m{v}_{\mathrm{lower}}\\ S{R}_{\mathrm{step}}\times \mathrm{Ceil}\left(max\text{\_}mv/S{R}_{\mathrm{step}}\right)+S{R}_{\mathrm{offset}},m{v}_{\mathrm{lower}}<max\text{\_}mv\le m{v}_{\mathrm{upper}}\\ S{R}_{\mathrm{upper}},\mathrm{otherwise}\end{array}\right.$$

(13)

in which the parameters SR_lower, SR_upper, SR_step, and SR _offset are resolution dependent. For our simulation, we set SR_lower equal to 4 for CIF and 26 for HD (720P) resolution. Meanwhile, we set SR_upper, SR_step, and SR _offset equal to 32, 4, and 4 for CIF resolution and equal to 72, 8, and 2 for HD (720P) resolution. Meanwhile, we set mv_lower and mv_upper equal to 2 and 24 for CIF resolution and 24 and 64 for HD (720P) resolution.

Finally, the SR is selected by choosing the minimum SR between max_avail_SR and SR from Figure 2, for MB coding.

4.5. Summary of the algorithm

Figure 4 shows the proposed B-R-D optimized algorithm that can be combined with existing ME algorithms to make them BW scalable. This algorithm first models the available BW with its RDG and then predicts and allocates the BW in an R-D optimized sense to determine the available SR. The whole algorithm is repeated for all inter-coded frames in a GOP and consists of four steps, as described below.

Figure 4

Flowchart of the B-R-D optimized modeling method.

Step 1. Initialization: Create the BW budget from (7) for all MBs in a GOP.

Step 2. BW evaluation in an R-D sense: Evaluate the RDG in terms of the consumed BW as shown in (8) and (9) to model the BW in a R-D sense.

Step 3. BW prediction and allocation with the smoothness constraint: From the RDG obtained from step 2 and the available BW, the BW for the next coded MB is predicted in (10) to (12) and allocated as described in Section 4.3 to keep the video quality as smooth as possible using the smoothness constraint.

Step 4. SR decision and refinement: According to the available BW from step 3, the SR of next coded MB is determined and refined in (13) for ME execution.

5. Simulation results

5.1. Simulation conditions

The proposed algorithm was implemented in the H.264/AVC reference software, JM [23], for performance evaluation. The simulation conditions are CIF-sized test sequences with a baseline profile, no R-D optimization, one reference frame, a full-search algorithm as well as an Enhanced Predictive Zonal Search (EPZS) algorithm [24] for ME, IPPP sequences, 30 frames/s, and 16 frames per GOP. All of the block matching algorithms were implemented using Visual C++ on a PC with a 2.66 GHz Intel^® Core™ 2 Duo CPU.

In the following simulations, we classify the corresponding BW conditions into two patterns: a constant data BW pattern and a variable data BW pattern. Both patterns provide the same amount of reference block data for the same SR ± R. However, the constant data BW pattern will assume that the available BW is constant and fixed during ME operations, which in turn assumes that the available BW is sufficient and implies that the video encoder does not have a BW constraint during the video encoding process. Meanwhile, the variable data BW pattern will assume that the available BW is variable during ME operations, which assumes that the available BW is insufficient and implies that the video encoder is BW constrained during the video encoding process. The constant data BW pattern is the scenario used in traditional ME design, which does not consider the other components, while the variable data BW pattern simulates the scenario where the BW is changing due to situations like simultaneous coding and decoding (defined as SCD mode) in a video phone or different low power modes (defined as LP mode) for mobile applications. The SCD mode assumes the decoding uses merged sequences from Stefan, Akiyo, and Football (interleaved high-motion and low-motion sequences) and sets the scene cut at a multiple of 32 frames. With the above interleaved decoded sequence, the available BW for encoding will change dynamically, as shown in Figure 5a. Figure 5b shows the LP mode with a descending trend in data BW in a power aware system. In the following simulations, we assume the SR for the search algorithm is ± R for the constant data BW pattern R and the variable data BW pattern case.

Figure 5

Variable data BW pattern with ± 8 SR for: (a) the SCD mode and (b) the LP mode.

To show the benefit of the proposed scheme, we tested three different BW adaption schemes in the following simulations. The first scheme, denoted as fixed-SR, is for ME without any BW adaption scheme. Thus, the total BW for ME is equally distributed for all MB coding, and its SR setting is constant for the entire coding time. The second scheme, denoted as simple-SR, is for ME with a simple BW adaption scheme. Its BW adaption equally distributes the available data BW to all MBs in a period, as in the fixed-SR case, but the distribution will be changed when the available BW changes. Thus, its SR adapts as well. This adaption does not consider the used BW or the related R-D information. The final scheme, denoted as BRD-SR, is the proposed B-R-D optimized BW-scalable method.

5.2. B-R-D performance evaluation

Tables 1, 2, 3, 4, and 5 show the simulation results for the constant and variable BW patterns with the different BW adaption schemes. Figure 6 shows the average BW per frame for the high-motion Stefan sequence with the quantization parameter set to 28.

Table 1 Performance comparison with the fixed-SR scheme for CIF resolution

Table 2 Performance comparison with the simple-SR scheme for CIF resolution in the SCD mode

Table 3 Performance comparison with the simple-SR scheme for CIF resolution in the LP mode

Table 4 Execution-time comparison with the fixed-SR scheme for CIF resolution

Table 5 Performance comparison with the fixed-SR scheme for 720P resolution.

Figure 6

Constant BW patterns with SR equal to: (a) ± 8 (b) ± 16 (c) ± 24 and variable BW patterns with SR equal to (d) ± 8 (e) ± 16 (f) ± 24.

For the constant BW pattern case, Table 1 illustrates that the full search ME with the proposed BRD-SR scheme can attain similar quality performance as the that with the fixed-SR scheme in the low-motion sequence (Akiyo sequence) and the medium-motion sequence (Foreman sequence), but with less BW. In case of low-motion sequence, the proposed algorithm can save 35-83% of the BW with different SRs. For the medium-motion sequence, our algorithm can save 4-45% of the BW. For the high-motion sequence (Stefan sequence), our algorithm can save an average bit-rate of up to 13% and increase the PSNR by up to 0.1 dB under the low SR constraint. Also, the simulation shows similar results as that in the full search algorithm by applying our proposed algorithm to the fast algorithm, the EPZS algorithm, which is due to our effective SR adjustment. For a fair comparison, the presented BW has considered data reuse [11] in the overlapped region between search points, and thus, only new data that are not in the local buffer will be loaded from external memory and counted in the BW usage. In summary, the proposed algorithm can save data BW for the full search and EPZS algorithms as well.

For the variable BW pattern case, Tables 2 and 3 compare the results between the BRD-SR scheme and the simple-SR scheme in the SCD and LP modes. All of these results show trends in R-D performance and BW saving similar to those in Table 1. In summary, these results show our algorithm with B-R-D optimization can better utilize the BW for ME computation and achieves better performance than the fixed-SR and simple-SR schemes.

Table 4 shows the execution-time of the proposed algorithm and compares it to the fixed-SR scheme with the constant BW pattern. The results are similar to those found with the simple-SR scheme in the variable BW pattern case. Our proposed algorithm slightly improves execution time. However, the saving is not directly proportional to BW saving due to the calculation overhead of the MB-level BW-scalable scheme. These overheads can be reduced with further software optimization or better hardware implementation of the existing ME engine.

Table 5 shows the simulation results for the HD resolution videos and a comparison of the proposed scheme with the fixed-SR scheme. The simulation conditions are three 720P-sized video sequences with a baseline profile, no R-D optimization, one reference frame, IPPP sequences, 30 frames/s, and 16 frames per GOP. All of the simulation results show similar savings to those found with CIF resolution, which are listed in Table 1. This proves the applicability of the proposed algorithm on larger sized video sequences.

6. Conclusion

In this article, we propose a BW-scalable approach for an ME algorithm to maximize the R-D performance while dynamically allocating the available BW. Compared to the traditional methods, our algorithm could save up to 70% of the BW with a full-search algorithm and 65% of the BW with the EPZS algorithm with an average SR size of ± 16 for low-motion CIF resolution sequences. Compared to either the full search or EPZS algorithm, our proposed algorithm can save up to 70% of the BW with an SR size of ± 56 for HD (720P) resolution video. These savings come from appropriate MB-level BW allocation. In addition, while coding high-motion sequences, the simulation result shows our design could save an average bit rate of up to 13% and increase the average PSNR by up to 0.1 dB with similar BW usage for CIF resolution. The proposed design can be combined with current ME designs. Further study can be done by incorporating this work into the rate-control scheme or other resource constrained algorithms for better performance.