Robust motion estimation on a low-power multi-core DSP
© Igual et al.; licensee Springer. 2013
Received: 31 January 2013
Accepted: 10 April 2013
Published: 10 May 2013
This paper addresses the efficient implementation of a robust gradient-based optical flow model in a low-power platform based on a multi-core digital signal processor (DSP). The aim of this work was to carry out a feasibility study on the use of these devices in autonomous systems such as robot navigation, biomedical assistance, or tracking, with not only power restrictions but also real-time requirements. We consider the C6678 DSP from Texas Instruments (Dallas, TX, USA) as the target platform of our implementation. The interest of this research is particularly relevant in optical flow scope because this system can be considered as an alternative solution for mid-range video resolutions when a combination of in-processor parallelism with optimizations such as efficient memory-hierarchy exploitation and multi-processor parallelization are applied.
Motion estimation has been deeply investigated during the last 50 years; however, it is still considered by the scientific community as an emerging field of special interest due to the plethora of applications that supports the interpretation of the real world, such as navigation, sports tracking, surveillance, video compression, robotics, vehicular technology, etc. It is also useful in the neuroscience field, where the task of modeling neuromorphic algorithms and systems, which fit well according to the human brain evidences, is an open and common research problem.
Motion estimation determines motion vectors and describes the transformation of an entire two-dimensional (2D) image into another, usually taken from contiguous frames in a video sequence using pixels or specific parts such as shaped patches or rectangular blocks.
Motion relies on three dimensions, but images are a projection of the three-dimensional scene onto a two-dimensional plane, therefore posing a mathematically ill-posed problem [1–3], usually known as ‘aperture problem’. To overcome these drawbacks, external knowledge regarding the behavior of objects, such as rigid body constraints or other models that might approximate the motion of a real video camera, becomes necessary. These models are based on the motion of rotation, translation, and zoom, in all three dimensions.
The optical flow paradigm is not exactly the same concept as motion estimation, although they frequently come up associated. Optical flow is the apparent motion of image objects or pixels between frames . Two assumptions are usually applied to optical flow :
Brightness constancy: although the 2D position of the image discriminant characteristics, such as brightness, color, etc., may change, they keep their value constant over time. Algorithms for estimating optical flow exploit this assumption in various ways to compute a velocity field that describes the horizontal and vertical motions of every pixel in the image.
Spatial smoothness: it appears from the observation that pixels in the neighborhood usually belong to the same surface and are inclined to present the same image motion.
Optical flow has many drawbacks that increase the burden of estimating it. For instance, the optical flow is ambiguous in homogeneous image regions due to the brightness constancy assumption. Additionally, in real scenes, the assumption is violated at the motion boundaries as well as by occlusions, noise, illumination changing, reflections, shadows, etc. Therefore, only the synthetic-made motion can be recovered with no ambiguity. These two assumptions may lead to errors in the flow estimates.
There are a number of common examples which deliver a non-null value for motion estimation but a zero value for optical flow, e.g., a rotating sphere under constant illumination. Similarly, a static sphere with changing light will deliver optical flow, while the motion field remains null , or an old barber pole in motion that shows a real velocity field perpendicular to the estimated optical flow.
Pattern-matching methods  are probably the most intuitive methods. They operate by comparing the positions of image structure between adjacent frames and inferring velocity from the change in location. The aim of block-matching methods is to estimate motion vectors for each macro-block within a specific and fixed search window in the reference frame. These exhaustive or semi-exhaustive search algorithms match all macro-blocks within a search window in the reference frame to estimate the optimal macro-block in order to fit with the minimum block-matching error metric.
Motion energy methods are probabilistic methods that use space-time oriented filters tuned to respond optimally to specific image velocities. Banks of such filters are used to respond to a range of visual motion possibilities . Therefore, motion estimation is not unique for every single stimulus. These methods usually work under Fourier space.
Gradient-based or differential technique family uses derivatives of image intensity in space and time. Combinations and ratios of these derivatives yield explicit measures of velocity [2, 5]. The particular implementation of the algorithm used in this paper belongs to this family and is based on Johnston’s work [6, 7]. The multi-channel gradient model (McGM) was developed as part of a research effort aimed at improving our understanding of the human visual system. This model also allows us to make predictions that can be tested through psychophysical experimentation as separate motion illusions that are observed by humans in experiments .
One of the main drawbacks of the McGM model is the high hardware requirements needed to achieve real-time processing. On one hand, McGM presents an uptrend in temporal data storage which is translated into non-negligible memory requirements; on the other hand, optical flow processing requires important computational capabilities to meet real-time requirements. Previous works [9, 10] have fulfilled those requirements by means of exploitation of inherent data parallelism of McGM using both modern multi-core processors and hardware accelerators such as field-programmable gate arrays (FPGAs) and graphics processing units (GPUs).
The limitation in power consumption of current embedded devices makes it necessary to consider energy-related issues in the implementation of optical flow algorithms. There are in the literature ad hoc solutions to solve the motion estimation problem with power constraints. As an example, there are countless proposals under low-power conditions for pattern-matching family algorithms, but most are in the video compression field [11, 12]. Another approach with central processing units (CPUs)  presents a parallel scheme applied to a model based on well-known Lucas-Kanade approach, which reduces power consumption in terms of thermal design power (TDP) and still meets the real-time requirements when low-power chipsets (TDPs of 20 to 30 W) are used. Moreover, Honegger et al.  implement a low-power stereo vision system with FPGA based on the Nios II processor (Altera, San Jose, CA, USA). Furthermore, processor manufacturers are now concerned for concepts such as green computing. The aim is to develop more efficient chips not only in terms of performance rates (throughput measured in terms of floating-point operations per second (FLOPS) or Mbits per second) but also energy efficiency . Besides modern and efficient multi-core CPUs, hardware accelerators such as GPUs or Intel MIC (Intel Corp., Santa Clara, CA, USA), or reconfigurable devices (FPGAs), one of the latest additions on specific-purpose architectures applied to general-purpose computing are low-power digital signal processors (DSPs). One of the primary examples in this field is the C6678 multi-core DSP from Texas Instruments (TI; Dallas, TX, USA) that combines a theoretical peak performance of 128 GFLOPs (billions of floating-point operations per second) with a power consumption of roughly 10 W per chip. Besides, one of the most appealing features is the ease of programming, adopting well-known programming models for sequential and parallel implementations.
Our contribution provides an efficient implementation for an optical flow gradient-based model using a low-power DSP exploiting different levels of parallelism. To the best knowledge of the authors, this is the first attempt to use a DSP architecture to implement a robust optical flow gradient-based model. There are only few approaches existing in the literature exploiting gradient-based motion estimation methods in DSP platforms as the one proposed by Shirai et al.  in early 1990s, implementing the classical method of Horn-Schunck algorithm  using many boards with a TMS320C40 DSP each. This algorithm supplements optical flow constraint with regularizing smooth terms, while our work uses spatio-temporal constancy. Besides, performance and/or energy consumption is not considered in that work. Rowenkap et al.  implemented in 1997 the same algorithm as the previous work, using the same DSP and reaching up a throughput of 5 frames per second (fps) and 15 fps (for 1282 image resolution) when using one and three DSPs, respectively. The last work considered is the neuromorphic implementation of Steiner  that uses the Srinivasan algorithm  on a dsPIC33FJ128MC804 processor; this algorithm is based on simple stage procedure of image interpolation.
Exploit loop-level parallelism by means of a very long instruction word (VLIW) processor capability available in TI’s C6678 DSP.
Take advantage of data-level parallelism by means of multi-media extensions capability.
Make use of thread-level parallelism available in TI’s C6678 multi-core DSP.
Exploit the memory system hierarchy with the efficient use of cache levels and on-chip shared memory.
Our experimental evaluation includes a comparison of the DSP implementation with other state-of-the-art architectures, including general-purpose multi-core CPUs and other low-power architectures.
The rest of the paper is organized as follows. Section 2 moves through a specific neuromorphic model and describes the particularities of each stage. Section 3 gives an overview of the DSP architecture, together with the main motivations for choosing this platform for our motion estimation approach. In Section 4, we give details about the specifics of the implementation of McGM on the DSP and provide an experimental analysis of the implementation. Finally, Section 5 provides some concluding remarks and outlines future research lines.
2 Multi-channel gradient model
The McGM model, proposed by Johnston et al. [6, 7], implements a processing vision scheme described by Hess and Snowden , combining the interaction between ocular vision and brain perception and simplifying the human vision model . In order to solve the problems with the basic motion constraint equation, many gradient measurements have been introduced (Gaussian derivatives) into the velocity measure via Taylor expansion representation of the local space-time structure.
2.1 FIR-filtering temporal
In practice, for an input movie of N frames with n x×n y resolution, this stage produces approximately (N−L)×n x×n y×n Temp_filt temporal data as indicated in Figure 2 as T 1, T 2, and T 3 (n Temp_filt=3) in temporal filtering stage; these intermediate data structures are provided to the spatial filtering stage as inputs.
2.2 FIR-spatial derivatives
being σ the variance in normal distribution.
From the point of view of data-path processing, for n Spat_filters Gaussian filters (see Figure 2), this stage generates (N−L)×n x×n y×n Temp_filters×n Spat_filters output data.
2.3 Steering filtering
From the data-path perspective, this is the most memory-consuming and computational-demanding stage. Resource consumption is closely related to the number of orientations to consider, which is denoted by n θ s orientations in Figure 2. More specifically, the amount of data produced at this stage is quantified close to ((N−L)×n x×n y×n Temp_filters×n Spat_filters×n θ s).
2.4 Product and Taylor and quotients
From the point of view of dataflow, McGM changes its trend at this point and starts to converge, which means a considerable reduction in the amount of data to compute. Data stored is approximately (N−L)×n x×n y×n θ s×6.
2.5 Velocity primitives
Modulus and phase extraction corresponds to the final velocity vector, which is computed from the velocity components previously calculated.
Velocity primitives are allocated with (N−L)×n x×n y×n θ s×4 data.
2.6 Modulus and phase
Lastly, modulus and phase are size of (N−L)×n x×n y (one piece of data per input pixel).
3 Overview of the C6678 DSP architecture
The C6678 digital signal processor from Texas Instruments is a high-performance, low-power DSP with floating-point capabilities . It presents eight C66x VLIW cores and runs at 1 Ghz. The whole device dissipates a maximum power of 10 W. Besides low-power, high-performance, and floating-point capabilities, one of the strengths of the C6678 device is the amount of standard peripherals it supports: PCIe interface to communicate with a CPU host, Serial Rapid I/O, and Hyperlink for fast- and low-latency inter- and intra-chip communication, or direct memory access (DMA) to overlap computation with transfers between the external memory and on-chip memory.
3.1 C66x core architecture
The C66x core illustrated in Figure 3 is the base of the multi-core C6678 DSP architecture. It is implemented as a VLIW architecture, taking advantage of different levels of parallelism:
Instruction-level parallelism. In the core, eight different functional units are arranged in two independent sides. Each one of the sides has four processing units, namely L, M, S, and D. The M units are devoted to multiplication operations. The D unit performs address calculations and load/store instructions. The L and S units are reserved for additions and subtractions, logical, branch, and bitwise operations. Thus, this eight-way VLIW machine can issue eight instructions in parallel per cycle.
Data-level parallelism. The C66x instruction set (ISA) includes single-instruction multiple-data (SIMD) instructions that operate on 128-bit vector registers. More precisely, the M unit, performs four single-precision (SP) multiplications (or one double precision (DP) multiplication) per cycle. L and S units carry out two SP additions (or one DP addition) per cycle. Thus, the C66x is ideally able to perform eight single-precision multiply-add (MADD) operations in 1 cycle. In double precision, this number is reduced to two MADDs in 1 cycle. With eight C66x cores, a C6678 processor running at 1 GHz yields 128 SP GFLOPS or 32 DP GFLOPS. All floating-point operations support the IEEE754 standard.
Thread-level parallelism. It can be exploited by running different threads across the cores of the DSP. In our case, we will use OpenMP as the tool to manage thread-level parallelism.
3.2 Memory hierarchy
The memory hierarchy for the C6678 device is shown in Figure 3 (left). L1 cache is divided into 32 KB of L1 program cache and 32 KB of L1 data cache per core. There is also 512 KB of L2 cache per core. Both L1 data cache and L2 memory can be configured either as random-access memory (RAM), cache, or part RAM/part cache. This provides additional capability of handling memory and can be exploited by the programmer. There is an on-chip shared memory of 4,096 KB accessible by all cores, known as multi-core shared memory controller (MSMC) memory, and an external 64-bit DDR3 memory interface running at 1,600 MHz with ECC support.
3.3 Programming the DSP
TI’s DSPs run a lightweight real-time native operating system called SYS/BIOS. A C/C++ compiler is provided as part of the development environment. The C/C++ compiler eases the porting effort of virtually every existing C/C++ code to the architecture. To improve the efficiency of the generated code for each TI architecture, the compiler provides optimization techniques in the form of #pragmas and intrinsic SIMD instructions to fully exploit the core architecture and extract all the potential performance without resorting to assembly programming.
The compiler supports OpenMP 3.0 to allow rapid porting of existing multi-threaded codes to multi-core DSPs. The OpenMP runtime performs the appropriate cache control operations to maintain the consistency of the shared memory when required, but special precaution must be taken to keep data coherence for shared variables, as no hardware support for cache coherence across cores is provided.
3.4 Work environment
All codes were evaluated using a TMDXEVM6678LE evaluation module that includes an on-board C6678 processor running at 1 GHz. The board has 512 MB of DDR3 RAM memory available for image storage or generation. Our tests were developed on top of SYS/BIOS using the OpenMP implementation from Texas Instruments, MCSDK version 2.1, and Code Generation Tools version 7.4.1 with OpenMP support enabled. Single-precision floating-point arithmetic was used for all the experiments. We have not observed any precision issue in our DSP implementations compared with previous results in other architectures [9, 24, 25]. Therefore, our experimental section will be focused exclusively on a performance analysis instead of a qualitative analysis of the obtained numerical results.
4 Implementation and experimental results
In this section, we present relevant algorithmic and implementation details of each stage of the McGM method. Whenever possible, we provide a list of incremental optimizations applied in order to improve the performance of our implementation on the multi-core DSP. Due to the high-computational requirements of the first three stages of the algorithm (temporal filtering, spatial filtering, and steering), we will focus on those parts. However, some notes about the last stages, together with experimental results, are also given.
The optimizations proposed are DSP specific and address four of the most appealing features of the architecture: instruction, data, and thread parallelism extraction, and the exploitation of the flexibility of the memory hierarchy, plus the usage of DMA to overlap computation and communication.
4.1 Relevant parameters for McGM
Main parameters involved in the McGM algorithm
Input frame set
Number of input frames
Depends on input
n x×n y
Depends on input
322 to 1282
Window (temporal convolution)
15 to 23
Number of temporal filters
Window (spatial convolution)
15 to 23
Number of spatial filters
Number of orthogonal orders
n θ s
Number of angles
6 to 24
In Table 1, we also add four different parameter configurations that will be used for global throughput evaluation. Although all experimental results are reported for video sequences with square frames, our implementation is prepared for non-squared images, and no qualitative differences in the performance results have been observed.
4.2 McGM implementation on the DSP
Basic implementation. We establish a baseline C implementation for comparison purposes. It includes the necessary compiler optimization flags and common optimization techniques to avoid unnecessary calculations and benefit from data locality and cache hierarchy. No further DSP-specific optimizations are applied in the code of this naive implementation.
DMA and memory-hierarchy optimization. One of the strengths of the DSP is the ability of explicitly managing on-chip memory levels (L1 cache, L2 cache, and MSMC memory). Thus, one can define buffers, assign them to a particular memory-hierarchy level (using the appropriate #pragma annotations in the code), and perform data copies between them as necessary. In addition, DMA capabilities are offered in order to overlap data transfers between memory levels and computation.
The usage of blocking and double-buffering is required. This involves the allocation of the current block of each frame to be processed and the next block which is being transferred through DMA while CPU computation is in progress. This technique effectively hides memory latencies, improving the overall throughput. In our case, we have mapped the temporal buffers that accommodate blocks of the input frames to the on-chip MSMC memory, in order to improve memory throughput in the computation stage.
Loop optimization. VLIW architectures require a careful loop optimization in order to let the compiler effectively apply techniques such as software pipelining, loop unrolling, and data prefetching . In general, the aim is to keep the (eight) functional units of the core fully occupied as long as possible. To achieve this goal, the developer guides the compiler about safe loop unrolling factors, fixed unroll counts (using appropriate #pragma constructions), or pointer disambiguation (using restrict keyword on those pointers that will not overlap during the computation) by means of the mentioned tags or pragmas. Even though this type of optimizations is not critical in superscalar processors that defer the extraction of instruction-level parallelism to execution time, it becomes crucial for VLIW architectures, even more for algorithms heavily based on loops as McGM. We have performed a full search to find the optimal unroll factor for each loop in the algorithm.
SIMD vectorization. As mentioned in Section 3, each C66x core is able to execute single-cycle arithmetic and load/store instructions on vector registers up to 128-bit wide. Naturally, this feature is supported at ISA level and can be programmed using intrinsics . In McGM, data parallelism is massive and can be exploited by means of SIMD instructions in many scenarios. Intermediate data structures are stored using single-precision floating point (32-bit wide).
Thus, in the convolution step, input data can be grouped and processed in a SIMD fashion using 128-bit registers (usually referred as quad registers) for multiplications and 64-bit registers for additions. Given that the C66x architecture can execute up to eight SP multiplications (four per each M unit) and eight SP additions (two per each L and S unit), each core can potentially execute up to eight SP multiplication-additions per cycle if SIMD is correctly exploited. At this stage, we load and operate on four consecutive pixels of the image, unrolling the corresponding loop by a factor 4. Special caution must be taken in order to meet the memory alignment restrictions of the load/store vector instructions; to meet them, we apply zero-padding to the input image when necessary, according to its specific dimensions.
Loop parallelization. Up to this point, all the optimizations have been focused on exploiting parallelism at core level. The last stage of the optimization involves the exploitation of thread-level parallelism to leverage the multiple cores in the DSP. The parallelization is carried out by means of OpenMP. Special care must be taken with shared variables, as no cache coherence is automatically maintained. Thus, data structures must be zero-padded to fill a complete cache line and to avoid false sharing, and explicit cache write-back and/or invalidate operations must be performed in order to keep coherence between local memories to each core.
4.2.1 Stage 1: temporal filtering
Algorithm and implementation
In order to obtain the temporal derivative of the image, it is necessary to perform a convolution of each image sequence with each one of the three temporal filters obtained (low-pass and two band-pass filters.)
Algorithm 1 outlines the basic behavior of the temporal filtering stage. Usually, for all stages, the calculation of the corresponding filter is performed off-line if necessary, prior to computation. As the number of temporal filters usually remains constant and is reduced (i.e., n Temp_filters = 3), performance rates of this stage greatly depend on the window size (L) in which we apply the temporal filters and on frame dimensions (n x×n y).
Algorithm 1 temp_filt = stage I (frames, N , L , n Temp_filters, α , τ )
As output, n Temp_filters matrices of the same dimensions as each input frame are generated as a result of the convolution of each frame with the corresponding convolution filter. These matrices will be the input for the second stage (spatial filtering.)
DSP optimizations and performance results
At this stage, the critical factors affecting performance are frame size (n x×n y) and temporal window size (L). In general, for a fixed L, throughput decreases for increasing frame dimensions. For a fixed frame dimension, the impact of increasing the window size is also translated into a decrease in performance, although not in a relevant factor.
Independently from the evaluated frame resolution and window dimensions, core-level optimizations (usage of DMA, loop optimizations, and SIMD vectorization) are translated into performance improvements between ×1.5 and ×2, depending on the specific selected parameters. When OpenMP parallelization is applied, the throughput improvement yields between ×5.5 and ×7 compared with the optimized sequential version. In general, the throughput obtained by applying the complete set of optimizations improves the original basic implementation in a factor between ×7 and ×14. We would like to remark the multiplicative effects observed when both in-core and multi-core optimizations are carried out.
4.2.2 Stage 2: spatial filtering
Algorithm and implementation
From the algorithmic point of view, spatial filtering does not dramatically differ from the previous stage, see Algorithm 2. For each one of the spatial filters generated a priori and each one of the temporal-filtered frames, we apply a bi-dimensional convolution. Note that the amount of generated data increases compared with that received from the previous stage in a factor of n Spat_filters. The window size in the convolution (T parameter) is the key in terms of precision and performance. As a result of this stage, we obtain a set of intermediate spatially filtered frames that will be provided as an input to the steering stage.
Algorithm 2 spat_filt = stage II (temp_filt, N , L , n Temp_filters, n Spat_filters, T )
DSP optimizations and performance results
Besides the basic implementation derived from the algorithmic definition of the stage, our optimizations (loop optimization, vectorization, and parallelization) are focused on the bi-dimensional convolution kernel in order to adapt it to DSP architecture specifications. More specifically, we leverage the separability of the bi-dimensional convolution to perform and highly optimize one-dimensional (1D) vertical and horizontal convolutions, applying optimizations at instruction level (loop unrolling), data level (vectorization in the 1D convolution loop body), and thread level across cores (through OpenMP).
At this stage, frame size (n x×n y) and spatial window size (T) substantially impact performance rates. As for the previous stage, when fixing T, throughput decreases for increasing frame dimensions. However, for a fixed frame dimension, the impact of increasing the spatial window size is translated into higher throughput; from our analysis, our separate bi-dimensional convolution implementations attain better performance as window size increases, mainly due to the avoidance of memory latency effects. This improvement, though, is expected to stabilize for larger window sizes (that are usually not common in McGM).
Core-level optimizations are translated into performance improvements between ×1.6 and ×2.2, depending on the evaluated frame and window dimensions. The thread-level parallelization yields an improvement between ×5 and ×6.5 when comparing with the optimized sequential version. In general, the throughput obtained by applying the complete set of core-level optimizations and thread-level parallelization improves the original basic implementation in a factor between ×8 and ×13.
4.2.3 Stage 3: steering filtering
Algorithm and implementation
Algorithm 3 describes the necessary steps to perform the steering stage in the McGM method. Basically, the algorithm proceeds by applying a convolution between each spatial-filtered frame obtained from the previous stage (I in the algorithm), and an oriented filter F θ previously calculated. The response of each one of the temporal- and spatial-filtered frames to this oriented filter will be the output of this stage.
Algorithm 3 R = stage III (spac_filt, N , L , n Temp_filters, n Spat_filters, n Ortho_Orders, n θ s)
DSP optimizations and performance results
The optimizations applied to this stage are in the same way as those presented from the previous stages. Data parallelism is heavily exploited when possible, and loops are optimized after a deep search of the optimal unrolling parameters. OpenMP is used to extract thread-level parallelism and leverage the power of the eight cores in the C6678.
Special caution must be taken at this stage with memory consumption, as it reaches the maximum memory requirements of the McGM algorithm. More specifically, at this point, both the spatial-filtered frames and their steering filtering must coexist in the memory. However, this potential issue is conditioned by input algorithm parameters which are known beforehand.
At this stage, the factors affecting performance are frame size (n x×n y) and number of orientations (n θ s). For a fixed number of angles, throughput decreases for increasing frame dimensions. For a common resolution, increasing the number of angles considered also yields higher throughput. Core-level optimizations are more significant here, being the reason the higher arithmetic intensity in the loop bodies. These optimizations yield performance improvements between ×4 and ×5, depending on the evaluated frame dimensions and number of angles. The thread-level parallelization yields an improvement between ×1.5 and ×2.5 taking as a reference the optimized sequential version, with higher improvements as the number of orientations is increased. In general, the throughput obtained by applying the complete set of optimizations outperforms the basic implementation in a factor between ×10 and ×12.5.
4.2.4 Final stages
4.3 Global throughput results and real-time considerations
Throughput of the DSP implementation of McGM for different parameter configurations
In general, throughput is reduced for increasing frame resolutions. While this rate is high for the minimum tested resolution (up to 650 fps for 32×32 frames), it dramatically decreases for larger frames, achieving a minimum of 9.74 fps for the largest resolution tested (128×128). Differences between several parameter configurations are specially significant for small frame dimensions but not critical for the rest. Comparing the global performance results with those for each one of the stages presented in Figures 4, 5, and 6, the main insight is that the steering stage is the clear limiting factor. Global throughput is far from that attained in the temporal and spatial filtering stages (that were in the order of thousands of frames per second, depending on the resolution) and closer to that attained for the steering stage. This confirms the time breakdown detailed in Figure 7, which illustrates that 90% of the overall execution time is devoted to this stage.
Throughput and power efficiency of McGM implementations on different architectures, using Conf. 4 and different frame sizes
Power efficiency (kpps/watt)
TI DSP C6678 processor (eight cores) at 1 GHz with 512 Mbytes of RAM.
Two Intel Xeon X5570 (eight cores in two sockets) at 2.93 GHz with 24 Gbytes of RAM.
Intel Atom D510 (two cores) at 1.66 GHz with 2 Gbytes of RAM.
ARM Cortex A9 (two cores) at 1 GHz (built by TI) with 1 Gbyte of RAM.
The table also reports the TDP in order to give an overview of the peak power consumption for each one of the platforms. Note that the TI C6678 DSP can be considered as a low-power architecture, especially compared with the Intel Xeon (10 vs. 190 W when the two sockets of the latter are used). However, it is still far from the reduced power dissipated by the ARM Cortex A9.
Clearly, the multi-threaded implementation of McGM on the eight cores of the Intel Xeon yields the highest throughput rate from all the evaluated frame dimensions. For input images of 128×128 pixels, the throughput rate is roughly 21 fps. When only one core of the Intel Xeon is used, this rate is reduced to 4 fps. Our optimized implementation on the C6678 DSP outperforms the sequential results on the Intel Xeon, achieving a peak rate for the largest tested frame dimensions of 9.74 fps. Considering a rate around 20 fps, acceptable for being considered as real-time processing (performance rates meeting real-time processing are in italic in Table 3), the parallel implementation on the Intel Xeon can attain real-time on resolutions up to 128×128, meanwhile the TI DSP can attain real-time processing on up to 96×96 frame dimensions. Given the scalability observations extracted from our experimental results, we do not observe any relevant limitation for better performance results in future multi-core DSP architectures, possibly equipped with a larger number of cores.
4.4 Power efficiency considerations
This throughput rates must be considered in the context of the real power dissipated by each platform. To illustrate the power efficiency of each platform when executing McGM, Table 3 also provides a comparative analysis of the efficiency of each architecture in terms of thousands of pixels processed per second (kpps) per watt. The best power efficiency ratios are indicated with superscript letters in the table. Note that even though the ultra-low-power ARM is the most efficient architecture for the smallest input images (32×32), the TI DSP is clearly the most efficient platform for larger images. In this sense, the TI DSP offers a trade-off between performance and power that can be of wide appeal for those applications and scenarios in which power consumption is a restriction, but real time is still a requirement for medium/large image inputs.
General-purpose multi-core architectures deliver lower rates in terms of power efficiency but are a requirement if real-time processing is needed for the largest tested images. Compared with the other two low-power architectures (Intel Atom and ARM Cortex A9), real-time processing is only achieved for low-resolution images (32×32 in both cases). Thus, our DSP implementation, and the DSP architecture itself, can be considered as an appealing architecture not only when low power is desired but also when throughput is a limiting requirement.
In this paper, we have presented a detailed performance study of an optimized implementation of a robust motion estimation algorithm based of a gradient model (McGM) on a low-power multi-core DSP. Our study reports a general description of each stage of the multi-channel algorithm, with several optimizations that offer appealing throughput gains for a wide range of execution parameters.
We do not propose the TI DSP architecture as a replacement of high-end current architectures, like novel multi-core CPUs or many-core GPUs, but as an attractive solution for scenarios with tight power-consumption requirements. DSPs allow trade-off between performance, precision, and power consumption, with clear gains compared with other low-power architectures in terms of throughput (fps). In particular, while real-time processing is attained only for low-resolution image sequences on current low-power architectures (typically 32×32 frames), our implementations elevates this condition up to images with resolution 96×96 or higher, depending on the inputs execution parameters. These results outperform those on a single core of a general-purpose processor and are highly competitive with optimized parallel versions in exchange of a dramatic reduction in power requirements.
These encouraging results open the chance to consider these architectures in mobile devices where power consumption is a severe limiting factor, but throughput is a requirement. Our power consumption considerations are based on estimated peak dissipated power as provided by manufactures in the processor specifications. Nevertheless, to be more accurate in terms of power consumption, we will consider as future work a more detailed energy evaluation study, offering real measurements at both core and system level.
This work has been supported by Spanish Projects CICYT-TIN 2008/508 and TIN2012-32180.
- Huang CL, Chen YT: Motion estimation method using a 3D steerable filter. Image Vision Comput 1995, 13(1):21-32. 10.1016/0262-8856(95)91465-PView ArticleGoogle Scholar
- Lucas BD, Kanade T: An iterative image registration technique with an application to stereo vision. In Proc. of 7th Int. Joint Conf. on Artificial Intelligence (IJCAI ’81). San Francisco, CA, USA: Morgan Kaufmann Publishers Inc; April 1981:674-679.Google Scholar
- Oh H-S, Lee H-K: Block-matching algorithm based on an adaptive reduction of the search area for motion estimation. Real-Time Imaging October 2000, 6: 407-414.View ArticleGoogle Scholar
- Sun D, Lewis JP, Michaelj: Black. Learning optical flow. In Proc. ECCV. Rhode Island 02912, USA: Brown University, Providence; 2008:83-97.Google Scholar
- Baker S, Gross R, Matthews I: Lucas-kanade 20 years on: a unifying framework: part 3. Int J Comput Vis 2002, 56: 221-255.View ArticleGoogle Scholar
- Liang X, McOwan PW, Johnston A: Biologically inspired framework for spatial and spectral velocity estimations. J. Opt. Soc. Am. A April 2011, 28(4):713-723.View ArticleGoogle Scholar
- Benton CP, McOwan PW, Johnston A: Robust velocity computation from a biologically motivated model of motion perception. Proc R Soc B 1999, 266: 509-518. 10.1098/rspb.1999.0666View ArticleGoogle Scholar
- Johnston A, Clifford CW: A unified account of three apparent motion illusions. Vision Res April 1995, 35(8):1109-1123.View ArticleGoogle Scholar
- Ayuso F, Botella G, Garcia C, Prieto M, Tirado F: GPU-based acceleration of bio-inspired motion estimation model. Concurrency and Computation: Practice and Experience, In pressGoogle Scholar
- Juan GB, García Ríos A, Rodriguez-Alvarez M, Vidal ER, Meyer-Bäse U, Molina MC: Robust bioinspired architecture for optical-flow computation. IEEE Trans. VLSI Syst 2010, 18(4):616-629.View ArticleGoogle Scholar
- Dhoot C, Mooney VJ, Chowdhury SR, Chau LP: Fault tolerant design for low power hierarchical search motion estimation algorithms. In VLSI-SoC. Los Alamitos, CA (USA): IEEE Computer Society; 2011:266-271.Google Scholar
- De Vleeschouwer C, Nilsson T: Motion estimation for low power video devices. In ICIP (2). Los Alamitos, CA (USA): IEEE Computer Society; 2001:953-956.Google Scholar
- Anguita M, Díaz J, Ros E, Fernandez-Baldomero FJ: Optimization strategies for high-performance computing of optical-flow in general-purpose processors. IEEE Trans. Circuits Syst. Video Techn 2009, 19(10):1475-1488.View ArticleGoogle Scholar
- Honegger D, Greisen P, Meier L, Tanskanen P, Pollefeys M: IROS. IEEE; 2012:5177-5182.Google Scholar
- Subramaniam B, Wu-chun Feng: The Green Index: A Metric for Evaluating System-Wide Energy Efficiency in HPC Systems. In 8th IEEE Workshop on High-Performance, Power-Aware Computing (HPPAC). Los Alamitos, CA (USA): IEEE Computer Society; May 2012.Google Scholar
- Shirai Y, Miura J, Mae Y, Shiohara M, Egawa H, Sasaki S: Moving object perception and tracking by use of dsp. In Computer Architectures for Machine Perception, 1993. Proceedings. Los Alamitos, CA (USA): IEEE Computer Society; Dec 1993:251-256.View ArticleGoogle Scholar
- Horn BKP, Schunck BG: Determining optical flow. Artif. Intell 1981, 17: 185-203. 10.1016/0004-3702(81)90024-2View ArticleGoogle Scholar
- Rwekamp T, Platzner M, Peters L: Specialized architectures for optical flow computation: A performance comparison of asic, dsp, and multi-dsp. In Proceedings of the 8th ICSPAT 1997, 829-833.Google Scholar
- Steimer A: Global optical flow estimation by linear interpolation algorithm on a DSP microcontroller. In Master Thesis. ETH Zurich. Switzerland; October, 2011.Google Scholar
- Srinivasan MV: An image-interpolation technique for the computation of optic flow and egomotion. Biol. Cybernetics 1994, 71: 401-415. 10.1007/BF00198917View ArticleGoogle Scholar
- Snowden RJ, Hess RF: Temporal frequency filters in the human peripheral visual field. Vision Res 1992, 32(1):61-72. 10.1016/0042-6989(92)90113-WView ArticleGoogle Scholar
- Koenderink JJ: Optic flow. Vision Res 1996, 26: 161-180.View ArticleGoogle Scholar
- TMS320C6678: Multicore Fixed and Floating-Point Digital Signal Processor. , February 2012. Texas Instruments Literature Number: SPRS691C http://www.ti.com/lit/ds/sprs691c/sprs691c.pdf Google Scholar
- Ayuso F, Botella G, Garcia C, Prieto M, Tirado F: GPU-based acceleration of bioinspired motion estimation model. In WPABA 2011. DC, USA: IEEE Computer Society Washington; 2011.Google Scholar
- Ayuso F, Botella G, Garcia C, Prieto M, Tirado F: GPU-based signal processing scheme for bioinspired optical flow. In 21st Int. Conf. on Field Programmable Logic and Applications, Workshop on Computer Vision on Low-Power Reconfigurable Architectures, 2011. Los Alamitos, CA (USA): IEEE Computer Society; 2011-2011. 09/2011 (2011)Google Scholar
- Introduction to TMS320C6000 DSP optimization , October 2011. Texas Instruments Literature Number: SPRABF2 http://www.ti.com/lit/an/sprabf2/sprabf2.pdf
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.