Modeling of Video Sequences by Gaussian Mixture: Application in Motion Estimation by Block Matching Method
 Abdenaceur Boudlal^{1},
 Benayad Nsiri^{2}Email author and
 Driss Aboutajdine^{1}
https://doi.org/10.1155/2010/210937
© Abdenaceur Boudlal et al. 2010
Received: 7 October 2009
Accepted: 26 April 2010
Published: 30 May 2010
Abstract
This article investigates a new method of motion estimation based on block matching criterion through the modeling of image blocks by a mixture of two and three Gaussian distributions. Mixture parameters (weights, means vectors, and covariance matrices) are estimated by the Expectation Maximization algorithm (EM) which maximizes the loglikelihood criterion. The similarity between a block in the current image and the more resembling one in a search window on the reference image is measured by the minimization of Extended Mahalanobis distance between the clusters of mixture. Performed experiments on sequences of real images have given good results, and PSNR reached 3 dB.
Keywords
1. Introduction
Motion estimation is the process which generates the motion vectors that determines how each motion compensated prediction frame is created from the previous frame. It examines the movement of objects in an image sequence to try to obtain vectors representing the estimated motion. Motion compensation uses the knowledge of object motion obtained to achieve data compression.
Motion estimation plays a key role in many video applications, such as framerate video conversion, video retrieval, video surveillance, and video compression.
The key issue in these applications is to define appropriate representations that can efficiently support motion estimation with the required accuracy.
In interframe coding, motion estimation and compensation have become powerful techniques to eliminate the temporal redundancy due to high correlation between consecutive frames [1].
 (1)
Objects move in translation in a parallel plane to the camera plane, that is, the effects of camera zoom and object rotations are not considered.
 (2)
Illumination is spatially and temporally uniform.
 (3)
Occlusion of one object by another and uncovered background are neglected.
Several motion estimation approaches have been proposed so far in the open literature such as pelrecursive algorithms, frequency domain techniques, optical flow, and block matching methods.
Pelrecursive Algorithms rely on iterative refining of motion estimation for individual pels by gradient methods that enable to predict recursively the displacement of each pel from its neighbouring pels. These algorithms involve more computational complexity and less regularity and are therefore difficult to realize in hardware [5]. Frequency motion estimation techniques are mainly used for the global motion estimation. The most known frequency technique is the phase correlation method that capitalizes on the wellknown Fourier shift theorem which states that shifts in the spatial domain correspond to linear phase changes in the Fourier domain [6–8]. Optical flow estimation ensures high accuracy for scenes with small displacements but fails when the displacements are large. In general, these methods suffer from the aperture problem because each neighbourhood of pixels can have a different motion in the image [9–11]. Block Matching Algorithms estimate motion on the basis of rectangular blocks and produce one motion vector for each block. These algorithms are more suitable for a simple hardware realization because of their regularity and simplicity [12].
This paper is organized as follows. In Section 2, the modeling and parameter estimation of Gaussian mixtures is briefly described. The distance measure between Gaussian mixtures models is studied in Section 3. We describe our approach in Section 4. Section 5 presents simulation results under and without influence of noise. Some concluding remarks are given in Section 6.
2. Modeling and Parameter Estimation of Gaussian Mixtures
where n is the dimensionality of the vector x, is the mean vector, and is the covariance matrix assumed to be positive definite. For clarity, we let be the collection of all the parameters in the mixture, that is,
where are the posterior probabilities.
As EM is highly dependent on initialization, the first set of parameters selection is very important for EM algorithm. If the initial parameters are not well selected, the algorithm may converge into local maxima points. The convergence properties of EM algorithm over Gaussian Mixture Model have been extensively studied in [19, 20].
3. Distance Measures between Gaussian Mixtures Models
However, this measure creates a singularity for singular covariance matrices. In practical problems it often appears in learning such models mixture. The acquired covariance matrix are not always conditioned and their inversion creates a problem. In our implementation, we replace the inverse of singular covariance matrix by its pseudoinverse. Singular value decomposition is used for the calculation of the pseudoinverse. Roundoff errors can lead to a singular value not being exactly zero even if it should be. Tolerance parameter places a threshold when comparing singular values with zero and improves the numerical stability of the method with singular or nearsingular matrices.
4. Approach and Conception of the Proposed Method
4.1. The Cost Function
When modeling by a mixture of two Gaussian distributions, the cost function is defined by the Extended Mahalanobis distance between the components of strong weights and the components of weak weights .
4.2. Steps of the Proposed Method
 (1)
Each block in the reference image or the current image is modeled by a mixture of three Gaussian distributions. This modeling consists in estimating the parameters of the mixture (weight, means vectors, and covariance matrix).
 (2)
The Parameters are sorted based on their weights in mixture. This allows the identification of the components of weak weights, the components of medium weights, and the components of strong weights.
 (3)Research of minimal interblocks distance (reference/current).
 (a)The Extended Mahalanobis distances between a block of the current image and all blocks in a search window in the reference image are stored in the matrices , , and .

(M_{1}) matrix contains the values of Extended Mahalanobis distances between the components of weak weights.

(M_{2}) matrix contains the values of Extended Mahalanobis distances between the components of medium weights.

(M_{3}) matrix contains the values of Extended Mahalanobis distances between the components of strong weights.

 (b)
The value of the minimal distance of the three matrices , , and corresponds to the most similar block in reference image.
 (a)
4.3. Practical Considerations
Matrices , , and show the Extended Mahalanobis distances between a block of the current image and all blocks in a search window in the reference image of the Foreman sequence.
Matrix contains the distances between the components of weak weights.
Matrix contains the distances between the components of medium weights.
Matrix contains the distances between the components of strong weights.
The value of the minimum distance of the three matrices , , and is equal to 0.93 corresponding to the first line second column indices in the matrix . These indices correspond to the most similar blocks in the reference image.
For these types of Foreman sequence, about 80% minimum distances are in the matrix (distances between the components of weak weights). This percentage mainly depends on statistical characteristics of pixels in the image.
5. Experimental Results
 (i)
Method: exhaustive blockmatching (full search) is the most obvious candidate for a search technique for finding the best possible weight in the search area.
 (ii)Classical criterion methods.
 (a)
Sum of Absolute Differences "SAD".
 (b)
Sum of Square Error "SSE".
 (c)
Normalized CrossCorrelation "NCC".
 (a)
 (iii)
Method proposed criterion: minimization of Extended Mahalanobis distance between mixture of two and three Gaussian distributions ("GMM2" and "GMM3").
 (iv)
Precision: pixel.
 (v)
 (vi)
5.1. Simulation Results without Noise Influence
5.1.1. Objective Evaluation
where is the measured PSNR for frame , and is the total number of frames. We will compare the "GMM3" and "GMM2" methods against the "SAD", "SSE", and "NCC" methods. In addition, the PSNR comparison among the five algorithms will be introduced.
Average PSNR Values for test images.
Sequences  Algorithm  PSNR [dB] 

"Hand"  "CORR"  22.49 
"SAD"  21.73  
"SEE"  22.31  
"GMM2"  24.05  
"GMM3"  24.64  
"Foreman"  "CORR"  27.04 
"SAD"  28.60  
"SEE"  28.53  
"GMM2"  29.00  
"GMM3"  29.58  
"coastguard"  "CORR"  23.37 
"SAD"  23.21  
"SEE"  23.93  
"GMM2"  24.31  
"GMM3"  24.86  
"Soccer"  "CORR"  24.08 
"SAD"  23.87  
"SEE"  24.21  
"GMM2"  24.80  
"GMM3"  25.18  
"Football"  "CORR"  24.26 
"SAD"  24.24  
"SEE"  24.88  
"GMM2"  25.22  
"GMM3"  25.27  
"Cones"  "CORR"  21.63 
"SAD"  21.64  
"SEE"  22.20  
"GMM2"  22.64  
"GMM3"  23.00 
5.1.2. Subjective Evaluation
5.2. Simulation Results under Influence of Noise
6. Conclusion and Perspective
In this paper we have modeled sequence images blocks by a mixture of two and three Gaussian distributions and have used block matching criterion based on Mahalanobis distance minimization between the clusters of mixture to estimate motion. This technique has been compared to other equivalent methods in the literature. The simulation confirms that the proposed technique allows the significant PSNR gains. These gains can be observed in terms of both the perceptual quality and the PSNR of the restored images. However, this technique requires more computations. It might be necessary to further reduce computations to fit real time requirements. Parameters estimation of Gaussian mixture consists of repetitive operations which could greatly benefit from some existing architectures to perform repetitive tasks efficiently. A forthcoming work will be devoted to improve the speed of execution and further increase performance by modeling the blocks of the image by a Gaussian mixture where the number of clusters varies.
Declarations
Acknowledgments
The authors would like to thank Mohamed Zyoute, Fatima Boudlal, Fadoua AtaaAllah Nizar Bennani, and Hatim Chergui for their help.
Authors’ Affiliations
References
 Lucas B, Kanade T: An iterative image registration technique with an application to stereo vision. Proceedings of the DARPA Image Understanding Workshop, 1981 121130.Google Scholar
 Dufaux F, Moscheni F: Motion estimation techniques for digital TV: a review and a new contribution. Proceedings of IEEE 1995, 83(6):858876. 10.1109/5.387089View ArticleGoogle Scholar
 Tziritas G, Labit C: Motion analysis for image sequence coding. In Advances in Image Communication. Volume 4. Elsevier, Amsterdam, The Netherlands; 1994.Google Scholar
 Stiller C, Konrad J: Estimating motion in image sequences, a tutorial on modeling and computation of 2D motion. IEEE Signal Processing Magazine 1999, 16(4):7091. 10.1109/79.774934View ArticleGoogle Scholar
 Estrela V, Rivera LA, Beggio PC, Lopes RT: Regularized pelrecursive motion estimation using generalized crossvalidation and spatial adaptation. Proceedings of the 16th Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI '03), 2003 331338.View ArticleGoogle Scholar
 Argyriou V, Vlachos T: Performance study of gradient correlation for subpixel motion estimation in the frequency domain. IEE Proceedings: Vision, Image and Signal Processing 2005, 152(1):107114. 10.1049/ipvis:20051073Google Scholar
 Ertürk S: Digital image stabilization with subimage phase correlation based global motion estimation. IEEE Transactions on Consumer Electronics 2003, 49(4):13201325. 10.1109/TCE.2003.1261235View ArticleGoogle Scholar
 Essannouni F, Oulad Haj Thami R, Salam A, Aboutajdine D: A new fast full search block matching algorithm using frequency domain. Proceedings of the 8th International Symposium on Signal Processing and Its Applications (ISSPA '05), 2005 2: 559562.Google Scholar
 Heeger DJ: Optical flow using spatiotemporal filters. Proceedings of the 1st International Conference on Computer Vision, 1988 181190.Google Scholar
 Alvarez L, Weickert J, Sánchez J: Reliable estimation of dense optical flow fields with large displacements. International Journal of Computer Vision 2000, 39(1):4156. 10.1023/A:1008170101536View ArticleMATHGoogle Scholar
 Jacobson L, Wechsler H: Derivation of optical flow using a spatiotemporalfrequency approach. Computer Vision, Graphics, & Image Processing 1987, 38(1):2965. 10.1016/S0734189X(87)801524View ArticleGoogle Scholar
 Zhu C, Qi WS, Ser W: A new successive elimination algorithm for fast block matching in motion estimation. Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '04), 2004 3: 733736.Google Scholar
 Xu JB, Po LM, Cheung CK: Adaptive motion tracking block matching algorithms for video coding. IEEE Transactions on Circuits and Systems for Video Technology 1999, 9(7):10251029. 10.1109/76.795056View ArticleGoogle Scholar
 Dempster AP, Laird NM, Rubin DB: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical, Series B 1977, 39: 138.MathSciNetMATHGoogle Scholar
 Neal R, Hinton G: A view of the em algorithm that justifies incremental, sparse, and other variants. In Learning in Graphical Models. Edited by: Jordan MI. Kluwer Academic Publishers, Dordrecht, The Netherlands; 1998.Google Scholar
 Afify M: Extended baumwelch reestimation of Gaussian mixture models based on reverse jensen inequality. Proceedings of the 9th European Conference on Speech Communication and Technology, 2005 11131116.Google Scholar
 McLachlan GJ, Peel D: Finite Mixture Models. John Wiley & Sons, New York, NY, USA; 2000.View ArticleMATHGoogle Scholar
 SanjayGopal S, Hebert TJ: Bayesian pixel classification using spatially variant finite mixtures and the generalized EM algorithm. IEEE Transactions on Image Processing 1998, 7(7):10141028. 10.1109/83.701161View ArticleGoogle Scholar
 Redner RA, Walker HF: Mixture densities, maximum likelihood and the EM algorithm. SIAM Review 1984, 26(2):195239. 10.1137/1026034MathSciNetView ArticleMATHGoogle Scholar
 Xu L, Jordan MI: On convergence properties of the EM algorithm for Gaussian mixtures. Neural Computation 1996, 8(1):129151. 10.1162/neco.1996.8.1.129View ArticleGoogle Scholar
 Younis K, Karim M, Hardie R, Loomis J, Rogers S, DeSimio M: Cluster merging based on weighted Mahalanobis distance with application in digital mammography. Proceedings of the IEEE National Aerospace and Electronics Conference (NAECON '98), July 1998 525530.Google Scholar
 Packwood RA, Steliaros MK, Martin GR: Variable size block matching motion compensation for objectbased video coding. Proceedings of the 6th IEE International Conference on Image Processing & Its Applications, July 1997, Dublin, Ireland 5660.View ArticleGoogle Scholar
 Barjatya A: Block matching algorithms for motion estimation. IEEE Transactions Evolution Computation 2004, 8(3):225239. 10.1109/TEVC.2004.826069View ArticleGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.