Quantization Effects and Stabilization of the Fast-Kalman Algorithm

Abstract

The exact and actual cause of the failure of the fast-Kalman algorithm due to the generation and propagation of finite-precision or quantization error is presented. It is demonstrated that out of all the formulas that constitute this fast Recursive Least Squares (RLS) scheme only three generate an amount of finite-precision error that consistently propagates in the subsequent iterations and eventually makes the algorithm fail after a certain number of recursions. Moreover, it is shown that there is a very limited number of specific formulas that transmit the generated finite-precision error, while there is another class of formulas that lift or "relax" this error. In addition, a number of general propositions is presented that allow for the calculation of the exact number of erroneous digits with which the various quantities of the fast-Kalman scheme are computed, including the filter coefficients. On the basis of the previous analysis a method of stabilization of the fast-Kalman algorithm is developed and is presented here, a method that allows for the fast-Kalman algorithm to follow very difficult signals such as music, speech, environmental noise, and other nonstationary ones. Finally, a general methodology is pointed out, that allows for the development of new algorithms which, intrinsically, suffer far less of finite-precision problems.