Hirschman Optimal Transform Block LMS Adaptive Filter

In this paper, we derive a "convolution theorem" suitable for the Hirschman optimal transform (HOT), a unitary transform derived from a discrete-time, discrete-frequency version of the entropy-based uncertainty measure first described by Hirschman (1957). We use the result to develop a fast block-LMS adaptive filter which we call the HOT block-LMS adaptive filter. This filter requires slightly less than half of the computations that are required for the FFT block-LMS adaptive filter. The simulations show that the convergence rates of both the HOT and FFT block-LMS adaptive filters are similar.


INTRODUCTION
The HOT is a recently developed discrete unitary transform that uses the orthonormal minimizers of the entropy-based Hirschman uncertainty measure [1].This measure is different from the energy-based Heisenberg uncertainty measure that is only suited for continuous time signals.The Hirschman uncertainty measure uses entropy to quantify the spread of discrete-time signals in time and frequency [2].Since the HOT bases are among the minimizers of the uncertainty measure, they have the novel property of being the most compact in discrete time and frequency.The fact that the HOT basis sequences have many zero-valued samples, and their resemblance to the DFT basis sequences, makes the HOT computationally attractive.Furthermore, it has been shown recently that a thresholding algorithm using the HOT yields superior frequency resolution of a pure tone in additive white noise to a similar algorithm based on the DFT [3].
The main theorem in [1] describes a method to generate an 2 N K = -point ortho-normal HOT basis, where K is an integer.
A HOT basis sequence of length 2 K is the most compact bases in the time-frequency plane.The 3 2 -point HOT matrix is Eq. (1) indicates that the HOT of any sequence is related to the DFT of some polyphase components of the signal.In fact, we called this property the "1 and ½ dimensionality" of the HOT in [2].Consequently, for this paper, we will use the terms HOT and DFT of the polyphase components interchangeably.The 2 K -point HOT requires fewer computations than the 2 K -point DFT.We used this computational efficiency of the HOT to implement fast convolution algorithms in [4].When K is a power-of-2 integer, then 2 2 log K K (complex) multiplications are needed to compute the HOT, which is half that is required when computing the DFT.
In this paper, we use the computational efficiency of the HOT to implement a fast block-LMS adaptive filter.The fast Block-LMS adaptive filter was first proposed [5] to reduce computational complexity.Our proposed HOT Block-LMS adaptive filter requires less than half of the computations required in the corresponding FFT Block-LMS adaptive filter.This significant complexity reduction could be important in many real time applications.Of course, the complexity reduction comes at some performance cost.
In Section 2, we develop the notion of convolution with the HOT.In Section 3, we develop the HOT Block-LMS algorithm.Section 4 contains our convergence analysis.Simulations are provided in Section 5.There we also examine the performance and costs in some detail.Finally, we conclude.

HOT AND CIRCULAR CONVOLUTION
Suppose that u and h are two signals of length where ( ) i U k is the DFT of ( ) i u l .In matrix form, Eq. ( 2) can be written ( )  , ( ) diag , , … and F K is the Kpoint DFT matrix.Eq. (3) relates the DFT of the signal u to its HOT.Therefore, circular convolution in the HOT domain is given by 1 The ⋅ means element-wise multiplication.Eq (4) is not in the form that we require.To find the j th polyphase component of the output K j F y , we divide it into the following K equations: 1 0 0 where Eq. ( 5) simplifies to Because the DFT matrix is unitary we have Let H i be the diagonal matrix corresponding to K i F h .Then the element-wise multiplication in Eq. ( 8) can be replaced with conventional matrix multiplication.Finally we combine the K equations in (8) where ( ) From Eq. ( 9) one can see that calculating the HOT of the output y requires 3 6 log K K K + multiplications are required.Asymptotically in K , we see that the HOT could be three times more efficient than the DFT.

THE HOT BLOCK-LMS ADAPTIVE FILTER
In a block adaptive filter, the adaptation proceeds block-byblock with the weight update equation ( ) where d and y are the desired and output signals, respectively, u is the vector that contains the input samples in the k th block, L is the block size or the filter length, and the error is . The FFT is commonly used to efficiently calculate the output of the filter and the sum in the update equation.For our proposed HOT Block-LMS algorithm, we use the update equation: We assume that the block size or the filter length is 2 2 K (our reason for this assumption will become clear) and the block size is equal to the filter order.The parameter j determines which polyphase component of the error signal is being used in the adaptation.This parameter can be changed from block to block.Therefore, in the adaptation we are only using one polyphase component, rather than the whole signal.The output and the sum in this update equation are efficiently calculated: a) Append the weight vectors with 2 2 K zeros (the resulting vector is now 2 K points long as required in the HOT definition) and find its HOT.b) Use the inverse HOT and Eq. ( 9) to calculate the j th polyphase component of the circular convolution.The j th polyphase component of the output can be found by discarding the first half of the j th polyphase component of the circular convolution.c) Use the inverse HOT and Eq. ( 9) to calculate the j th polyphase component of the circular convolution.The j th polyphase component of the output can be found by discarding the first half of the j th polyphase component of the circular convolution.d) Calculate the j th polyphase component of the error, insert a block of 2 K zeros, up-sample by K, then calculate its HOT.
Fig. 1 HOT Block-LMS Algorithm.e) Circularly flip the vector in b) and then compute its HOT.f) Compute the sum in the update equation using Eq. ( 9)this sum is that of the first half elements of the circular convolution between the vectors in part e) and d).
The HOT Block LMS adaptive filter is shown in Figure 1.Now we look at the computational cost of the algorithm and compare it to that of the FFT block adaptive algorithm.Parts a), b), and e) require multiplications -asymptotically more than twice as many.Therefore, by using only one polyphase component for the adaptation in a block, the computational cost can be reduced by a factor of 2.5.While this complexity reduction comes at the cost of not using all available information, our proposed algorithm provides better estimates than the LMS filter.The reduction of the computational complexity in our algorithm comes from using the polyphase components of the input signal to calculate one polyphase component of the output using HOT.It is worth mentioning that the fast exact LMS adaptive algorithm (FELMS) [6] also reduces the computational complexities by finding the output via processing the polyphase components of the input.However, the computational complexity reduction of the FELMS is less than that found in the FFT and HOT block adaptive algorithms because the FELMS is designed to have exact mathematical equivalence to, and hence the same convergence properties as, the conventional LMS.

COVERGENCE ANALYSIS
In this section we look at the convergence analysis of the HOT Block-LMS in time domain.We will assume small step size.The HOT-LMS minimizes the cost, which is the average of squared errors in the j th error polyphase component.From the statistical LMS theory [7], the HOT Block-LMS can be analyzed using the stochastic difference equation , Q is the eigenvector matrix of the input autocorrelation matrix, o w is the optimal Wiener solution, and Λ is the diagonal matrix that contains the eignvalues of the input autocorrelation matrix.The driving force ( ) k φ for the HOT Block-LMS is given by where o e is the error produced by the Wiener filter.It is easily where o J is the minimum MSE produced by the Wiener filter.The mean square of the th l component of ( ) where l λ is the th l eigenvalue of the input autocorrelation matrix.Therefore, the average time constant of the HOT Block-LMS is III 1307 The misadjustment can be calculated using the equation The average time constant of the HOT Block-LMS is the same as that of the FFT Block-LMS.However the HOT Block-LMS has K times higher misadjustment than the FFT Block-LMS.Similar to the FFT Block-LMS [8], the convergence of the HOT Block-LMS can be improved by equalizing the modes of the filter by using different step size for each mode.The suitable step size for each mode can be found by estimating the power in each HOT sample using 2 ( ) ( 1) (1 ) ( ) , where ( ) i U k is the i th HOT sample of the input of the filter and γ is a constant close to, but less than 1.The step size for the i th mode is given by , ( ) where α is a constant.As we will show in the next section, by assigning to each weight an individual step size, the convergence rate improvement of the HOT Block-LMS is similar to the improvement that can be achieved by the FFT Block-LMS.

SIMULATIONS
In this section we simulate the HOT Block-LMS adaptive filter to investigate its performance and compare its convergence rate with the LMS and FFT Block-LMS.The input of the adaptive filter was generated by coloring unit variance white noise using the FIR filter H(z) = 0.1 + 0.2 z --1 + 0.3z -2 + 0.4z --3 + 0.4z -4 + 0.2z -5 + 0.1z-6.The desired input was generated using the linear model d(n)= T o w u(n)+e o (n), where e o (n) is the measurement white noise with variance of 10 -4 .The learning curves of the LMS, the FFT Block-LMS, and the HOT Block-LMS are shown in Figure 2. The step size was adjusted such that all of the adaptive filters converge at the same rate when they are fed with white noise (the corresponding Figure is not included for the lack of space).
The learning curves show that the convergence rate of the computationally more efficient HOT Block-LMS is close to that of the FFT Block-LMS.This is due to the ability of the HOT basis to approximately diagonalize the autocorrelation matrix of the input.

CONCLUSIONS
This paper has proposed the computationally efficient HOT Block-LMS algorithm.In addition to its efficiency, the simulations show that its convergence rate is close to that of the FFT Block-LMS.Our next goal is to quantitatively determine the convergence rate as has been done with the FFT Block LMS [9].Fig. 2 The learning curves of the LMS, the FFT Block-LMS, and the HOT Block-LMS.
y is their circular convolution.Then, in the frequency domain we can write ( ) ( ) ( )Y k U k H k = .To replace the DFT with the HOT we need the relationship between the DCT and HOT of the signals u, h, and y.Define ( where