On the Convergence of Iterative Receiver Algorithms Utilizing Hard Decisions

The convergence of receivers performing iterative hard decision interference cancellation (IHDIC) is analyzed in a general framework for ASK, PSK, and QAM constellations. We ﬁrst give an overview of IHDIC algorithms known from the literature applied to linear modulation and DS-CDMA-based transmission systems and show the relation to Hopﬁeld neural network theory. It is proven analytically that IHDIC with serial update scheme always converges to a stable state in the estimated values in course of iterations and that IHDIC with parallel update scheme converges to cycles of length 2. Additionally, we visualize the convergence behavior with the aid of convergence charts. Doing so, we give insight into possible errors occurring in IHDIC which turn out to be caused by locked error situations . The derived results can directly be applied to those iterative soft decision interference cancellation (ISDIC) receivers whose soft decision functions approach hard decision functions in course of the iterations.


Introduction
In this paper receivers performing iterative hard decision interference cancellation (IHDIC) are analyzed.In such receivers, hard estimates are generated for each symbol to be estimated and used for cancellation of interference.This is performed during several iterations.Algorithms based on IHDIC have been proposed for linear modulation as well as direct-sequence code-division multiple-access (DS-CDMA) systems.It has to be emphasized that IHDIC was basically proposed by Hopfield [1,2] and Hopfield and Tank [3] , but his primary intention was to find a neural network which provides a content-addressable memory.The structure found by Hopfield is also referred to as Hopfield network and belongs to the group of neural networks; that is, IHDIC may also be seen as a Hopfield network.
A comprehensive overview of neural networks is given in [4][5][6].Mainly, real-valued systems with symbols being elements of a binary real-valued alphabet X, for example, X = {0, 1} or X = {−1, +1}, have been considered.Analysis of convergence and stability of Hopfield networks has been carried out in [7][8][9][10], where it is shown that a Hopfield network always can find a local minimum of an optimization problem of the form a = arg min a∈X V r − T a 2 (1) in several iterations where a ∈ X V denotes a hypothesis column vector of size V, T is a V × V matrix, and r is a column vector of V observations.| • | denotes the norm of a vector.Obviously, when considering communications problems it is desirable to find the global minimum of (1) which can be done with the optimum receiver resulting in a very high complexity in most cases.However, it may also be advantageous to find a local minimum that is close to the global one with low complexity as the vicinity to the global minimum ensures that the found solution differs only slightly from the optimum one.Further analysis on the number of stable states, the convergence time, the domain of attraction of the stable states, and so forth, can be found in [11][12][13][14][15].

EURASIP Journal on Advances in Signal Processing
For linear modulation-based transmission, IHDIC has been studied, for example, in [16,17] for channel equalization where the parallel update scheme was used in conjunction with MMSE filtering.IHDIC for multiuser detection applied to DS-CDMA-based transmission utilizing up to 2 iterations has been proposed, for example, in [18][19][20][21].In [19] the serial update scheme was compared by means of simulations with the parallel one and the superiority of the former could be shown.In these publications mostly binary phase-shift keying (BPSK) symbols have been assumed.In [22] IHDIC is used for pilot channel cancellation.IHDIC using the serial and the parallel update scheme with several iterations is considered for DS-CDMA multiuser detection, for example, in [23][24][25][26].Basic work on parallel interference cancellation in DS-CDMA systems with hard decision functions can also be found in [27,28] whereas in [29] an information theoretic analysis is conducted.
However, already Hopfield proposed to use a so-called sigmoid function instead of hard limiters for continuoustime neural networks in [1][2][3].Such a sigmoid function like the "tanh(•)" law proposed in [3] may also be seen as a soft decision function.The advantage of using soft estimates in place of hard decisions in receivers with feedback was first discovered in a work by Taylor [30] on transmission with linear modulation, where a suboptimum soft estimate of the transmitted symbols was used in the feedback section of a decision-feedback equalization unit.
However, since the early 1990s it was known in neural network theory that the performance of a Hopfield network can be improved if the slope of the "tanh(•)" function is increased in course of iterations [5].Later, this technique has been adopted for algorithms performing iterative soft decision interference cancellation (ISDIC) for DS-CDMA and has been refined by adapting the slope of the "tanh(•)" function using an estimate of the average power of residual interference after cancellation [31].
In [32][33][34] IHDIC applied to multiuser detection is compared with ISDIC and the superiority of the latter is shown.For more information on ISDIC for linear modulation and DS-CDMA-based transmission the reader is referred to [31,[35][36][37][38].
Though ISDIC is proven to be a superior receiver scheme compared to IHDIC, an analysis of the convergence of IHDIC is valuable.As mentioned above, best performance of ISDIC can be achieved if the slope of the soft decision function is adjusted in course of iterations.Thus, if the employed soft decision functions approach hard decision functions in course of iterations, then ISDIC behaves like IHDIC.Consequently, the convergence behavior of corresponding ISDIC schemes equals that of IHDIC in the last iterations and the results of this paper can be applied.
The paper is organized as follows.First, we introduce the system model in Section 2. In Section 3, we review IHDIC and Hopfield networks.In several publications it has been observed by simulations that the parallel update scheme may lead to cycles in the estimated values in course of iterations.Based on findings in neural network theory, we prove in Section 4 that IHDIC always converges to a fixed solution for the serial update scheme and amplitude-shift keying (ASK), phase-shift keying (PSK), or quadrature amplitude modulation (QAM) constellations.In Section 5, we prove that IHDIC converges to cycles of length 2 for the parallel update scheme.Note that this also includes the case where the estimates remain constant in course of iterations.In Section 6, we visualize the convergence of IHDIC with the aid of simulations and concluding remarks are given in Section 7.

System Model
For IHDIC algorithms we use a matrix-vector notation in the discrete-time domain.All signals are represented by their complex-valued baseband equivalents.An uncoded packet transmission with complex-valued symbols a ν ∈ X which are stacked in a vector a = [a 1 , . . ., a ν , . . ., a V ] T is considered ((•) T : transposition).X denotes the signal alphabet which may contain the elements from an ASK, PSK, or QAM signal constellation.We define a transmission matrix T of size L×V, whose entries are determined by the transmission pulses and/or the respective channel impulse response (CIR).Then, the received signal vector r = [r 1 , . . ., r l , . . ., r L ] T of size L can be expressed as where n = [n 1 , . . ., n l , . . ., n L ] T contains discrete-time complex white Gaussian noise values n l with variance σ 2 n .Obviously, the system model according to (2) includes a transmission with linear modulation or DS-CDMA as special cases.

Mode of Operation of IHDIC and Hopfield Networks
Both IHDIC and Hopfield networks operate in an iterative fashion.In each iteration μ (μ > 0) of the Hopfield network a new hard estimate a ν,μ is calculated for each symbol index ν ∈ {1, . . ., V} according to Here, H (•) denotes the hard decision function for the used modulation scheme.The vector a ν,μ contains hard estimates of interfering symbols and is built according to The estimates in vector a ν,μ are utilized in (3) to cancel interference effective on the symbol a ν taking the effective transmit pulses in matrix T into account.Subsequent matched filtering with 1/ρ ν,ν (T (:,ν) ) H and application of the hard decision function H (•) yields a new hard estimate a ν,μ in iteration μ ((•) H : Hermitian transposition).T (:,ν) denotes the νth column of matrix T and contains the effective transmission pulse of symbol a ν which has energy ρ ν,ν = (T (:,ν) ) H • T (:,ν) .For H = 0 the Hopfield network performs serial updating of the estimates and for H = −1 a parallel update is done.In the former case, always the latest estimates are utilized whereas in the latter case exclusively the estimates of the last iteration μ − 1 are exploited for calculation of a new estimate.The values a ν,μ are initialized according to a ν,0 = 0 for all ν.
The algorithm stops, if the estimates a ν,μ do not change from iteration μ to the next one μ + 1 for all symbol indices ν ∈ {1, . . ., V}, that is, or the iteration number exceeds a prescribed limit μ max .The last iteration performed by the IHDIC scheme is denoted with μ stop .Then IHDIC delivers the final estimates a ν = a ν,μstop for all ν ∈ {1, . . ., V}.

Convergence of IHDIC and Hopfield Networks for Serial Update
In Hopfield network theory only real-valued systems with symbols being elements of a binary real-valued alphabet X have been considered.Convergence and stability analysis for Hopfield networks in serial and parallel update mode have been carried out in [1-3, 5, 6, 8-10].In this and the next section, we give a generalization for complex-valued communications systems and ASK, PSK, and QAM symbol alphabets.
Theorem 1.For serial update, a Hopfield network or IHDIC, respectively, always converges to a fixed state in course of iterations for ASK, PSK, or QAM symbols.
Proof.Convergence of a Hopfield network for serial update (H = 0) can be proven by analyzing with after each estimate a ν,μ has been updated.It can be shown that the error energy E remains constant or decreases having updated an estimate.As E is bounded from below, that is, E ≥ 0, the serial update rule ensures convergence of the Hopfield network.
Let us assume that an update of an estimate has just been done in the Hopfield network yielding the squared norm according to (6) which is indicated by E .The next estimate to be updated is denoted by a ν0,μ and the corresponding squared norm after updating this symbol is indicated by E .Without loss of generality it is assumed that 1 < ν 0 ≤ V to ease notation.For the following analysis we introduce the vector (8) which contains the latest estimates after updating the estimate on a ν0 in iteration μ.With help of ( 2) and ( 6) the difference E − E can be written as where the MF estimate after cancellation of interference (cf. (3)) and the crosscorrelation values of the effective transmission pulses have been used.Obviously, in (9) the quantity corresponds to the metric of a maximum-likelihood hypothesis test [39], where a ν0,μ is an observed value and a ν0,μ ∈ X for μ > 0 is the hypothesis.The set X may contain an arbitrary ASK, PSK, or QAM constellation.As the hard decision applied in the Hopfield network (cf. (3)) and (10), always minimizes (12) and EURASIP Journal on Advances in Signal Processing holds, using ( 9), ( 14), and ρ ν0,ν0 > 0 it can be concluded that It has to be emphasized, that a ν0,μ−1 is generated based on a ν0,μ−1 and therefore not necessarily minimizes | a ν0,μ − a ν0,μ−1 | 2 .This means either E remains constant from one iteration to the next (cf.( 15)) or it decreases (cf.( 16)) and as E is bounded from below, the algorithm converges for the serial update scheme.Hence, a Hopfield network with serial update always converges to a minimum of E in (6) but not necessarily to the global one for ASK, PSK, or QAM symbols.The example in Figure 1 also visualizes this behavior.
In Figure 1, a convergence chart is given for the selected parameters.Obviously, there is a similarity to EXIT charts (cf.[40]).Figure 1 is easily constructed from ( 17) and (18).To ease representation, the iteration index μ is omitted.This means a 1 and a 2 in Figure 1 display the current hard estimates a 1,μ and a 2,μ in each iteration μ.It can be seen that starting from the initialization (0, 0) the IHDIC algorithm directly finds the correct solution (1, 1).However, given that the algorithm has to start with a 2 < −2/3 it is easily understood that IHDIC would converge to the estimates (−1, −1) and would remain there.Hence, the estimates (−1, −1) establish a kind of locked error situation.This means that given some errors in the hard estimates a ν,μ , it is possible that cancellation using erroneously detected symbols causes an amount of interference that causes other symbols to be detected erroneously which in turn causes the first group of symbols to be detected in the same wrong way.We denote this case as locked error situation [41].
Figure 1: Analysis of a locked error situation with convergence chart formed by two nested hard decision functions for BPSK and IHDIC with serial update: initialization (a) converges to the wrong solution (−1, −1), whereas for initialization (b) the correct solution (1, 1) is found.Therefore, solution (−1, −1) corresponds to a locked error situation.
However, decision functions have not been taken into account in the analysis resulting in an easier tractability via eigenvalues.In this section, we analyze the convergence of IHDIC including decision functions.For binary real-valued alphabets X the convergence behavior of Hopfield networks for parallel update has been analyzed, for example, in [9].

Theorem 3. A Hopfield network or IHDIC, respectively, employing a parallel update rule converges either to a stable state or to cycles of length 2 for ASK, PSK, or QAM symbols if all effective transmission pulses of the involved symbols have equal energy. For PSK symbols a Hopfield network or IHDIC, respectively, employing a parallel update rule converges either to a stable state or to cycles of length 2 even if all effective transmission pulses of the involved symbols have not equal energy.
Proof.The proof is similar to that in Section 4 where an energy function is defined which is shown to be bounded from below and nonincreasing during the iterations.
First Part.We first focus on the first part of the theorem and assume that all effective transmission pulses of the involved symbols have equal energy, that is, For analysis of the convergence of a Hopfield network with parallel update the new auxiliary energy function is introduced only for analysis of the convergence behavior where and the vector a μ according to (7) are used.Obviously, E μ is bounded from below as the entries in the vectors a μ and a μ−1 are complex-valued but finite.In the following, the difference E μ+1 − E μ is analyzed: where a vector containing all MF outputs after interference cancellation in the Hopfield network is used.Note that the condition in ( 19) is necessary for derivation of ( 23) from (22).With this condition we obtain Re Like in (12) the metric of a maximum-likelihood hypothesis test [39] a μ+1 − a μ+1 2 (27) appears, where a μ+1 is an observed vector and a μ+1 ∈ X V for μ ≥ 0 is the hypothesis.As the vector-based hard decision made in the Hopfield network always minimizes (27) with the inequality is valid as a μ−1 is generated based on a μ−1 and therefore not necessarily minimizes | a μ+1 − a μ−1 | 2 .Note that the set X may contain an arbitrary ASK, PSK, or QAM constellation.Therefore, using (24) and (30) it can be concluded that This means neglecting the first value of the energy E 0 = 0 which is related to the initialization a ν,0 = 0 for all ν, the energy function decreases monotonically (cf.( 32)) in the first iterations and will always converge to a value where it remains constant as it is bounded from below.In this state, according to (31) cycles of length 2 occur, which includes the case where a μ remains constant, that is, Second Part.To prove the second part of the theorem, we focus on PSK and do not apply any restrictions to the energy ρ ν,ν of the effective transmission pulses of the involved symbols.For analysis we modify the auxiliary energy function (cf.(20)) to Similarly to (24) we calculate the difference E μ+1 − E μ which leads to with a μ+1 according to (25) and D ρ according to (21), but without any restrictions to the energies ρ 1,1 , . . ., ρ V,V .In the Hopfield network or IHDIC, respectively, the hard decision is based on a μ+1 , However, for PSK is valid, as some scaling with positive real-valued coefficients in diagonal matrix D ρ does not change the result of the hard decision.Hence, the hard decisions in vector a μ+1 also similarly to (30) which leads again to (31) and (32).Hence, for PSK symbols a Hopfield network employing a parallel update rule converges either to a stable state or to cycles of length 2 even if the effective transmission pulses of the involved symbols have not equal energy.
Example 4. In the following, the occurrence of cycles of length 2 is visualized by an example.In Figure 2, a convergence chart is given based on the parameters of the example in Section 4 (cf.Figure 1).However, now the parallel update scheme is used in IHDIC.Again, we assume a BPSK transmission, that is, a ν ∈ X = {−1, +1}, with two transmitted symbols (V = 2).In Figure 2, a 1 and a 2 represent the current hard estimates a 1,μ and a 2,μ in each iteration μ.Obviously, for parallel update, the estimates (1, −1) lead to (−1, 1) which in turn results in the estimates (1, −1) in the next step.Hence, a cycle of length 2 is established.But as proven above, also stable solutions may occur for the parallel update as they can be interpreted as a special case of cycles.In the example in Figure 2, the estimates (1, 1) and (−1, −1) correspond to stable solutions of the parallel update scheme.received energy per information bit, N 0 : single-sided power spectral density of noise).In Figure 3 transmission using 4ASK symbols with V = 128 users is considered.Figure 4 shows results for 8PSK transmission with V = 64 users and Figure 5 visualizes the possible convergence behavior for 16QAM transmission with V = 64 users.The displayed BER has been determined by evaluating the received symbols of all V users for a certain channel use.As proven above and visualized in Figures 3, 4, and 5, a Hopfield network with parallel update converges to a cycle of length 2 or to a stable state where the relation a μ+1 = a μ = a μ−1 holds.A stable state can be a locked error situation or the correct solution.

Simulation Results
In case of convergence to the correct solution the BER curves stop in the logarithmic representation of Figures 3, 4, and 5 at a certain iteration index as BER = 0 for higher iteration indices.

Concluding Remarks
In this paper iterative hard decision interference cancellation (IHDIC) has been studied.An overview of IHDIC applied to linear modulation and DS-CDMA-based transmission systems, respectively, has been given and the relation to Hopfield neural network theory has been pointed out.Convergence of IHDIC has been analyzed in a general framework for ASK, PSK, and QAM constellations.It has been proven that IHDIC with serial update scheme always converges to a stable state in the estimated values in course of iterations and that IHDIC with parallel update scheme converges to cycles of length 2. Furthermore, we gave insight into possible errors occurring in IHDIC which turned out to be caused by locked error situations.The derived results can be directly transferred to those iterative soft decision interference cancellation (ISDIC) receivers whose soft decision functions approach hard decision functions in course of the iterations.

6 EURASIPFigure 2 :
Figure2: Analysis of a cycle of length 2 with convergence chart formed by two nested hard decision functions for BPSK and IHDIC with parallel update: in course of iterations the estimates (1, −1) lead to (−1, 1) which in turn results in (1, −1) (cf.also Figure1).

Figures 3 , 4 ,Figure 3 :
Figures 3,4,and 5 show the possible convergence behavior of IHDIC with parallel update for a synchronous DS-CDMA transmission for 4ASK, 8PSK, and 16QAM, respectively.Each figure displays the progress of the bit error ratio (BER)

Figure 4 :Figure 5 :
Figure 4: Possible convergence behavior for IHDIC with parallel update for 10 log 10 (E b /N 0 ) = 15 dB, synchronous DS-CDMA transmission over an AWGN channel with V = 64 users employing 8PSK and complex-valued random spreading sequences with spreading factor L = 256.