A Novel Pseudoerror Monitor

The error rate (ER) is a crucial criterion in evaluating the performance of a digital communication system. Many ER estimation methods have been described in the literature. Among them, the pseudoerror monitoring solution has attracted special attention due to its consistent performance in di ﬀ erent environments and distinctive blind estimation capability, that is, estimating the ER without needing any prior knowledge of the transmitted information. In this paper, a novel pseudoerror monitor (PEM) design, the kernel PEM, is developed. Incorporating the strength of the probability density function (pdf) approximation technique, the proposed design has remarkable advantage of being able to produce statistically consistent ER estimate within a much shorter observation time. Simulation results are given in support of this claim.


INTRODUCTION
One of the primary goals of a digital communication system is to provide users with reliable data transmission service. Being the most straightforward measure of the reliability of data transmission, not surprisingly, the error rate (ER) has been widely recognized as a crucial criterion in evaluating the performance of a digital communication system. Many ER estimation methods have been described in the literature, for example, the error counting solution [1], the parameter estimation solution [1,2,3,4,5], the probability density function (pdf) approximation-based solution [6,7,8], the pseudoerror monitoring solution [1,9,10,11,12,13,14,15], and so forth. Among them, the pseudoerror monitoring scheme has attracted special attention due to its distinctive blind estimation capability and consistent performance in various environments. The conventional pseudoerror monitor (PEM) designs, however, require a relatively long observation time to produce statistically reliable estimates at low ERs. In this study, a novel PEM design, termed kernel PEM, has been developed. By exploiting the pdf approximation technique, the proposed design successfully reduces the observation time without degrading the overall quality of the ER estimate.
This paper is organized as follows. In Section 2, the principle of the pseudoerror monitoring approach is introduced. In Section 3, the kernel density-approximation technique is reviewed. Section 4 describes the kernel PEM design, sum-marizes its advantages, and proposes an iterative method to attain the optimum estimation. Simulation results are given in Section 5 to demonstrate the superiority of the proposed design over its conventional counterparts. Section 6 concludes this paper.

PSEUDOERROR MONITORING
In pseudoerror monitoring, the observed events that are relatively more likely to be erroneous are treated. These events are not necessarily the real transmission errors. The most direct benefit of this strategy is to relieve the error counting monitor from the high dependence on the prior knowledge of the transmitted information. Furthermore, the observation time needed for generating statistically consistent ER estimate can be reduced significantly too.
In conventional pseudoerror monitoring, several secondary transmission channels are constructed, and controlled amounts of signal degradations are introduced (or the error criteria are released), to make the error events occur more frequently. Such errors are often referred to as pseudo errors. As a consequence, the ER is amplified and a sufficiently large number of pseudo errors can be recorded within a much shorter observation time. The estimates of the pseudoerror rates (PERs), resulted from counting the numbers of pseudo errors, are then extrapolated to estimate the ER.
The accuracy of the ER estimate calculated as above is dependent on the extrapolation method used. A simple and generally acceptable extrapolation can be performed by treating the logarithmic ER as a linear function of a suitably defined degradation parameter, such as the signal degradation factor [9]. For secondary channels with signal degradation factors of d 1 and d 2 , we can extrapolate the PER estimates P d1 and P d2 , respectively, to have the desired ER estimate P 0 as follows: Many PEM designs have been described in the literature. These schemes face the same challenge when they are applied to fast-varying channels, that is, the long observation time. This problem can be relieved by adding in more signal degradations or further relaxing the error criteria. However, since the discrepancy between the extrapolation and the actual error pattern can be too big sometimes, this solution may suffer a serious drop in the estimation accuracy. In some cases, the resultant ER estimate may be too biased to be useful to serve as a performance indicator.

KERNEL DENSITY FUNCTION APPROXIMATION
The subject of density function approximation has long been a hot research topic in statistics and it has been studied extensively in the literature (see [16,17] and the references therein). Among the existing solutions, the kernel approximation method is the most widely studied and perhaps the most successful method in practice. A kernel pdf estimator can be constructed as follows: where x is the random variable of interest, X i is the ith sample of x, n is the number of the samples used for the approximation, h is a positive smoothing parameter, f is the approximate of the actual pdf f , and K is a kernel function that satisfies Function K is usually, but not always, selected to be a density function, such as the standard Gaussian function. It follows from (2) that the density approximate f is also a density function. The value of h determines the amount of details of the samples that will be masked in the approximation process. If h is set too small, the spurious fine structure will become visible, and if h is set too large, some important features of the distribution will be obscured. The optimum value of h is affected by many factors, for example, the choice of the kernel, the actual density, the criterion used to evaluate the pdf approximate, and so forth. If the concerned statistics is a Gaussian distribution with a variance of σ 2 , the optimum smoothing parameter for the standard Gaussian kernel can be found to be [16] where h o is optimum in the sense of minimizing the mean integrated square error (MISE), that is, Obviously, the MISE criterion measures the global accuracy of the resultant pdf approximate.

Principle
The pdf approximation technique can be readily applied in ER estimation as follows: where P sm is the probability that the mth (m = 0, 1, . . . , M − 1) symbol is transmitted, x m is the corresponding decision statistics, f m is the pdf approximate of x m , and ER m denotes the error region of x m . Assume that all the M symbols are equiprobable, that is, P sm = 1/M, and they suffer the same degree of corruption during the transmission, that is, f m can only be identified by its mean value. The ER estimator in (6) can be accordingly simplified to where x is an arbitrary decision statistics. The ER can now be estimated in two successive steps: approximate the pdf of a decision statistics, and then calculate its integration over the relevant error region. Rather than using some specific types of events as the error counting method and the conventional pseudoerror monitoring method do, the density approximation-based scheme exploits the information carried by all the observations. Consequently, it cuts down the cost on the observation time significantly.
Although it seems possible to estimate the ER directly by integrating the pdf approximate obtained over the real-error region, this solution, termed kernel real-error monitoring, is not feasible in practice. The ER estimate obtained in this way is very sensitive to the authenticity of the error decisions. It follows that in order to produce a good ER estimate, the transmitted information must be known a priori. That condition is hardly possible in practice.
The conventional pseudoerror monitoring solution described previously works successfully in blind ER estimation, but fails to provide sufficient reduction in the observation time. The kernel real-error monitoring solution, on the other side, may reduce the observation time, but it is incapable of giving satisfactory performance in blind state. The idea of the  proposed kernel pseudoerror monitoring solution is to combine the strengths of the two methods to generate a fast and reliable blind ER estimation. In this scheme, the pdf approximate is used to calculate a number of PER estimates, and these values are then extrapolated in the same way as in the conventional pseudoerror monitoring method to give the desired estimate. Figure 1 shows the typical structure of a kernel PEM that uses the threshold modification technique to generate the pseudo errors. In this case, the PER estimates are obtained by integrating the unique pdf approximate over a set of predefined pseudoerror regions. By substituting (2) and the expression of the standard Gaussian kernel into (7), we can express PER estimate P rk as follows: where {r k , k = 1, 2} are the modified thresholds. As is shown in the above equation, the PER estimates can be calculated directly from the samples. Therefore it is not necessary to derive the explicit expression of the pdf approximate. Note that modifying the threshold is in effect equivalent to adding in some amount of signal degradation. For a binary phase shift keying (BPSK) system that is solely corrupted by additive white Gaussian noise (AWGN), the equivalent degradation factor d rk corresponding to the modified threshold r k is where µ is the mean value of the decision statistics and r 0 is the original threshold. It follows from (1) that If the signal degradation technique is applied to generate the pseudo errors, the resultant kernel PEM takes the form of (1). The PER estimates are now the results of integrating a series of pdf approximates, corresponding to different signal degradation factors, over an identical error region. Clearly, this scheme incurs a higher implementation cost. In the rest of the paper, the former monitor structure is further investigated.

Comparison with conventional schemes
The error counting estimation maps the ER domain [0, 1] to a set of discrete values {k/n, k = 0, 1, . . . , n}, where k is the number of the recorded errors. Apparently, in this solution, the sample size n must be far greater than the reciprocal of the ER, so as to avoid trivial results of zero. In [10,18], it has been suggested that more than ten error events should be recorded within each run of estimation, which places very high demands on the observation time at low ERs. The conventional PEM designs exploit the error counting method in estimating the PERs, and accordingly, inherit its disadvantage as well. Although the exploitation of the ER extrapolation technique provides a certain degree of ER amplification and relaxes the requirement for long observation, it is inadequate for extremely low ERs. Consider a BPSK system that is solely corrupted by AWGN and assume that the signalto-noise ratio (SNR) per bit is 12 dB (corresponding to ER 9.0 × 10 −9 ). The modified threshold is taken to be 0.1 (corresponding to an ER amplification factor of 22.4). It can be easily verified that the observation time should be greater than 5.0 × 10 7 sampling intervals. Even if a wider pseudoerror region is used to have an ER amplification factor as large as 1000, the scheme will still need about 1.1 × 10 6 samples to produce acceptable results. The kernel ER estimation method, on the other side, maps the ER domain to a continuous subset [P h , 1 − P h ], where P h is the ER estimate for clean signal, and, as can be seen from (8), it is equal to Q(|µ|h −1 ). Theoretically, the kernel estimation method may provide nontrivial estimate for arbitrarily low ERs. In this sense, it is not constrained by the requirement to have a certain smallest number of samples. This attractive feature is inherited by the kernel PEM design and makes it distinctively more competitive than the conventional methods in fast-varying channels.
In addition, by mapping the infinite ER domain to a finite number of values, the error counting solution and thus the conventional PEM schemes unavoidably incur the ER ambiguity, that is, the inability to discriminate closely-spaced ERs. The minimum ER distance that can be discriminated is n −1 . This problem is, at least theoretically, obviated from the pdf approximation-based solutions, in which one-to-one mappings are built up between the actual ERs and the ER estimates obtained.
The superiority of the proposed kernel PEM design is also evident by its flexibility in adjusting the operation of the monitor. Since the objective of estimating the ER is to provide a reliable indicator of the system performance, the consistency of the ER estimate is usually more important than the absolute value of the ER itself [1]. In conventional PEM designs, other than increasing the observation time, the only method of improving the consistency is to define wider pseudoerror regions, or equivalently, add in larger amount of signal degradation. As has been mentioned earlier, this approach may introduce unbearable bias, and in some cases, it may even lead to misjudgement of the system performance. In the kernel PEM scheme, better consistency is the immediate outcome of using a larger smoothing parameter. Although it also suffers certain loss of accuracy, this approach is advantageous in not needing to change the orders of the ER estimates, that is, lower ERs are mapped to smaller values and vice versa. Consequently, in the proposed scheme, the increment of the estimation bias will not show distinctive destructive effect on the final evaluation of the system performance. Moreover, the adoption of a narrower pseudoerror region reduces the error introduced by linear extrapolation, and this may be helpful in counteracting the loss of accuracy caused by oversmoothing the samples.

Optimum smoothing parameter
For a given operational environment and an observation time, the performance of a kernel PEM is determined mainly by the value of the smoothing parameter and the size of the pseudoerror regions. The former factor dominates the statistical properties of the pdf approximate, while the latter determines the amount of error introduced by the integration in PER estimation and the extrapolation in ER calculation. Since controlling the smoothing effect is more flexible, effective, and reliable, it is highly recommended to be used as the main means of adjusting the behavior of the monitor. Modifying the thresholds, on the other side, should be kept out of consideration unless the previous scheme alone cannot fulfill the requirement. In this study, we discuss the optimum smoothing effect for fixed modified thresholds, that is, fixed setting of the pseudoerror regions.
The smoothing parameter given in (4) works quite well in the simulations conducted. However, it requires the variance of the noise to be known a priori, otherwise, a relatively costly noise variance estimator has to be implemented. Furthermore, inaccurate knowledge or estimate of the variance may seriously degrade the performance of the monitor. To obviate these problems, a suboptimum value has been proposed in [6], which relates the smoothing effect to the sample size Although this formula is simple to use, it is often unable to provide sufficient smoothing effect. As a consequence, the resultant ER estimate will contain considerable variation. Other than using a rough approximation as in (11), the difficulties associated with noise variance estimation can be overcome by searching for the optimum parameter directly as follows: initiate the monitoring and set h to a relatively large value, for example, n −1/5 ; decrease h iteratively, each time by a small step size, until the minimum of a predefined cost function is reached. The cost function should be selected with respect to the specific requirement. In this study, the mean square error (MSE) of the logarithmic PER estimate is used. Since the estimate of the larger of the two PERs to be exploited in the extrapolation contains comparatively negligible error, without loss of generality, the smaller PER is assumed to be P r1 and it is used to form the cost function C, that is, C = MSE log P r1 = bias 2 log P r1 + var log P r1 , (12) where bias log P r1 = E log P r1 − log P r1 , var log P r1 = E log 2 P r1 − E 2 log P r1 . (13) The value of P r1 can be obtained from the error counting approach, which provides an unbiased estimate of the ER (or PER).
To reduce the observation time taken by the error counting estimation, we can consider regulating the variance of the PER estimate and searching for the smallest parameter that satisfies the consistency requirement. Other factors, such as the statistical average of the distance between the estimates of two given ERs, the probability that the ER estimate goes out of a predefined confidence range, and so forth, may also be taken into consideration in order to produce the most desirable result. It should be reminded that due to the practical constraint of the limited precision on computation, a kernel ER estimator can also give a trivial estimate. In that case, the use of a larger parameter value becomes necessary.

SIMULATION RESULTS
In the simulations conducted, the transmitted signal is assumed to be BPSK modulated and the amplitude of the signal component at the receiver is normalized to one. Figure 2 shows the performance of the kernel PEM in AWGN channel, where the SNR per bit is assumed to be 10 dB and the sample size n is fixed at 2000. In Figure 2a, the modified thresholds r 1 and r 2 are set to 0.1 and 0.2, respectively, and the smoothing parameter h is set to 0.04, which is optimum in the sense of minimizing the MSE of the estimate of P r1 and is obtained using the iterative method described previously. Figure 2b shows the effect of using a larger smoothing parameter, where h is redefined to be 0.1 while r 1 and r 2 take the same values. Figure 2c illustrates the effect of using wider pseudoerror regions, where r 1 and r 2 are set to 0.2 and 0.4, respectively, and h takes the corresponding optimum value 0.035. For ease of comparison, the theoretical ERs are displayed in the figures with dashed lines. As is clearly illustrated, the consistency of the ER estimate can be enhanced by increasing the value of the smoothing parameter or by extending the coverage of the pseudoerror regions.
The result obtained with a threshold modification monitor is shown in Figure 3, where the operation conditions remain unchanged, and n, r 1 , and r 2 are set to 10000, 0.2, and 0.4, respectively. It can be seen that although the observation time is much longer and the pseudoerror regions are much wider, the conventional monitor is still unable to    compete with the proposed method. This is shown by the broken points in the figure, which represent trivial ER estimates.
The effectiveness of the proposed solution is not restricted to Gaussian statistics. Figure 4 shows its performance in the presence of a random interference signal, where the SNR per bit and the signal-to-interference ratio are both assumed to be 10 dB and the monitor used is identical to that used in Figure 2a.

CONCLUSION
By combining the strengths of the conventional PEM and the kernel real-error monitor, the proposed kernel PEM has been shown to perform better than both. Compared with the conventional PEM, the proposed monitor is superior in that it significantly reduces the observation time. Compared with the kernel real-error monitor, the proposed method has a better performance in blind state. Overall, the kernel PEM design has great potential to be applied in practice to offer fast and statistically consistent blind ER estimate.