The Effects of Noise on Speech Recognition in Cochlear Implant Subjects: Predictions and Analysis Using Acoustic Models

Cochlear implants can provide partial restoration of hearing, even with limited spectral resolution and loss of fine temporal structure, to severely deafened individuals. Studies have indicated that background noise has significant deleterious effects on the speech recognition performance of cochlear implant patients. This study investigates the effects of noise on speech recognition using acoustic models of two cochlear implant speech processors and several predictive signal-processing-based analyses. The results of a listening test for vowel and consonant recognition in noise are presented and analyzed using the rate of phonemic feature transmission for each acoustic model. Three methods for predicting patterns of consonant and vowel confusion that are based on signal processing techniques calculating a quantitative difference between speech tokens are developed and tested using the listening test results. Results of the listening test and confusion predictions are discussed in terms of comparisons between acoustic models and confusion prediction performance.


INTRODUCTION
The purpose of a cochlear implant is to restore some degree of hearing to a severely deafened individual. Among individuals receiving cochlear implants, speech recognition performance varies, but studies have shown that a high level of speech understanding is achievable by individuals with successful implantations. The speech recognition performance of individuals with cochlear implants is measured through listening tests conducted in controlled laboratory settings, which are not representative of the typical conditions in which the devices are used by the individuals in daily life. Numerous studies have indicated that a cochlear implant patient's ability to understand speech effectively is particularly susceptible to noise [1,2,3]. This is likely due to a variety of factors, such as limited spectral resolution, loss of fine temporal structure, and impaired sound-localization abilities.
The manner and extent to which noise affects cochlear implantee's speech recognition can depend on individual characteristics of the patient, the cochlear implant device, and the structure of the noise and speech signals. Not all of these relationships are well understood. It is generally presumed that increasing the level of noise will have a negative effect on speech recognition. However, the magnitude and manner in which speech recognition is affected is more ambiguous. Particular speech processing strategies may be more resistant to the effects of certain types of noise, or noise in general. Other devices parameters, such as the number of channels, number of stimulation levels, and compression mapping algorithms, have also been shown to influence how speech recognition will be affected by noise [4,5,6]. The effects of noise also depend on the type of speech materials and the linguistic knowledge of the listener. With all of these interdependent factors, the relationship between noise and speech recognition is quite complex and requires careful study.
The goals of this study were to analyze and predict the effects of noise on speech processed by two acoustic models of cochlear implant speech processors. The listening test was conducted to examine the effects of noise on speech recognition scores using a complete range of noise levels. Information transmission analysis was performed to illustrate the results of the listening test and to verify assumptions regarding the acoustic models. The confusion prediction methods were developed to investigate whether a signal processing algorithm would predict patterns of token confusion similar to those seen in the listening test. The use of the similarities and differences between speech tokens for prediction of speech recognition and intelligibility has a basis in previous studies. Müsch and Buus [7,8] used statistical decision theory to predict speech intelligibility by calculating the correlation between variations of orthogonal templates of speech tokens. A mathematical model developed by Svirsky [9] used the ratio of frequency-channel amplitudes to locate phonemes in a multidimensional perceptual space. A study by Leijon [10] used hidden Markov models to approximate the rate of information transmitted through a given acoustic environment, such as a person with a hearing aid.
The motivation for estimating trends in token confusions and overall confusion rate, based solely on information in the processed speech signal, is to enable preliminary analysis of speech materials prior to conducting listening tests. Additionally, a method that estimates token confusions and overall confusion rate would have applications in the development of speech processing methods and noise mitigation techniques. Sets of processed speech tokens that are readily distinguishable by the confusion prediction method should also be readily distinguishable by cochlear implantees, if the prediction method is well conceived and robust.
The rest of this paper is organized as follows. Section 2 discusses the listening test conducted in this study. The experimental methods using normal-hearing subjects and the information transmission analysis of vowel and consonant confusions are detailed. Results, in the form of speech recognition scores and information transmission analyses, are provided and discussed. Section 3 describes the methods and results of the vowel and consonant confusion predictions developed using signal processing techniques. The methods of speech signal representation and prediction metric calculation are described, and potential variations are addressed. Results are presented to gauge the overall accuracy of the investigated confusion prediction methods for vowels and consonants processed with each of the two acoustic models.

LISTENING TEST
The listening test measured normal-hearing subjects' abilities to recognize noisy vowel and consonant tokens processed by two acoustic models. Using acoustic models to test normal-hearing subjects for cochlear implant research is a widely used and well-accepted method for collecting experimental data. Normal-hearing subjects provide a number of advantages: they are more numerous and easier to recruit, the experimental setups tend to be less involved, and there are not subject variables, such as experience with cochlear implant device, type of implanted device, cause of deafness, and quality of implantation, that affect individual patient's performance. Results of listening tests using normal-hearing subjects are often only indicative of trends in cochlear implant patient's performance; absolute levels of performance tend to disagree [1,11]. There are several sources of discrepancies between the performance of cochlear implant subjects and normal-hearing subjects using acoustic models, such as experience with the device, acclimation to spectrally quantized speech, and the idealistic rate of speech information transmission through the acoustic model. However, acoustic models are still an essential tool for cochlear implant research. Their use is validated by numerous studies where cochlear implant patient's results were successfully verified and by the flexibility they provide in testing potential speech processing strategies [12,13].

Subjects
Twelve normal-hearing subjects were recruited to participate in a listening test using two acoustic models for vowel and consonant materials in noise. Prior to the listening tests, subjects' audiograms were measured to evaluate thresholds at 250 Hz, 500 Hz, 1 kHz, 2 kHz, 4 kHz, and 8 kHz to confirm normal hearing, defined in this study as thresholds within two standard deviations of the subject group's mean. Subjects were paid for their participation. The protocol and implementation of this experiment were approved by the Duke University Institutional Review Board (IRB).

Speech materials
Vowel and consonant tokens were taken from the Revised Cochlear Implant Test Battery [14]. The vowel tokens used in the listening test were {had, hawed, head, heard, heed, hid, hood, hud, who'd}. The consonants tested were {b, d, f, g, j, k, m, n, p, s, sh, t, v, z} presented in /aCa/ context. The listening test was conducted at nine signal-to-noise ratios: quiet, +10 dB, +8 dB, +6 dB, +4 dB, +2 dB, +1 dB, 0 dB, and −2 dB. Pilot studies and previous studies in the literature [3,5,15,16] indicated that this range of SNRs would provide a survey of speech recognition ability over the range of scores from nearly perfect correct identification to performance on par with random guessing. Speech-shaped noise, that is, random noise with a frequency spectrum that matches the average long-term spectrum of speech, is added to the speech signal prior to acoustic model processing.

Signal processing
This experiment made use of two acoustic models implemented by Throckmorton and Collins [17], based on acoustic models developed in [18,19]. The models will be referred to as the 8F model and the 6/20F model, named for the number of presentation and analysis channels. A block diagram of the general processing common to both acoustic models is shown in Figure 1. With each model, the incoming speech is prefiltered using a first-order highpass filter with a 1 kHz cutoff frequency, to equalize the spectrum of the incoming signal. It is then passed through a 6th-order antialiasing Butterworth lowpass filter with an 11 kHz cutoff. Next, the filterbank separates the speech signal into M channels using 6th-order Chebyshev filters with no passband overlap. Each channel is full-wave rectified and lowpass filtered using an 8th-order Chebyshev with 400 Hz cutoff to extract the signal envelope for each frequency channel. The envelope is discretized over the processing window of length L using the root-mean-square value. The numbers of channels and channel cutoff frequencies for the two acoustic models used in this study were chosen to mimic two popular cochlear implant speech processors. For the 8F model, the prefiltered speech is filtered into eight logarithmically spaced frequency channels covering 150 Hz to 6450 Hz. For the 6/20F model, the prefiltered speech is filtered into twenty frequency channels covering 250 Hz to 10823 Hz, with linearly spaced cutoff frequencies up to 1.5 kHz and logarithmically spaced cutoff frequencies for higher filters. The discrete envelope for both models is calculated over a two-millisecond window, corresponding to 44 samples for speech recorded at a sampling frequency of 22050 Hz.
The model output is assembled by determining a set of presentation channels, the set of frequency channels to be presented in the current processing window, then amplitude modulating each presentation channel with a separate sinewave carrier and summing the set of modulated presentation channels. In each processing window, a set of N (N ≤ M) channels is chosen to be presented. All eight frequency channels are presented (N = M = 8) with the 8F model. With the 6/20F model, only the six channels with the largest amplitude in each processing window are presented (N = 6, M = 20). The carrier frequency for each presentation channel corresponds to the midpoint on the cochlea between the physical locations of the channel bandpass cutoff frequencies. The discrete envelopes of the presentation channels are amplitude modulated with sinusoidal carriers at the calculated carrier frequencies, summed, and stored as the model output.

Procedure
The listening tests were conducted in a double-walled soundinsulated booth, separate from the computer, experimenter, and sources of background noise, with stimuli stored on disk and presented through headphones. Subjects recorded their responses using the computer mouse and graphical user interface to select what they had heard from the set of tokens. Subjects were trained prior to the tests on the same speech materials processed through the acoustic models to provide experience with the processed speech and mitigate learning effects. Feedback was provided during training.
Testing began in quiet and advanced to increasingly noisy conditions with two repetitions of a randomly ordered vowel or consonant token set for training, followed by five repetitions of the same randomly ordered token set for testing. The order of presentation of test stimuli and acoustic models were randomly assigned and balanced among subjects to neutralize any effects of experience with the previous model or test stimulus in the pooled results. Equal numbers of test materials were presented for each test condition, defined by the specific acoustic model and signal-to-noise ratio.

Results
The subjects' responses from the vowel and consonant tests at each SNR for each acoustic model were pooled for all twelve subjects. The results are plotted for all noise levels in Figure 2. Statistical significance, indicated by asterisks, was determined using the arcsine transform [20] to calculate the 95% confidence intervals. The error bars in Figure 2 indicate one standard deviation, which were also calculated using the arcsine transform. The vowel recognition scores show that the 6/20F model significantly outperforms the 8F model at all noise levels. An approximately equivalent level of performance was achieved with both acoustic models on the consonant recognition test, with differences between scores at most SNRs not statistically significant. Vowel recognition is heavily dependent on the localization of formant frequencies, so it is reasonable that subjects using the 6/20F model, with 20 spectral channels, perform better on vowel recognition.
At each SNR, results of the vowel and consonant test were pooled across subjects and tallied in confusion matrices, with rows corresponding to the actual token played, and columns indicating the token chosen by the subject. An example confusion matrix is shown in Table 1. Correct responses lie along the diagonal of the confusion matrix. The confusion matrices gathered from the vowel and consonant test can be analyzed based on the arrangement and frequency of incorrect responses. One such method of analysis is information transmission analysis, developed by Miller and Nicely in [21]. In each set of tokens presented, it is intuitive that some incorrect responses will occur more frequently than others, due to common phonetic features of the tokens. The Miller and Nicely method groups tokens based on the common phonetic features and calculates information transmission using the mean logarithmic probability (MLP) and mutual information T(x; y) which can be considered the transmission from x to y in bits per stimulus. In the equations below, p i is the probability of confusion, N is the number of entries in the matrix, n i is the sum of the ith row, n j is the sum of the jth column, and n i j is a value from the confusion matrix resulting from grouping tokens with common phonetic features: The consonant tokens were classified using the five features in Miller and Nicely-voicing, nasality, affrication, duration, and place. Information transmission analysis was also applied to vowels, classified by the first formant frequency, the second formant frequency, and duration. The feature classification matrices are shown in Table 2. Information transmission analysis calculates the transmission rate of these individual features, providing a summary of the distribution of incorrect responses, which contains useful information unavailable from a simple token recognition score. Figure 3 shows the consonant feature percent transmission, with percent correct recognition or "score" from Figure 2 included, for the 6/20F model and 8F model. The plots exhibit some deviation from the expected monotonic result; however, this is likely due to sample variability and variations in the random samples of additive noise used to process the tokens. It appears that increasing levels of noise deleteriously affect all consonant features for both acoustic models. It is interesting to note that consonant recognition scores for the 6/20F model and 8F model are nearly identical, but feature transmission levels are quite different. The differences in the two acoustic models result in two distinct sets of information that result in approximately the same level of consonant recognition. A previous study by Fu et al. [3] performed information transmission analyses on consonant data for 8-of-8 and 6-of-20 models and calculated closely grouped feature transmission rates at each SNR for both models, resembling the 8F results shown here. Both Fu et al. models as well as the 8F model in this study have similar model bandwidths, and it is possible that the inclusion of higher frequencies in the 6/20F model and their effect on channel location and selection of presentation channels results in the observed spread of feature transmission rates. Further comments on these results are presented in the discussion.   The patterns of feature transmission are much more consistent between the two acoustic models for vowels, as shown in Figure 4. The significantly higher vowel recognition scores at all noise levels using the 6/20F model translate to greater transmission of all vowel features at all noise levels. Hence, the better performance of the 6/20F model is not due to more effective transmission of any one feature.

CONFUSION PREDICTIONS
Several signal processing techniques were developed in the context of this research to measure similarities between processed speech tokens for the purpose of predicting patterns of vowel and consonant confusions. The use of the similarities and differences between speech tokens has a basis in previous studies predicting speech intelligibility [7,8], and investigating the perception of speech tokens presented through an impaired auditory system [10] and processed by a cochlear implant [9].
The three prediction methods that are developed in this study use two different signal representations and three different signal processing methods. The first method is token envelope correlation (TEC), which calculates the correlation between the discrete envelopes of each pair of tokens. The second method is dynamic time warping (DTW) using the cepstrum representation of the speech token. The third prediction method uses the cepstrum representation and hidden Markov models (HMMs). These three methods provide for comparison a method using only the temporal information (TEC), a deterministic measure of distance between the speech cepstrums (DTW), and a probabilistic distance measure using a statistical model of the cepstrum (HMM).

Dynamic time warping
For DTW [22], the (ith, jth) entry in the prediction metric matrix is the value of the minimum-cost mapping through a cost matrix of Euclidean distances between the cepstrum coefficients of the ith given token and the jth response token. To calculate the (ith, jth) entry in the prediction metric matrix, the cepstrum coefficients are computed from energynormalized speech tokens. A cost matrix is constructed from the cepstrums of the two tokens. Each row of the cost matrix specifies a vector of cepstrum coefficients calculated during one window of the given signal, each column corresponds to a vector of cepstrum coefficients calculated during one window of the response signal, and the entry in the cost matrix is a measure of distance between the two vectors. In this project, the coefficient vector differences were quantified using the Euclidean distance d 2 (x, y), The minimum-cost path is defined as the contiguous sequence of cost matrix entries from (1,1) to (N, M), where N is the length of the given token cepstrum and M is the length of the response token cepstrum, such that the sum of the sequence entries is minimized. To reduce the complexity of searching for the minimum-cost path, sequence steps are restricted to three cases: horizontal (n, m + 1), vertical (n + 1, m), and diagonal (n + 1, m + 1). Additionally, since the shortest path from (1,1) to (N, M) will be nearly diagonal, the cost matrix entry is multiplied with a weighting parameter in the case of a diagonal step, to prevent the shortest path from becoming the default minimum-cost path. The value for the weighting parameter, equal to 1.5 in this study, can be increased or decreased resulting in a lesser or greater propensity for diagonal steps.
Next, the cumulative minimum-cost matrix D i j containing the sum of the entries for the minimum-cost path from (1,1) to any point (n, m) in the cost matrix is calculated. Given the restrictions on sequence-step-size, sequence step direction, and weighting parameter, the cumulative cost matrix is calculated as

Token envelope correlation
For TEC, the (ith, jth) entry in the prediction metric matrix is the normalized inner product of the discrete envelopes of two processed speech tokens that have been temporally aligned using dynamic time warping. The discrete envelope was originally calculated as a step in the acoustic model processing. The discrete envelope used in TEC is similar to the discrete envelope calculated in the acoustic model, with a lower cutoff frequency on the envelope extraction filter. The cepstrums of the ith processed given token and the jth processed response token are used in the DTW procedure to calculate the minimum-cost path for the two tokens. The minimum-cost path is then used to temporally align the two discrete envelopes, addressing the issue of different token lengths in a more elegant manner than simple zero padding. Using DTW to align the signals injects flexibility in the alignment to account for potential listener ambiguity regarding the starting point and pace of the speech token. After alignment of the given token and response token, the final value of the prediction metric can be calculated as where x i is the discrete envelope of the ith given token, y j is the discrete envelope of the jth response token, and M i, j is the (ith, jth) entry in the prediction metric matrix.

Hidden Markov models
The third prediction method is based on hidden Markov models (HMMs) [22,23]. Using HMMs, the (ith, jth) entry in the prediction metric matrix is the log-likelihood that the cepstrum of the ith given token is the observation produced by the HMM for the cepstrum of the j th response token. To calculate the (ith, jth) entry in the prediction metric matrix using HMMs, a continuous-observation HMM was trained for each speech token using a training set of 100 tokens. All training data were collected from a single male speaker in quiet. HMMs were trained for different numbers of states Q and numbers of Gaussian mixtures M, with Q ranging from two to six and M ranging from two to four. Training was performed using the expectation-modification method to iteratively determine the parameters that locally maximize the probability of the observation sequence. The state transition matrix and Gaussian mixture matrix were initialized using random values. A k-means algorithm was used to initialize the state-observation probability distributions. The probability of an observation was determined using the forward algorithm [23] where O i is the ith element in the observation sequence, q T = S i indicates that the model is in the ith state at time T, and λ are the HMM parameters.

Prediction performance
The accuracy of each prediction method was verified using the vowel and consonant confusion matrices generated in the listening test as basis for comparison. The confusion matrices at each of the eight noise levels and in quiet were pooled to produce a general pattern of confusions independent of any specific noise level. Combining the confusion matrices across noise levels was justified by information transmission analyses, which indicated that increasing the amount of additive noise most significantly affected the rate of confusions rather than the pattern of confusions. The first test of confusion prediction performance gauged the ability to predict the most frequent incorrect responses (MFIRs). The prediction of MFIRs was measured in terms of successful near predictions, defined as the case where one token in the set of MFIRs matches one token in the predicted set of MFIRs. Sets of two tokens were used for vowel near predictions (25% of possible incorrect responses), three tokens for consonants (23% of possible incorrect responses). For example, if the two MFIRs for "head" were "hid" and "had," then either "hid" or "had" would have to be one of the two predicted MFIRs for a successful near prediction. Measuring prediction performance using near predictions satisfies the objective of predicting patterns in the confusions, rather than strictly requiring that the predicted MFIR was indeed the most frequent incorrect response. The purpose of measuring near predictions is to test whether the methods are distributing the correct tokens to the extremes of the confusion response spectrum. Figure 5 shows the percentages for successful near prediction of the MFIR tokens for each acoustic model and token set. Percentages of successful near prediction were calculated out of possible nine trials for vowels (N = 9) and fourteen trials for consonants (N = 14). Near-perfect performance is achieved using DTW. The HMM method performs at a similarly high level. The TEC method consistently underperforms the two methods utilizing the cepstrum coefficients for confusion prediction. Chance performance is also shown for comparison.
The second test of confusion prediction performance analyzed the ability of each method to discern how frequently each individual token will be confused, as represented by the main diagonal of the confusion matrices. Rather than predicting the absolute rate of confusion, which would be dependent on noise level, the test evaluates the accuracy of a predicted ranking of the tokens from least to most recognized, or most often to least-often confused.
To calculate the predicted ranking of the individualtoken recognition rates, the off-diagonal values in each row of the prediction metric matrix were averaged and ranked, as a means of evaluating each token's uniqueness. The more separation between the played token and the set of incorrect responses, where separation is measured by the prediction metrics, the less likely it is that an incorrect response will occur. Ranking of 8F vowel experiment percent correct TEC R 2 = 0.1111 Figure 6: Scatter plot for 8F vowel predicted rankings using TEC versus actual recognition rankings. Includes regression line and R 2 value, corresponding to top-left value in Table 3a.
The fit of the predicted token recognition rankings to the actual recognition rankings was represented using linear regression. The coefficient of determination R 2 [24] was calculated for the linear regression of a scatter plot with one set of rankings plotted on the ordinate and another on the abscissa. R 2 values were calculated for two different sets of scatter plots. The first set of scatter plots was created by plotting the predicted recognition rankings and token length rankings against the true recognition rankings. A ranking of token lengths was included to investigate any potential effects of token length on either the calculation of the prediction metrics or the listening test results. Figure 6 displays an example scatter plot for TEC predicted 8F vowels rankings including the regression line and R 2 value. Each token is represented by one point on the chart. The x-axis value is determined by token rank in terms of recognition rate in the listening test, and the y-axis value is determined by the token's predicted recognition ranking using TEC. Similar scatter plots (not shown) were created for the other prediction methods. All of the R 2 values with listening test rankings on the x-axis are shown in Table 3a. A second set of scatter plots was created by assigning token length rankings to the x-axis, rather than listening test rankings, and using predicted rankings and listening test rankings for the y-axis values (Table 3b). Table 3 shows the R 2 values for the two different methods of plotting. With the percent correct plotted on the xaxis, the HMM is shown to perform very well for vowel recognition rankings with either acoustic model. DTW and HMM perform similarly on 8F consonants, but not at the level of HMM on vowels. HMM performance is weaker for 6/20F consonants than for 8F consonants. Predicted recognition rankings for any material set using TEC do not appear promising. Investigating the potential relationship between token length and predicted recognition rankings leads to the observation that HMM predicted rankings for vowels with both acoustic models and DTW predicted rankings for 6/20F vowels appear to correspond to token length. The true recognition ranking also appears related to length for 8F vowels. The relationship between HMM predicted rankings and token length can potentially be explained by the structure of the HMM. The state transition probabilities are adapted to expect tokens of a certain length; longer or shorter tokens can cause state transitions that are forced early or delayed. This would affect the calculated log-likelihood values, and could result in artifacts of token length in the predicted recognition rankings.
The third task tested whether the performance gap seen in the listening test between the token sets with different materials and acoustic models was forecast by any of the prediction methods. DTW was the only method that appeared to have any success predicting the differences in token correct identification for the different acoustic models and token sets. The token identification trend lines for vowels and consonants are shown in Figure 7a. The overall level of token recognition for any combination of token set and acoustic model was predicted with DTW by averaging the off-diagonal prediction metrics. The average confusion distance is plotted as a constant versus SNR in Figure 7b since the metric is not specific to the performance at any particular noise level, and indicates that the pattern of the trends of recognition levels is reasonably well predicted.
Predicted trends for TEC and HMM are not shown, but did not accurately indicate the trends in the listening test. The failure of TEC at the third task supports the conclusion that the strictly temporal representation lacks sufficient distinguishing characteristics. Since the measure for this task is essentially an average of the token recognition rankings calculated in the second task, another measure of prediction performance for which TEC scored poorly, the poor performance using TEC for this task is not surprising. However, the HMM prediction metric performed very well on the first two tasks. Based on that performance, the failure of HMMs was unexpected, especially with the accuracy of the predicted trends using DTW.

DISCUSSION
Information transmission analysis using the method developed by Miller and Nicely [21] calculates how effectively the two acoustic models transmitted the features of vowels and consonants. The increased spectral resolution of the 6/20F model, credited for the better performance of the 6/20F model for vowel token recognition, also appeared in the information transmission results, with proportionally greater transmission of both the F1 and F2 features. The results of the consonant feature analyses are more difficult to classify. A reasonable hypothesis would be that the 8F model should more effectively transmit broadband features, since it has a continuous frequency spectrum with greater bandwidth than the 6/20F model. The 6/20F model should better transmit frequency-specific consonant features due to greater frequency resolution. However, many outcomes from the consonant feature transmission analysis disagree with this hypothesis. Affrication, a broadband feature, is transmitted with similar efficiency by both acoustic models. Voicing is relatively narrowband and suspected to be more effectively transmitted by the 6/20F model; however, it is also transmitted with similar efficiency by both acoustic models. The 6/20F model transmits place and duration more effectively than the 8F model. Duration is essentially a temporal feature, and differences between the acoustic models should not affect transmission of this feature. An acoustic description of the effect of place is very complex and difficult to describe in general terms for all tokens. Place can appear as a broadband or narrowband feature in different regions of the frequency spectrum. The 8F model was more efficient at transmitting the nasality feature. With regards to the 6/20F model, it is possible that six spectral channels do not provide sufficient information to maximally transmit some of the consonant features, whereas for vowels, only a few channels are required for transmission of the formants. This examination of the information transmission analysis results can benefit from the observation that the vowel and consonant features are not independent of each other. For example, the vowel feature duration, a temporal feature, was much more effectively transmitted, with a difference of approximately 50% transmission across all noise levels, by the 6/20F model than by the 8F model; however, the two models have the same temporal resolution. The increased spectral resolution would have legitimately increased transmission of the formant features, resulting in a reduced number of incorrect responses in the listening test, which would in turn raise the calculated transmission of duration information as a side effect. It is expected that some of the calculated percent transmission of features for consonants may also reflect strong or weak performance of other features, or could potentially be influenced by unclassified features. Analysis of the feature classification matrices could help explain potential relationships between the calculated values for feature transmission.
The results of the confusion predictions indicate that analysis of the differences between tokens can provide insight to the token confusions. The three tasks used in this study to analyze the performance of the confusion predictions investigate prediction of trends along the rows, the diagonal, and the overall separation of prediction metrics, providing a multifaceted view of the accuracy of the overall token confusion pattern. The two methods utilizing the cepstrum coefficients for representing the speech token outperformed the method using strictly temporal information in all three tests. The experiment setup and speech-shaped noise characteristics, either of which could potentially affect patterns of token confusion, were not considered in the prediction metric calculations. Expanding the prediction methods to include such additional factors could improve the accuracy of confusion pattern prediction.
Not considering the effects of the noise characteristics and experiment setup also resulted in symmetric prediction metrics matrices calculated using DTW and TEC. This is not entirely consistent with the results of the listening test, however the results presented in this study using DTW indicate that symmetry does not prohibit prediction of trends in token confusion. The procedure for calculating the prediction metric with each prediction method included steps to normalize the outcome for tokens of different lengths, to emphasize the differences within the speech signals and minimize any effect of differences in token length. However, Table 3 indicates that token length may have been used by the listening test participants to distinguish the speech tokens. Reinserting some effect of token length in the calculation of the prediction metrics or removing token length as a factor in the listening test may also improve confusion prediction accuracy.
In summary, this study presented results of a listening test in noise using materials processed through two acoustic models mimicking the type of speech information presented by cochlear implant speech processors. Information transmission analyses indicate different rates of transmission for the consonant features, likely due to differences in spectral resolution, number of channels, and model frequency bandwidth, despite similar speech recognition scores. The development of signal processing methods to robustly and accurately predict token confusions would allow for preliminary analysis of speech materials to evaluate prospective speech processing and noise mitigation schemes prior to running listening tests. Results presented in this study indicate that measures of differences between speech tokens calculated using signal processing techniques can forecast token confusions. Future work to improve the accuracy of the confusion predictions should include refining the prediction methods to consider additional factors contributing to token confusions, such as speech-shaped noise characteristics, experiment setup, and token length.