Procedure for the steadystate verification of modulationbased noise reduction systems in hearing instruments
 Jesko G Lamm^{1}Email author,
 Anna K Berg^{1},
 Christian M Künzler^{1},
 Bernhard Kuenzle^{1} and
 Christian G Glück^{1}
https://doi.org/10.1186/168761802011100
© Lamm et al; licensee Springer. 2011
Received: 30 January 2011
Accepted: 10 November 2011
Published: 10 November 2011
Abstract
Hearing instrument verification involves measuring the performance of modulationbased noise reduction systems. The article proposes a systematic procedure for their verification. The procedure has the potential for application in the verification of other signal processing systems, because it is independent of the hearing instrument domain. Its key concept, the separation of abstract and concrete design of test signals, has been adopted from the embedded systems domain. Specifically for modulationbased noise reduction systems in hearing instruments, the article shows a complete implementation of the verification procedure, proposing improvements of existing measurement techniques. To fully cover the verification procedure, a new measurement approach based on maximum length sequences and DFT processing is introduced, revisiting concepts of system identification that came up in the 1970s. These can easily be used with the computational resources of today's microcomputers. Sample measurements with existing hearing instruments demonstrate the verification procedure with different measurement techniques.
Keywords
1 Introduction
This article describes the verification of a noise reduction subsystem within the fully integrated hearing instrument. By verification we mean the confirmation of compliance with the specified requirements, here, by measurement. This means that the scope of this article is limited to the measurement of system responses rather than a clinical verification of the noise reduction functionality under test.
Measuring system responses with test signals is a typical problem of system identification and has been solved with measurement techniques based on test signals that meet some typical requirements regarding their power spectrum and their amplitude distribution. Particularly, the minimization of peaks has been of interest with regard to the fact that practical systems have a limited dynamic range. However, the synthesis of test signals that allow enforcing a signal feature like modulation has only recently been proposed [4, 5].
This article puts the synthesis techniques of prior work into the context of systematic verification, focusing the socalled coverage of the system's input parameters. We show how to systematically design sets of test signals that drive the system under test into a number of different states, allowing to confirm a complete verification of the subsystem of interest.
A process for achieving the systematic verification is needed. While processes targeted at test coverage have been described for the verification of purely softwareoriented systems (e.g., [6]) or simple signal processing systems like e.g., control units in automotive technology ([7, 8]), there has not yet been such work for systems that should provide intense digital signal processing, like e.g., noise reduction subsystems in hearing instruments.
This article introduces test design techniques for signal processing systems by combining existing test processes from the embedded systems domain with signal design techniques from system identification into a novel verification procedure. Its key concept is to obtain an abstract description of test sequences first, in order to derive concrete test signals in a second step. Since these will have to be synthetic in order to match the criteria defined by the test sequence, they are not suitable for testing the system under realistic conditions.
It is therefore a prerequisite for the procedure to have requirements toward the system under test stated in a technical, measurable way. A typical application is the regression testing in product development where the performance characteristics of the system under development are reassessed after implementation changes. Typically, additional tests under realistic conditions (e.g., a clinical trial) are needed before a product can be released, but these are out of scope of the presented procedure.
We believe that the verification procedure is applicable to different kinds of signal processing systems, because one of its essential partsthe abstract description of test signals that can be implemented with different synthesis techniquesis independent of the kind of application, but also because applications other than hearing instruments have to deal with subsystems similar to a noise reduction, e.g., being based on signal features (like e.g., music classification for portable devices [9]) or having to adapt their processing based on information encoded in the signal (like e.g., voice activitydependent transmission systems in telephony [10]). Therefore the procedure itself will be described independently of our application area, hearing instruments.
After introducing definitions of terms and concepts as well as the proposed verification procedure, the article will report experiments demonstrating the procedure with modulationbased noise reduction subsystems of hearing instruments. In contrast to previously reported measurements for verifying these [4, 5, 11, 12], the ones presented here are based on a design for test coverage that is derived from the requirements toward the subsystem under test.
2 Definitions
2.1 Definitions related to signal processing
2.1.1 Signals
This section defines different kinds of signals to be used in measurements.

A signal whose amplitude has only two discrete values is called a binary signal.

A perfect sequence is a stimulus whose spectral components are constant over the whole Nyquist frequency range (see e.g., [13] for a more formal definition of a perfect sequence).
2.1.2 Frequency response measurements
NLMSbased measurement
As an improvement of the least mean squares (LMS) algorithm that has been introduced by Widrow and Hoff [14], Nagumo and Noda [15] have introduced the normalized least mean squares (NLMS) algorithm. It iteratively approximates the impulse response h(n) of the system under test with tap weight factors ĥ (n) of an adaptive filter whose frequency response can be used as an estimate of the system's frequency response H(f).
DFTbased measurements
Differential measurements
Observing the frequency response of one of a linear system's subsystems is possible by differential measurements[4], i.e., a combination of two separate measurements with an identical stimulus, once with the subsystem of interest activated and once having it deactivated. Dividing the frequency responses of both measurements can show the effect of the subsystem of interest on the frequency response of the whole system.
2.1.3 Modulation/modulation frequency
Here, n is the sample index, f_{ s }is the sampling rate, f_{m,b}is the modulation frequency in subband b, and λ_{ b }is a bandlimited stationary signal with a constant envelope over time (which can in some cases only be approximated, but is indeed achieved with the binary signals we will discuss later). Considerations on the signal's band limits will follow further below.
2.2 Definitions related to verification

Test coverage would ideally describe the percentage of the system's input or state parameter range used during tests. In the case of a signal processing system dealing with quasicontinuous signals, the range of possible input signals is dramatically large and has to be constrained to a moderate number of test signals for practical testing. The selection of tests is here based on the hypothesis that one test signal from a certain class  an equivalence class  is sufficient to test the whole class. This hypothesis shall be called uniformity hypothesis[18] in the following. Test coverage in the context of this article denotes the percentage of equivalence classes rather than the percentage of possible signals that is reached by the test. State space coverage is not considered.

A test step is a time interval during a test (definition based on [8]).

A test sequence is a composition of test steps that cover certain equivalence classes, optionally together with a specification of transitions between them. Note that for simplicity, this article does not distinguish between test sequences and test cases, as does [8].
3 Verification procedure
 1)
Identifying the requirements against which to test
 2)
Designing tests
 3)
Performing tests
3.1 Identifying requirements against which to test
Requirements engineering (e.g., [19]) typically ensures that testable requirements are available. However, this matter will not be covered here in more detail, because it is not actually relevant for this article how the requirements specification has been established. Here, it is important to have such a specification and, based on it, identify those requirements that are within the scope of the test.
3.2 Designing tests
3.2.1 Describing abstract test sequences with regard to test coverage
Test design should use a method that can ensure the desired test coverage. In the domain of signal processing systems, we propose the classification tree method for embedded systems (CT/ES) from [7, 8]: the input domain of the system under test is partitioned into equivalence classes according to the original classification tree method from [20], then test sequences are defined in order to cover them with test steps that are abstract, i.e., independent of concrete test signals.
Finding suitable equivalence classes is a key to an appropriate test design; therefore a good starting point is helpful. We expect the identified requirements according to Sect. 3.1 to be a suitable starting point, because they may give hints about the most important input parameters that have to be considered in partitioning the system's input domain. A main reason for this: we expect the main functionality of the signal processing systems targeted here to be a processing of input signals. The requirements thus have to specify how these input signals have to be processed and thus make statements about the system's input domain.
The classification tree representation of equivalence classes, test sequences and test steps enables the assessment of test coverage and the further elaboration on the test, i.e., the verification of the test design and the synthesis of concrete stimuli that comply to it by covering the corresponding equivalence classes of the system's input space.
It may not be possible to have the classification tree method cover all system parameters specified by the requirements according to Sect. 3.1. Therefore the test designer should also identify those tests that are needed in addition to the ones from the identified test sequences in order to verify each requirement with at least one test.
3.2.2 Selecting the synthesis procedure for implementing the concrete test signals
This section discusses different stimuli from system identification and their use as a basis for synthesizing concrete test signals that match the abstract signal description as to the previous section. These signals should be designed for real systems whose usable signal range has its lower limit in a noise floor and its upper limit at a certain maximum level that is given by limited word lengths in the digital domain and/or limited amplitudes in the analog domain. Ideal stimuli would therefore have a white power spectrum, such that the spectral components of the background noise are negligible compared to those of the stimulus at any frequency. To provide good signaltonoise performance within the given level limitations, the peak factor [21] of the stimulus should also be small. Obviously, binary signals have a minimum peak factor, but are reported to oppose challenges to some digitaltoanalog converters [22] and cannot match every given power spectrum. Therefore, different kinds of signals will be considered in the following.

Discreteinterval binary signals[23, 24] result from algorithms that search a certain set of continuoustime binary signals for those ones whose power spectrum approximately matches a specified one. We define that a discreteinterval binary sequence (DIBS) is the discretetime representation of a discreteinterval binary signal.

Binary maximum length sequences (binary msequences) contain all possible sequences of storage initialization in a binary shift register of length L, except the initialization of all storages with zero  resulting in a sequence length of 2^{ L }1 [22]. For system identification they are usually synthesized with computer programs [25, 26] rather than with shift registers. Binary msequences are perfect sequences and have a minimum peak factor.

A periodic multisine signal with a predefined discrete power spectrum can be obtained by adding sine waves of different frequencies. Their amplitudes result from the desired spectrum; phases, however, can be varied, e.g., for minimizing the signal's peak factor [21, 23, 27, 28]. Signal synthesis is most efficiently done using Fast Fourier Transform methods [29, 30].
Sect. 4.5 exemplarily demonstrates the different stimuli that have been described with sample measurements. Their performance in these measurements will be discussed in Sect. 4.6. A more general discussion of stimuli can be found in the literature of system identification (e.g., [29]).
3.2.3 Selecting the measurement technique
There are different techniques for measuring the frequency response H(f) of the system under test: for example, the measurement based on the adaptive LMS algorithm (Figure 2 bottom left) of H(f) and the straightforward computation of an estimate of H(f) from the signals x and y based on the DFT (Figure 2 bottom right).
The impulse response of a system under test can be time varying and it may be desired to track the corresponding variations over time. The NLMS algorithm can achieve this under certain conditions and is therefore a common choice in transfer function measurements (e.g., [13]).
The DFTbased measurements require a steadystate condition of the system under test. The used test signals should have spectral components that are constant [16] over the frequency range of interest. They should also be periodic [4], which can avoid leakage errors [31] in processing based on the DFT, if the DFT window length is a multiple of the period length [29]. If this match of lengths is not possible, zero stuffing  the insertion of additional zeros into the DFT frame  can adjust the signal frame to the DFT frame. It has been shown, however, that this may reduce the measurement precision compared to a situation with matched lengths [32]. As a consequence, it shall be a prerequisite for all further considerations about DFTbased processing that the DFT window length matches the period length of the used stimulus. In case of signals whose length is not a power of two, this may mean that the Fast Fourier Transform algorithm cannot be used. Even in these cases, we expect the computation time to be sufficiently short, based on the assumption that measurement data will be postprocessed with the computational power of a modern desktop computer.
Both LMSbased and DFTbased measurements ideally need stimuli with a white power spectrum. The DFTbased measurements only have optimum precision when used with periodic stimuli, whereas the LMS does not require periodicity of signals. The most important criterion for selecting the measurement approach is the time variance of the system under test: while LMSbased measurements can handle time variance under certain conditions, the DFTbased measurements only work with a timeinvariant system. DFTbased measurements have the advantage that no convergence of an iterative algorithm is needed. This makes the measurement window for a given frequency resolution small and thus the time resolution high.
3.3 Performing tests
How to perform the tests is dependent on the chosen test design. We can therefore not state a general flow of activities for this part of the procedure. We rather use typical experiments to demonstrate the step of performing tests. This will be done within the next section.
4 Measurements
This section demonstrates the application of the verification procedure in the hearing instrument domain, based on experiments with hearing instruments. The design of experiments is given by the proposed verification procedure. The device under test and the measurement setup will be presented in the following sections.
4.1 Device under test
In all experiments, the device under test was a hearing instrument with a modulationbased noise reduction subsystem. Most of the devices used for the experiments below were part of recent test plans at the Bernafon laboratories, allowing us to perform most of the shown experiments within the regular test plans of the laboratory. As a consequence, different experiments have been performed with different hearing instrument models, because test plans do not necessarily foresee to sequentially perform all test cases with the same one. The noise reduction subsystems of the used hearing instruments were equivalent and thus satisfy the same requirements and design.
Requirements toward the noise reduction subsystem
ID  Text  Ref. 

1  The noise reduction shall apply attenuation  
1.1  Attenuation shall depend on modulation  
1.1.1  The dependency between modulation depth (m) and attenuation (a) shall be as follows: $\frac{a}{dB}=\{\begin{array}{ll}{A}_{0}\hfill & ;m\le {M}_{1}\hfill \\ {A}_{0}\cdot \left(1\frac{m{M}_{1}}{{M}_{2}{M}_{1}}\right)\hfill & ;m\in ]{M}_{1},{M}_{2}[\hfill \\ 0\hfill & ;\text{else}\hfill \end{array}$ (See Figure 1)  
1.1.2  The noise reduction shall be sensitive to modulation frequencies in range [112 Hz]  [1] 
1.2  Attenuation shall be applied in individual subbands  [3] 
1.2.1  The crossover frequencies shall be { List of frequencies }  
1.3  The attenuation by the noise reduction shall be superposed linearly with other attenuations in the system 

The gray blocks compose a functional model of the noise reduction subsystem in the notation of the Simulink^{®} software.

The unfilled blocks indicate the IDs of requirements from Table 1 that are fulfilled by the associated functional blocks.
The functionality of the noise reduction subsystem according to Figure 4 is explained in [3] and will only be briefly summarized here: The block "Filter" extracts subband contents of the input signal for each subband individually and feeds these into block "Compute modulation" to estimate modulation depths according to Equation 2. The block "Compute attenuation" determines attenuation as a function of modulation depth according to Figure 1. This attenuation will be applied in block "Apply attenuation" together with other attenuation in the system, which was zero for all experiments except the last one where it resulted from a transient noise reduction system to be described later. The block "Synchronize" ensures that the signal in the lower signal path is delayed by the group delay of the upper path in order to ensure that the processing in block "Apply attenuation" will be based on correctly timed information.
4.2 Test setup
The setup for performing the designed test consisted of a combination of offtheshelf hardware and software as well as customized computer programs. This section describes each of them.
4.2.1 Infrastructure for test design
An abstract description of test sequences was done using the tool CTE^{®}[33, 34] that supports the earliermentioned CT/ES method. The MATLAB^{®} technical computing environment was used to synthesize binary msequences according to [26]. One of its thirdparty toolboxes, the Frequency Domain Identification Toolbox (FDIDENT, [35]), was used for synthesizing discreteinterval binary sequences based on [23] and multisine signals according to [28].
4.2.2 Infrastructure for test execution
Note that the described test setup differs from the usual condition in which a hearing instrument is worn, because the effect of the human head on the sound field from the sound source is not taken into account. The acoustic effect of the human head in wearing the hearing instrument has thus been neglected here, but it could easily be modeled by putting the hearing instrument under test on an artificial head within the test box.
The test system [36] was implemented using a National Instruments PXI™ system running customized computer programs based on National Instruments LabVIEW. The test system was equipped with a NIPXI 4461 analog input/output card that can play test signals originating from a hard disk, where they have been stored after creating them with the MATLAB^{®} technical computing environment. The signals were presented via a digitaltoanalog converter (D/A) of the input/output card, an audio amplifier (Amp. A) and the loudspeaker of the measurement box (L_{1}), while recording the hearing instrument's output via the measurement microphone (M_{1}), a microphone preamplifier (Amp. B) and an analogtodigital converter (A/D) of the input/output card.
The recorded digital data were stored in a file on a hard disk that could be read by the MATLAB^{®} technical computing environment for further processing. The sampling rate for both playing and recording signals was set to 22,050 Hz. The test system ensured synchronous playback and recording.
4.3 Test design

Input parameters (symbolized by rectangles) are the modulation depths (brief: modulations) in the different subbands, based on requirement 1.1 and 1.2 in Table 1.

Equivalence classes (symbolized by range expressions in square brackets) have been derived from requirement 1.1.1 in Table 1.

Test sequences ("1", "2", "3",...) and test steps ("1.1", "1.2",...) are denoted by short verbal descriptions in the column on the left.

Filled circles on the grid show that a test step should cover a certain equivalence class.

A diagonal straight line between two circles denotes a gradual transition of the used test signal between different equivalence classes.
The circles and their connection lines in Figure 6 are an abstract description of test signals that should be suited for verifying most multiband modulationbased noise reduction systems.
Figure 6 shows two kinds of test sequences: On the one hand, a static test (1) that covers the extreme modulation classes of very low and very high modulation for all subbands, and on the other hand dynamic tests (2 to x, one per subband) that gradually vary modulation within the intermediate modulation range ]M_{1}, M_{2}[. Together, these test sequences achieve sufficient test coverage: since all equivalence classes have at least one circle vertically below them, all equivalence classes are covered by tests.
The abstract test description from Figure 6 should now be mapped to concrete test signals that are used for frequency response measurements based on a suitable measurement technique. Although NLMSbased measurements are a common way of measuring acoustic frequency responses (e.g., [13]), we chose DFTbased measurements, because of the possibility to achieve high time resolution, which were required in one of the experiments. As a consequence, test signals had to be periodic.
Since system tests will acoustically stimulate the system under test, we would theoretically have to describe acoustic signals here, which are in continuous time. However, since the native format of the given test system is a digital waveform, we describe signals in discrete time. All stated sampling rates refer to the test system, not to the system under test.
The parameter φ_{ b }on which the above signal depends via Equation 3 was left variable to allow for experimenting with different values of it.
The test steps from Figure 6 never require more than one subband at a time to have a modulation outside the range [M_{2}, M_{max}]. Using only maximum modulation to cover the equivalence class of that range, one can use the signal θ_{ b }from Equation 7 to establish all test steps from Figure 6, if a suitable modulation of signal σ_{ b }is chosen in the one subband whose modulation falls into another equivalence class (note that for each test step in Figure 6, there is maximum one definition of such a subband).
So far, the described stimuli therefore cover all requirements from Table 1, except number 1.1.2, 1.2.1 and 1.3. These requirements can be covered with a simple measurement approach that does not require an abstract test design. This will be demonstrated in Sect. 4.5.
4.4 Test procedure
For all experiments, the gain in the hearing instrument under test was set 20 dB below the maximum offered value to reduce nonlinearities. Unless stated differently, all adaptive features of the hearing instrument, apart from noise reduction, were turned off for all test runs. The hearing instrument was furthermore configured for linear amplification, this means that there was no dynamic range compression.
Before each experiment, the test system was calibrated using builtin functionality, in order to ensure that transfer characteristics of all equipment in the signal path, particularly the acoustic transducers, were compensated in the digital signal processing of the test system. This ensured that the power spectra encoded in audio files of the input and output signals were equivalent to the acoustic power spectra at the input and output transducers of the device under test.
According to the earliermentioned differential measurement approach, two DFTbased measurements were performed per stimulus: first with the noise reduction subsystem of the hearing instrument switched off, and second while having it switched on.
In using the above equation, measurement samples ${Y}_{k}^{\left(\mathsf{\text{off}}\right)}\left(n\right)=0$ would have been treated as invalid samples and discarded from the result to avoid division by zero, though in practice, such samples did not occur during the experiments that were made.
4.5 Experiments
4.5.1 Verification of crossover frequencies
The signal p was chosen to be a periodically repeated binary msequence of 1,023 samples period length, and the time T that passes between the start of the measurement and the application of the signal p was set to 40s. The inherent assumption of this procedure is: after time T the noise reduction subsystem has settled to steady state and maintains it while the signal s continues, such that a frequency response can be measured, based on the stimulus p. The signal applied before time T contains a pure tone to stimulate the noise reduction subband around frequency f_{0}, added to a modulated signal that is constructed in a way similar to Equation 3. Based on empirical investigation of the procedure, the parameters a and b were chosen such that the pure tone's level was 15 dB higher than the level of the remaining signal components before time T, in order to make the unmodulated signal the dominant stimulus around frequency f_{0}. The modulation frequency was set to f_{ m }= 4Hz, because this is the frequency at which the modulation spectrum of speech has its peak [1].
The frequency f_{0} was varied stepwise within the bandwidth of the system under test. The step width was chosen between 40 and 250Hz, depending on the bandwidth of the noise reduction subbands in the given frequency region.
For each measurement, the signal p(n) was the input signal of the system under test between time T and the end of the measurement, allowing for frequency response measurements. Differential DFTbased measurements were performed to obtain the frequency response of the noise reduction subsystem immediately after time T. Averaging of the data resulting from three subsequent periods of the measurement stimulus was used for obtaining a smoothed frequency response [37]. The measured frequency responses were postprocessed by a human observer: it was necessary to discard duplicate responses of the same subband as well as invalid measurements that were caused by f_{0} inbetween two subbands triggering the noise reduction subsystem in both of them. Afterward, the observer could determine crossover frequencies by graphically intersecting the frequency responses of two adjacent subbands.
4.5.2 Verification of the frequency response for a static modulation pattern
Parametrization of measurements
Measurement number  Choice of parameter φ_{i} as a function of subband number b  Type of signal λ_{ b }and ν_{ b } 

1  ∀_{ i }: φ_{ i }= 0  DIBS 
2  ∀_{ i }: φ_{ i }= 0  multisine 
3  ${\phi}_{i}=\left\{\begin{array}{cc}\hfill \frac{\pi}{2}\hfill & \hfill ;i\ne b\hfill \\ \hfill 0\hfill & \hfill ;i=b\hfill \end{array}\right.$  multisine 
For each measurement, the test stimulus was presented during at least 15 s. Differential measurements of the noise reduction subsystem's frequency response were made, averaging five DFT windows. These windows were taken from the last 5 s of the test run in order to observe the steadystate condition.
Observations: From the measured responses, only the one of Figure 9d has the correct subband attenuation in the sense that it produces the expected 3 dB corner frequencies. Furthermore, Figure 9b shows a side effect, the attenuation in a subband not adjacent to subband number b (as indicated with an arrow on the figure).
4.5.3 Investigation of the side effect in the DIBSbased measurement
Measurement data from different runs of measurement number 1 according to Table 2 with different hearing instruments were analyzed in order to investigate the side effect according to Figure 9b. The hearing instruments had been set to different noise reduction configurations during the measurements. The measurement data were subdivided according to the sensitivity to unmodulated noise of the used noise reduction configuration. The higher the configured first knee point "M_{1}" according to Figure 1, the more sensitivity to unmodulated noise was assumed in the classification. As a result, three classes were obtained: a "Low" class for low sensitivity, a "Med" class for medium sensitivity, and a "High" class for high sensitivity.
Side effects in dependency of modulation sensitivity
Hearing instrument  Percentage of subbands with side effect  

in "Low" class (%)  in "Med" class (%)  in "High" class (%)  
1  0  0  47 
2  0  7  60 
3  0  0  60 
4  0  7  67 
5  0  20  80 
4.5.4 Verification of attenuation with varying modulation
4.5.5 Verification of the superposition with transient noise reduction attenuation
Transient noise reduction systems target a special kind of noise that has been reported to be one reason for annoyance among hearing instrument users [38, 39]: nonspeech transient noises, i.e., signals with a fast change of level over time.
Since transient noise reduction systems should act in addition to traditional noise reduction, they are a good example for the superposition of additional attenuation according to requirement 1.3 from Table 1.
This section proposes a test that addresses the stated requirement. Since the scope of this article is limited to steadystate responses of modulationbased noise reduction subsystems, we will not cover the verification of a transient noise reduction subsystem here. Rather, we show how to test that the traditional noise reduction subsystem has maintained its steadystate performance after a short intervention of a superposed transient noise reduction subsystem.
The challenge here is to insert a transient event into the test stimulus, but still enabling the observation of the frequency response. Although the given test case looks like a good application area for timefrequency approaches like wavelets, we have chosen to stay with the verification techniques that have been presented so far, because we consider the typical mutual exclusion of precise time resolution and precise frequency resolution in timefrequency techniques as a problem for the given test case.
Here $\mathsf{\text{sine}}\left(n\right)=sin\left(2\pi \frac{{f}_{0}}{{f}_{s}}n\right)$; p is a periodically repeated, 1,023 samples binary msequence of 70 dB SPL, n is the sample index, f_{ s }is the sampling rate, f_{0} is the frequency of the stimulating sine (here: f_{0} = 1, 094 Hz), and f_{ m }is the modulation frequency for modulating the background noise, a and b were chosen such that the level of a · sine (n) was 65 dB SPL and was 15 dB higher than the level of $b\cdot \left[1+cos\left(2\pi \frac{{f}_{m}}{{f}_{s}}n\right)\right]\cdot p\left(n\right)$. The modulation frequency was set to f_{ m }= 4 Hz, as earlier. The coefficient c was chosen such that the corresponding sine signal had a level of 90 dB SPL. Furthermore, T_{1} = 40.0s; T_{2} = 40.1s, and the total stimulus duration was 42.1 s.
The stimulus s from Equation 12 uses a sine signal as a basis for stimulating both the noise reduction subsystem as well as the transient noise reduction subsystem: while a steadystate presentation of the sine signal would be sufficient for triggering the noise reduction, the dramatic change of the sine amplitude at time T_{1} is required to trigger the transient noise reduction. One should note that sinebased signals would not necessarily be the optimum stimuli in testing transient noise reduction systems alone. In this case, however, the sine signal was chosen in order to make mainly the behavior of the modulationbased noise reduction subsystem predictable.
A sample hearing instrument with a modulationbased noise reduction subsystem and a transient noise reduction subsystem was used for measurements based on the stimulus s from Equation 12. Differential DFTbased frequency response measurements were performed during the time after T_{2}. They were differential in the sense that both noise reduction and transient noise reduction were disabled for the "off" measurement and were enabled for the "on" measurement.
The tester can verify that the noise reduction subsystem's response in the top subfigure equals the expected steadystate response. The tester can also verify that the envelope of the signal in the bottom subfigure is attenuated compared to cases with an inactive noise reduction subsystem. The additional amount of attenuation shall be exactly the steadystate attenuation of the noise reduction subsystem to pass the test of correct superposition of both subsystems.
4.6 Discussion
The verification of isolated requirements like the subbands' crossover frequencies or the superposition of noise reduction effect with effects of other subsystems could successfully be demonstrated. The core requirements about the modulationbased behavior of the noise reduction subsystem, however, lead to measurements exposing side effects or imprecise results, like shown in Figure 9. Therefore, the results shown in that figure need to be discussed further.
We assume that an inadequate test signal rather than a problem in the system under test explains the side effect that has been marked by an arrow in Figure 9b. Our theory is that the side effect occurs due to the side lobe energy according to Figure 3, which shows the spectrum level of one of the signals ν_{ b }involved in the measurement according to Figure 9b. It can easily be seen that the side effect occurs in the same region where the unwanted side lobe in the power spectrum of the DIBS stimulus is present. An explanation for the side effect is that the unmodulated side lobe energy of the stimulus targeted at subband b enters the subband that spans the frequencies at which the side effect can be observed. Since the stimulus targeted at these frequencies is unmodulated, the sidelobe energy can make the effective signal in that subband less modulated: it disturbs the modulation pattern. According to this theory, particularly noise reduction subsystems configured to have a high sensitivity to unmodulated noise would apply attenuation in the subband corresponding to the frequency range of the side lobe energy, which is consistent with the observations reported in Table 3.
The two white areas in the middle of Figure 12 display a modulation valley; this means a part of the signal in which the cosine in Equation 3 is close to 1. During a modulation valley in subbands other than b, the chirplike signal of subband b has most of its energy close to its upper band limit. Our theory is that this disturbs the modulation pattern of subband b+1 via nonidealities of band split filters in the noise reduction subsystem under test, and, in consequence, leads to application of undesirable attenuation in subband b + 1.
In the measurement according to Figure 9d, the phase shift ∀_{i≠b}: φ_{ i }= π/2 in the modulating cosine has moved the modulation valley to a point in time at which most energy in each subband is present close to the subband's center frequency. This means that, during a modulation valley in most of the subbands, the unmodulated energy of subband b is not dense close to this subband's corner frequencies, where it could have an impact on other subbands. Our theory is that the unmodulated signal in subband b cannot disturb the other subbands' modulation patterns with φ_{ i }= π/2, because at points in time at which the energy in subband b can have an impact on other subbands, these now have modulation peaks, which are only affected in a negligible way if additional energy enters the subband. This can explain why the measurement result according to Figure 9d is closest to the expected response of the subsystem under test, as shown in Figure 9a.
5 Limitations
In order to scope the application range of the presented work, this section discusses the limitations of both the general procedure that has been proposed and the concrete techniques for stimulusbased measurement we demonstrated in the application to noise reduction subsystems of hearing instruments.
5.1 Limitations of the procedure
5.1.1 Limitations of the scope
The presented procedure targets those kinds of signal processing systems whose processing functionality is based on input signals. This means that systems without signal inputs cannot be verified based on the procedure. This would for example disqualify a system for speech synthesis based on text files as a candidate for applying the procedure.
Since the procedure is based on synthetic stimuli, it cannot assess the performance of the system under test in realistic conditions. Therefore it can only be used in monitoring the technical performance of the system, but cannot be used in determining how well the system fulfills user needs.
The application range of synthesized test signals is in general limited to the one system the stimuli have been targeted at, because the procedure does not address the effect of implementation changes in the system under test on the measurement results obtained with the synthetic stimuli. This means that the validity of test signals has to be reassessed on product improvements affecting one or more of the subsystems that contribute to the processing of the test signals in the system under test.
5.1.2 Limitations of the concept
The procedure is based on the uniformity hypothesis and can therefore only produce valid test results if the partitioning of the system's input domain into equivalence classes holds. Especially for systems with highly nonmonotonic inputoutput relationships we expect difficulty in finding suitable equivalence classes.
The procedure also does not guarantee to find test signals for every given system, because it cannot automatically derive test signals from system specifications, meaning that the equations for signal synthesis have to be invented by the test designer.
5.2 Limitations of the demonstrated synthesis and measurement techniques
5.2.1 Limitations of the scope
As mentioned in the introduction, the presented procedure cannot replace clinical trials of a noise reduction subsystem in a hearing instrument. Whereas the procedure is generic enough to support various noise reduction algorithms, the exemplarily demonstrated test signals are based on a modulation frequency of 4 or 5.4 Hz, and are thus only suitable for testing noise reduction algorithms with time constants much greater than 250 or 190 ms, respectively. Furthermore, the used measurement procedure assumes a system that is timeinvariant or has at least reached a steadystate condition in which it behaves like a timeinvariant system. The exemplarily demonstrated tests are thus inappropriate for time varying systems.
5.2.2 Limitations of the concept
Adaptive features other than the one under test may need to be turned off during the measurement, because it is difficult or even impossible to find stimuli that keep all other subsystems in steadystate as a reaction to the stimuli.
Furthermore, the synthesis of test signals is dependent on the design. For example, if crossover frequencies of the demonstrated noise reduction subsystem would change in the course of product innovation, then test signals would need to be resynthesized, based on the new set of crossover frequencies.
6 Conclusion and outlook
We have proposed a procedure for the design of test signals targeted at obtaining test coverage in the measurementbased verification of signal processing systems. The procedure is based on identifying requirements toward the system under test and verifying if they have been met. A main goal is test coverage regarding the input parameters of the system under test. Where achieving test coverage is nontrivial, the procedure foresees the separate steps of first describing abstract test sequences in terms of equivalence classes of input parameters to be covered, and secondly synthesizing concrete measurement stimuli to be used with a particular measurement technique.
The procedure has been explored using the stimulusbased verification of modulationbased noise reduction subsystems in hearing instrument as a sample application. All requirements that had been stated for the sample subsystem could be covered with a test.
The comparison of different stimuli showed that some stimuli are more exposed to producing side effects in certain tests than others. For example, it can be concluded from the results that multisine synthesis procedures are a good basis for the synthesis of stimuli, if their chirplike nature is accounted for in test signal design. On the other hand it can be concluded that DIBSbased stimuli can produce side effects if they are used for narrowband synthesis.
Test signal synthesis was based on the assumption that the noise reduction subsystem under test maintains its steadystate on the one hand during the stimulation with a modulated signal of 4 or 5.4Hz modulation frequency and on the other hand after the stimulation with a pure tone that is replaced by a different signal during the actual measurement. This assumption holds for slow noise reduction subsystems, but certainly not for all known noise reduction concepts.
Even though the proposed verification procedure may require new design of stimuli on implementation changes, its abstract way of describing tests provides flexibility with regard to changing assumptions: if the synthesized stimuli or the chosen measurement technique are no longer suited for a given noise reduction technology (e.g., because it does not provide sufficient linearity or timeinvariance), then the test design can still be used, and only the choice of stimuli and/or measurement technique has to be reconsidered. For example, NLMSbased measurements can be considered if the system under test is timevariant or too nonlinear for the application of the DFTbased measurements we explored here.
Declarations
7 Acknowledgements
The authors would like to thank Mr. Miquel Sans, Bernafon AG, for contributing his knowledge and some indispensable ideas to the verification techniques related to transient noise reduction systems. They also would like to thank The MathWorks, Inc. for supporting their work on this article.
Authors’ Affiliations
References
 Holube I, Hamacher V, Wesselkamp M: Hearing instrumentsnoise reduction strategies. In Proceedings 18th Danavox Symposium: Auditory Models and Nonlinear Hearing Instruments. Kolding, Denmark; 1999:359377.Google Scholar
 Hamacher V, Chalupper J, Eggers J, Fischer E, Kornagel U, Puder H, Rass U: Signal processing in highend hearing aids: state of the art, challenges, and future trends. EURASIP J Appl Signal Process 2005, (18):29152929.Google Scholar
 Schaub A: Digital Hearing Aids. Thieme Medical Publishers, New York; 2008.Google Scholar
 Lamm JG, Berg AK, Glück CG: Synthetic stimuli for the steadystate verification of modulationbased noise reduction systems. EURASIP J Adv Signal Process 2009. 2009Google Scholar
 Lamm JG, Berg AK, Glück CG: Synthetic signals for verifying noise reduction systems in digital hearing instruments. In EUSIPCO 2008: Proceedings of the 16th European Signal Processing Conference. Lausanne, Switzerland; 2008.Google Scholar
 Richardson DJ, Clarke LA: A partition analysis method to increase program reliability. In Proceedings of the 5th International Conference on Software Engineering. IEEE, San Diego; 1981:244253.Google Scholar
 Conrad M: Systematic testing of embedded automotive softwarethe classificationtree method for embedded systems (CTM/ES).In Perspectives of ModelBased Testing, ser Dagstuhl Seminar Proceedings Edited by: E Brinksma, W Grieskamp, J Tretmans. Schloss Dagstuhl, Germany; 2005. no 04371 (Dagstuhl, Germany): Internationales Begegnungs und Forschungszentrum für Informatik (IBFI) [Online] [http://drops.dagstuhl.de/opus/volltexte/2005/325]Google Scholar
 Conrad M: Modellbasierter Test eingebetteter Software im Automobil. TU Berlin, Deutscher Universitätsverlag; 2004. PhD thesisView ArticleGoogle Scholar
 Blume H, Haller M, Botteck M, Theimer W: Perceptual feature based music classificationa DSP perspective for a new type of application. International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation, 2008. SAMOS 2008 2008, 9299.View ArticleGoogle Scholar
 Gierlich HW: New measurement methods for determining the transfer characteristics of telephone terminal equipment. In Proceedings IEEE International Symposium on Circuits and Systems. San Diego; 1992:20692072.Google Scholar
 Hoetink AE, Körössi L, Dreschler WA: Classification of steadystate gain reduction produced by amplitude modulation based noise reduction in digital hearing aids. Int J Audiol 2009, 48: 444455. 10.1080/14992020902725539View ArticleGoogle Scholar
 Bentler R, Chiou LK: Digital noise reduction: an overview. Trends Amplif 2006,10(2):6782.Google Scholar
 Antweiler C, Antweiler M: System identification with perfect sequences based on the NLMS algorithm. Int J Electron Commun 1995,49(3):129134.Google Scholar
 Widrow B, Hoff M: Adaptive switching circuits. IRE WESCON Convention Record, part 4 1960, 96104.Google Scholar
 Nagumo J, Noda A: A learning method for system identification. IEEE Trans Automat Contr 1967,12(3):282287.View ArticleGoogle Scholar
 Barker HA, Davy RW: System identification using pseudorandom signals and the discrete Fourier transform. Proc IEE 1975,122(3):305311. 10.1049/piee.1975.0084Google Scholar
 Nichols ST, Dennis LP: Estimating frequency response function using periodic signals and the FFT. Electron Lett 1971,7(22):662663. 10.1049/el:19710452View ArticleGoogle Scholar
 Bougé L, Choquet N, Fribourg L, Gaudel MC: Test sets generation from algebraic specifications using logic programming. J Syst Softw 1986, 6: 343360. 10.1016/01641212(86)90004XView ArticleGoogle Scholar
 Bahill AT, Dean FF: Handbook of Systems Engineering and Management. Wiley; 2009:205266. ch. 4 Discovering System Requirements)Google Scholar
 Grochtmann M, Grimm K: Classification trees for partition testing. Softw Test Verif Reliab 1993.,3(6382):Google Scholar
 Schroeder MR: Synthesis of low peak factor signals and binary sequences with low autocorrelation. IEEE Trans Inf Theory 1970, 16: 8589. 10.1109/TIT.1970.1054411View ArticleGoogle Scholar
 Müller S, Massarani P: Transferfunction measurements with sweeps. J Audio Eng Soc 2001,49(6):443471.Google Scholar
 Van den Bos A, Krol RG: Synthesis of discreteinterval binary signals with specified Fourier amplitude spectra. Int J Contr 1979,30(5):871884. 10.1080/00207177908922819MathSciNetView ArticleMATHGoogle Scholar
 Paehlike KD, Rake H: Binary multifrequency signalssynthesis and application. In Proceedings of the 5th IFAC Symposium on Identification and System Parameter Estimation. Volume 1. Darmstadt, Germany; 1979:589596.Google Scholar
 Godfrey KR, Tan AH, Barker HA, Chong B: A survey of readily accessible perturbation signals for system identification in the frequency domain. Contr Eng Pract 2005,13(11):13911402. 10.1016/j.conengprac.2004.12.012View ArticleGoogle Scholar
 Buracas GT, Boynton GM: Efficient design of eventrelated fMRI experiments using msequences. Neuroimage 2002,16(3 Part 1):801813.View ArticleGoogle Scholar
 Hartmann WM, Pumplin J: Periodic signals with minimal power fluctuations. J Acoust Soc Am 1991,90(4):19861999. 10.1121/1.401678View ArticleGoogle Scholar
 Van der Ouderaa E, Schoukens J, Renneboog J: Peak factor minimization using a timefrequency domain swapping algorithm. IEEE Trans Instrum Meas 1988,37(1):145147. 10.1109/19.2684View ArticleGoogle Scholar
 Pintelon R, Schoukens J: System Identification: A Frequency Domain Approach. IEEE Press, New York; 2001.View ArticleMATHGoogle Scholar
 Wellstead PE: Pseudonoise test signals and the Fast Fourier Transform. Electron Lett 1975,11(10):202203. 10.1049/el:19750154View ArticleGoogle Scholar
 Ljung L: System Identification. Prentice Hall; 1999.View ArticleMATHGoogle Scholar
 Lamb JD, Rees D: Digital processing of system responses to pseudorandom binary sequences to obtain frequency response characteristics using the fast fourier transform. The Use of Digital Computers in Measurement, IEE Conference Publication 1973, 103: 141146.Google Scholar
 Classification Tree Editor (CTE^{ © })Razorcat Development GmbH, Berlin; [http://www.razorcat.com]
 Mischke J: Classification Tree Editor for Embedded Systems, ISBN 3831118485. Books on Demand GmbH; 2001.Google Scholar
 Kollár I: Frequency domain system identification toolbox V3.3 for Matlab. Gamax Ltd, Budapest; 2004.Google Scholar
 Lamm J, Glück C, R Wey: Effiziente Verifikation mit hörbarem Erfolg. Swiss Eng stz 2010, 04/2010: 2223.Google Scholar
 Bendat JS, Piersol AG: Random DataAnalysis and Measurement Procedures. Wiley; 2000.MATHGoogle Scholar
 Chalupper J, Powers TA: New algorithm is designed to take the annoyance out of transient noise. Hear J 2007,60(7):42,4446,48.View ArticleGoogle Scholar
 Keidser G, O'Brien A, Latzel M, Convery E: Evaluation of a noisereduction algorithm that targets nonspeech transient sounds. Hear J 2007.,60(2): 29,32,34,3839Google Scholar
 Oppenheim AV: Speech spectrograms using the fast Fourier transform. IEEE Spectr 1970,7(8):5762.View ArticleGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.