Research on structural sound source localization method by neural network

Abstract

To solve problems related to much calculation to adapt to complex scenes in traditional structural sound source localization, this paper proposes a method based on neural network. The structural sound source at other positions was stimulated by successively striking 36 grid centers on the surface of the plate. The time delay between different accelerometer signals was considered as the input, and the location of the predicted sound source was considered as the output. The influence of the number of test sets and epoch training times on sound source localization accuracy was discussed. These results show that with the increase in the epoch training times, the number of test set decreases, and the number of training set increases, increasing the sound source localization accuracy of backpropagation neural network. However, these error conditions will frequently appear due to the overfitting phenomenon. When the epoch is trained to 50,000 times, and the quantity of the test set is 4, the backpropagation neural network has the best localization accuracy with an order of magnitude of 10⁻³ in error, and the localization error scope of the plate is between 0.01 and 0.1 m.

1 Introduction

Sound source localization is an indispensable part of structural health monitoring research. A structural health monitoring system has been established for large mechanical equipment and platforms to monitor whether the structure is abnormal. Therefore, it is crucial to diagnose and repair platform equipment and mechanical facilities to accurately identify and locate the anomalous sound source of the monitored system. For example, spacecraft in orbit may collide with space debris and need to lock the impact position as fast as possible. Therefore, it is imperative to realize the fast and accurate positioning of the impact position for spacecraft safety [1]. The loose parts monitoring system (LPMS) [2] applied in nuclear power plants is used to monitor the screws, pins, and other parts of the primary circuit of nuclear reactors to avoid the degradation of reactor safety performance.

There are three traditional sound source location methods: calculating arrival time difference [3, 4], the circle intersection location method [5], and the scanning legal location method [6]. It is necessary to measure the time difference between sensor signals accurately and then determine the location of the collision source by the wave velocity and time difference. The time difference is estimated mainly by the time domain root mean square method, generalized cross-correlation method, and Hilbert envelope peak method. Among them, the generalized cross-correlation method (GCC) is one of the most commonly used delay estimation algorithms. It is widely used in sound source location in the air [7]. Li et al. [8] used the improved generalized cross-correlation method to realize the positioning of airborne sound source signals under the condition of low SNR. Kocur et al. [9] obtained the distribution positions of abnormal sound sources by X-ray tomography scanning through time reverse modeling based on the positioning method of wave propagation in structures. Gibbons et al. [10] proposed the spectral ratio method, which judges difference in wave properties according to the waveform data on arrival of the sound source signal to locate the position of sound source. Marxim and Mohanty [11] combined the empirical mode method and generalized cross-correlation method to improve the delay estimation accuracy of underwater acoustic signals and the accuracy of sound source positioning. Unlike air and water media, wave propagation in solid structures is more complex and changeable. The stress wave generated after collision includes the shear wave, longitudinal wave, bending wave, and other components. The combination of multiple waves will affect accurate estimation of the time delay. Among them, the curved wave is the main component with high amplitude and dispersion. Yang et al. [12] used the first peek of the Hilbert envelope as the signal arrival point to calculate the time delay between different sensors when tapping the scale. Ji et al. [13] combined wavelet denoising with the Hilbert envelope algorithm to achieve collision location on the plate. Saragiotis [14] proposed a sound source location method based on the sensor position and wave propagation speed based on the arrival time difference and time measured simultaneously. Gollob and Kocur [15] located the signal position of the sound source by analyzing the accurate wave velocity distribution and propagation path.

In addition, some researchers have proposed a new method of sound source location based on the wave propagation law. The beamforming method [16] and others can be applied to locate sound sources in three-dimensional space, which is more complex than that in a two-dimensional plane structure. Zhou et al. [17] established the space–time relation equation between the position of the sound source and the measurement point based on the sensor coordinates, wave arrival time and wave speed, and other information in the case of known wave propagation velocity (for uniform media) and constructed a set of three-dimensional sound source positioning methods suitable for consistent press. Bi et al. [18] proposed a plan combining time-reversal analysis with an equivalent sound source method with the help of a spherical array composed of 48 microphones. The results show that this method can locate sound sources in an arbitrarily shaped shell and has a good localization effect for sound sources with frequencies above 1000 Hz. Tsangouri et al. [19] formed three-dimensional concrete beam cracks by large-scale production in the laboratory, evaluated the location of possible sound sources, and found that the location results were significantly affected by the characteristics of the concrete structure. Gollob and Kocur [20] proposed a multipath method of sound source location analysis based on a nonuniform wave velocity model given the complex propagation characteristics of waves in concrete structures. This method achieves the expected positioning accuracy, but a significant wave path deviation is caused by cracks and large gaps in the concrete, which leads to an increase in the positioning error.

In the above ideas of sound source location, the result of sound source location in the structure is often affected by the calculation accuracy of time delay estimation, wave velocity, and other variables in the structured medium. Under the condition of low SNR, it is difficult to measure the signal delay of the sound source accurately, and the propagation law of different wave types in the structure is very different. Due to appearing of the dispersion phenomenon, the wave propagation velocity is different for other frequency signals, affecting the location accuracy of collision sources. In addition, when the elastic wave propagates in the structure, its velocity is also affected by the thickness of the flexible body, material properties, and other factors. Because the propagation law of waves has not been wholly clarified, the technical route of sound source location based on the transmission characteristics of structural waves has not been constituted. In recent years, application of neural network in environmental monitoring, seismic research, environmental monitoring, and other fields has been widely concerned. Research on structural sound source localization method by neural network has gradually been developed. Wu et al. [21] used backpropagation neural network for damage location detection of building structures. Povich and Lim [22] used three-layer backpropagation network to identify damage location of a 20-span plane truss. Boffa et al. [23] used neural network to locate impact events of aerospace structural components. It mainly has analyzed birds’ impact, tools falling, and debris impact generating vibration and impact in different modal frequency range. Also, authors discussed various combination influence of experimental sensors arrangement on positioning results. These results are verified accuracy of neural network prediction and can avoid solving wave velocity in different frequency. Worden and Staszewski [24] applied neural network to determine location of acoustic source from structural response recorded by sensors attached to surface. By this method, authors successfully find location of acoustic source generated under this impact. It also effectively improves prediction accuracy of acoustic source location.

From the perspective of a neural network, through the network layer of the training setup between the sensor signal eigenvalue and the mapping relationship between the positioning location (characteristic values of the input layer node corresponding to the eigenvalue of the sensor signal and the output layer node corresponding to estimated positioning point), this article effectively avoids the time delay and the complex calculation of wave velocity between these two variables. Finally, the sensor input characteristic signal to the location of the prediction point has a good data training effect.

2 Establishment of experimental model

2.1 Approach to solving problems

Because of the complex problem of structural sound source location, because of the influence of many factors, and because most of these factors have different degrees of nonlinear correlation, the application of neural networks can deal with this kind of problem well. Among them, the error backpropagation neural network has a solid non-mapping ability, which can be used to predict the structural sound source location. In this paper, the elastic wave transmission path, using the transmission rule in tablets as a “black box,” is determined from the angle of the artificial neural network by analyzing the relation between the input signal and output signal data characteristics and finding the rule of the signal connection structure to realize sound source localization to solve the problem of beamformer structures.

2.2 Introduction of backpropagation neural network

The backpropagation neural network is a widely used neural network model. It trains multilayer feedforward neural networks using the error backpropagation algorithm based on the error backpropagation principle. The backpropagation neural network changes the traditional network structure and introduces new layering and logic. The learning and training process is divided into forwarding computation and backward computation.

In the process of forwarding computation propagation, the input is transmitted from the input layer to the output layer after processing of hidden layer neurons, and each layer of neurons only affects the computational state of the next layer of neurons. The final output value and the error between the output value and the actual value can be obtained by calculating in network layers. If error between the final output result and the real value exceeds the allowable range, the back-calculation propagation will process. At this point, the error value will propagate from the output layer to the input layer. By apportioning the error to each network unit, the signal between each layer unit can be obtained. Meanwhile, the connection weight and bias value between network layers can be updated according to the error signal. The propagation process will be repeated alternately until the global mistake of the neural network tends to a given minimum value. In repeated training, the backpropagation neural network adopts the gradient descent method to make the weight change along the negative gradient direction of the error function and converge to the point of minimum value. Finally, the training result is close to the actual value [25].

• (1) Forward calculation of the propagation process [26]. During forward computation propagation, x_i represents the input signal value, w_i is the connection weight, and b_i represents the offset. Therefore, the output value h_i of a neuron is used, which is equal to its input value x_i. The connection weight is as following Eqs. (1) and (2):

  $${\text{Net}}_{i} = \sum\limits_{i = 1}^{n} {\left( {x_{i} w_{i} + b_{i} } \right)}$$

  (1)
  $$h_{i} = f\left( {{\text{Net}}_{i} } \right)$$

  (2)

In the above Eq. (3), if f uses the hyperbolic tangent function as the activation function, then:

$$f\left( {{\text{Net}}_{i} } \right) = \frac{{e^{{Net_{i} }} - e^{{ - Net_{i} }} }}{{e^{{Net_{i} }} + e^{{ - Net_{i} }} }}$$

(3)

In the output layer, the output layer unit generally uses the linear function (rectified linear activation) as the activation function. Therefore, the output values are equal to the weighted sum of the input values. As shown in Eq. (4), for the output, its value is y_m:

$$y_{m} = \mathop \sum \limits_{j} \left( {h_{j} v_{mj} } \right)$$

(4)

• (2) Reverse calculation of the propagation process. First, the error between the actual value and the expected output value of the network is calculated, and the root means square function MSE is used as the error function. Then, the error e is as following Eq. (5):

  $$e = \frac{1}{2}\sum\limits_{m} {\left( {y^{{\prime }} - \widetilde{y}} \right)}^{2}$$

  (5)

The backpropagation neural network uses the gradient descent method to reduce the weight. The partial derivative of each neuron unit in the output layer is calculated using the error value. The weight adjustment amount of each time is as following Eq. (6):

$$\Delta w_{mj} = - \mu \frac{\partial e}{{\partial w_{mj} }} = - \mu \frac{\partial e}{{\partial y_{mj} }}\frac{{\partial y_{mj} }}{{\partial w_{mj} }}$$

(6)

In Eq. (6), μ is the learning rate, and 0 < μ < 1. According to Eq. (6), the error value of each neuron in the output layer and the output value of each neuron in the hidden layer can be used to modify the connection weight. After calculating the error term of each layer, backpropagation can be carried out to obtain the error of each layer again. Meanwhile, the partial derivative of parameters of each layer is calculated, and the parameter values are updated to reduce the absolute error.

2.3 Establishing a neural network model

Based on the above process, this paper constructs backpropagation neural network structure shown in Fig. 1. Since input data contain seven components and output data contain two components, the number of input layers of constructed backpropagation neural network is seven, the number of output layers is two, respectively. Input parameter is time difference of arrival corresponding to peak value of wave, and output parameter is predicted plate position coordinate. The hidden layer contains several neurons. Through continuous training and debugging of neural network, it is shown that the middle-hidden layer is set as one layer, and the hidden layer contains one hundred hidden units. The hidden layer makes use of the S-shaped activation function with hyperbolic tangent function. The output layer uses the linear activation function with rectified linear activation function. The learning rate could change with the increase in iterations changeable manually. The specific formula is 0.9999 ^ (epoch/2). The loss index evaluated function used in backpropagation neural network model is the root mean square error function (MSE).

Fig. 1

Backpropagation neural network structure model

3 Experiment of structural sound source localization based on backpropagation neural network

3.1 Arrangement of experiment

An acceleration sensor mounted on the flat plate is used to receive the signal of striking the flat plate.

The experimental platform of the flat plate experiment consists of the test object, acceleration sensor, data acquisition equipment, and computer. The empirical thing is a Q235 steel plate. The size of the steel plate is 1.3 m × 0.8 m × 0.008 m, and rubber pads are installed at the four edges of the steel plate to buffer and isolate vibration. In Fig. 2, it shows the connection and site layout of the experimental device. To avoid the reflection of the boundary, 36 grid areas were drawn in the middle of the steel plate with a square size of 0.1 m × 0.1 m, and the center of each grid was hammered with a strong hammer. The grid number and acceleration sensor position are shown in Fig. 3, and the data sampling rate is 65536 Hz.

Fig. 2

Experimental device connection and site layout

Fig. 3

Schematic diagram of flat plate meshing

3.2 Procedure of the experiment

The specific steps of the sound source positioning experiment are as follows:

• (1) Grid division: According to positioning accuracy requirements, grids are divided on the impacted object and numbered. In the experiment, the grids are divided into 0.1 m × 0.1 m.

• (2) Data calibration: The calibration data were obtained by the hammer percussion experiment, and unknown collision signals of different percussion points were obtained by tapping the center point of each grid.

• (3) Data processing: The Hilbert envelope method was used to obtain the peak times of the collision signal to sensors #1, #2, #3, #4, #5, #6, #7, and #8, which were recorded as t₁, t₂, t₃, t₄, t₅, t₆, t₇, and t₈, respectively.

• (4) Input data determination: The arrival time difference t of the signal characteristic peak values of sensors in different channels is calculated, respectively, as t₁₂, t₂₃, t₃₄, t₄₅, t₅₆, t₆₇, and t₇₈ (where t₁₂ = t₁–t₂, t₂₃ = t₂–t₃, t₃₄ = t₃–t₄, t₄₅ = t₄–t₅, t₅₆ = t₅–t₆, t₆₇ = t₆–t₇, and t₇₈ = t₇–t₈).

• (5) Output data determination: (X_k, Y_k) is used to represent the grid coordinates of the tap (where k = 1,2…, 36).

4 Results and analysis of experiment

The training data can be obtained by tapping the grid on the plate. The relationship between the predicted position and the actual tap point position is shown in Tables 1, 2 and 3.

Table 1 Loss errors of different epochs when the number of test sets was 10

Table 2 Loss function diagram of different number of test sets when the epoch was 50,000

Table 3 Predicted location coordinates

4.1 Error analysis

By constantly adjusting the number of test sets and the times of epoch training, different loss function images can be obtained, as shown in Figs. 4 and 5. In Fig. 4, it shows the trend of the training error and test error changing with the epoch times when the test set was 10, and Fig. 5 shows the curves of the training error and test error in different test sets when the number of epochs was 50,000.

Fig. 4

Loss function diagram of different epochs when the test set was 10

Fig. 5

Variation curves of final training error and test error on different epochs when the test set was 10

4.1.1 Optimization of epoch

As seen from Fig. 4, during the first 150 epochs of training, the descending gradient of the training error and test error was pronounced, which decreased from 1.14e−1 to approximately 3.60e−2. When the epoch training times changed from 150 to 200 times, the error decreased slightly from 3.60e−2 to 2.94e−2. When the epoch training times changed from 200 to 5000 times, the training error and test error did not change, and the error value changed from 2.90e−2 to near 2.60e−2. When the epoch times increased from 5000 to 10,000, the training error and test error values decreased gently from 2.60e−2 to approximately 1e−2. When the number of epochs risen from 10,000 to 50,000, the error decreased. When the number of epochs increased from 10,000 to 20,000, the error value decreased from 1e−2 to approximately 6e−3, showing an apparent decreasing trend of loss error. When the epoch times risen from 20,000 to 50,000, the error value decreased from 6e−3 to about 5e−3, and the loss error trend was not noticeable. When the number of epochs increased from 50,000 to 100,000, the training error decreased from 4e−3 to approximately 1e−3. However, it was evident that when the number of epochs increased from 60,000 to 100,000, the test error risen to 6.82e−3. When the epoch times increased from 100,000 to 200,000, the curve of the training error and test error changed a little, and the final training error and test error were 1.49e−3 and 5.71e−3, respectively.

According to the images of the training error and test error given in Fig. 4, the final training error and test error values under different epoch training times can be obtained when the number of test sets is 10, as shown in Table 1.

The final loss error data given in Table 1 were used to draw the absolute training error and test error change curves under different numbers of epochs when the number of test sets was 10. As seen from Fig. 5, with the increase in epoch times, the loss error curve showed a downward trend overall, and the changes in the test error and training error were consistent. The change curve was mainly divided into three stages: rapid decline, slow decline, and gentle fluctuation. When the number of epochs increased from 100 to 10,000, the loss error decreased rapidly. When the number of epochs varied from 10,000 to 50,000, the decreasing trend of loss error became slow. When the number of epochs increased from 50,000 to 200,000, the loss error curve showed fluctuated characteristics. This is because when the epoch training times reached an enormous value (100,000 and 200,000), the loss error was in a small range. At this point, the backpropagation neural network model is prone to overfitting when the epoch training times increase, resulting in increased training errors and test errors. Therefore, when the number of test sets was ten, and the number of epochs was 50,000, the training error and test error were small.

4.1.2 Optimization of the number of test sets

As seen in Fig. 6, when the number of training iterations is 50,000, the final training error and test error will change as the number of test sets changes.

Fig. 6

Loss functions of different number of test sets when the epoch was 50,000

The curve's trend in Fig. 6 shows that the training error and test error decreased significantly during the first 150 epochs of training. When the epoch training times changed from 150 times to 15,000 times, the loss error curve entered the stage of gentle fluctuation and slow decline. In the platform stage, the loss error hardly changed. In the slow descent stage, the loss error decreased slowly. When the epoch training times increased from 15,000 to 50,000 times, the change degree of the loss error was also small, but when the test set was minimal, the final loss error increased.

According to the images of the training error and test error given in Fig. 6, the loss function curves under different test sets can be obtained when the number of epochs is 50,000, as shown in Table 2.

Using the final loss error data given in Table 2, the loss error curves of different numbers of test sets were drawn when the number of epochs was 50,000. It can be seen from Fig. 7 that the test error is higher than the training error. When the number of test sets changes from 1 to 2, the training error remains the same, the training error value is approximately 4.1e−3, and the test error increases from 5.14e−3 to 8.97e−3. When the number of test sets changes from 2 to 3, the training error and test error decrease. When the number of test sets changed from 3 to 7, the training error increased slightly but remained near 2e−3. The test error showed a trend of gradual increase from 3.77e−3 to 8.02e−3. When the number of test sets changes from 7 to 8, the training error increases from 2.62e−3 to 4.18e−3, and the test error decreases from 8.02e−3 to 4.89e−3. When the number of test sets changes from 8 to 10, the training error first slips and then increases, and the training error finally reaches 3.86e−3. The test error first increases and then falls, and the test error finally comes to 5.69e−3.

Fig. 7

Loss functions of different number of test sets when the epoch was 50,000

According to the above analysis, it can be seen that when the epoch training times were fixed at 50,000 and the number of test sets was 4, the test errors and training errors were minor. The predicted coordinates were close to the coordinates of the actual sound source position.

4.2 Analysis of localization results

Through the learning and training of the backpropagation neural network model mentioned above, the coordinates of the sound source location predicted by the neural network can be obtained when the number of epochs is 50,000 and the number of test sets is 4, as shown in Table 3.

Table 3 shows the fundamental error values between the predicted position and the actual sound source position. The fundamental minimum error is 0.01 m, the fundamental maximum error is 0.10 m, and the fundamental average error is 0.05 m. Using backpropagation neural network training, the positioning error accuracy can be obtained, and the positioning error is less than 0.10 m. The error distribution diagram between the predicted position and the actual sound source position is shown in Fig. 8.

Fig. 8

Error distribution between the predicted position and the sound source position

Overall, the error between the predicted position and the actual position of the sound source is between 0.01 and 0.10 m, and the error range is less than one grid size. The positioning accuracy of the sound source in the structure is high.

5 Conclusions

In this paper, a backpropagation neural network is designed, the delay difference signal between acceleration sensors is used as the input, and the number of input units is 7. Specific conclusions are as following:

• (1) With the increase in the neural network epoch training times, the loss error of the backpropagation neural network decreases overall. The loss error curve can be divided into three stages: rapid decline, slow decline, and delicate fluctuation. After entering the gentle fluctuation stage, increasing the epoch training times is prone to overfitting, which leads to an increase in the training error and test error value of the neural network model.

• (2) In the case of small samples, the decrease in the number of test sets and the increase in the number of training sets can reduce the training error, improve the training effect of the backpropagation neural network, and improve the positioning accuracy. However, when the number of test sets is small (1 or 2), with the increase in the epoch training times, the backpropagation neural network model will show extensive error results due to overfitting.

• (3) When the epoch training times was 50,000 and the number of test sets was 4, the backpropagation neural network achieved the highest positioning accuracy level, with an order of magnitude of e−3 in error, and the positioning error range of each grid on the plate ranged from 0.01 to 0.10 m.

• (4) Although the data used for training and testing in this paper are small, a backpropagation neural network is used to ensure the prediction accuracy of the location coordinates of the knock points based on the adjustment of the epoch training times and test sets, thus achieving better positioning of the structural sound sources, which has excellent application and research value.

• (5) This paper mainly discusses the influence of some parameters in neural network structure on positioning results, such as the number of iterations, the number of test set. From previous analysis, higher iteration times play a positive role in positioning accuracy with neural network structure model in this paper. A larger sample dataset can be better verified accuracy, portability, and stability of this neural network model in the paper.

Reduction of signal-to-noise ratio (SNR) with increase in background noise, it will lead to increase in acoustic source location error. For example, Boffa et al. [23] studied the influence of noise on signals. In this paper, by manually adding zero-mean Gaussian noise to normal signal and repeating above positioning steps, it is found that positioning results in low signal-to-noise ratio environment will gradually deteriorate with continuous increase in noise level. Therefore, it can be concluded that although this methodology is applicable to noise pollution signals with relatively high signal-to-noise ratio, it is necessary to filter noise when the signal-to-noise ratio is lower than a certain level. Under the condition of low signal-to-noise ratio, it is difficult to accurately measure time delay of sound source signal, and further research is needed.

In addition, similar existing work is compared with this paper. She et al. [25] proposed an acoustic source location method based on backpropagation neural network. Under time difference of receiving indoor sound as the input, acoustic source location was as the output. Experimental results show that backpropagation neural network model with seven neurons and three hidden layers is more suitable for this research scenario, and accuracy of location can reach a high level. Authors in this document have studied indoor acoustic source location problems. The document is similar to background of this article. With time difference of arrival of wave crest as input parameter, it is mainly aimed at the flat structure. Results of structural sound source location are analyzed to achieve better positioning accuracy.

Sheng [27] designed acoustic source location model based on generalized cross-correlation and convolution neural network. Through extracting feature vectors of different dimensions from four-channel acoustic source signal, authors design five-layer, nine-layer, eleven-layer, and thirteen-layer dual-channel convolution neural network structure with different depths for experimental verification, it is shown that the nine-layer network structure has better accuracy of acoustic source event detection and location estimation compared with other three networks. Compared with this document, this structure of neural network designed in the paper is simpler and more convenient. Also, better positioning effect can be achieved.