Pavement crack characteristic detection based on sparse representation

Abstract

Pavement crack detection plays an important role in pavement maintaining and management. The three-dimensional (3D) pavement crack detection technique based on laser is a recent trend due to its ability of discriminating dark areas, which are not caused by pavement distress such as tire marks, oil spills and shadows. In the field of 3D pavement crack detection, the most important thing is the accurate extraction of cracks in individual pavement profile without destroying pavement profile. So after analyzing the pavement profile signal characteristics and the changeability of pavement crack characteristics, a new method based on the sparse representation is developed to decompose pavement profile signal into a summation of the mainly pavement profile and cracks. Based on the characteristics of the pavement profile signal and crack, the mixed dictionary is constructed with an over-complete exponential function and an over-complete trapezoidal membership function, and the signal is separated by learning in this mixed dictionary with a matching pursuit algorithm. Some experiments were conducted and promising results were obtained, showing that we can detect the pavement crack efficiently and achieve a good separation of crack from pavement profile without destroying pavement profile.

Introduction

In the life cycle of pavement, there will be various pavement distresses due to the burden of vehicles and natural causes. The pavement distress will affect the lifespan of the pavement, vehicles energy assumption, transportation efficiency, and the transportation safety [1]. Among the various pavement distresses, pavement cracking data is the most important element for quantifying the condition of pavement surface [2, 3], so it is crucial to detect and recognize the pavement cracking automatically and accurately before repairing them.

There are some methods for detecting pavement cracking. The traditional method is to detect by human vision, but manual surface distress survey are subject to many limitation such as non-repeatability, subjectivity, and high personal costs [4]. Manual procedures are time-consuming, and present substantial differences between evaluations of different raters [5]. In recent years, several 2D image analysis-based pavement crack detection techniques were proposed [6–10]. One major issue with pure 2D video-based systems is their inability to discriminate dark areas not caused by pavement distress such as tire marks, oil spills, shadows, and recent fillings [11]. Moreover, the shadows and poor illumination are also major problems for daytime operation though they can be overcome using additional lighting systems or by acquiring data in the night after sunset [12].

Pavement crack detection technique based on the 3D laser is a recent trend. The measurement principle of pavement crack detection based on the 3D laser technique is shown in Figure 1. A laser stripe is emitted from a structured light source, and is projected onto the detected object. Since the surface of the detected object is not on the plane of the detection platform, the stripes from the lamp house will produce a deformed line on the across the detected object. By using the CCD camera to capture the deformed line, we can obtain the contour of the detected object by analyzing the deformed line [13]. In the practical application of pavement detection, the stripes from light source will deform after going through the pavement cracks (Figure 2), thus the 3D information in both x and y dimensions (surface) and z dimensions (depth) of the crack can be analyzed based on these deformed stripes. The advantage of using 3D laser technique is that it is insensitive to the disturbance such as tire marks, oil spills, and shadows. A crack with some shadows is shown in Figure 3, which contains some shadows, tire marks, and one transverse crack marked in green box. The longitudinal laser stripe deforms across transverse crack and the deformation part is marked in green Oval box. From Figure 3, it is obvious that the 3D laser based technique can effectively detect the crack accurately in case of shadows.

Figure 1

Illustration of 3D laser detection system.

Figure 2

Pavement scanned by laser stripes. ( a) the stripes from the light source deform across the pavement cracks; ( b) the corresponding profile from picture (a), the elevation coordinate for each profile is relative in this Figure.

Figure 3

Pavement scanned by laser stripes with shadows.

Since each laser profile acquired by 3D laser detection system has its own characteristic in terms of profile shape, crack shape, the number of cracks and signal-to-noise ratio, the signal processing techniques are adapted to each laser profile to extract the features of the crack. As the technology protection, few literatures describe algorithms in detail in this area. Bursanescu used a special filter to avoid the noise and extract the crack. The filter uses an adaptive width mobile widow, and width is self-adjusting. We cannot find more details in his papers [5, 11, 14]. Laurent introduced the algorithm for the detection of cracks, which is the valley detection of candidate cracks in the individual pavement profiles [15–17]. This simple technique is fast and easy to implement, but it cannot achieve a good separation of crack from pavement profile.

For profile signal, the main profile signal is varying slowly, which spreads over the whole observing period; Crack signal has sharp edges and performs different shapes, which belongs to narrow-scale signal. Wavelets can detect the location of cracks accurately due to its good time-frequency characteristic, but the wavelet base is fixed which cannot match the crack shape well. So after analyzing the characteristics of main profile and cracking, we constructed a mixed over-complete dictionary according to main profile and crack characteristics and proposed a novel method based on sparse representation for crack detection and the mainly pavement profile extraction without noise. Some experiments were conducted and promising results were obtained, showing that we can detect the pavement crack accurately and achieve a good separation of crack from pavement profile.

Pavement crack and profile detection based on sparse representation

Sparse representation of signals

Sparse representation of signals has become a major area of research in the last few years [18, 19]. Using an over-complete dictionary matrix D ∈ R^{n× K} that contains K atoms, { d _j } _{j = 1} ^K , as its columns, it is assumed that a signal y ∈ Rⁿ can be represented as a sparse linear combination of these atoms. The representation of y may either be exact y = Dx, or approximate, y ≈ Dx, satisfying ‖ y − Dx‖₂ ≤ ϵ. The vector x ∈ R _K displays the representation coefficients of the signal y. This, sparest representation, is the solution of either

$$\underset{x}{min}{‖x‖}_{0}subjecttoy=Dx$$

(1)

or

$$\underset{x}{min}{‖x‖}_{0}subjectto\Vert y-Dx{\Vert }_2\le ϵ$$

(2)

where ‖.‖₀ is the l⁰ norm , counting the non zero entries of a vector [20].

For our profile signal f contains crack signal s _C and main profile signal s _Prof two layers as a linear combination, we propose to seek the sparsest of all representation over the mixed dictionary. Thus we need to solve

$$\left\{x_c^{\mathit{opt}},x_p^{\mathit{opt}}\right\}=\underset{\left\{x_c^{\mathit{opt}},x_p^{\mathit{opt}}\right\}}{Arg}min\left({‖x_c‖}_0+{‖x_p‖}_0\right)subjectto:={\Phi }_c{x}_c+{\Phi }_p{x}_p$$

(3)

where Φ _c is the crack dictionary, Φ _p is the main profile dictionary, x _c and x _p are the coefficients in the corresponding dictionaries.

Characteristics of main profile signal and crack signal

The profile signal acquired by 3D laser detection technology as shown in Figure 4.can be expressed as follows:

$$f=s_R+s_C+s_{\mathit{bump}}+s_{\mathrm{Prot}}+s_{\mathrm{Prof}}+s_n$$

(4)

where f is profile signal, s _C is crack signal, s _Prof is main profile signal, s _n is noise. where f is profile signal, s_Prof is the main profile signal, s _R , s _C ,s _bump ,s_Prot , and s _n represent the rut signal, crack signal, bump signal, pothole signal, and noise, respectively. This paper mainly studies the characteristics of cracks and main profiles and how the crack to be separated from the main profile.

Figure 4

Pavement profile signal.

Since longitudinal main profile signal s _Prof is used to calculate the international roughness index (IRI), it should not contain distress and noise. Therefore, it is necessary to analyze the characteristics of s _C and s _Prof and separate s _C from f without destroying s _Prof .

s _C has the following characteristics: clear and sharp edges, a direction below the horizontal surface, different narrow-scale shapes. In order to calculate the crack width and location accurately and achieve a good match with the shape of crack, we observed a large amount of crack data. Figure 5 clearly shows the basic shapes of crack, while most cracks perform asymmetrical form of these basic shapes due to the rain erosion, sand filling, etc.

Figure 5

The basic shapes of crack.

s _Prof is a low-frequency curve as the red curve shown in Figure 6, spreads over all the observing periods and performs to be the profile of pavement without distress and noise.

Figure 6

Red curve is the s _rof of real profile signal.

Mixed over-complete dictionary

The success of sparse representation application depends on how to pick the suitable dictionary which is employed to sparsely describe the signal. Based on the difference between s _C and s _Prof , it is possible to detect s _C and s _Prof by performing two different transformations. That is to say, we try to find two dictionaries Φ _c and Φ _p in line with the real signal to separate the s _C and s _Prof from f .

The four-point curve is a function of a vector x , which depends on four scalar parameters a , b , c , and d . As shown in Figure 7, this function is flexible, which can construct different shapes of the cracks using four-point transform. The mathematical function model of the four-point curve is represented as follow:

$$f\left(x,a,b,c,d\right)=\left\{\begin{array}{l}0,x\le a\\ \frac{x-a}{b-a},a\le x\le b\\ 1,b\le x\le c\\ \frac{d-x}{d-c},c\le x\le d\\ 0,d\le x\end{array}\right\}$$

(5)

Figure 7

Atoms in four-point function over-complete dictionary. The y-axis is the normalized amplitude.

Certainly, we construct the over-complete dictionary Φ _c with four-point curve function which can efficiently express the sharp edges and diversity of crack. The parameters a and d locate the “feet” of the trapezoid and the parameters b and c locate the “shoulders”. As for our work in this article, the scales ( d – a) of the four-point curve range from 1 to 5 and the shifts are densely sampled from 0 to L – 1 for each scale, where L is the length of signal.

As shown in Figures 6 and 8a, exponential function with large-scale can construct main profile. The mathematical function model of the exponential function could be represented as follow:

$${\phi }_k\left(t\right)=A_{k}exp\left(-{\left(\frac{t-m_k}{n_k}\right)}^2\right)k=0,1,2\text{,}$$

(6)

where m is the position information of exponential function, n is the scale information, A is a normalization factor. When m and n change in different areas separately, we can have different profiles. As for our work, m ranges from 0 to L – 1, where L is the length of signal, and the n ranges from 1 to 800.

Figure 8

The framework of the algorithm used in this paper.

Finally, the mixed over-complete dictionary Φ is composed of exponential function Φ _p and trapezoidal membership function Φ _c , which can be denoted by Φ=Φ _p + Φ _c , and all the atoms in over-complete dictionary are normalized.

Signal separation by the matching pursuit method

The matching pursuit (MP) algorithm [21] iteratively projects a signal onto a given dictionary and chooses the dictionary atom that best matches the signal in each iteration. We use this algorithm to decompose signal into sparse expression iteratively in the mixed over-complete dictionary Φ. Figure.9 shows the framework of the algorithm based on MP used in this article. The more detailed algorithm is given below:

• 1. Initialization: Set k=1, S⁽⁰⁾=0,R⁽⁰⁾=S, x _ck =0, where k is the number of iteration, S is the profile signal to be decomposed, R is the residual signal during the iterations; the superscript is the iteration number; x _ck is the coefficients in Φ _c ;δ_{c min} and δ_{c max} are the threshold of inner product between residual signal and each atom in crack dictionary Φ _c ;δ _p is the threshold of inner product between residual signal and each atom in crack dictionary Φ _p

• 2. Crack feature extraction with MP:

  • 2.1 -find the atom in Φ _c with maximum inner product in each iteration, i.e. $\left|〈R^{\left(k-1\right)},{\Phi }_{\mathit{cj}}〉\right|=sup\left|〈R^{\left(k-1\right)},{\Phi }_c〉\right|$, $x_{\mathit{ck}}=ma{x}_j\left|〈R^{\left(k-1\right)},{\Phi }_{\mathit{cj}}〉\right|$ where Φ _cj is the j th atom in Φ _c ,x _ck is the coefficients.

  • 2.2 -If ${\delta }_{min}\le \left|〈R^{\left(k-1\right)},{\Phi }_{\mathit{cj}}〉\right|\le {\delta }_{max}$, $S^k=S^{\left(k-1\right)}+x_{\mathit{ck}}{\Phi }_{\mathit{ck}}$, $R^{\left(k\right)}=S-S^{\left(k\right)}$,k++, go to 2.1. Else, no crack, go to Step3.

  • 2.3 -Through step 2.1 and step 2.2, S can be expressed as $S=S_{\mathit{residue}}+{\displaystyle \sum _{k=1}^m{x}_{\mathit{ck}}{\Phi }_{\mathit{ck}}}$m is the total number of iterations.

• 3. Main profile reconstruction with MP:

  • 3.1 - Set k=1, $S_{\mathit{residue}}=S-{\displaystyle \sum _{k=1}^m{x}_{\mathit{ck}}{\Phi }_{\mathit{ck}}}$, $R_{\mathit{residue}}^{\left(0\right)}=S_{\mathit{residue}}$

  • 3.2 -Project S _residue on a dictionary Φ _p and find the atom in Φ _p with maximum inner product in each iteration, i.e. $\left|〈{R_{\mathit{residue}}}^{\left(k-1\right)},{\Phi }_{\mathit{pj}}〉\right|=sup\left|〈{R_{\mathit{residue}}}^{\left(k-1\right)},{\Phi }_p〉\right|$, $x_{\mathit{pk}}={max}_j\left|〈{R_{\mathit{residue}}}^{\left(k-1\right)},{\Phi }_{\mathit{pj}}〉\right|$ where Φ _pj is the j th atom in Φ _p , x _pk is the coefficients.

  • 3.3 -If $\left|〈{R_{\mathit{residue}}}^{\left(k-1\right)},{\Phi }_{\mathit{pj}}〉\right|\le {\delta }_p$, ${S_{\mathit{residue}}}^k={S_{\mathit{residue}}}^{\left(k-1\right)}+x_{\mathit{pk}}{\Phi }_{\mathit{pk}}$, $R_{\mathit{residue}}^{\left(k\right)}=S_{\mathit{residue}}-S_{\mathit{residue}}^{\left(k\right)}$,k ++, go to 3.2. Else, go to step 4.

• 4. Finally, S can be expressed as follow:

  $S={\displaystyle \sum _{k=0}^m{x}_{\mathit{ck}}{\Phi }_{\mathit{ck}}}+{\displaystyle \sum _{k=0}^n{x}_{\mathit{pk}}{\Phi }_{\mathit{pk}}}+\sigma$, m is the iteration number of crack feature extraction, n is iteration number of main profile reconstruction, σ includes noise and approximation error.

Figure 9

Simulation signal of pavement profile. (a) simulation mainly profile signal (b) simulation pavement distress (c) noisy signal with SNR = 8db (d) simulation signal of pavement profile.

Experimental results

The following experiments are designed to examine the performance of the proposed approach for a good separation of the crack and main profile.

Comparison experiment with wavelet and median filtering method

In order to verify the effectiveness of our method, we construct the simulation signal of pavement profile. The simulation signal is presented in Figure 8d, where the profile is composed of main profile simulated by exponential function (Figure 8a), pavement distress (Figure 8b) and white Gaussian noise (SNR = 8db).

Figure 10a presents cracks obtained by wavelet and median filter method, where the db4 wavelet has been used to remove the profile noise and median filter has been used to extract the crack feature. Due to the limitation of this algorithm, the cracks in Figure 10a have distortion, which also contains some information about main profile and noise. The processed crack output by sparse representation method is shown in Figure 10(b). It is observed that cracks have clearly been detected by sparse representation method.

Figure 10

Comparison result of crack extraction. (a) Traditional method ( b) The sparse representation method.

Figure 11 presents main profile extraction results. From Figure 11, it can be seen that the main profile in Figure 11a have distortion, which will affect the results of the subsequent calculation of IRI. The processed main profile output by sparse representation method is shown in Figure 11b. It is observed that this method can not only remove the noise, but also maintain the shape of original main profile well.

Figure 11

Comparison result of main profile extraction. (a) Traditional method (b) The sparse representation method.

Figure 12, Table 1 and Table 2 presents extracted crack accuracy result. From Figure 12, Table 1 and Table 2 it can be seen that both methods can check the location of cracks accurately, but traditional method cannot detect the width and depth accurately.

Figure 12

Comparison result of extracted crack accuracy. (a) Traditional method (b) The sparse representation method.

Table 1 Disease characteristic parameter of the left crack in Figure 8 d (d)

Table 2 Disease characteristic parameter of the right crack in Figure 8 d (d)

From the above experiment results, it is clear that the sparse representation method outperforms the wavelet and median filtering method not only in crack detection and main profile reconstruction but also a good separation between crack and main profile.

Comparison experiment with wavelet and Gabor dictionary

We also did comparison experiment with wavelet and Gabor dictionary. The experimental parameters of the wavelet method are as follows: the db4 wavelet has been used to extract crack and make seven-layer wavelet decomposing; The parameters of Gabor dictionary are as follows:$g_r\left(t\right)=\frac{1}{\sqrt{s}}g\left[\frac{t-u}{s}\right]cos\left(vt+w\right)$is the functional form, γ = ( s,  u,  v,  w) = ( a^j, pa^jΔu,  ka^−jΔv,  iΔw) is time-frequency parameters of Gabor atom, in which a = 2, Δu = 1/2, Δv = π, Δw = π/6, 0 <  j ≤ log ₂N, 0 <  p ≤ N 2^−j+1, 0 < k ≤ 2^j+1,0 ≤ i ≤ 12; Figure 13 presents cracks obtained from Figure 8 d) by wavelet and Gabor dictionary and Figure 14 is an enlarged image of two cracks in Figure 13.

Figure 13

Comparison result of extracted crack accuracy.

Figure 14

Enlarged image of cracks in Figure. 12 a left crack Figure 12b right crack.

From Figures 13 and 14, it can be seen that the cracks detected by wavelet method and Gabor dictionary have distortion, which will affect the results of the subsequent main profile. It is observed that Gabor dictionary can match triangular shape of the crack very well, but when it detects trapezoidal shape of the crack, the match result is not good.

Experiment result of actual pavement crack

The experiment was carried out to examine the efficiency of the proposed technique for crack detection and main profile reconstruction. From Figure 15a, we can see that the horizontal light deforms when it encounters crack. After this image is processed by center coordinates extraction, calibration, we can obtain the profile of the pavement. As shown in Figure 15b, this profile includes main profile, noise, and pavement crack. Then we adapt the sparse representation method to the profile signal to extract the features of crack and reconstruct the main profile. From Figure 15c, we can see that this method not only detects the location of crack accurately, but also matches the shape of crack very well. Figure 15d shows the reconstructed main profile by sparse representation method, in which the blue line is the original profile signal and the red line is the reconstructed main profile without crack and noise. From Figure 15d, we can see that the reconstructed main profile not only removes the crack without destroying the main profile, but also retains the original profile information.

Figure 15

The experiment result of actual pavement crack.

In order to verify the consistency between the crack shape extracted by our algorithm and the actual crack shape, we use waveform similarity to evaluate our method. Assuming that the actual crack waveform is x( t), the crack waveform extracted by our algorithm is y( t), similarity R can be expressed as:

$$R=\frac{{\displaystyle \sum \left[x\left(t\right)\times y\left(t\right)\right]}}{\sqrt{{\displaystyle \sum x{\left(t\right)}^2\times {\displaystyle \sum y{\left(t\right)}^2}}}}$$

(7)

We captured 39 images with cracks randomly, and use sparse representation method to extract cracks. Experiment result was shown in Figure 16. From Figure 16, we can see that similarities R are all concentrated between 0.94 and 1, and the extracted crack waveform is good consistency with actual waveform.

Figure 16

Similarity between the actual crack waveform and the extracted crack waveform.

Accuracy experiment

In the area of pavement crack detection, crack location and width are the two important indicators. So the purpose of our experiment is to verify the accuracy of the extracted crack location and width with our method. As shown in Figure 17, we processed an adjustable gauge block with different groove width (4, 6 and 8mm, the accuracy is 0.1mm), to simulate the pavement cracks with different width (4, 6 and 8mm). The experiment was divided into two groups. We made the opening direction of groove parallel with the longitudinal direction of pavement in one group, i.e. let the transverse laser stripe project onto the groove of gauge block. In the other group, we made the opening direction of groove parallel with the transverse direction of pavement, i.e. let the longitudinal laser stripe project onto the groove of gauge block. Then the 3D laser scanning system is used to detect the groove of gauge block with the speed of 80km/h and the results are shown in Table 3. From Tables 3 and 4, we can see that, whether with the transverse or the longitudinal laser stripe, our method can detect the location and width of crack accurately, the indication error is not greater than 1mm.

Figure 17

Accuracy experiment of crack detection (a) gauge block (mm) (b) experimental image.

Table 3 The first group of experiment accuracy with longitudinal direction

Table 4 The second group of experiment accuracy with transverse direction

Conclusions

In this article, a novel method based on sparse representation is developed to detect pavement cracks and reconstruct the main pavement profile. The key for cracks separation from main profile is based on the features of the mixed over-complete dictionary, which consists of two kinds of atoms, one for crack representation and another for main profile representation. In this study, atoms of trapezoidal membership function are adopted to represent crack, and exponential function for main pavement profile. Compared to the wavelet and median filtering method, the cracks extracted by our method can match the shape of crack very well, which cannot damage the information of the main profile signal. Some outdoor and accuracy experiments were conducted and promising results were obtained, showing that this method cannot only detect the position of pavement crack efficiently and achieve a good separation of crack from pavement profile, but also reconstruct main profile very well. Because MP is very time consuming when the greedy exhaustive search in the whole huge over-complete dictionary adopted and it is still a challenging problem. In the future work, we will use computer grid technology to improve computational efficiency.