An improved denoising method for eye blink detection using automotive millimeter wave radar

Abstract

With the development of radar technology, the automotive millimeter wave radar is widely applied in the fields including internet of vehicles, Artificial Intelligence (AI)-based autonomous driving, health monitoring, etc. Eye blink, as one of the most common human activities, can effectively reflect the person’s consciousness and fatigue. The contacted eye blink detection often leads to uncomfortable experience and the camera-based eye blink detection has privacy issues. As an alternative, the non-contacted eye blink detection based on automotive millimeter wave radar resolves the aforementioned issues and has been received much attention. This paper proposes an eye blink detection method using the frequency modulated continuous wave radar. Firstly, the position of the person’s head is estimated by carrying out fast Fourier transform on the intermediate frequency signal, and the signals of the range bins at the head are extracted. Then, the complete ensemble empirical mode decomposition with adaptive noise algorithm is applied to decompose the eye signals into a series of intrinsic mode functions (IMFs), and the singular value decomposition is adopted to constrain the selection and reconstruction of the useful IMFs related to the eye blink signal. Finally, the short-time Fourier transformation and cell average constant false alarm rate are applied to detect the eye blink behavior. Experiments are carried out to validate the effectiveness of the proposed eye blink detection method.

1 Introduction

The Internet of Things (IoT) involves many aspects and can be applied in some fields such as energy, transportation, and manufacturing [1, 2]. It can effectively promote the intelligent development of these areas, and make the limited resources more reasonable to use [3]. With the development of the IoT, the millimeter wave radar sensing [4] has been received widespread attention. In recent years, the automotive millimeter wave radar [5, 6] has been rapidly penetrated and developed in the fields of internet of vehicles [5], Artificial Intelligence (AI)-based autonomous driving [6], etc. The automotive millimeter wave radar with multiple transmitting antennas and multiple receiving antennas can reasonably allocate resources according to the different requirements, so as to realize corresponding functions [7, 8]. And with the increasing demand for health monitoring [9, 10], physiological signs detection based on automotive millimeter wave radar has been received a lot of attention. Physiological signals, such as heart signal, breath signal, blink signal, etc., can reflect the fatigue, attention, stress, or consciousness level of the person [11, 12]. Since the eye blink motion is one of the most natural and frequent human activities, the eye blink detection is an effective way to measure fatigue and concentration. Hence, the eye blink detection has been researched widely [13, 14]. Electro-Oculogram [15], as a common blink detection method, mainly relies on the contacted devices. The eye blink motion is detected by attaching electrodes to the human skin near the eyes to measure potential changes between the electrodes. However, attaching electrodes to the skin causes abrasion, leading to uncomfortable user experience.

On the other hand, non-contacted eye blink detection usually relies on vision devices [16, 17]. It applies the camera to capture the image sequences that contain eye blink motion, and achieves eye blink detection by using computer vision technology. AL-Gawwam et al. [17] use facial feature trackers to localize the contours of the eyes and eyelids. They measure the range between the eyelids to obtain the opening state of the eyes, and the rapid change of the range between the eyelids is detected as an eye blink. Although the vision-based eye blink detection method improves user’s natural experience, the expensive cost, light sensitivity and privacy issues should be addressed.

As an alternative, the non-contacted eye blink detection based on radar resolves the aforementioned issues. The Doppler sensors are widely applied for eye blink detection [18,19,20]. Specifically, Tamba et al. [18] apply the Doppler sensor with setting the thresholds of the blink width and height for each person to reduce the influence of individual differences. Kim [19] adopts the principal component analysis to distinguish conscious and unconscious eye blink using a 5.8 GHz Doppler sensor radar. Yamamoto et al. [20] estimate the eye blink duration time by analyzing the eyelids, closing and opening behavior on the spectrograms. Compared to Doppler sensors, the frequency modulated continuous wave (FMCW) radar at the millimeter frequency range has significant advantages in the fields of target detection [21], vital signs detection [22], driver’s behavior detection [23], hand gesture recognition [24], and so on. Cardillo et al. [25] apply a 120 GHz FMCW radar to realize the head motion and eye blink detection. However, the authors only use the range information of the eye blink motion, the Doppler information is not mentioned.

Therefore, in this paper, we focus on the non-contacted eye blink detection and propose an eye blink detection method based on complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) with singular value decomposition (SVD) denoising using FMCW radar. The main contributions of this paper are concluded as follows:

Firstly, the intermediate frequency (IF) signal is obtained by mixing and filtering the eye blink data. Then, the person’s head position is estimated by performing fast Fourier transform (FFT) on the IF signal, and the signals in the position interval are extracted.

Secondly, we propose an eye blink signal reconstructed method by the CEEMDAN algorithm combining with the SVD. The extracted signal is decomposed into a series of intrinsic mode functions (IMFs) by CEEMDAN algorithm. Then, SVD is applied to constrain the selection and reconstruction of the useful IMFs for eye blink signal reconstruction.

Thirdly, the short-time Fourier transformation (STFT) is performed on the reconstructed eye blink signal, and the eye blink detection is realized by the cell average constant false alarm rate (CA-CFAR).

Finally, we carry out a series of experiments to verify the effectiveness of the proposed eye blink detection method. Experimental results show that the proposed method can successfully detect the eye blink motion.

The rest of this paper is organized as follows. In Sect. 2, the principle of FMCW radar is described. In Sect. 3, the proposed eye blink detection method is introduced in detail. The experimental results are analyzed and discussed in Sect. 4. The conclusion is drawn in Sect. 5.

2 FMCW radar principle

In this section, the principle of FMCW radar is first described (shown in Fig. 1). The radar system mainly contains several parts such as signal source, transmitting antenna (TX), receiving antenna (RX), mixer, low-pass filter (LPF), ADC sampler (A/D), etc. The signal source is responsible for generating the FMCW signal. The TX and RX are responsible for the transmission and reception of the signal. The received echo signal and the transmitted signal are mixed by the mixer, and passed through a LPF to obtain the IF signal. Finally, the IF signal is sampled by A/D sampler for further processing.

Fig. 1

The system structure of FMCW radar

Fig. 2

The schematic diagram of the radar signal

In this paper, we adopt sawtooth modulation and the transmitter transmits modulated sawtooth microwave (shown in Fig. 2) [23], which is expressed as

$$\begin{aligned} {S_{\mathrm{T}}}\left( t \right) = {A_{\mathrm{T}}}{{\mathrm{e}}^{j2\pi \left( {{f_{\mathrm{c}}}t + \int _0^t {{f_{\mathrm{T}}}\left( \tau \right) {\mathrm{d}}\tau } } \right) }}, \end{aligned}$$

(1)

where ${f_{\mathrm{c}}}$ is the carrier frequency, ${f_{\mathrm{T}}}(\tau ) = S \cdot \tau$ indicates that the frequency of the transmitted signal within a period of time, $S = {B \big /{{T_{\mathrm{c}}}}}$ is the slope of the chirp signal, B is the maximum bandwidth of the signal, ${T_{\mathrm{c}}}$ is the pulse width of the chirp signal, and ${A_{\mathrm{T}}}$ is the amplitude of the transmitted signal.

Let R be the range from the FMCW radar to the person’s head. The received echo signal can be expressed as

$$\begin{aligned} {S_{\mathrm{R}}}\left( t \right) = {A_{\mathrm{R}}}{{\mathrm{e}}^{j2\pi \left[ {{f_{\mathrm{c}}}\left( {t - {t_{\mathrm{d}}}} \right) + \int _0^t {{f_{\mathrm{R}}}\left( \tau \right) {\mathrm{d}}\tau } } \right] }}, \end{aligned}$$

(2)

where ${A_{\mathrm{R}}}$ is the amplitude of the received signal, ${t_{\mathrm{d}}} = {{2R} \big /c}$ is the signal delayed time from transmission to reception, c is the light speed, ${f_{\mathrm{R}}}(t) = S \cdot (t - {t_{\mathrm{d}}}) + \Delta {f_{\mathrm{d}}}$, and $\Delta {f_{\mathrm{d}}}$ is Doppler shift.

The received signal ${S_{\mathrm{R}}}(t)$ and the transmitted signal ${S_{\mathrm{T}}}(t)$ are sent to the mixer and passed through a LPF, then the IF signal can be expressed as

$$\begin{aligned} {S_{{\mathrm{IF}}}}\left( t \right) = \frac{1}{2}{A_{\mathrm{T}}}{A_{\mathrm{R}}}{{\mathrm{e}}^{j2\pi \left[ {{f_{\mathrm{c}}}{t_{\mathrm{d}}} + \left( {{f_{{\mathrm{IF}}}} - \Delta {f_{\mathrm{d}}}} \right) t} \right] }}, \end{aligned}$$

(3)

where ${f_{{\mathrm{IF}}}} = S \cdot {t_{\mathrm{d}}}$. Specifically, the eye blink signal is contained in the IF signal. Therefore, to detect the eye blink, the IF signal needs to be processed and analyzed.

3 Methods

In this section, we present the proposed eye blink detection method in detail. The flowchart of the proposed method is shown in Fig. 3. Firstly, the range FFT is performed on the IF signal to determine the position interval of the human head, and the signals in this position interval are extracted. Then, the extracted signal is decomposed into a series of IMFs by CEEMDAN algorithm. Next, the SVD is applied on each IMF signal to constrain the selection and reconstruction of the useful IMFs to reconstruct the eye blink signal. Finally, the STFT is performed on the reconstructed eye blink signal and the CA-CFAR is applied to detect the eye blink.

Fig. 3

The flowchart of the proposed method

3.1 Signal extraction

We place the FMCW radar directly in front of the person’s head and eyes, so the received signals mainly include the eye blink signal, the signals from other facial parts and noise signals. Therefore, we need to obtain the all signals located at the distance R for extracting the eye blink signal. Then, we firstly need to estimate the position information of the head. According to ${t_{\mathrm{d}}} = {{2R}\big / c}$ and ${f_{{\mathrm{IF}}}} = S \cdot {t_{\mathrm{d}}}$, the corresponding relationship between R and ${f_{{\mathrm{IF}}}}$ is

$$\begin{aligned} R = \frac{{c \cdot {f_{{\mathrm{IF}}}}}}{{2S}} = \frac{{c \cdot {T_{\mathrm{c}}}}}{{2B}}{f_{{\mathrm{IF}}}}. \end{aligned}$$

(4)

Therefore, to obtain the range information of the head, it is necessary to analyze the IF signal and estimate the frequency ${f_{{\mathrm{IF}}}}$. Firstly, the IF signal is sampled by A/D and the FFT is performed on sampling points of each chirp to obtain range information of the head. We assume for simple calculation, the sampled signal and transformed signal after FFT are [24]

$$\begin{aligned} {S_{{\mathrm{IF}}}}\left( n \right)= & {} \sum \limits _{m = 0}^{M - 1} {\sum \limits _{n = 0}^{N - 1} {{{\mathrm{e}}^{j2\pi \left[ {{f_{\mathrm{c}}}{t_{\mathrm{d}}} + \Delta {f_{\mathrm{d}}}m{T_{\mathrm{c}}} + \left( {{f_{{\mathrm{IF}}}} - \Delta {f_{\mathrm{d}}}} \right) n{T_{\mathrm{s}}}} \right] }}} },\end{aligned}$$

(5)

$$\begin{aligned} {S_1}\left( p \right)= & {} \sum \limits _{m = 0}^{M - 1} {\sum \limits _{n = 0}^{N - 1} {{{\mathrm{e}}^{j2\pi \left\{ {{f_{\mathrm{c}}}{t_{\mathrm{d}}} + \Delta {f_{\mathrm{d}}}m{T_{\mathrm{c}}} + \left[ {\left( {{f_{{\mathrm{IF}}}} - \Delta {f_{\mathrm{d}}}} \right) {T_{\mathrm{s}}} - \frac{p}{N}} \right] n} \right\} }}} }, \end{aligned}$$

(6)

where M is the number of transmitted chirps, N is the number of sampling points, $p = 0,1, \ldots ,N - 1$, the ${T_{\mathrm{s}}}$ is the sampling time and ${T_{\mathrm{c}}} = N{T_{\mathrm{s}}}$. Therefore, when the absolute value $\left| {{S_1}} \right|$ wants to reach the maximum, it needs to satisfy $p = \left( {{f_{{\mathrm{IF}}}} - \Delta {f_{\mathrm{d}}}} \right) N{T_{\mathrm{s}}}$. Because of ${f_{{\mathrm{IF}}}} \gg \Delta {f_{\mathrm{d}}}$, we can roughly obtain ${f_{{\mathrm{IF}}}} \approx {p \big / {N{T_{\mathrm{s}}}}}$ after FFT and the range value of the head can be obtained according to Eq. (4). Then, the signals located at the estimated range are extracted as the signal $x\left( t \right)$ that includes the eye blink signal, the signals from other facial parts and noise signals.

$$\begin{aligned} x(t) = {{\mathrm{e}}^{j2\pi \left[ {{f_{\mathrm{c}}}{t_{\mathrm{d}}} + \Delta {f_{\mathrm{d}}}t + \left( {{f_{{\mathrm{IF}}}} - \Delta {f_{\mathrm{d}}}} \right) {n_1}{T_{\mathrm{s}}}} \right] }}, \end{aligned}$$

(7)

where $t = m{T_{\mathrm{c}}}$, $m = 0,1, \ldots ,M - 1$, and $p = {n_1}$ while $\left| {{S_1}} \right|$ reaches the maximum.

In addition, considering the sampling of FMCW radar signal, the maximum measuring range can be expressed as

$$\begin{aligned} {R_{{\mathrm{max}}}} = \frac{{c{T_{\mathrm{c}}}}}{{2B}}{F_{\mathrm{s}}}. \end{aligned}$$

(8)

The maximum measuring range is 8.55 m when the parameters of FMCW radar are adopted in the experiments.

3.2 CEEMDAN algorithm

Due to the interference of the face and the environmental noise, the eye blink signal is too weak to be detected. Therefore, it is necessary to remove the interference and the noise to enhance the eye blink signal. The empirical mode decomposition (EMD) algorithm [26] is usually applied to denoise the weak vital signals. EMD decomposes the raw signal into several IMFs, and it realizes denoising by removing the noisy IMFs [26]. However, the EMD algorithm usually leads to the problem of the modal aliasing. Towards this end, the ensemble empirical modal decomposition (EEMD) algorithm [27] adds different white noises into the raw signal, and performs multiple EMD on the noisy signal and averages the IMFs of multiple EMD to obtain the final IMFs. Furthermore, the SVD is often used in conjunction with EEMD for constraining the selection and reconstruction of the useful IMFs [28]. However, the EEMD algorithm cannot completely eliminate the influence of white noise on the decomposition results. The CEEMDAN algorithm [29] adds a finite number of adaptive white noises at each stage of the decomposition process of EMD, which effectively solves the problems of EEMD. Therefore, we apply the CEEMDAN algorithm to decompose the extracted signal for signal denoising.

The white noise ${n^g}\left( t \right)$, $g = 1,2, \ldots ,{\mathrm{G}}$ with a standard normal distribution is added to the extracted signal $x\left( t \right)$, and the EMD is applied on the added signal to obtain the ${{\mathrm{IMF}}}_1^g\left( t \right)$. Then, ${{\mathrm{IMF}}_1}\left( t \right)$ can be computed as

$$\begin{aligned} {{\mathrm{IMF}}_1}\left( t \right) = \frac{{\mathrm{1}}}{{\mathrm{G}}}\sum \limits _{g = 1}^{{\mathrm{G}}} {{\mathrm{IMF}}}_1^g\left( t \right) , \end{aligned}$$

(9)

where ${{\mathrm{IMF}}_1}\left( t \right)$ represents the first IMF component, and the first remaining component ${r_1}\left( t \right)$ can be derived as

$$\begin{aligned} {r_1}\left( t \right) = x\left( t \right) - {{\mathrm{IMF}}_1}\left( t \right) . \end{aligned}$$

(10)

Then, the white noise is continuously added the remaining component ${r_1}\left( t \right)$ for obtaining ${{\mathrm{IMF}}_2}\left( t \right)$. This step is repeated until the remaining component is a monotonic function. Finally, the extracted signal $x\left( t \right)$ is decomposed into a series of IMFs using CEEMDAN algorithm, that is

$$\begin{aligned} x\left( t \right) = \sum \limits _{j = 1}^J {{{\mathrm{IMF}}_j}\left( t \right) } + {r_J}\left( t \right) , \end{aligned}$$

(11)

where J is the number of IMFs, ${\mathrm{IM}}{{\mathrm{F}}_j}$ is the j-th IMF, and ${r_J}\left( t \right)$ is the residual component of the signal after decomposing. The process of CEEMDAN algorithm is concluded in Table 1. It can be seen from Table 1 that the signal is decomposed into J IMFs and a residual component ${r_J}\left( t \right)$ by CEEMDAN algorithm.

Table 1 IMFs extraction with CEEMDAN algorithm

Each IMF represents different frequency component of the IF signals. Assuming that the useful signal and noise are distributed in different IMFs, the useful IMFs can be reconstructed by removing the noisy IMFs expressed as follows:

$$\begin{aligned} {\hat{x}}\left( t \right) = \sum \limits _{j = 1}^{{J_{\mathrm{u}}}} {{\mathrm{IM}}{{\mathrm{F}}_j}\left( t \right) }, \end{aligned}$$

(12)

where ${\hat{x}}\left( t \right)$ is the denoised signal, and ${J_{\mathrm{u}}}$ is the number of the useful IMFs.

3.3 Eye blink signal reconstruction using SVD

After CEEMDAN decomposing, the SVD [28] is adopted to constrain the selection and reconstruction of the IMFs. By applying the SVD, the IMFs related to the eye blink are selected and reconstructed, so as to obtain the eye blink signal from the extracted signal. In addition, the interference is removed in the selection process of IMFs, and each selected IMF is denoised during the reconstruction process.

Since the extracted signal is first decomposed into several IMFs by the CEEMDAN algorithm, and each IMF can be expressed as

$$\begin{aligned} {{\mathbf{M }}_j} = \left[ {{\mathrm{IM}}{{\mathrm{F}}_j}\left( 1 \right) ,{\mathrm{IM}}{{\mathrm{F}}_j}\left( 2 \right) , \ldots ,{\mathrm{IM}}{{\mathrm{F}}_j}\left( N \right) } \right] , \end{aligned}$$

(13)

where N is the temporal sampling number of the IMF.

Then, the Hankel matrix ${{\mathbf{H }}_j}$ of each IMF can be constructed as

$$\begin{aligned} {{\mathbf{H }}_j} = \left[ {\begin{array}{{cccc}} {{\mathrm{IM}}{{\mathrm{F}}_j}\left( 1 \right) }&{}{{\mathrm{IM}}{{\mathrm{F}}_j}\left( 2 \right) }&{} \cdots &{} {{\mathrm{IM}}{{\mathrm{F}}_j}\left( n \right) }\\ {{\mathrm{IM}}{{\mathrm{F}}_j}\left( 2 \right) }&{}{{\mathrm{IM}}{{\mathrm{F}}_j}\left( 3 \right) }&{} \cdots &{}{{\mathrm{IM}}{{\mathrm{F}}_j}\left( {n + 1} \right) }\\ \vdots &{} \vdots &{} \vdots &{} \vdots \\ {{\mathrm{IM}}{{\mathrm{F}}_j}\left( m \right) }&{}{{\mathrm{IM}}{{\mathrm{F}}_j}\left( {m + 1} \right) }&{} \cdots &{}{{\mathrm{IM}}{{\mathrm{F}}_j}\left( N \right) } \end{array}} \right] = {{\mathbf{T }}_H}{{\mathbf{M }}_j}, \end{aligned}$$

(14)

where $m = \left\lfloor {{N \big /2}} \right\rfloor + 1$, $n = N - m + 1$, ${{\mathbf{T }}_H}$ is the Hankel matrix construction operator, and └┘ means Floor operation. By applying the SVD to ${{\mathbf{H }}_j}$, ${{\mathbf{H }}_j}$ can be expressed as

$$\begin{aligned} {{\mathbf{H }}_j} = {{\mathbf{U }}_{m \times m}}{{\varvec{\Sigma }}_{m \times n}}{\mathbf{V }}_{n \times n}^{\mathrm{{T}}}, \end{aligned}$$

(15)

where ${\mathbf{U }}$ and ${\mathbf{V }}$ are the orthogonal matrices, ${\varvec{\Sigma }} = {\mathrm{diag}}\left( {{\sigma _1},{\sigma _2}, \ldots ,{\sigma _r}} \right)$, r and ${\sigma _i}$ are the rank and the singular value of ${{\mathbf{H }}_j}$, respectively.

The singular values represents the signal coherence in each IMF. The larger singular value represents the effective signal with better coherence. Therefore, the difference of the signal coherence is used as the criterion to select useful IMFs to reconstruct the eye blink signal.

The normalized singular spectrum energy $p\left( j \right)$ of the j-th IMF is

$$\begin{aligned} p\left( j \right) = \frac{{E\left( {{{\mathbf{H }}_j}} \right) }}{{\sum \nolimits _{j = 1}^J {E\left( {{{\mathbf{H }}_j}} \right) } }}, \end{aligned}$$

(16)

where $E\left( {{{\mathbf{H }}_j}} \right) = \sum \nolimits _{i = 1}^r {\sigma _i^2}$ is the singular spectrum energy.

Based on the energy probability theory [28], the energy probability of the singular spectrum $q\left( j \right)$ is defined as follows:

$$\begin{aligned} q\left( j \right) = - p\left( j \right) \lg \left[ {p\left( j \right) } \right] . \end{aligned}$$

(17)

Since the singular spectrum energy of the useful IMFs is different with the noisy IMFs [28], it can be regarded as the criterion for selecting the useful IMFs.

Moreover, due to the influence of white noise, the selected IMFs may retain the noise which may have the same frequency as the useful signals. In generally, the r singular values are sorted in descending order, and the k largest singular values are remained to reconstruct the Hankel matrix, that is

$$\begin{aligned} {\varvec{\Sigma }} = {\mathrm{diag}}\left( {{\sigma _1},{\sigma _2}, \ldots ,{\sigma _k}} \right) , \end{aligned}$$

(18)

where the rank of ${\varvec{\Sigma }}$ is k.

Then, the new Hankel matrix ${{{\hat{\mathbf{H }}}}_j}$ can be reconstructed as

$$\begin{aligned} {{{\hat{\mathbf{H }}}}_j} = {{\mathbf{U }}_{m \times m}}{{\varvec{\Sigma }}_{m \times n}}{\mathbf{V }}_{n \times n}^{\mathrm{{T}}} = {{\mathbf{T }}_k}{{\mathbf{H }}_j}, \end{aligned}$$

(19)

where ${{\mathbf{T }}_k}$ represents the reconstruction operator of the new Hankel matrix ${{{\hat{\mathbf{H }}}}_j}$.

Therefore, the final reconstructed eye blink signal $s\left( t \right)$ after denoising can be written as

$$\begin{aligned} s\left( t \right) = \sum \limits _{j = 1}^{{J_u}} {{\mathbf{T }}_H^{- 1}{{\mathbf{T }}_k}{{\mathbf{T }}_H}{{\mathbf{M }}_j}}, \end{aligned}$$

(20)

where ${\mathbf{T }}_H^{- 1}$ is the inverse of the Hankel matrix construction operator.

Furthermore, the process of the selection and reconstruction of IMFs using SVD is shown in Table 2. It can be seen from Table 2 that the SVD is applied to all IMFs, and the useful IMFs related to the eye blink are selected through the energy probability of the singular spectrum $q(j)$. The eye blink signal is finally reconstructed.

Table 2 IMFs selection and reconstruction using SVD

3.4 Eye blink detection by STFT combining with CA-CFAR

Although the eye blink signal has been reconstructed, it is still difficult to detect the eye blink in the time domain. Fortunately, the time-frequency analysis is usually performed for achieving the eye blink detection [30,31,32].

In this paper, the STFT is applied on the reconstructed eye blink signal $s\left( t \right)$, that is

$$\begin{aligned} S\left( {t,f} \right) = \int _{ - \infty }^\infty {s\left( \tau \right) h\left( {\tau - t} \right) } {{\mathrm{e}}^{ - 2\pi f\tau }}{\mathrm{d}}\tau , \end{aligned}$$

(21)

where $h\left( {\tau - t} \right)$ is the window function.

After performing STFT, the time-frequency spectrum can be obtained. The Doppler feature of eye blink can be observed on the time-frequency spectrum, but it does not mean that the eye blink detection has been realized. Therefore, to achieve the eye blink detection, the CA-CFAR algorithm [33] is adopted to perform on the time-frequency spectrum.

For the CA-CFAR detector, noise samples are extracted from both leading and lagging cells around the cell under test (CUT). The noise power can be estimated as [34]

$$\begin{aligned} {P_n} = \frac{1}{I}\sum \limits _{i = 1}^I {{y_i}}, \end{aligned}$$

(22)

where ${P_n}$ is the estimated noise power, I is the number of training cells and ${y_i}$ is the sample in each training cell.

Then, the detection threshold can be given by

$$\begin{aligned} T = \alpha {P_n}, \end{aligned}$$

(23)

where T represents the detection threshold and ${\alpha }$ is a scaling factor. Then, we compute the power result for each CUT. If the calculation result of one CUT exceeds the threshold T, this CUT is considered as an eye blink motion [35].

Fig. 4

The experimental scene

4 Experimental results and discussion

In this section, we first introduce the FMCW radar and radar parameters used in the experiment. We carry out a series of experiments to verify the effectiveness of the proposed eye blink detection method.

4.1 Experimental specification

The experimental scene is shown in Fig. 4, where the person’s head and eyes are located at R (${R \approx 20}$ cm) directly in front of the FMCW radar. The radar adopted in the experiment is Texas Instruments AWR1642 as shown in the red device in Fig. 4, and it has two transmitting and four receiving antennas. The green equipment behind AWR1642 is DCA1000 and it is used for data acquisition.

The central frequency of the FMCW radar is 77 GHz, and the bandwidth is 4 GHz. The sampling frequency of the A/D sampler is set to 2 MHz. The pulse width of chirp is set to 114 microseconds ($\mu$s) and the time of one frame is set to 100 milliseconds (ms). We use 30 frames to collect the data of eye blink motion and the collection time is 3 seconds (s). Moreover, each frame contains 255 chirps and each chirp contains 200 sampling points. The parameters of FMCW radar used in the experiments are listed in Table 3.

Table 3 Parameters setting for eye blink detection

Fig. 5

The result of range estimation

4.2 Experimental results and discussion

During the eye blink data collection stage, the head of the person is located stably in front of the FMCW radar, and the eyes are looking at the antennas of the radar. In the experiment, the IF data of eye blink in 3 s is collected. Then, by performing FFT on the IF signal, the range between the head and FMCW radar is estimated. The estimated result of range is shown in Fig. 5. It can be seen from Fig. 5 that the head is approximately 20 cm in front of the radar, and keeps stable, which is consistent with the set conditions.

Fig. 6

Decomposition result of CEEMDAN algorithm

Fig. 7

The singular values of each IMF

Since the signals at the range bin with strongest spectrum contain the eye blink motion, they are extracted for eye blink detection. In order to reduce the time complexity of CEEMDAN decomposition, the extracted signal is averaged on every continuous five chirps. It should be noted that the time of one chirp is 114 μs, so the time of five chirps is 570 μs. The signals remain relatively stable in a short time. Then, the processed signal is decomposed by the CEEMDAN algorithm. The decomposition result is shown in Fig. 6. It can be seen from Fig. 6 that the IF signal is decomposed into seven IMFs and one residual signal. These IMFs are sorted from high frequency to low frequency.

Fig. 8

The energy probability of the singular spectrum of each IMF

Fig. 9

The comparison result of the reconstructed signal and the averaged signal

Subsequently, the Hankel matrix of each IMF is constructed and SVD is applied to obtain the corresponding singular values, and the singular values of each IMF are as shown in Fig. 7. We also show the energy probability of the singular spectrum of each IMF in Fig. 8. It can be observed from Figs. 7 and 8 that the singular values and the singular spectrum energy probability of IMF5 to IMF7 are larger than other IMFs, and the respective frequencies of IMF5 to IMF7 are calculated which are close to 0 Hz. This is because the low frequency signals reflected from the face are more intense and the eye blink is so weak that the singular spectrum energy probability caused by the eye blink motion is small. Therefore, according to the principle of the CEEMDAN algorithm, we want to remove IMF5 to IMF7 for removing interference from the other facial parts, and select IMF1 to IMF4 as the useful IMFs to reconstruct the eye blink signal. In fact, the IMFs are selected for reconstruction with the rule that the energy probability of the singular spectrum is less than the threshold, where the threshold is set as the averaged value of the sum of the energy probabilities of the singular spectrum of all IMFs. In addition, it is noted that in the reconstruction process, the k largest singular values are selected for each useful IMF reconstruction. The selected IMFs are denoised during the reconstruction process.

Furthermore, the reconstructed eye blink signal, the IF signal and useful IMFs are shown in Fig. 9. The blue line represents the IF signal and the red line represents the reconstructed eye blink signal. The yellow line represents the useful IMFs which is the summation of IMF1 to IMF4. It can be seen from Fig. 9 that the IF signal has two obvious changes in amplitude, which are located at about 40 chirps and 1060 chirps, respectively. The amplitude change may be caused by the hardware device, the noise or the eye blink motion. By observing the IF signal and useful IMFs, it can be found that CEEMDAN algorithm can effectively denoise the signal by removing the noisy IMFs. However, the two amplitude changes are still obvious, which means that the useful IMFs still contain the noise. Compared with the useful IMFs in Fig. 9, the amplitude change located at about 1060 chirps can be more clearly observed in the reconstructed eye blink signal, while the amplitude change at about 40 chirps is weakened. This is because the noise contained in each selected IMF is further removed during the reconstruction of the selected IMFs, so as to enhance the eye blink signal. Therefore, the amplitude change at about 40 chirps may be affected by the hardware device and noise, while the amplitude change at about 1060 chirps may be caused by the eye blink motion.

Fig. 10

Time-frequency spectrum with eye blink once

Fig. 11

Time-frequency spectrum with eye blink twice

Then, STFT is performed on the reconstructed eye blink signal to obtain the time-frequency spectrum. We denote the result of STFT performing on IF signal by IF + STFT. The CEEMDAN + STFT represents the result of STFT performing on the reconstructed useful signal of IMF1 to IMF4 by CEEMDAN algorithm, and the result of STFT performing on the reconstructed eye blink signal by CEEMDAN algorithm with SVD is denoted by CEEMDAN + SVD + STFT. To validate the effectiveness of the proposed algorithm, we compare the time-frequency spectrum with eye blink once and twice in 3 seconds, and the results are shown in Figs. 10 and 11, respectively. It can be observed from Figs. 10a and 11a that the strong static interferences with zero frequencies from the face cause the eye blink motion difficult to observe. Since the CEEMDAN algorithm can remove the noise and interference of the signal, the strong static interference from the face can be eliminated and the eye blink signal can be reconstructed. Therefore, the time-frequency spectrum of CEEMDAN + STFT can be easy to detect the eye blink motion as shown in Figs. 10b and 11b.

Although the zero-frequency components can be effectively removed, the noise contained in each IMF cannot be eliminated. The CEEMDAN algorithm combining with SVD is proposed to reconstruct the eye blink signal, so as to remove the noise in each selected IMF. In fact, we can see from Figs. 10c and 11c that the interference is effectively eliminated, and the eye blink motion can be observed clearly.

Finally, the CA-CFAR is performed on the time-frequency map to realize the blink detection. The detection results are shown in Figs. 10d and 11d. It can be clearly observed that there is one blink motion in 3 s in Fig. 10d and there are two blink motions in 3 s in Fig. 11d. The experimental results show that the results of the eye blink detection are consistent with the actual situation, so as to validate the effectiveness of the proposed method.

5 Conclusion

In this paper, we proposed an eye blink detection method using 77 GHz FMCW radar. Firstly, the FFT was performed on the IF signal to obtain the position of the head. The signals located in the position were extracted and the extracted signal was averaged every continuous five chirps. Then, the CEEMDAN algorithm was applied to decompose the averaged signal into several IMFs, and the eye blink signal was reconstructed by using SVD to constrain the selection and reconstruction of the useful IMFs. Finally, the eye blink detection was realized by performing STFT and CA-CFAR on the reconstructed eye blink signal. Furthermore, the experimental results proved the effectiveness of the proposed eye blink detection method. In the future, we will combine the eye blink detection with machine learning to achieve eye blink classification or fatigue detection.