An adaptive dynamically weighted median filter for impulse noise removal

A new impulsive noise removal filter, adaptive dynamically weighted median filter (ADWMF), is proposed. A popular method for removing impulsive noise is a median filter whereas the weighted median filter and center weighted median filter were also investigated. ADWMF is based on weighted median filter. In ADWMF, instead of fixed weights, weightages of the filter are dynamically assigned with the results of noise detection. A simple and efficient noise detection method is also used to detect noise candidates and dynamically assign zero or small weights to the noise candidates in the window. This paper proposes an adaptive method which increases the window size according to the amounts of impulsive noise. Simulation results show that the AMWMF works better for both images with low and high density of impulsive noise than existing methods work.


Introduction
Image processing is used in many scientific fields such as astronomy, aerospace, photogrammetry, particle physics, biology, medical sciences, geology, and science of materials [1]. Removing noise in an image is important for improving image quality. Some filters like H-infinity filter have been applied to practical industrial systems [2][3][4]. Usually, noise originates from taking pictures through a defective sensor or transmitting images through a noisy channel [5]. Such noise can be categorized into many different types by probability distributions, such types being impulsive noise, Gaussian noise, Rayleigh noise, and Laplacian noise, to name a few [4,6,7]. Impulse noise is a random variation of image intensity. The presence of impulse noise in image replaces intensities of corrupted pixels by extreme values. Mathematical model of impulse noise is given in (1) with probability p 1 L−1; with probability p 2 x; with probability 1−p 1 −p 2 8 < : ð1Þ where x denotes noise-free pixels of corrupted image.
Addition of impulse noise to images or frames of a video is a problem that occurs in TV [8], CCTV, and MRI [6] due to many reasons like unreliable sensor, power, and channel. Because impulsive noise seriously deteriorates picture quality and disrupts image processing, an efficient noise-removal algorithm is a core image processing technology.
Applying linear filters to image corrupted by impulsive noise leads to image blurring [9,10]. To address the problem of impulsive noise, researchers focused on applying nonlinear filters. Tukey [11] introduced a powerful nonlinear filter called median filter, and many improved filters have been proposed by other researchers. Huang et al. [12] described an efficient method to apply Median to 2-D images. Yin and Yang [13], Ko and Lee [14] proposed weighted median, and center weighted median filters respectively that apply weights to the window. Median filter with adaptive window size was used by Hwang and Hadded [15], Chang et al. [16], Juneja and Mohana [17] and Ibrahim et al. [18] to propose adaptive median filter (AMF) and its different types.
Toh and Isa [19] proposed a modified adaptive median filter that extends window size when current window does not have any noise-free pixel. Bhateja et al. [20] proposed a method similar to [19]; however, the maximum allowable size of window became 9 × 9. If 9 × 9 window surrounding noisy pixel does not have any known noise-free pixel, it replaces the noisy pixel with previously processed or previous noise-free pixel. Bhadouria et al. [21] proposed a median-based method that extends the window up to maximum allowable size of 9 × 9 when there is no noisefree pixel in the window. If 9 × 9 window does not have noise-free pixels, it replaces the corrupted pixel by mean of all pixels in 3 × 3 window. Afzal et al. [22] simply replaces the corrupted pixel with the next noise-free pixel.
Many researches used two-stage approach that first detects location of impulse noise and then apply median or other filters derived from median on those detected locations only to avoid blurring. To detect impulsive noise, Vijaykumar et al. [23] used absolute difference of all the pixels of 3 × 3 window with the center pixel and decides the center pixel to be noisy if less than two differences are greater than threshold. This algorithm fails in regions with high density of noise as more than two differences will be greater that threshold. Jampur et al. [24] and Luo [25] used fuzzy-based techniques to detect impulse noise and then applies heuristic median and median filters respectively on those detected pixels. Heuristic median filter uses average of non-noisy pixels that results in an average filter which causes blurring. Luo [25] uses degree of membership in replacing noisy pixel that may result in a value that does not exist in that region.
Duan and Zhang [26] divides the sorted window across every pixel in two intervals [0, med] and [med, 255] and compare the pixel intensity with locations corresponding to maximum difference in those two intervals to select noise candidates. Noise candidates are further processed with four convolutional kernels as used by Zhang and Karim [27] along the four directions. In algorithm proposed by Zhang and Karim [27], if a particular direction have excess of noise, the absolute difference will be smaller in that direction which will results the value to be smaller than the threshold, resulting in detection as nonnoisy pixel. Ng and Ma [28] first detects noise candidate the same as Duan and Zhang [26] did; it then applies the same technique to noise candidates but using 3 × 3 window this time to have final selection of noisy pixels. Lee [29] uses absolute difference of median of 5 × 5 window and average of 3 × 3 window with center pixel and compares them with a threshold. It is a time efficient algorithm for low-density noise. Ghanekar [30] detects candidates by comparing pixel with extreme values in a 3 × 3 window, then performs the final detection by choosing minimum and maximum locations among locations of four maximum differences, and compares pixel with intensity values at those minimum and maximum locations.
Some researchers used techniques other than median filter for removing impulse noise. Deng and An [31] and Sadri et al. [32] used wavelet. Wang et al. [33] and Liu et al. [34] used dictionary learning-based impulse noise removal techniques. Dictionary learning-based techniques are very complex and result in changing properties of input image as they replace patches of input image with patches stored in predefined dictionary. Wavelet-based methods, on the other hand, are far more complex. Median-based methods are simple methods that are powerful for removal of impulse noise and preserve edges well.
In this paper, adaptive dynamically weighted median filter (ADWMF) is proposed which combines weighted median filter with an impulse noise detection algorithm. Previous switching filters use static window size for a noisy image whose size depends on the total noise density of the image. Such techniques fail to remove noise when certain regions have comparatively excessive amount of noise due to uneven distribution of noisy pixels in the corrupted image. Conventional adaptive filters, on the other hand, extend window size when there is no noisefree pixel in window or median of window is a noisy pixel. Such algorithms do not consider performance when extending the window size. Instead of using fixed window size for an image, the proposed method changes window size according to the amount of noisy pixels present in a region. The proposed method uses an efficiently modified version of simple noise detection method, and it is used to propose the ADWMF. After detecting impulsive noise candidates, the weighted median filter is modified by dynamically assigning zero weights to the noise candidates in the window. Finally, this paper proposes an adaptive filter which increases the window size according to the amounts of impulsive noise candidates in a region. Window size in each region is determined by using number of noisy pixels in that region. Thresholds for changing window size according to number of noisy pixels in a region are defined after extensive simulations that ensure best possible results on the basis of minimum mean square error. Simulation results show that ADWMF works better for both images with low and high density of impulsive noise than existing methods work.
The rest of this paper is organized as follows: Section 2 describes some of the conventional methods. Section 3 describes proposed impulse noise detection algorithm, proposed dynamically weighted median filter, and proposed adaptive dynamically weighted median filter (AMWMF). Section 4 shows the results and discussion of the proposed filter followed by conclusion in Section 5.

Problem statement
Before introducing some conventional and proposed filters, it is suitable to introduce some notations that will be used in this paper. Image will be represented by notation I. X(i, j) will be used to represent an input pixel on which a particular impulse noise removal filter will be applied; notation Y(i, j) will be used to represent the output of a particular filter to replace X(i, j). A (2N + 1) × (2N + 1) window that is used across X(i, j) for a particular filter is represented by W and is given in (2) So the pixels in window W can be represented by X(i + q, j + t). Median filter proposed by Tukey [8] is given in (3) Adaptive median filter (AMF) [12,13,17,18] uses median filters adaptively. It mostly increases the window size by comparing median value with extreme values of image. The problem with AMF is that there might have background pixels which equal the extreme values of image. In such a case, AMF will keep extending its window size until the window contains a median value that is not equal to one of two extreme values. To improve the performance, weighted median (WM) filter [13] was proposed by assigning weightage to each location with the help of weighted window. WM value of a pixel X(i, j) is where ◊ is the repetition operator such that Figure 1 shows the weightage windows which are used for cases of 3 × 3 and 5 × 5 WM filters. It can be observed that locations near to center have high amount of weights compared to the locations that are away from the center. The sum of all weights in window should be odd. The center weighted median (CWM) [14] is a special case of WM which assigns weight to center pixel only. Duan et al. [35] proposed a method which uses a rank ordered mean (ROM) filter based on a noise detection. Steps for noise removal are as follows: ð Þ 2 −1 or X(i, j) < r 1 , then select it as noisy pixel and apply ROM filter to it by skipping remaining steps for noise detection. If X i; j ð Þ ¼ r 2Nþ1 ð Þ 2 −1 or X(i, j) = r 1 , then select X(i, j) as noise candidate and move to the second step. If none of the above conditions are satisfied, select the pixel as noise-free pixel.
2) Take the sum of absolute difference of X(i, j) with elements in horizontal, vertical, main diagonal, and auxiliary diagonal with respect to X(i, j) and store them in m 1 , m 2 , m 3 , and m 4 respectively. Calculate m = min[m 1 , m 2 , m 3 , m 4 ]. 3) If m > T, then pixel X(i, j) is a noisy pixel, and else, it is a noise-free pixel. 4) Apply ROM filter to pixels that are detected as a noise.
Since ROM filter uses average of two values in window, it blurs the image. Also, if a certain direction among four directions have excess of noise, the sum of absolute difference along that direction will be less than T. Noise detection algorithm will select X(i, j) as noisefree pixel no matter it is noisy or not.
Ibrahim et al. [18] modified AMF to propose an adaptive filter named simple adaptive median filter (SAMF). SAMF uses non-noisy pixels for calculating median value of selected window. It first detects noisy pixels, extends the window if numbers of non-noisy pixels are less than 8, and replace the detected noisy pixel with median of nonnoisy pixels in the window. Another modification is the automatic evaluation of minimum window size like where K is the number of detected noisy pixels in image, and ⌊⌋ corresponds to rounding off to closest integer value. If number of "noise-free pixels" is even, median value will be  FC  TL  FC  TL  FC  TL  FC  TL  FC  TL  FC  TL  FC  TL  FC  TL  FC  replaced by the average of two values that does not exist in its neighborhood region.

Proposed work
Proposed filter consists of two steps, noise detection and noise removal. Both stages are incorporated to each other. Firstly, impulse noise is detected by the proposed efficient method, and one of 3 × 3, 5 × 5, and 7 × 7 dynamically weighted median filters (DWMs) is applied according to the number of noisy pixels in windows.

Proposed impulse noise detection
A new simple impulse noise detection technique is proposed by modifying impulse detection method proposed by Ghanekar [30]. Figure 2 shows the whole flowchart of the proposed impulse noise detection method. The method in [30] is divided in two stages. First stage is based on noise candidate detection, and its steps are given as: 1) Move 3 × 3 window across every pixel x(m, n) at location (m, n). 2) For each window W 3 m;n ð Þ , order the pixels in ascending order to get R(m, n) = [r 1 (m, n), r 2 (m, n), ……, r 9 (m, n)].
3) If x(m, n) = r 1 (m, n) or x(m, n) = r 9 (m, n), then pixel x(m, n) is a noisy candidate, and else, it is a noise free pixel.
Second stage is based on final selection of noisy pixels by processing noise candidates using steps given below 1) Move 11 × 11 window W 11 m 1 ;;n 1 ð Þ across all noisy candidates at locations (m 1 , n 1 ), and sort the window to get (8) as 2) Compute the distance vector D(m 1 , If (x(m 1 , n 1 ) < w min or x(m 1 , n 1 ) > w max ), select x(m 1 , n 1 ) as noisy pixel, where w min = r (p + 1) and w max = r t . p = min[i, j, k, l] and t = max[i, j, k, l].
Method in [30] is an efficient algorithm, but its second stage has a minor problem that diverts true detection from 100% accuracy. Let us consider an example shown below. A 5 × 5 window is used instead of 11 × 11 for better understanding.  18  20 18 20 18  255 0  18 18 18  18  18 0  20 18  20  20 18 20 18  20 18 20 20 18 : If pixel x(m 1 , n 1 ) in a region with uniform values is affected by pepper noise, then D(m 1 , n 1 ) will contain less than four non-zero values as shown below. In this case, d max1 = 255 , d max2 = 18 , d max3 = 2, and d max4 = 0 with i = 24 , j = 3 , k = 16, and l = 1. Hence, p = 1 will results in w min = r 2 = 0. Since the condition x(m 1 , n 1 ) < w min is not satisfied, the algorithm will results in detection of x(m 1 , n 1 ) as noise-free pixel. To avoid this drawback of detection method in [30], an extra step was added to algorithm in [30] after application of fourth step of stage.
if dp m 1 ; The results of proposed impulse noise detection were compared with boundary discriminative noise detection (BDND) [28] algorithm and Ghanekar method [30]. For comparison purpose, BDND [28] algorithm was selected because it provides 100% true detection whereas Ghanekar method [30] was selected to show improvement in results. Performance of an impulse noise detection algorithm depends on its capability to detect all the true positives effectively in the presence of false positives as less as possible. Because if a noise detector detects too much false positives, non-noisy pixels at the locations of false positives will be treated as noisy pixels; hence, their intensity values will be replaced by a new value causing the distortion of information along with increase in image blurring. Table 1 shows false caught (FC) and true left (TL) comparison of BDND [28], Ghanekar [30], and proposed

Adaptive dynamically weighted median filter
Weighted median (WM) filter discussed in Section 2 is followed in this paper. Proposed algorithm addresses a problem that is faced by WM filter in case of presence of noise in nearest neighborhood of noisy pixel. In weighted median filter, if center pixel is noisy or its nearest neighbors are noisy pixels, those noisy pixels will get high weightage and hence will be repeated more. In such case, probability of selection of noisy pixel as median value becomes higher. Even if size of window increases in case of selection of 0 (lowest intensity) or L − 1(highest intensity) as median value, window will keep increasing until the sum of weightage for non-noisy pixels exceeds the sum of weightage of noisy pixels. A new dynamically weighted median filter (DWMF) assigns weightage of 0 to those locations in a W*W window that are detected as noisy pixels using proposed impulse noise detection algorithm. Before discussing steps for proposed adaptive dynamically weighted median filter (ADWMF), some statistics and  calculations that were helpful in proposing adaptive weighted median filter are discussed.

Window selection
Automatic selection of weightage window can be made using 2D Gaussian surface. Gaussian function has the property that its intensity increases as move towards center. Figure 3 shows the process of obtaining 5 × 5 weightage window using 5 × 5 Gaussian surface. Figure 3a shows the intensity values of 5 × 5 Gaussian surface. Intensity values (X) in Fig. 3b were sorted in ascending order as Weights were assigned to those locations using All those weights assigned to sorted elements were placed in corresponding locations of weightage window according to original locations of sorted elements of 5 × 5 Gaussian surface to get weightage window shown in Fig. 3c. Weightage windows for other window sizes were also obtained using Gaussian surface in the same way. Figure 4 shows the flowchart of dynamically weighted median filter (DWMF). Weighted median filter is modified with the help of impulse noise detection algorithm to propose DWMF. Binary image I b and noisy image I are input of DWMF. Binary image I b can be obtained using proposed impulse noise detection algorithm. W*W patches, W n and W b , are selected across all detected noisy pixels in both noisy image I and binary image I b respectively. Weightage window W weights of size W*W is calculated, but those locations of W weights are removed where W b have the values of 1 (if all entries of W b are 1, W weights is replaced by W b to avoid exception). In such way, detected noisy pixels get weightage of 0. Assigned weights in W weights are shifted if gaps are observed in W weights due to removal of elements at noisy locations. For example, if all elements that correspond to weightage of 4 are detected as noisy pixels, weightage of 4 is removed from W weights . Hence, there is a jump of 2 is observed when moving from weightage of 3 to 5 in W weights . In such case, weights are reassigned to decrease the number of repetitions. The modified window is summed, and weight of one of the highest weights is incremented in case if the sum is even. The condition of odd sum is imposed in order to avoid having averaged value of two pixels as a result of DWMF. Final weightage window W R is created after confirming the odd sum of repeated window to create repetition array A R . Noisy pixel is replaced by median value of A R .

Statistics calculation for adaptive criteria
As it is discussed in Section 1, using the same window for an image on the basis of total noise density of image results in failure of removal of noise in case when some regions of image contain more noisy pixels compared to other. However, extending window adaptively without considering performance might not be a good option as well. Considering these problems, the proposed ADWMF uses windows of different sizes according to the number of noisy pixels present in a region. Window is extended when numbers of noisy pixels are greater than threshold set for the current window. These thresholds are defined on the basis of minimum mean square error. Binary image containing locations of detected noisy pixels was obtained using the proposed impulse noise detection. Using that binary image, numbers of detected noisy pixels in 3 × 3, 5 × 5, and 7 × 7 windows were counted to apply the corresponding DWMF filter. The formula for mean square error used is given as where I = Image n = Number of pixels k = kth noisy pixel I k = Original kth pixel value. I k = Recovered kth pixel value. Equation (11) was applied to all 11 test images. Average MSEs were calculated for median, weighted median, and DWMF with 3 × 3, 5 × 5, and 7 × 7 windows to see the possibility of achieving low MSE with combination of hybrid filter of more than one filter or switching between different window sizes, but MSE for DWMF was smaller as compared to median    and weighted median filters. For example, to calculate statistics for DWMF 3 × 3, a 3 × 3 window was moved across every detected noisy pixel and numbers of detected noisy pixels were calculated in that window. All pixels which have the same number of noisy pixels in a 3 × 3 window were grouped together. Similarly, other cases which have different numbers of noisy pixels from 2 to 9 were also calculated and grouped to calculate MSEs together. Similarly for 5 × 5 and 7 × 7 windows, the cases which have different numbers of noisy pixels were in group. Table 2 shows the average MSEs with applications of median, weighted median, and DWMF using 3 × 3, 5 × 5, and 7 × 7 windows for selected number of noisy pixels. From the table, we can find that the DWMF provides better performances for any window size than the other filters. Since the number of noisy pixel cannot exceed the total number of pixels of a window, the maximum number of noisy pixels in a 3 × 3 window becomes 8. MSE for all filters with all three window sizes were examined to see the possibility of proposing a hybrid filter, but it can be seen from the table that proposed DWMF have smaller MSE for all cases. Salt and pepper noise with 30, 50, and 90% densities were added to all 11 test images to obtain statistics for improving. Table 3 shows the selected filters which provide the best performances according to the different numbers of noisy pixels. Figure 5 shows the flowchart of the proposed algorithm. Proposed filter consists of two steps, noise detection and noise removal. Both stages are incorporated to each other. Result of impulse noise detection is a binary image with values of 0 for non-noisy region and values of 1 for noisy pixel region. Numbers of noisy pixels are counted across all detected pixels using 3 × 3 and 5 × 5 windows, and then, a dynamically weighted median filter (DWM) with a specific window size is adaptively applied according to the presence of number of noisy pixels in a region. Impulse noise detection is not only used to count    the number of noisy pixels but also used in assigning weightage of 0 to detected noisy locations in 3 × 3, 5 × 5, and 7 × 7 windows. Thresholds given in Table 3 are used to propose ADWMF. Steps for ADWMF are given as:  Table 3 and apply 5 × 5 DWMF and 7 × 7 DWMF respectively. Subtract I 5 bDWM from I b after applying 5 × 5 DWMF so that 7 × 7 DWMF does not reprocess pixels processed by 5 × 5 DWMF. 4 Results and discussion Figure 6 shows the set of images used for comparisons of algorithms. Performance evaluation was made on the basis of comparison of peak signal-to-noise ratio (PSNR) and mean structural similarity (MSSIM) of adaptive DWMF with methods proposed by adaptive median filter (AMF), adaptive weighted median filter (AWMF), Ibrahim et al. [19] (AdFuz), Bhadouria [21], and Bhateja et al. [20].  [20]. e SAMF. f Bhadouria [21]. g Proposed ADWMF Fig. 7 Zoomed results with 30% impulsive noise. a Original image. b AMF. c AWMF. d Bhateja [20]. e SAMF. f Bhadouria [21]. g Proposed ADWMF Khan and Lee EURASIP Journal on Advances in Signal Processing (2017) 2017:67

Proposed adaptive filter
Page 11 of 14 PSNR calculation was done using (12) as given below Tables 4, 5, and 6 show the PSNR comparisons of proposed ADWMF with conventional methods for 30, 50, and 90% noise densities respectively. Adaptive weighted median filter (AWMF) is an adaptive form of simple weighted median filter. PSNR comparisons show that the proposed method outperforms all other methods for both low-and high-density noise. Tables 7, 8, and 9 show the MSSIM comparisons of all methods. The proposed method outperformed all other methods in both PSNR and MSSIM comparisons.
Tables 10, 11, and 12 show the CPU elapsed time comparisons to show the complexity of all methods. For 30 and 50% noise densities, the proposed method provides slightly higher complexity compared to other methods; however, for 90% noise, the proposed method needs more time since 7 × 7 window is used too often. Figures 7, 8, and 9 show the zoomed results of all methods. SAMF among conventional methods performs well for low-density noise; however, for higher densities, it provides artifacts along edges. For 90% noise density, AMF and AWMF fail to remove noise, whereas SAMF, Bhadouria [20], and Bhateja [21] restore all noisy pixels, however, at the cost of blurring along edges. The proposed method not only preserves edges for low density of noise but also performs very well for noise density as high as 90%. Conventional method totally fails to preserve regions that have pure white or pure black background as in Fig. 8, because pure white or pure black regions provide confusion as they have same intensity as of the impulse noise; however, the proposed method, irrespective of intensity values, provides edge and detail preservation in all cases.

Conclusions
A new impulse noise removal filter, dynamically weighted median filter (DWMF) along with modified impulse noise detection method, is proposed in this paper. Conventional modified versions of switching median filters extend window size either according to the total noise density of image that results in failure to remove noise from regions that have excessive noise in a region due to uneven distribution of noise in different regions. Conventional versions of adaptive median filters extend its window size either when none of noise-free pixel is available or when median filter of a window is a noisy pixel. Such approaches does not consider the performance analysis. Proposed method overcomes problems mentioned above by considering both different amounts of noisy pixels in each region as well as ensuring the best performance. The proposed method first detects locations of noisy pixels in image followed by noise removal stage. Proposed impulse noise detection algorithm is a modified version of impulse noise detection algorithm proposed by Ghanekar [30]. Weighted median filter is modified by assigning weights dynamically by combining it with proposed impulse noise detection technique to propose DWMF. Impulse noise detection algorithm is not only used for assigning weights dynamically Fig. 9 Zoomed results with 90% impulsive noise. a Original image. b AMF. c AWMF. d Bhateja [20]. e SAMF. f Bhadouria [21]. g Proposed ADWMF but also used with DWMF to propose a new adaptive filter named adaptive dynamically weighted median filter (ADWMF). Experiments were performed to evaluate the thresholds at which DWMF should switch to a larger window to provide better performance. Statistics of those experiments were used to propose ADWMF. Switching of window for ADWMF is made on the basis of number of detected noisy pixels in a specific region instead of applying DWMF of a specific size for whole image on the basis of total noise density present in it. ADWMF can be applied to any kind of image whether effected by low or high density of noise. It is not necessary for noise to be evenly distributed in image; ADWMF can restore the image even if some portions have access of noise density. Both objective and subjective simulations were performed which show that ADWMF outperforms conventional methods.
In future, after researching about the possibility where the proposed noise detection and filtering approach can be applied to some practical industrial processes [36,37], it might be investigated.