A New Switching-Based Median Filtering Scheme and Algorithm for Removal of High-Density Salt and Pepper Noise in Images

Abstract

A new switching-based median filtering scheme for restoration of images that are highly corrupted by salt and pepper noise is proposed. An algorithm based on the scheme is developed. The new scheme introduces the concept of substitution of noisy pixels by linear prediction prior to estimation. A novel simplified linear predictor is developed for this purpose. The objective of the scheme and algorithm is the removal of high-density salt and pepper noise in images. The new algorithm shows significantly better image quality with good PSNR, reduced MSE, good edge preservation, and reduced streaking. The good performance is achieved with reduced computational complexity. A comparison of the performance is made with several existing algorithms in terms of visual and quantitative results. The performance of the proposed scheme and algorithm is demonstrated.

1. Introduction

Images are often corrupted by impulsive noise in addition to several other types of noise. There are two models of impulsive noise, namely, salt, and pepper noise and random valued impulse noise. Salt and pepper noise is sometimes called fixed valued impulse noise producing two gray level values 0 and 255. Random valued impulse noise will produce impulses whose gray level value lies within a predetermined range. For example, if gray level exceeds a value , it is a positive impulse ( to 255); if gray level is less than it is a negative impulse (0 to ). Impulse noise is caused by faulty camera sensors, faults in data acquisition systems, and transmission in a noisy channel. Median filtering has been established as a reliable method to remove impulse noise without damaging edge details [1, 2]. The Standard Median Filter (SMF) is effective only at low noise densities. Several methods have been proposed for removal of impulse noise at higher noise densities [3–5]. Recently, computational complexity has become an important consideration in impulse noise removal. Use of a small size fixed window in median filtering keeps the computational load a minimum. However, small window size leads to insufficient noise reduction. Switching-based median filtering has been proposed as an effective alternative for reducing computational complexity. This method involves detection of noisy pixels prior to processing, and filtering is applied only to corrupted pixels while leaving uncorrupted pixels intact. Several switching-based methods have been proposed [6–21]. A recent method named Decision Based Algorithm (DBA) is one of the fastest methods and it is an efficient algorithm capable of impulse noise removal at noise densities as high as 80% [16, 17]. A major drawback of this algorithm is streaking at higher noise densities. The median filter not only smoothes the noise in homogeneous regions but it also tends to produce regions of constant or nearly constant intensity. The shape of these regions depends on the geometry of the filter window. They are usually streaks (linear patches) or amorphous blotches. These side effects of the median filter are highly undesirable, because they are perceived as either lines or contours that do not exist in the original image. The probability that two successive outputs of the median filter have the same value is quite high

(1)

when the input is a stationary random process. When the window size "" tends to infinity, this probability tends to 0.5. Streaking and blotching are undesirable effects. Postprocessing of the median filter output is desirable. A better solution is to use other nonlinear filters based on order statistics, which have better performance than median filter with reduced streaking and computational complexity. Streaking cannot be neglected particularly in high-density noise situations where a large number of pixels in a processing window are noisy pixels. One strategy, which is the simplest, is to replace the corrupted pixel by an immediate uncorrupted pixel. When window is moved to the next position, a similar situation arises. The replacement involves repetition of the uncorrupted pixel. This repetition causes streaking. In several algorithms such as adaptive algorithms and robust estimation algorithms, this repetition is less frequent and therefore is not as visible as in case of DBA. This paper introduces a new switching-based median filtering scheme and algorithm for removal of impulse noise with reduced streaking under the constraint of reduced computational complexity. The algorithm is also expected to provide good noise performance and edge preservation. This paper considers salt and pepper type impulse noise [12–17].

2. Switching-Based Median Filters

Switching-based median filters are well known. Identifying noisy pixels and processing only noisy pixels is the main principle in switching-based median filters. There are three stages in switching-based median filtering, namely, noise detection, estimation of noise-free pixels and replacement. The principle of identifying noisy pixels and processing only noisy pixels has been effective in reducing processing time as well as image degradation. The limitation of switching median filter is that defining a robust decision measure is difficult because the decision is usually based on a predefined threshold value. In addition the noisy pixels are replaced by some median value in their vicinity without taking into account local features such as presence of edges. Hence, edges and fine details are not recovered satisfactorily, especially when the noise level is high. In order to overcome these drawbacks. Chan et al. [16] have proposed a two-phase algorithm. In the first phase an adaptive median filter is used to classify corrupted and uncorrupted pixels. In the second phase, specialized regularization method is applied to the noisy pixels to preserve the edges besides noise suppression. The main drawback of this method is that the processing time is very high because it uses very large window size. There are several strategies for identification, processing, and replacement of noisy pixels. The simplest strategy is to replace the noisy pixels by the immediate neighborhood pixel. The DBA [17] employs this strategy wherein the computation time is the lowest among several standard algorithms even at higher noise densities. A disadvantage of this strategy is increased streaking. It is highly desirable to limit streaking which degrades the final processed image. This is indeed a challenging task under the constraint that the processing time be kept as low as possible while preserving edges and removing most of the noise.

3. New Switching-Based Median Filtering Scheme

This paper develops a new switching-based median filtering scheme for tackling the problem of streaking in switching-based median filters with minimal increase in computational load while preserving edges and removing most of the noise. The new scheme employs linear prediction in combination with median filtering. The proposed scheme is based on a new concept of substitution prior to estimation.

A linear predictive substitution of noisy pixels prior to estimation is proposed. The new scheme consists of four stages, namely, detection, substitution, estimation, and replacement in contrast to the existing schemes which work with three stages, namely, detection, estimation, and replacement.

Stage 1 takes pixels of the input image and identifies pixels corrupted by salt and pepper noise. Salt and pepper noise produces two-level pixels, namely, 0 and 255 and, therefore, identification is straightforward.

Stage 2 employs a simple modified first-order linear predictor whose output is used as a substitution for noisy pixels. It should be stated here that the linear predictor is not used as an estimator in strict sense. This new use of linear predictor is developed in the next section.

Stage 3 estimates denoised pixels. In order to preserve edges, a median filtering is employed that is based on L-estimators [1, 2]. The name L-estimators comes from linear combination of order statistics. An L-estimator can be defined as

(2)

where is the th order statistic of the observation data. The performance of an L-estimator depends on its weights which are some fixed coefficients.

Stage 4 replaces noisy pixels by the estimated pixels.

The methods chosen in each stage are strongly influenced by the goals, namely, good noise performance, reduced streaking, edge preservation, and minimal computational complexity.

4. Linear Predictive Substitution of Noisy Pixels

We consider the case where an image is corrupted by salt and pepper noise at high noise density levels such that more than half of the pixels inside a window (2D-representation) or inside an array (1D representation) are impulses of value 0 or 255. Noise-free pixels take on values between 0 and 255. For the purpose of analytical treatment, let be a set consisting of original noise-free image pixels and the median of Let be a set in which are noise-free pixels, and are pepper noise pixels. Let be the median of For simplicity, it is assumed that the elements of the set are arranged in ascending order of the values of the pixels. Let be substituted by a new set and be the median of The first elements are noise-free pixels from set and the rest of the elements from are substitution pixels for the noisy pixels These substitution pixels are derived from noise-free image pixels as developed in Section 5. In the case of high density noise levels above 50 percent, the median is also a noisy pixel. Let by and be replaced by

Proposition.

If more than half of the elements in the set are outliers, then

(3)

where represents the norm in L1 sense.

Proof.

is an impulse not correlated with because the errors due to faulty operations do not depend on the original signal. Let be the autocorrelation Let be a substitute sample derived from one or more of the noise-free image pixels such that is a prediction. Let be the cross-correlation Now, . If , then impulse noise sample is correlated with and is not correlated with which is a contradiction. This is true for the subsequent elements in the sets and Therefore, . In other words, we propose that in the case of high density impulse noise levels, the median of a substitute set derived from noise-free pixels of the original set according to a predescribed rule that enhances correlation results in a denoised pixel

The next section develops a method for deriving substitute pixels for impulse noise pixels of a given corrupted image.

5. A Low-Order Recursive Linear Predictor from Finite Data

Linear prediction is the problem of finding the minimum mean square estimate of using a linear combination of the past signal values from to The most commonly used forward one step Finite Impulse Response (FIR) linear predictor of order is given by

(4)

where are the coefficients of the prediction filter. The solution is given by the Wiener-Hopf [18] equation

(5)

where is an autocorrelation matrix, is predictor coefficient vector, and is autocorrelation vector. The autocorrelation is defined as

(6)

is defined as for to It is assumed that signal values are real. Consider the set and let be substituted by which is a prediction from or all previous elements. Let so that is the new substitute pixel for Now, let be substituted by the prediction . Again, let . We substitute for and so on. The new set is now wherein are substitution pixels for noisy pixels by linear prediction from noise-free pixels. Rewriting as we have This is the substitution set introduced in Section 4.

The substitution concept proposed in this section requires a recursive-type prediction. One ideal approach is to start from a causal Infinite Impulse Response (IIR) linear predictor [18]. Suppose that the image can be modeled as an Auto Regressive Moving Average (ARMA) process with a known power spectrum such that where is the minimum phase spectral factor and is the variance of the white noise driving the model. The causal Infinite Impulse Response (IIR) predictor is given by which, in time domain, becomes

(7)

In image processing with a short finite data, assumption of a power spectrum with known characteristics is generally not possible. The predictor coefficients can be determined from autocorrelation of the available data where signal model is not available. This is a reasonable approach in realistic situations [18].

Let be a prediction from one or more noise-free pixels. An outlier (a salt or pepper noise pixel) is substituted by . This is acceptable because has some correlation with previous data and, therefore, is a better candidate than an impulse. After substitution, let be treated as an image pixel-free of impulse noise corruption. Let be Define

(8)

Let a first-order recursive linear predictor be defined as . The error due to prediction is . Minimization of the square of the error leads to where The above procedure is repeated for all impulse corrupted pixels. All of the substitute pixels are obtained by this procedure. The resulting set is a substitute set for in this new scheme and not an estimate. We have proved in Section 4 that a subsequent optimization by median filtering of the substitute set takes the current noisy pixel closer to original noise-free image pixel. One of the computationally simplest optimizations that preserve edges is median filtering and, therefore, the resulting substitute pixel set Z is filtered using median operation, which is an L1 optimization in Maximum Likelihood sense. Figure 1 shows the flow chart of the proposed scheme.

Figure 1

Flowchart of the proposed scheme.

There are several advantages of the proposed scheme. In DBA the current noisy pixel under processing is replaced with the median of the processing window. If the median itself is corrupted, then the median is replaced by a previously processed neighborhood pixel. At higher noise densities most of the pixels will be corrupted necessitating repeated replacement. This repeated replacement produces streaking. The proposed method avoids this.

In robust statistics estimation filter [19–21], the current noisy pixel under processing is replaced by an image data estimated using an estimation algorithm. But the computation time is much longer. It will be demonstrated in Section 7 that the linear prediction substitution followed by median filtering as introduced by this paper can overcome the problem of streaking and blur while the computational complexity is reduced in comparison with robust statistics estimation filter.

6. The Proposed Noise Removal Algorithm

Let denote the image corrupted by salt and pepper noise. For each pixel a 2-D sliding window of size is selected in such a way that the current pixel lies at the centre of the sliding window. The proposed algorithm first detects the noisy pixel. If the current processing pixel lies inside the dynamic range [] then it is considered as a noise-free pixel. Otherwise it is considered as a noisy pixel and replaced by a value using the proposed linear prediction algorithm.

Step 1.

A 2-D window "" of size is selected. Assume that current pixel under processing is

Step 2.

If is an uncorrupted pixel and it is left unchanged and the window slides to the next position.

Step 3.

Else is a corrupted pixel and go to Step 10.

Step 4.

Store all the elements of "" in a 1-D array "".

Step 5.

Sort the 1-D array "" in ascending order.

Step 6.

For each pixel in "" of value "255" moving from left to right, replace by a predicted value which is given by where and are autocorrelation for lags 1 and 0.

Assuming stochastic approximation for maintaining simplest computational complexity

(9)

If substitute by (This is a special case when the pixel is a salt noise pixel having the value 0.)

Step 7.

For each pixel in "" of value "0" moving from right to left, replace by a predicted value which is given by, where

(10)

If substitute by (This is a special case when the pixel is a pepper noise pixel having the value 255.)

Step 8.

The new array is Sort the 1-D array "" with predicted values and find the median value.

Step 9.

Replace the current pixel under processing by the above median value.

Step 10.

Steps 1 to 3 are repeated until processing is completed for the entire image.

7. Illustration of the Proposed Algorithm

Each and every pixel of the image is checked for the presence of salt and pepper noise pixel. During processing if a pixel element lies between "0 and 255", it is left unchanged. If the value is 0 or 255, then it is a noisy pixel and it is substituted by a substitution pixel.

Array labeled displays an image corrupted by salt and pepper noise.

Array labeled depicts the current processing window and a pepper noise pixel. The square shown in solid line represents the window; and element inside the circle represents a pepper noise pixel

If the current pixel under processing is between 0 and 255, it is left unchanged. Otherwise it will be replaced by a new pixel value estimated using the proposed algorithm. For this purpose, the elements inside processing window are arranged as an array and sorted in ascending order

Check for the pixel elements of value "255" starting from the left. If the pixel value is "255", then that value will be substituted by a predicted value from the immediate neighborhood pixel. Array Z_Aillustrates this. The element inside the circle is the substitute pixel for the pepper noise pixel. This is repeated for all the pixels having the value "255". Array is sorted again to find the median. This is shown as array The element encircled is the median

Finally, the current noisy pixel in the window in array is replaced with the new median value. The final processed array is shown as

The element encircled in array is the final estimate of the pepper noise pixel of array In the proposed algorithm, a window will slide over the entire image. Computation complexity is minimum with a fixed window. This procedure is repeated for the entire image. Similar procedure can be adopted for the salt noise substitution, estimation, and replacement.

8. Simulation Results and Discussion

In this section, results are presented to illustrate the performance of the proposed algorithm. Images are corrupted by uniformly distributed salt and pepper noise at different densities for evaluating the performance of the algorithm. Three images are selected. They are Lena, Cameraman, and Boat image. A quantitative comparison is performed between several filters and the proposed algorithm in terms of Peak Signal-to-Noise Ratio (PSNR), Mean Square Error (MSE), Image Enhancement Factor (IEF), Mean Structural SIMilarity (MSSIM) Index, and computational time. The results show improved performance of the proposed algorithm in terms of these measures. Matlab R2007b on a PC equipped with 2.21 GHz CPU and 2 GB RAM has been used for evaluation of computation time of all algorithms.

The performance of the algorithm for various images at different noise levels from 70% to 90% is studied, and results are shown in Figures 2–7. The metrics for comparison are defined as follows:

(11)

where is the original image, is the restored image, and is the corrupted image. The Structural SIMilarity index between the original image and restored image is given by SSIM [21] where and are mean intensities of original and restored images, and are standard deviations of original and restored images, and are the image contents of th local window, and is the number of local windows in the image. Figure 2 displays the original and corrupted images of Lena.jpg image. Figure 4 displays the original and corrupted images of Boat.gif image. Figure 6 displays the original and corrupted images of Cameraman.tif image.

Figure 2

(a) Original Lena image. (b) Image corrupted by 70% noise density. (c) Image corrupted by 80% noise density. (d) Image corrupted by 90% noise density.

Figure 3

R esults of different filters for Lena image. (a) Output of SMF. (b) Output of PSMF. (c) Output of AMF. (d) Output of DBA. (e) Output of REMF. (f) Output of PA. Row 1–Row 3 show processed results of various filters for Lena.jpg image corrupted by 70%, 80%, and 90% noise densities.

Figure 4

(a) Original Boat image. (b) Image corrupted by 70% noise density. (c) Image corrupted by 80% noise density. (d) Image corrupted by 90% noise density.

Figure 5

R esults of different filters for Boat image. (a) Output of SMF. (b) Output of PSMF. (c) Output of AMF. (d) Output of DBA. (e) Output of REMF. (f) Output of PA. Row 1–Row 3 show processed results of various filters for Boat.gif image corrupted by 70%, 80%, and 90% noise densities.

Figure 6

(a) Original Cameraman image. (b) Image corrupted by 70% noise density. (c) Image corrupted by 80% noise density. (d) Image corrupted by 90% noise density.

Figure 7

Results of different filters for Cameraman image. (a) Output of SMF. (b) Output of PSMF. (c) Output of AMF. (d) Output of DBA. (e) Output of REMF. (f) Output of PA. Row 1–Row 3 show processed results of various filters for Cameraman.tif image corrupted by 70%, 80%, and 90% noise densities.

In Figures 3, 5 and 7, the first column represents the output of Standard Median Filter (SMF) [4], second column represents the output of Progressive Switching Median Filter (PSMF) [14], third column represents the output of Adaptive Median Filter (AMF) [16], and fourth column represents the output of Decision-Based Algorithm (DBA) [17]. Fifth column represents the output of Robust Estimation Median Filter (REMF) [19] and the sixth column represents the output of the Proposed Algorithm (PA). Tables 1–6 display the quantitative measures. SMF replaces the current pixel by its median value irrespective of whether a pixel is corrupted or not. Therefore, the performance is poor. PSMF has slightly improved performance but its noise removing capacity is very poor at higher noise densities. AMF exhibits improved performance but due to its adaptive nature the computation complexity is much higher. DBA has very good noise removing capability and good edge preservation at higher noise densities but it produces streaking at higher noise densities. REMF has improved performance than DBA but its computational complexity is much higher. Figures 8–11 display the quantitative performance of the various algorithms for cameraman image. It can be observed that the proposed algorithm removes noise effectively even at higher noise levels and preserves the edges and reduces streaking which is a major drawback of DBA while maintaining lower computational complexity when compared to adaptive algorithm and robust statistics-based algorithms. Figure 12 represents the computation time required at various noise densities for different algorithms on cameraman image, and the results are also tabulated in Table 7.

Table 1 PSNR and MSE for various filters for Lena image at different noise densities.

Table 2 IEF and MSSIM for various filters for Lena image at different noise densities.

Table 3 PSNR and MSE for various filters for Boat image at different noise densities.

Table 4 IEF and MSSIM for various filters for boat image at different noise densities.

Table 5 PSNR and MSE for various filters for Cameraman image at different noise densities.

Table 6 IEF and MSSIM for various filters for cameraman image at different noise densities.

Table 7 Comparison of PSNR and CPU time in seconds for cameraman image.

Figure 8

Noise density versus PSNR for cameraman image.

Figure 9

Noise density versus MSE for cameraman image.

Figure 10

Noise density versus IEF for cameraman image.

Figure 11

Noise density versus MSSIM for Cameraman image.

Figure 12

Noise density versus computation time in seconds for Cameraman image.

In the proposed method, replacement by immediate neighborhood is avoided by substitution of noisy pixels potential candidates based on linear prediction. Since linear prediction is employed prior to any processing, repetition of the same pixel is avoided as window is moved from one position to the next position. This eliminates streaking. In the standard switching median filtering except DBA, estimation of noise-free pixels takes considerable time on account of mathematical criteria employed. This time increases significantly in adaptive based estimation techniques. In the proposed filter, the estimation is not based on explicit computation of estimation criteria; instead a median filtering replaces estimation. This is the main reason for reduction in computational complexity. Extra computation necessitated by low-order linear prediction is significantly smaller than techniques employing rigorous estimation schemes. The DBA which is one of the fastest algorithms (which also avoids estimation) involves three median sorting, namely, right sorting, left, and diagonal sorting. In the proposed filter there is only two sortings. Therefore introduction of first-order linear prediction only slightly increases the computation time compared with DBA but much lower than other filters. The proposed algorithm can be a good compromise in preference to the adaptive algorithm, DBA, and robust statistics-based algorithm.

9. Conclusion

A new switching-based median filtering scheme and an algorithm for removal of high-density salt and pepper noise in images is proposed. The algorithm is based on a new concept of substitution prior to estimation in contrast to the standard switching-based nonlinear filters. Noisy pixels are substituted by prediction prior to estimation. A simple novel recursive linear predictor is developed for this purpose. A subsequent optimization by median filtering results in final estimates. The performance of the algorithm is compared with that of SMF, PSMF, AMF, DBA, and REMF in terms of Peak Signal-to-Noise Ratio, Mean Square Error, Mean Structure Similarity Index, and Image Enhancement Factor and Computational time. Both visual and quantitative results are demonstrated. The results show that the notable features of the proposed algorithm are reduced streaking at high noise densities compared to DBA which is one of the fastest algorithm and reduced computational complexity compared to adaptive and robust algorithms. The proposed algorithm can be a good compromise for salt and pepper noise removal in images at high noise densities. However, further reduction in computational complexity is desirable.