An iterative enhanced super-resolution system with edge-dominated interpolation and adaptive enhancements

Abstract

For super-resolution (4K × 2K) displays, super-resolution technologies, which can upsample videos to higher resolution and achieve better visual quality, become more and more important currently. In this paper, an iterative enhanced super-resolution (IESR) system which is based on two-pass edge-dominated interpolation, adaptive enhancement, and adaptive dithering techniques is proposed. The two-pass edge-dominated interpolation with a simple and regular kernel can sharpen visual quality while the adaptive enhancement can provide high-frequency perfection and the adaptive dithering conveys naturalization enhancement such that the proposed IESR system achieves better peak signal-to-noise ratio (PSNR) and exhibits better visual quality. Experimental results indicate that the proposed IESR system, which improves PSNR up to 28.748 dB and promotes structural similarity index measurement (SSIM) up to 0.917611 in averages, is better than the other existing methods. Simulations also exhibit that the proposed IESR system acquires lower computational complexity than the methods which achieve similar visual quality.

1 Introduction

Currently, the super-resolution displays with 4K × 2K pixels are vigorously available in the commercial market; however, the existing TV programs are mostly with either standard definition (SD) with 640 × 480 or high definition (HD) with 2K × 1K resolution. In other words, there are almost no super-resolution programs to match up with 4K × 2K TV displays. Thus, the super-resolution technologies, which can upsample SD or HD videos to higher resolution, become more and more important for current applications. Super-resolution (SR) is a technique to recover a higher resolution image from a given low-resolution (LR) image. Simply, the SR algorithm could be treated as an interpolation method to enhance the resolution of images or videos. The interpolated image usually could still lose some detailed information. For 4K × 2K TV displays, it is noted that the SR algorithms should consider real-time implementation issues.

In the literatures, the SR algorithms can be classified into interpolation-based, reconstruction-based, and learning-based approaches to solve the problem of recovering detailed information extracted from the low-resolution image. The interpolation-based approach involves in the prediction of the unknown pixels by filtering processes. Based on the concept of ideal low-pass filtering, the interpolation methods [1] always need to consider the balance of computational complexity and reconstruction quality. The linear, bi-cubic, and cubic spline interpolations [2] are the possible means for reducing the complexity. To prevent filtering pixels across edges, numerous edge-directed interpolation methods are proposed [3-11].

The reconstructed-based approach generates high-resolution images by exploiting the information from a set of successive low-resolution images in the same scene but with sub-pixel displacements. In the wavelet domain [12], the LR image is considered as the lower sub-band of the wavelet-transformed high-resolution (HR) image. However, they are difficult in estimating the unknown coefficients of the other three higher wavelet sub-bands due to their independencies. Instead of the frequency domain methods, the most contemporary methods turn their attentions to solve the problem in the spatial domain [13]. The back projection algorithm iteratively projects the error between the simulated and input LR images to estimate HR error by iteratively minimizing the reconstruction error [14]. However, many jaggy artifacts along the edges may affect the quality of images. To reduce these artifacts, Dong et al [15] employed the nonlocal image redundancy to improve the quality of SR images. In the same time, it brings heavy computation complexity for updating the reconstruction error in each step. The projection onto convex sets (POCS) algorithms [16,17] applied to the input LR images could increase the solution of the element on the convex set. The maximum a posteriori (MAP) methods [18,19] adopt the associated probability of target high-resolution images to form a prior probability to refer the solution based on Bayesian inference.

The learning-based approaches [20,21] attempt to capture the correlation between LR and HR patches to exploit the redundant high-frequency information which is remained in HR training samples. Although these algorithms need a large number of databases to store millions of LR and HR patch pairs, the quality of reconstructed images can be improved even that the magnificent factor is large.

The dictionary learning-based denoising approach [22] used taxonomy based on image representations for a better understanding of state-of-the-art image denoising techniques. The multiresolution structure and sparsity of wavelets are used for nonlocal dictionary learning in each decomposition level of the wavelets [23]. The classification-based least squares trained filters on picture quality improvement algorithms are suggested [24].

In this paper, we propose an iterative enhanced super-resolution (IESR) system, which is based on two-pass edge-dominated interpolation by adding adaptive enhancement and dithering mechanisms. The proposed (IESR) system is based on iterative back projection concept [14]; however, the proposed two-pass edge-dominated interpolation consists of two adaptive fixed-structure filters. Besides, we further include the adaptive enhancement and adaptive dithering units to improve the quality of the HR image in iterative cycles. The overview of the proposed super-resolution system is addressed in Section 2. The edge-dominated interpolation methods will be described in Section 3 while the adaptive enhancement and adaptive dithering algorithms will be present in Section 4. In Section 5, the experimental results for verifying the proposed algorithms in comparison to the well-known super-resolution algorithms are demonstrated. Finally, the conclusions about this paper are addressed in Section 4.

2 Overview of the proposed super-resolution system

The block diagram of the proposed super-resolution system is shown in Figure 1. The flow diagram of the proposed super-resolution system is exhibit in Figure 2. The detailed descriptions of the proposed system are shown as the follows.

Figure 1

Block diagram of the proposed IESR system.

Figure 2

Flowchart of the proposed IESR system.

For later performance comparisons, as shown in Figure 1, the LR image, I ^L as the test image is obtained by down sampling a HR image, I ^H. For down sampling, we adopt Lanczos2 low-pass filtering to eliminate high-frequency components to prevent from the aliasing effect. The reconstruction kernel of Lanczos one-dimensional (1-D) low-pass filter is given as:

$$ L(x)\kern0.5em =\kern0.5em \left\{\begin{array}{c}\hfill nsin\left(\pi x\right) \sin \left(\pi x/n\right)/{\pi}^2{x}^2,\hfill \\ {}\hfill 0,\hfill \end{array}\begin{array}{c}\hfill \kern0.36em \mathrm{if}\kern0.24em 0\kern0.5em <\kern0.5em \left|x\right|\kern0.5em <a\hfill \\ {}\hfill \mathrm{otherwise},\hfill \end{array}\right. $$

(1)

where n is typically two or three. As shown in Figure 3, Lanczos2 filter is an approximation of the ideal low-pass filter, which is specified by sinc function.

Figure 3

The reconstruction kernel of Lanczos2 sinc low-pass filter.

For half-pixel interpolation, the 1-D Lanczos2 filtering process can be expressed by:

$$ {l}_0\kern0.5em =\kern0.75em -\kern0.5em 0.032\ {s}_{-2}\kern0.5em +\kern0.5em 0.284{s}_{-1}\kern0.5em +\kern0.5em 0.496{s}_0\kern0.5em +\kern0.5em 0.284{s}_1\kern0.5em -\kern0.5em 0.032{s}_2, $$

(2)

where s-₂, s-₁, s ₀, s ₁, and s ₂ denote the five consecutive inputs while l ₀ represents the co-located low-pass filtering result at s ₀. In (2), the coefficients are calculated by Lanczos kernel to predict half pixels through sinc functions. It is noted that the symmetrical extension of image pixels is conducted for the image borders while applying low-pass filtering. With the about filtering process of the SR image, we then perform the two-to-one down sampling for horizontal and vertical to get the LR image, which will be treated as the input for testing SR algorithms while the original HR image will be used for performance evaluation of the proposed and other super-resolution algorithms. It is noted that the LR images might not have their related HR image and the LR image obtained from the HR image could be different from the data generation exhibited in Figure 2 normally.

In the proposed super-resolution system, the input low-resolution image, I ^L is first upsampled by the proposed two-pass edge-dominate interpolation (TEI) to become the initially restored HR image, ${\tilde{I}}_{(0)}^H$ at k = 0. For the kth (k > 0) iteration, the adaptive enhancement and dither noise, ${D}_{(k)}^H$ is added to the previous (k − 1)th iteration result to obtain the kth restored HR image as:

$$ {\tilde{I}}_{(k)}^H={\tilde{I}}_{\left(k-1\right)}^H+{D}_{(k)}^H $$

(3)

The detailed descriptions about the adaptive enhancement and adaptive dithering to obtain ${D}_{(k)}^H$ will be addressed in Section 4.

Similar to data generation, the kth LR image, ${I}_{(k)}^L$ is computed by down sampling of the kth restored HR image, ${\tilde{I}}_{(k)}^H$ after Lanczos2 sinc filtering. The restored LR error image with respect to the low-resolution image, I ^L is computed by:

$$ {e}_{(k)}^L\kern0.5em ={I}_{(k)}^L\kern0.5em -{I}^L. $$

(4)

If the restored error, ${e}_T\kern0.5em =\kern0.5em \left\Vert {e}_{(k)}^L\right\Vert$ is less than a predetermined threshold ε or the number of iterations, k is equal to the maximum limited number M, the whole iterative super-resolution process will be terminated. Thus, the final restored HR image, ${\tilde{I}}_{(k)}^H$ will be the output HR image.

If the iterative process is not terminated, the LR error image ${e}_{(k)}^L$ is then upsampled by the proposed two-pass edge-dominated interpolation to become the HR error image, ${e}_{(k)}^H$. From ${e}_{(k)}^H$, we can estimate the enhancement HR image, ${E}_{(k)}^H$, and its dithering image, ${D}_{(k)}^H$ for the next iteration. The proposed TEI will be discussed in the next section.

3 Two-pass edge-dominated interpolation

The TEI is shown in Figure 4. To gain more accurate interpolation value, the YUV color space is adopted in the proposed method. The Y component, also called luma component, represents the details of the texture. Therefore, the edge-dominated weights of the Y component are used to perform the interpolations of U and V components. In other words, the dominated weights used for Y component will be directly adopted for U and V components to save the computation and raise the interpolation performance with respect to the TEI performed in the RGB space.

Figure 4

The proposed two-pass edge-dominated interpolation. Which is the simplified notation of the proposed TEI-based up sampling depicted in Figure 1.

Figure 5a shows the two-pass edge-dominated interpolation, which first performs diagonal interpolation and then vertical-horizontal interpolation. Figure 5b exhibits relationships among the known (black) pixels, the first-pass interpolated (yellow) pixels, and the second-pass interpolated (white) pixels.

Figure 5

Two-pass edge-dominated interpolation. (a) Flow diagram and (b) the known and interpolated pixels handled in the first and second passes.

The first pass is to perform diagonal pixel (yellow) interpolation by using the edge-dominated concept. For each target (yellow) pixel x ₀, there are four adjacent known (black) pixels, x ₁, x ₂, x ₃, and x ₄ from the LR image used to estimate it as:

$$ {x}_0\kern0.5em =\kern0.5em {\displaystyle {\sum}_{m=1}^4\kern0.5em {a}_m{x}_m/\kern0.5em {\displaystyle {\sum}_{m=1}^4\kern0.5em {a}_m}}, $$

(5)

where a _k for k = 1, 2, 3, and 4 are called as the edge-dominated weights for the target pixel. The edge-dominated weights can be computed by:

$$ {a}_m\kern0.5em =\kern0.5em {e}^{-{s}_m/c},\kern0.5em \mathrm{f}\mathrm{o}\mathrm{r}\kern0.5em m\kern0.5em =\kern0.5em 1,\kern0.5em 2,\kern0.5em 3,\kern0.5em \mathrm{and}\ 4, $$

(6)

where c is a control parameter and is set to 32 in this paper. To retrieve the edge information, we should compute the edge sensitivities s _m first. As shown in Figure 6, the edge sensitivities s _m are suggested as:

Figure 6

Diagonal interpolation and its associated weights and edge sensitivities: target pixel (red color) and known (black color) pixels.

$$ {s}_1\kern0.5em =\kern0.5em \left|{x}_1\kern0.5em -\kern0.5em {x}_3\right|\kern0.5em +\kern0.5em \gamma \left|{x}_5\kern0.5em -\kern0.5em {x}_4\right|\kern0.5em +\kern0.5em \gamma \left|{x}_6\kern0.5em -\kern0.5em {x}_2\right|, $$

(7)

$$ {s}_2\kern0.5em =\kern0.5em \left|{x}_2\kern0.5em -\kern0.5em {x}_4\right|\kern0.5em +\kern0.5em \gamma \left|{x}_7\kern0.5em -\kern0.5em {x}_1\right|\kern0.5em +\kern0.5em \gamma \left|{x}_8\kern0.5em -\kern0.5em {x}_3\right|, $$

(8)

$$ {s}_3\kern0.5em =\kern0.5em \left|{x}_1\kern0.5em -\kern0.5em {x}_3\right|\kern0.5em +\kern0.5em \gamma \left|{x}_2\kern0.5em -\kern0.5em {x}_9\right|\kern0.5em +\kern0.5em \gamma \left|{x}_4\kern0.5em -\kern0.5em {x}_{10}\right|, $$

(9)

$$ {s}_4\kern0.5em =\kern0.5em \left|{x}_2\kern0.5em -\kern0.5em {x}_4\right|\kern0.5em +\kern0.5em \gamma \left|{x}_1\kern0.5em -\kern0.5em {x}_{12}\right|\kern0.5em +\kern0.5em \gamma \left|{x}_3\kern0.5em -\kern0.5em {x}_{11}\right|, $$

(10)

where γ is a parameter to control the weights of reference paired pixels. If any subtracting paired pixels across an edge, the difference of two pixels stated in (7), (8), (9), or (10) becomes large. Physically, the larger the edge sensitivity s _m is, the less similarity to x _m will be.

After the first-pass interpolation, the rest (white) pixels are calculated in the second step. Of course, we will not only use the pixels (black) of the original LR image but also the pixels (yellow) obtained from the first step. As shown in Figure 7, to find the target pixel (the center white pixel), y ₀ in either horizontal or vertical interpolation, we also perform the edge-dominated interpolation as:

Figure 7

Vertical/horizontal interpolation and its associated weights and the edge sensitivities. Vertical/horizontal interpolation and its associated weights and the edge sensitivities of the target pixel y ₀ (white) and known (black and yellow) pixels.

$$ {y}_0\kern0.5em =\kern0.5em \left\{\begin{array}{c}\hfill \left({b}_1{y}_1\kern0.5em +\kern0.5em {b}_3{y}_3\right)/\left({b}_1\kern0.5em +\kern0.5em {b}_3\right),\mathrm{if}\kern0.4em {s}_h\kern0.5em >\kern0.5em {s}_v\hfill \\ {}\hfill \left({b}_2{y}_2\kern0.5em +\kern0.5em {b}_4{y}_4\right)/\left({b}_2\kern0.5em +\kern0.5em {b}_4\right),\kern1.60em \mathrm{if}\ {s}_h\kern0.5em <\kern0.5em {s}_v\hfill \\ {}\hfill {\displaystyle {\sum}_{n=1}^4{b}_n{y}_n/\kern0.5em {\displaystyle {\sum}_{n=1}^4{b}_n}},\kern2em \mathrm{otherwise}\hfill \end{array}\right., $$

(11)

where

$$ {b}_n\kern0.5em =\kern0.5em {e}^{-{D}_n/c},\kern0.75em n\kern0.5em =\kern0.5em 1,\kern0.5em 2,\kern0.5em 3,\kern0.5em 4, $$

(12)

and

$$ {D}_1\kern0.5em =\kern0.5em \left|{y}_1\kern0.5em -\kern0.5em {y}_3\right|\kern0.5em +\kern0.5em \left|{y}_5\kern0.5em -\kern0.5em {y}_4\right|\kern0.5em +\kern0.5em \left|{y}_6\kern0.5em -\kern0.5em {y}_2\right|, $$

(13)

$$ {D}_2\kern0.5em =\kern0.5em \left|{y}_4\kern0.5em -\kern0.5em {y}_2\right|\kern0.5em +\kern0.5em \left|{y}_1\kern0.5em -\kern0.5em {y}_7\right|\kern0.5em +\kern0.5em \left|{y}_3\kern0.5em -\kern0.5em {y}_8\right|, $$

(14)

$$ {D}_3\kern0.5em =\kern0.5em \left|{y}_1\kern0.5em -\kern0.5em {y}_3\right|\kern0.5em +\kern0.5em \left|{y}_4\kern0.5em -\kern0.5em {y}_{10}\right|\kern0.5em +\kern0.5em \left|{y}_2\kern0.5em -\kern0.5em {y}_9\right|, $$

(15)

$$ {D}_4\kern0.5em =\kern0.5em \left|{y}_4\kern0.5em -\kern0.5em {y}_2\right|\kern0.5em +\kern0.5em \left|{y}_{12}\kern0.5em -\kern0.5em {y}_1\right|\kern0.5em +\kern0.5em \left|{y}_{11}\kern0.5em -\kern0.5em {y}_3\right|, $$

(16)

The horizontal or vertical interpolation is determined by computation of s _h and s _v as shown in Figure 7. The horizontal and vertical sensitivities of y ₀ can be respectively formulized as:

$$ {s}_h\kern0.5em =\kern0.5em \left|{y}_4\kern0.5em -\kern0.5em {y}_2\right|\kern0.5em +\kern0.5em \left|{y}_{12}\kern0.5em -\kern0.5em {y}_1\right|\kern0.5em +\kern0.5em \left|{y}_1\kern0.5em -\kern0.5em {y}_7\right|\kern0.5em +\kern0.5em \left|{y}_{11}\kern0.5em -\kern0.5em {y}_3\right|\kern0.5em +\kern0.5em \left|{y}_3\kern0.5em -\kern0.5em {y}_8\right|, $$

(17)

and

$$ {s}_v\kern0.5em =\kern0.5em \left|{y}_1\kern0.5em -\kern0.5em {y}_3\right|\kern0.5em +\kern0.5em \left|{y}_5\kern0.5em -\kern0.5em {y}_4\right|\kern0.5em +\kern0.5em \left|{y}_4\kern0.5em -\kern0.5em {y}_{10}\right|\kern0.5em +\kern0.5em \left|{y}_6\kern0.5em -\kern0.5em {y}_2\right|\kern0.5em +\kern0.5em \left|{y}_2\kern0.5em -\kern0.5em {y}_9\right|. $$

(18)

It is noted that the computation of edge sensitivities and horizontal/vertical sensitivities, which involves the sum of selected absolute differences, can be dramatically reduced if we could properly reuse the absolute differences.

4 Image enhancement and dithering algorithms

The most edge-directed interpolation algorithms [3-11] including the proposed TEI method can successfully enlarge the low-resolution images to super-resolution ones with less artifacts along the texture edges. However, the loss of high-frequency components cannot be properly restored by general interpolation algorithms. To recover high-frequency parts of the original image, in this section, we further propose to utilize the adaptive enhancement (AE) and adaptive dithering (AD) algorithms for the Y component to further improve the quality of HR images iteratively.

From the LR reconstruction error, ${e}_{(k)}^L$ which is stated in (4), we can use the same two-pass edge-dominated interpolation to upsample ${e}_{(k)}^L$ to the HR reconstruction error, ${e}_{(k)}^H$ as shown in Figure 2. To eliminate the HR reconstruction error, of course, we should try to remove ${e}_{(k)}^H$ from the restored HR image. It is noted that ${e}_{(k)}^H$, which is recovered from the low-resolution error, only contains the low-frequency part of the HR reconstruction error.

In order to restore the high-frequency part, we suggest an adaptive high-pass image enhancement filter as:

$$ H(x)\kern0.5em =\kern0.5em 1\kern0.5em -\kern0.5em L(x)\kern0.5em =\kern0.5em \left\{\begin{array}{c}\hfill 1-n \sin \left(\pi x\right) \sin \left(\pi x/n\right)/{\pi}^2{x}^2,\hfill \\ {}\hfill 1,\hfill \end{array}\begin{array}{c}\hfill \kern0.36em \mathrm{if}\kern0.24em 0<\left|x\right|<a\hfill \\ {}\hfill \mathrm{otherwise},\hfill \end{array}\right. $$

(19)

which is the inverse of the reconstruction kernel of Lanczos sinc function. To obtain high-pass filtering, the associated Lanczos2 high-pass 1-D filtering result is given by:

$$ {h}_0\kern0.5em =\kern0.5em 0.032\ {s}_{-2}\kern0.5em -\kern0.5em 0.284{s}_{-1}\kern0.5em +\kern0.5em 0.504{s}_0\kern0.5em -\kern0.5em 0.284{s}_1\kern0.5em +\kern0.5em 0.032{s}_2, $$

(20)

where h ₀ represents the co-located high-pass filtering result at s ₀ . With symmetrical extension of image borders of ${\tilde{I}}_{(k)}^H$, the high-pass part of the kth restored HR image, ${h}_{(k)}^H$ can be retrieved.

Thus, the proposed adaptive enhancement algorithm is expressed by:

$$ {E}_{(k)}^H\kern0.5em =\kern0.5em \alpha (k){h}_{(k)}^H\kern0.5em -\kern0.5em \left(1\kern0.5em -\kern0.5em \alpha (k)\right){e}_{(k)}^H, $$

(21)

where α(k) is a decay function with k such that the high-frequency enhancement will be gradually reduced and the error compensation can be increased after iterations. The above HR enhancement algorithm can only recover the high-frequency components, whose magnitudes are partially distorted.

To further enhance the quality of the super-resolution image, we further add adaptive random Gaussian noise into ${E}_{(k)}^H$ to become:

$$ {D}_{(k)}^H={E}_{(k)}^H+\beta (k){\sigma}_q{n}_{(k)}^H, $$

(22)

where $\beta$(k) is a decay dithering function in the kth iteration such that the dithering will be gradually reduced after iterations. In (22), σ _q is the root mean square of ${E}_{(k)}^H$ in a q × q window centered at the dithering pixel, and the size of window is set to three in this paper. By borrowing the MAP concept [18,19], the prior probability of the high-resolution images is based on Gaussian noise model, ${n}_{(k)}^H=N\left(0,\;1\right)$ .If the termination is at the Mth iterations, the high-resolution image ${\tilde{I}}_{(M)}^H$ will be the output of the proposed IESR system.

5 Simulation results

Figure 8 shows all the test images, whose resolutions are from 2,048 × 2,560 to 6,000 × 4,190. More details of the resolution for each image can be found in Table 1. In the experiments, the original HR images are downscaled by factor 2 to generate the LR images with Lanczos filtering. For comparisons, two objective measures, peak signal-to-noise ratio (PSNR) and structural similarity index measurement (SSIM) are used to evaluate the performances of the super-resolution algorithms. The PSNR is the ratio of the maximum power and the noise power, which is defined as:

Figure 8

Test images (read from left to right in rows). (a) N1.bmp, (b) N2.bmp, (c) N4.bmp, (d) N5.bmp, (e) N6.bmp, (f) N7.bmp, (g) N8.bmp, (h) QFHD_P01.bmp, (i) QFHD_P03.bmp, (j) QFHD_P04.bmp, and (k) world_statelite.bmp.

Table 1 Quality evaluation of the proposed methods in PSNR (dB)

$$ \mathrm{PSNR}\kern0.5em =10 \log {}_{10}\frac{255^2}{\mathrm{MSE}}, $$

(23)

where

$$ \mathrm{M}\mathrm{S}\mathrm{E}\kern0.5em =\kern0.5em \frac{{\displaystyle {\sum}_{M,N}{\left[{\tilde{I}}_{(M)}^H\left(x,y\right)\kern0.5em -\kern0.5em {I}^H\left(x,y\right)\right]}^2}}{P\times N}, $$

(24)

which denotes the minimum square error between computed super-resolution image, ${\tilde{I}}_{(M)}^H$ and the original super-resolution image, I ^H(x, y), where M and N are the sizes of row and column, respectively, and (x, y) means the position of the pixel. The SSIM is another measurement system to compare the similarity of two images, which is defined as:

$$ \mathrm{SSIM}\left(x,y\right)\kern0.5em =\kern0.5em {\scriptscriptstyle \frac{\left(2{\mu}_c{\mu}_o\kern0.5em +\kern0.5em {c}_1\right)\left(2{\sigma}_{oc}\kern0.5em +\kern0.5em {c}_2\right)}{\left({\mu}_c^2\kern0.5em +\kern0.5em {\mu}_o^2\kern0.5em +\kern0.5em {c}_1\right)\left({\sigma}_c^2\kern0.5em +\kern0.5em {\sigma}_o^2\kern0.5em +\kern0.5em {c}_2\right)}}, $$

(25)

where μ _c and μ _o denote the average values and ${\sigma}_c^2$ and ${\sigma}_o^2$ are the variances of ${\tilde{I}}_{(M)}^H$ and I ^H(x, y), respectively, while σ _oc exhibits their covariance. The variables c ₁ and c ₂, which are used to avoid instability with weak denominator, are given by:

$$ {c}_1={\left({k}_1L\right)}^2\kern0.24em {c}_2={\left({k}_2L\right)}^2, $$

(26)

where L is dynamic range of the pixel values, like 255 for 8 bits. In (26), k ₁ and k ₂ are set to 0.01 and 0.03, respectively.

To exhibit the effectiveness of the proposal TEI, the AE, and AD algorithms are tested by various combinations, which are the TEI only, the ‘TEI + AE,’ and the ‘TEI + AE + AD’. For AE and AD algorithms, we suggest two decay functions. For the kth iteration, the linear decaying function is given as α ₁(k) = 0.5 − 0.1235(k − 1) while the power-of-two exponential decay function is defined as α ₂(k) = 2^− k.

For evaluation of ‘TEI only,’ ‘TEI + AE,’ and ‘TEI + AE + AD’ combinations, Tables 1 and 2 show PSNR (dB) and SSIM performances for these three combinations, respectively. All the proposed two-pass edge-dominated interpolation, adaptive enhancement, and adaptive dithering could help to achieve better performances in the proposed IESR system. The experimental results show that the power-of-two exponential decay function achieves better performance and less computation since it only involve bit-shift operations.

Table 2 Quality evaluation of the proposed methods with SSIM

Tables 3 and 4 show the PSNR and SSIM performances achieved by the proposed and the well-known SR algorithms, respectively. We observed that the proposed (TEI + AE + AD) method achieves the PSNR up to 36.643 dB and SSIM up to 0.983475, while the averaged PSNR is 28.748 dB and the averaged SSIM reaches to 0.917611. The proposed IESR system outperforms the other well-known super-resolution algorithms. Although the iterative back projection (IBP) [14] and the nonlocal back projection (NBP) algorithms [15] achieve similar results, the proposed method takes less execution time as shown in Table 5. For subjective comparisons, Figures 9 and 10 show the cropped super-resolution (SR) images for N1 and QFHD_P01 test images. The cropped images also show exhibit that the proposed methods outperform the exiting algorithms in visual quality.

Table 3 Performance comparison of the proposed and the well-known SR algorithms in term of PSNR (dB)

Table 4 Performance comparison of the proposed and the well-known SR algorithms in term in term of SSIM

Table 5 Computational time (seconds) required by the proposed algorithm and the nonlocal back project methods

Figure 9

Cropped SR (N1) images. (a) Ground truth and the reconstructed images by (b) NNI, (c) bilinear, (d) bi-cubic, (e) spare representation, (f) IBP, (g) NLIBP, and (h) the proposed methods.

Figure 10

Cropped (QFHD_P01) SR images. (a) Ground truth and the reconstructed images by (b) NNI, (c) bilinear, (d) bi-cubic, (e) spare representation, (f) IBP, (g) NLIBP, and (h) the proposed methods.

6 Conclusions

In this paper, a super-resolution algorithm based on edge-dominated interpolation adaptive enhancement and adaptive dithering is proposed. The edge-dominated interpolation can overcome the artifacts of interpolation such that we could have smoother results along the edges. The adaptive image enhancement algorithm can improve the distorted high-frequency parts while the adaptive dithering method can recover the loss of high-frequency components. In this paper, we only use Y component for edge detection, adaptive enhancement, and adaptive dithering such that we can reduce computation time and achieve better quality. The experimental results show that the proposed algorithm achieves PSNR up to 28.748 dB and SSIM up to about 0.918 in average while the computational time is also reasonably low for practical applications. Due to local data usage and regular structures in computation, the proposed super-resolution system is suitable for VLSI implementation.