A game-theoretic architecture for visible watermarking system of ACOCOA (adaptive content and contrast aware) technique

Abstract

Digital watermarking techniques have been developed to protect the intellectual property. A digital watermarking system is basically judged based on two characteristics: security robustness and image quality. In order to obtain a robust visible watermarking in practice, we present a novel watermarking algorithm named adaptive content and contrast aware (ACOCOA), which considers the host image content and watermark texture. In addition, we propose a powerful security architecture against attacks for visible watermarking system which is based on game-theoretic approach that provides an equilibrium condition solution for the decision maker by studying the effects of transmission power on intensity and perceptual efficiency. The experimental results demonstrate that the feasibility of the proposed approach not only provides effectiveness and robustness for the watermarked images, but also allows the watermark encoder to obtain the best adaptive watermarking strategy under attacks.

1. Introduction

Due to the advancement of digital technologies and rapid communication network deployment, a wide variety of multimedia contents have been digitalized which makes their duplication or circulation easy through both authorized and unauthorized distribution channels. With the advantages of effortless editing and digital data reproduction, the protection of the intellectual rights and the authentication of digital multimedia have become issues of great importance in recent years [1–3]. Among different techniques, visible watermarking schemes protect copyrights in a more active way since the logo watermark are generally embedded in the host image (Figure 1a). Such approach not only allows the observers to easily recognize the property owner of multimedia but also discourage the action of pirates.

Figure 1

The visible watermark embedding procedures. (a) An example of visible watermark embedding. (b) A generic visible watermark embedding diagram.

In this study, we have explored the inter-relationship between the image fidelity and robust requirement of visible watermarking and propose a powerful secure watermarking architecture which is based on game-theoretic methodology. The system provides an equilibrium condition solution for the copyright manager to make a decision by studying the effect of transmission power on intensity and perceptual efficiency. In addition, we have formulated the watermark embedding problem as a dynamic non-cooperative game with complete information [4]. Complete information requires that every player knows the strategies of the other players but not necessarily the actions. Under the complete information, we present a game-theoretic architecture as a watermarking game to analyze the different situation and get the best strategy between the embedding watermark energy and the perceptual translucence for visible watermark where the best strategy is defined by the Nash equilibrium of the game [4]. Tsai and Liu's research [5] has preliminary study for visible watermarking which only applies peak signal noise ratio (PSNR) and correlation for the payoff functions. However, visual image quality measure is very critical for visible watermarking and such an issue should be included and weighted during the algorithm design. Therefore, we here leverage the previous research of [5] not only to consider the above discussion but also improve the visible watermarking technique for a novel payoff function under the game-theoretic architecture.

The rest of this article is organized as follows. In section 2, related works about visible watermarking and game-theoretic architecture will be introduced briefly. In section 3, we will give the detailed description of the proposed watermarking algorithm called ACOCOA (adaptive content and contrast aware) and a power security watermarking architecture design. In section 4, numerical results with discussion will be presented. Finally, the conclusions and future works are in section 5.

2. Related works

2.1. Digital watermarking

Digital watermarking techniques are the process of possibly irreversibly embedding information into a digital signal and they are used to protect copyright of digital multimedia like sound, music, audio, images, or video files that have to be delivered for certain purpose, such as digital multimedia used in exhibition, digital library, advertisement, or distant learning web, while illegal duplication is forbidden.

A review of the literature indicates that the visible watermarking studies have captured significant attention since their applications meet the requirements of many media industries [2, 3].

Through the survey, Braudaway et al. [6] proposed one of the early approaches for visible watermarking by formulating the non-linear equation to divide the linear brightness scale into two regions and accomplish the brightness alteration in spatial domain. Meng and Chang [7] proposed an efficient compressed-domain content-based algorithm which applied the stochastic approximation model for Braudaway's method in the discrete cosine transform (DCT) domain by adding visible watermarks in video sequences. Kankanhalli et al. [8] proposed a coefficient modulation in the DCT domain where the scaling factors are calculated by exploiting the human visual system (HVS), to ensure that the perceptual quality of the host image is preserved. Mohanty et al. proposed a watermarking technique called dual watermark, which is a combination of a visible watermark and an invisible watermark in the spatial domain. The visible watermark is adopted to establish the owner's right to the image and invisible watermark is used to check the intentional and unintentional tampering of image [9]. Due to the watermark insertion is done in the spatial domain, the image fidelity and robustness under attacks is pretty low. Tsai and Lin have developed more advanced approach in [10] by considering the global and local characteristics of the host and watermark images in the discrete wavelet transform (DWT) domain. Consequently, Mohanty et al. [11] also proposed a mathematical modification model for exploiting the texture sensitivity of the HVS in DCT domain. The weakness of this approach is the necessity to keep the watermark secret which is very unrealistic for visible watermarking. Better design is achieved in [12] and the approach is leveraged in this research. Chen [13] has proposed a visible watermarking mechanism to embed a gray level watermark into the host image where the strength of the embedded watermark locally depends on the standard deviation of luminance.

Vehel and Manoury [14] proposed a method for digital image watermarking which is based on the modification of certain subsets of the wavelet packet decomposition (WPD) and the WPD is a generalization of the dyadic wavelet transform with low-pass subbands. Hu and Kwang implemented an adaptive visible watermarking in the wavelet domain by using the truncated Gaussian function to approximate the effect of luminance masking for the image fusion. Based on image features, they first classify the host and watermark image pixels into different perceptual classes. Secondly, they use the classification information to guide pixel-wise watermark embedding. In high-pass subbands, they focus on image features, while in the low-pass subbands, they use truncated Gaussian function to approximate the effect of luminance masking [15, 16]. Yong et al. [17] also proposed a translucent digital watermark in the DWT domain and use error-correct code to improve the ability of anti-attack.

Each of above mentioned schemes was not devoted to better feature-based classification and the use of sophisticated visual masking models. Huang and Tang [18] later presented a contrast sensitive visible watermarking scheme with the assistance of HVS. They utilized the contrast sensitive function (CSF) mask of the DWT domain with square function to determine the mask weights and at last they adjusted the scaling and embedding factors based on the block classification with the texture sensitivity of the HVS for watermark embedding. Tsai [12] improved their approach and further proposed a novel visible watermarking algorithm based on the content and contrast aware (COCOA) technique. He utilized the global and local characteristics of the host and watermark images and considered HVS model in the DWT domain by using the CSF, noise visibility function (NVF), and DWT basis amplitude modulation for the best quality of perceptual translucence and noise reduction.

In summary, Figure 1 describes the generic structure for visible watermark embedding processes. First, a host image (original image) directly embeds watermark in spatial domain or is transformed into frequency domain through the well-known spread spectrum approach [19], i.e., Discrete Fourier Transform (DFT), DCT, or DWT. However, the algorithms using transform domain approach develop more robust watermarking techniques than directly embedding watermark into the spatial domain [3, 18]. Consequently, coefficients are passed through a perceptual analysis block that determines how strong the watermark in embedding algorithm can be, so that the resulting watermarked image is acceptable. The watermark is embedded through using a well-designed algorithm based on mathematical or statistical model. If the host image is employed in frequency domain, the inverse spread spectrum approach is then adopted to obtain a watermarked image [2, 3]. The watermark extraction applies to the similar operations in embedding processes with reverse procedures.

Digital contents embedded with visible watermarks will overlay recognizable but unobtrusive copyright patterns to identify its ownership. Therefore, a visible watermarking technique should retain details of contents and ensure embedded patterns difficult or even hard to be removed, and no one could use watermarked data illegally. How to solve the conflict problem and to determine the best tradeoff between the intensity of embedded watermark and the perceptual translucence for adaptive visible watermark under intentional attacks is becoming a subject of importance [5, 12, 18]. In this article, we present a game-theoretic architecture to solve this gap by proposing the ACOCOA (adaptive content and contrast aware) algorithm that provides more flexible design for encoder to set the energy of embedding watermark. We will introduce the ACOCOA technique and a game-theoretic architecture for visible watermarking system in details.

2.2. Game theory

Game theory is the formal study of the conflict and cooperation. The concepts of a game-theoretic approach help to formulate structure, analyze and understand strategic scenarios, and make a decision whenever the actions of the several agents are interdependent [4]. Game theory aims to help us to understand the situations in which decision-makers interact. Therefore, decision-makers can better estimate the potential effects of their actions and then make the ideal decisions to avoid the conflict.

There are two types of game theory. One is non-cooperative game, which focuses on analyzing each game player to maximize their own profit. The other is the cooperative game, which concentrates on groups of players and may enforce cooperative behaviors. Game theory has applications in several fields, such as economics, auctions, bargaining, politics, law, biology, social network, and voting systems. Some games have been proposed and we will briefly address different game techniques here.

Cohen and Lapidoth [20] computed the coding capacity of the watermarking game for Gaussian covertext and squared-error distortions. Both the public version of the game (covertext known to neither attacker nor decoder) and the private version of the game (covertext unknown to attacker but known to decoder) are treated. Moulin et al. [21] proposed an information-theoretic analysis of information hiding. They describe the fundamental limits of information-hiding system, formulate the information-hiding problem as a communication problem, and seek the maximum rate of reliable communication through the communication system.

Among the various theories of game, Nash equilibrium is one of the most important and widespread equilibrium concepts in the twentieth century. Nash equilibrium is a solution concept of a game involving two or more players, in which each player is assumed to know the equilibrium strategies of the other players, and no player has anything to gain by changing only his or her own strategy unilaterally. If each player has chosen a strategy and no player can benefit by changing his or her strategy while the other players keep theirs unchanged, then the current set of strategy choices and the corresponding payoffs constitute Nash equilibrium [4]. Under such scenario, the situation of visible watermark embedding strategies against attacks can be formulated as a competition game based on the actions of encoder and attackers. Therefore, we proposed a secure watermarking system based on game-theoretic methodology to achieve the objective of watermarking management. The idea of Nash equilibrium is adopted to develop the solution for the non-cooperative problem. Section 3 will describe how we can apply such a concept to make the game design for making decision of the visible watermark embedding procedures.

2.3. Image quality measure

Image quality measure has become crucial for the most image processing application. It can evaluate the numerical error between the original image and the tested image. Several image quality measure metrics have been developed for incorporating the texture sensitivity of the HVS [22]. However, in the real world, there is yet no universal standard for an objective assessment of image quality. From the image visual quality study of [23], Ponomarenko et al. exploited the color image database TID2008 using a wide variety of known image quality metrics by the rank correlations of Spearman and Kendall. TID2008 database contains 1700 distorted images and 17 different types of distortions. They evaluated both full set of distorted test images in TID 2008 and for particular subsets of TID2008 that include distortions most important for digital image processing applications. Under their investigation, MSSIM, PSNR-HVS, and PSNR-HVS-M perform better correlation correspondence of HVS where PSNR-HVS and PSNR-HVS-M produce similar numerical results. In addition, VIF and WSNR show consistent presentation behavior under our study. Therefore, we will briefly explain several used metrics in this article including peak signal-to-Noise Ratios (PSNR), visual information fidelity (VIF), structural similarity (SSIM), mean structural similarity (MSSIM), the PSNR human visual system masking metric (PSNR-HVS-M), and weighted signal-to-noise ratio (WSNR) since several image quality measures will be adopted in the payoff function under the game-theoretic architecture. The formulas of VIF, SSIM, MSSIM, PSNR-HVS-M, and WSNR are explained in Appendix for details.

• (1) PSNR is the most commonly used quality measure for reconstruction of lossy compression codecs such as image compression, image distortion, and so on. The definition of PSNR is as following:

  $$\mathsf{\text{PSNR}}=10{log}_{10}\left(255^2∕\mathsf{\text{MSE}}\right)$$

  (1)

where MSE is the mean square error between original and tested images. In general, typical values for the PSNR in lossy image are between 30 and 50dB [24] and a higher PSNR means that the tested image is less degraded and provides a higher image quality.

• (2) VIF is based on local mutual information which measures how much information could flow from the reference image through the image distortion process to the human observer [22]. It uses natural scene statistics modeling in conjunction with an image-degradation model and the HVS model. The VIF measure can have values in the range [1], with VIF equal to 1 when the two compared images are identical.

• (3) SSIM is a method for measuring the similarity between original and tested images [25]. Typically, it is computed from three measurement comparisons: luminance, contrast and structure with the window sizes of 8 × 8. The window can be displaced pixel-by-pixel on the image but the authors propose to use only a subgroup of the possible windows to reduce the complexity of the calculation. In practice, one usually requires a single overall quality measure of the entire image; thus, the mean SSIM index is computed to evaluate the overall image quality. The SSIM can be viewed as a quality measure of one of the images being compared, while the other image is regarded as perfect quality. Similar to SSIM, the MSSIM [25] method is a convenient way to incorporate image details at different resolutions. The results of SSIM and MSSIM can be between 0 and 1, where 1 means excellent quality and 0 means poor quality.

• (4) PSNR-HVS-M is peak signal to noise ratio taking into account of CSF and between-coefficient contrast masking of DCT basis functions [26, 27]. Similar to PSNR, a higher PSNR-HVS-M value means that the tested image is less degraded.

• (5) WSNR [28] is a method, which uses the CSF as the weighting function by defining WSNR as the ratio of the average weighted signal power to the average weighted noise power. As HVS is not equally sensitive to all spatial frequencies, CSF is taken into account where CSF is simulated by a low-pass or band-pass frequency filter. Similar to PSNR, a higher WSNR value means that the tested image is less degraded.

3. The proposed approach

For visible watermarking techniques, robustness and translucence are generally required; but they are unfortunately conflicted with each other. If encoder increases the energy of watermark to improve its robustness against attack, the watermarked image will be more degraded under such a scenario. Therefore, it is necessary to find a balance position in order to keep the image quality acceptable. To figure out the ideal strategies in various situations by applying visible watermarking between encoder and attacker, an example is shown in Figure 2 where the amount of watermark embedding intensity increases, the quality of watermark logo also increases as well as the robustness against attacks. On the other hand, the attacker degradation intensity is decreased simultaneously. Accordingly, an equilibrium condition exists when the ideal strategies are encountered for both sides.

Figure 2

The illustration of equilibrium condition for the strategies between encoder and attacker.

In practice, the receiver will request the sender to send the watermarked image again if the received image quality is below an acceptable criterion. Such a condition forms a constraint for the application of visible watermarking since the image feasibility is essential to convince the receiver to take what is offered. In Figure 2, a horizontal dash line represents the acceptable image quality requirement where the equilibrium condition for both encoder and attack must above it. Otherwise, the attacked watermarked image will be rejected by the receiver. To fulfill our design methodology, we leverage the study of COCOA [12] to adaptive COCOA (ACOCOA) approach and develop a dynamic game-theoretic architecture for the watermark embedding problem which is described as a dynamic non-cooperative game with complete information [4]. The ideal strategy developed in Section. 3.2 is defined by the Nash equilibrium of the game [4]. The detailed information about ACOCOA will be explained in the following.

3.1. The ACOCOA (adaptive content and contrast aware) technique

HVS researches offer the mathematical models about how human sees the world. Psychovisual studies have shown that human vision has different sensitivity from various spatial frequencies. Tsai [12] proposed the COCOA algorithm with the consideration of HVS model by using the CSF and NVF for the best quality of perceptual translucence and noise reduction. However, the scaling factor α _λ,θ and embedding factor β _λ,θ of COCOA algorithm are based on the CSF perceptual importance and wavelet basis function amplitudes. They both need further flexibility to fit the dynamic adjustment under game-theoretic architecture where encoder can make different decisions. Therefore, we propose an ACOCOA technique which modifies the perceptual weighting as following:

$${\alpha }_{\lambda ,\theta }=1-0.7{\beta }_{\lambda ,\theta }$$

(2)

$${\beta }_{\lambda ,\theta }=\left\{\begin{array}{cc}\hfill 1-\mathsf{\text{NV}}{\mathsf{\text{F}}}_{x,y}\hfill & \hfill \mathsf{\text{, if }}1-\mathsf{\text{NV}}{\mathsf{\text{F}}}_{x,y}<P\times T_{\lambda ,\theta },\hfill \\ \hfill P\times T_{\lambda ,\theta }\hfill & \hfill \mathsf{\text{otherwise}}\hfill \end{array}\right.$$

(3)

$$T_{\lambda ,\theta }=\left\{\begin{array}{cc}\hfill A_{\lambda ,\theta }\hfill & \hfill \mathsf{\text{, if }}{A}_{\lambda ,\theta }<G_{\lambda ,\theta },\hfill \\ \hfill G_{\lambda ,\theta }\hfill & \hfill \mathsf{\text{otherwise}}\hfill \end{array}\right.$$

(4)

$$G_{\lambda ,\theta }=0.01+\frac{{\left(7.20-r_{\lambda ,\theta }\right)}^2}{{7.2}^2}$$

(5)

Here, T _λ,θ is the perceptual weight which is determined by basis function amplitudes and CSF masking in order to avoid adding too much energy in the low frequency subbands. r _λ,θ is the perceptual weight in [18]. λ is the DWT level and θ is the orientation, and NVF is the characteristic of the local image properties. P is the watermark weighting factor in the range of [1] where a higher P value means that host image has stronger watermark embedded. Table 1 shows A _λ,θ for a 5-level 9/7 DWT from [12]. Table 2 shows G _λ,θ values after a 5-level wavelet pyramidal decomposition, which are calculated by Equation 5. Figures 3 and 4 illustrate r _λ,θ and T _λ,θ values in different DWT level and orientation, respectively.

Table 1 Basis function amplitudes for a 5-level 9/7 DWT [12]

Table 2 CSF masking with 11 unique weights for a 5-level wavelet pyramidal decomposition

Figure 3

DWT CSF mask with 11 unique weights in different DWT level and orientation [18].

Figure 4

T _λ,θ in different DWT level and orientation.

In order to further improve the application of block classification by simply categorizing three type blocks in [18], the local and global characteristics in DWT domain is considered. In ACOCOA scheme, a stochastic image model for watermark embedding is adopted by using the NVF which characterizes the local image properties.

$$\mathsf{\text{NV}}{\mathsf{\text{F}}}_{x,y}=\frac{w\left(x,y\right)}{w\left(x,y\right)+{\sigma }_I^2}$$

(6)

$w\left(x,y\right)=\gamma {\left[\eta \left(\gamma \right)\right]}^{\gamma }\frac{1}{{∥r\left(i,j\right)∥}^{2-\gamma }}$ and ${\sigma }_I^2$ are the global variance of the cover image I, $\eta \left(\gamma \right)=\sqrt{\Gamma \left(3∕\gamma \right)∕\Gamma \left(1∕\gamma \right)}$, $\Gamma \left(t\right)={\int }_0^{\infty }{\mathsf{\text{e}}}^{-u}{u}^{t-1}\mathsf{\text{d}}u$ (gamma function) and $r\left(x,y\right)=\left(I\left(x,y\right)-\overline{I}\left(x,y\right)\right)∕{\sigma }_I$, γ is the shape parameter, and r(x, y) is determined by the local mean and the local variance. For most of real images, the shape parameter is in the range 0.3 ≤ γ ≤ 1.

In our scheme, we use the stationary GG model in the embedding stage, and the estimate shape parameter for γ = 0.65, and width of window is 1. Regarding the visible watermarking algorithm, the algorithm in [12] is modified based on the consideration of the image quality where the controlling parameters of watermark embedding are selected. The watermarking procedures are briefly described as following and the flow chart is shown in Figure 5.

Figure 5

The flow chart of the proposed visible watermarking approach.

• (1) The host color image is converted in the color space domain from RGB to YCrCb.

• (2) By using Bi9/7 filter, compute the 5-level 2-D wavelet coefficients of Y component from host color image and grayscale watermark image.

• (3) Modify the DWT coefficients of the host image by using the following equation:

  $$I_{x,y}^w={\alpha }_{\lambda ,\theta }\times I_{x,y}+\left({\beta }_{\lambda ,\theta }+\mathsf{\text{NV}}{\mathsf{\text{F}}}_{x,y}\times K\right)\times w_{x,y}$$

  (7)

Note: (x,y) indicates the spatial location. I and w are the decomposed wavelet coefficients of the host image and the watermark image. NVF _x,y is defined in Equation 6 and the relationship of α _λ,θ and β _λ,θ are defined in Equations 2 and 3. The constant K value is 0.08.

• (4) Inverse transform the DWT coefficients of the host image to obtain a watermarked image.

3.2. A game-theoretic architecture design for visible watermarking system

Take the ACOCOA algorithm as an example and the formula from Equation 7 where I _x,y , $I_{x,y}^w$, and w _x,y are the (x,y)th pixels of the host image, the watermarked image, and the visible logo image, respectively. α _λ,θ in Equation 2 and β _λ,θ in Equation 3 are the two weighting factors that contain the adjustable parameter value of P for host image and watermark intensity. While the image quality of $I_{x,y}^w$ is a constraint during the watermark embedding, the selection of α _λ,θ and β _λ,θ will be critical points since they will determine the expected image quality of $I_{x,y}^w$. After the watermark embedding stage, encoder will send the watermarked image to the receiver via Internet or other communication channels, while the attackers would try various ways to remove or destroy the watermark if they can intercept the transmission. Under such scenario, the robustness of the watermarking technique is essential to protect the intellectual property. Therefore, the visible watermark embedding action can be stated as a non-cooperative game where individual player decides the strategy to cope with the different situations.

We adopt the definition of Nash equilibrium in [29]. Suppose that there are N players in a game. Let X _i denote the set of possible strategies for player i. V_i(s₁, ... s _N ) denotes player i's payoff function where s₁, ... s _N are the strategies chosen by players 1, ..., N,respectively. An Nash equilibrium is a strategy profile $\left\{s_1^{*},...,s_N^{*}\right\}$ where $s_1^{*}\in X_i$ is the equilibrium strategy of player i and the function $f_i\left(x\right)=V_i\left(S_i^{*},...,S_{i-1}^{*},x,S_{i+1}^{*},...,S_N^{*}\right)$ is optimized, for all x ∈ X _i . That is, in Nash equilibrium, a player's equilibrium strategy is the best response to the belief where the other players will also adopt their Nash equilibrium strategies.

There are two stages in Nash equilibrium. First, each player's optimal strategy is identified in response to what the other players might do. This is done for every combination of strategies by the other players. Second, Nash equilibrium is identified when all players are playing their optimal strategies simultaneously, and every player's strategy is ideal given under the other players use their equilibrium strategy. If both the set of players and set of strategies are not infinite, at least one such equilibrium exists in any time.

This study proposes a security architecture of watermarking system, which is based on the game theory and extended from Figure 1 as the generic structure for visible watermark embedding processes. A game-theoretic architecture consists of four main parts where the roles and functions are defined below:

• (1) a set of players;

• (2) for each player, each has a set of strategies/actions;

• (3) for each player, there is existing a payoff function to evaluate the gain/profit associated with the adopted strategy/action;

• (4) for each player, there are a set of constraints.

Figure 6 demonstrates the complete flow diagram of the game-theoretic architecture design for two players--encoder vs. attacker for the visible watermarking technique. The encoder and attacker player will design a payoff function to estimate the gain/profit in order to select the best strategies/actions in the watermarking game. In the mean time, the acceptable image quality is the constraint for both players. That is, the system will request to recreate a watermarked image if the image quality is below the acceptable level. The detailed description of each parts of the game-theoretic architecture for visible watermarking is as following:

Figure 6

The complete flow diagram of visible watermarking system under the proposed game-theoretic architecture for two players.

• (1) Players

In this case, there are two players in the game security system. One player is the encoder player and the other one is the attacker player.

• (2) Strategies/actions

Due to the dynamic property during the watermark embedding stage, there are certain strategies/actions for each player to determine the best parameters based on its own interest. Let V _i and V _j denote the state of encoder and attacker players. The set of strategies for encoder player is V _i (s₁, ...s _N ) where s₁, ...s _N are N different parameter/strategy selections for watermarking algorithm. On the other hand, we assume that attacker adopts the technique to remove or destroy the watermark from the watermarked image. Here, the set of actions for attacker player is V _j (s₁, ...s _M ) where s₁, ...s _M are equivalent to M different parameter/strategy selections for attacking algorithm.

• (3) Payoffs

The payoffs represent the welfare of the players at the end of the game. They are on the basis of each player choosing his strategy and the payoff function of a player is defined as the total profit/gain. From encoder player point of view, the image quality between the host image and the watermarked image is critical since the encoder need to reserve the highest fidelity after watermark embedding. Based on the quality assessment metric study of Ponomarenko et al [23], we apply four quality assessment metrics that produce reasonably good results from [23], such as MSSIM, VIF, PSNR-HVS-M, and WSNR. In addition, the correlation between the logo watermark and the extracted watermark after attack is also important since the robustness of the watermark embedding technique is critical for the encoder player. Therefore, four image quality assessment metric and correlation functions will be adopted in the payoff function for encoder player.

The payoff function f ₁ of encoder player is defined as a weighted sum of the strategy profiles e^M(quality assessment metric) where m is from 1 to 4 and e⁵ (correlation). The complete formula of f₁ is shown in Equation 8

$$\begin{array}{l}{f}_1{}_{(N,M)}=W_1\times \left(\frac{1}{4}\right)\times {\displaystyle \sum _{m=1}^4\frac{{\text{e}}_{(N,M)}^m-\min ({\text{e}}_{(.,M\text{)}}^m)}{\max ({\text{e}}_{(.,M)}^m)-\min ({\text{e}}_{(.,M)}^m)}}\\ +W_2\times \frac{{\text{e}}_{(N,M)}^5-\min ({\text{e}}_{(.,M)}^5)}{\max ({\text{e}}_{(.,M)}^5)-\min (t_{(.,M)}^5)}\end{array}$$

(8)

where ${\mathsf{\text{e}}}_{\left(N,M\right)}^m=\mathsf{\text{quality assessment metric}}{\left(I,I_w\right)}_{N,M}^m$, ${\mathsf{\text{e}}}_{\left(N,M\right)}^5=\mathsf{\text{correlation}}{\left(\left(I_w-I\right),w\right)}_{N,M}$,0 ≤ W ₁ ≤ 1, 0 ≤ W ₂ ≤ 1, and W₁ + W₂ = 1.

e^m represents image visual quality metric where e¹ is MSSIM, e² the VIF, e³ the PSNR-HVS-M, and e⁴ the WSNR. W₁ and W₂ are the weighting parameters for image quality and the robustness of watermark respectively in Equation 8.

The meaning of ${\mathsf{\text{e}}}_{\left(.,M\right)}^m$ represents the payoff value of a certain M for whole set of N where N is from 1 to N_Max.

Note:

I is the original host image; w is the logo watermark; and I _w is watermarked image.

In order to achieve the objective of encoder player's evaluation, the payoff should get a balanced function value between the intensity of embedded watermark and the perceptual translucence for watermark. Therefore, the payoff function f₁ is defined as a normalized operation from four quality assessment metrics (MSSIM, VIF, PSNR-HVS-M, and WSNR) and correlation where the encoder's best strategy is $f_1^{*}=argmax{f_1}_{\left(.,M\right)}$.

In the similar way, the same quality assessment metrics (MSSIM, VIF, PSNR-HVS-M, and WSNR) used for the payoff function of the encoder are evaluated here for the attacker player since the image quality between the watermarked image and the attacked watermarked image is decisive for the receiver. That is, the attacker expects that the receiver will not be conscious of the action of attacks. Therefore, the image quality plays an important role for the payoff function f₂ of attacker player and the formula is defined in Equation 9. Compared Equations 8 with 9, there is no correlation component in Equation 9 since the attacker does not have the original watermark logo for comparison.

$$f_2{}_{(N,M)}=(\frac{1}{4})\times {\displaystyle \sum _{n=1}^4\frac{e_{(N,M)}^n-\min (e_{(N,\text{. )}}^n)}{\max (e_{(N,\text{. })}^n)-\min (e_{(N,\text{. })}^n)}}$$

(9)

where

$e_{\left(N,M\right)}^n=\mathsf{\text{quality assessment metric}}{\left(I_w,I_w^{\prime }\right)}_{N,M}^n$.

Note: eⁿrepresents image visual quality metric where e¹ is MSSIM, e² is VIF, e³ is PSNR-HVS-M, and e⁴ is WSNR.

The meaning of $e_{\left(N\mathsf{\text{,}}\mathsf{\text{.}}\right)}^n$ represents the payoff value of a certain N for whole set of M where M is from 1 to M_max.

Note: I _w is watermarked image and $I_w^{\prime }$ is the attacked watermarked image.

Accordingly, the payoff function f₂ is defined as a normalized operation from four quality assessment metrics where the attacker's best strategy is $f_2^{*}=argmin{f_2}_{\left(N,\mathsf{\text{.}}\right)}$.

• (4) The constraints

From the receiver point of view, the received image must be above an acceptable image quality which is the horizontal line as shown in Figure 2. This becomes the same requirement of the watermarking game for encoder and attacker to make an acceptable watermarked image to receiver. Therefore, the encoder's payoff function should be higher than average value with no attack which can be described as f_{1 (N,1)}≥ 0.5.. On the other hand, the attacker has various actions so we set a constraint μ value where μ defined in Equation 10 is the average value of attacker's payoff function in different strategies and actions.

$$\mu =\left\{\begin{array}{cc}\hfill \frac{1}{N\times M}\times \sum _{n=1}^N\sum _{m=1}^M{f}_{2\left(n,m\right)}\hfill & \hfill ,\mathsf{\text{if }}\mu >0.5\hfill \\ \hfill \mathsf{\text{0}}\mathsf{\text{.5}}\hfill & \hfill \mathsf{\text{, otherwise}}\hfill \end{array}\right.$$

(10)

• (5) Equilibrium condition

We adopt the concept of the Nash equilibrium and analyze the strategies/actions of the players in the watermarking system. If there has a solution profile $\left(f_1^{*},f_2^{*}\right)=\left(argmax\left(f_{1\left(.,M\right)}\right),argmin\left(f_{2\left(N,.\right)}\right)\right)$, we can say $\left(f_1^{*},f_2^{*}\right)$ is an equilibrium condition result of the game-theoretic architecture for visible watermarking.

4. Experimental results

The proposed ACOCOA visible watermarking algorithm and game-theoretic architecture have been implemented and intensively tested by using the commonly available color images from USC image database [30] with 512 × 512 images. The image quality metrics for the payoff function are available at the following website: MeTriX MuX Visual Quality Assessment Package [31]. The grayscale watermark of logo image adopted in the experiments is the school logo shown in Figure 1a. Different signal processing and geometric attacks have been thoroughly tested. Due to the limit of enough space to tabulate all attacks, the experimental results show similar behavior which provides the best selection of Nash equilibrium condition under different attacks. The performance analysis can be categorized as follows.

4.1. JPEG2000 compression

Here, we tabulate all details of strategies/actions for encoder and attacker using JPEG2000 compression as attacker's action. Such procedures can be applied to any different attack. The actions for encoder player are V _j (s₁, ...s _N ) where s₁, ...s _N are different watermark weightings of 0.0, 0.1, 0.2,..., 1.0 for β _λ,θ . On the other hand, the actions for attacker player are V _j (s₁, ...s _M ) where s₁, ...s _M are equivalent to compression ratio of no compression, 0.1, 0.09, ..., 0.01 for total 11 states. The meaning of compression ratio like 0.01 represents 100:1 between the uncompressed image and compressed image. Other settings from 0.1 to 0.02 are with the same operation.

It is the assumption here that the encoder knows the potential attack and it will apply the game theory to obtain the best strategy for watermark embedding. Through detailed examination, the watermark robustness plays an important role for the payoff function so we set the two weighting parameters W₂ = 0.6 and W₁ = 0.4 for Equation 8.

The performance summaries by different encoder's strategies and attacker's actions for Lena image of MSSIM, VIF, PSNR-HVS-M, WSNR, and Correlation are demonstrated in Figure 7. The results reveal that the values of the four image quality metrics and correlation are decreasing while the compression ratio is increasing. On the other hand, the correlation values are increasing while the embedded watermark is stronger for different encoder strategy. Table 3 illustrates the encoder's payoffs f₁(N,M) where N and M are from 1 to 11, respectively, and the best selection for each attacker action occurs among different encoder strategy. In the mean time, the best selection characterizes the goal of the encoder for not only achieving the highest perceptual image quality but also enduring the watermark robustness against the attacker.

Figure 7

Performance summaries by different encoder's strategies and attacker's actions for Lena image of (a) MSSIM, (b) VIF, (c) PSNR-HVS-M, (d) WSNR, and (e) Correlation.

Table 3 The encoder's payoffs and the best selection

From the attacker's viewpoint, it is reasonable to assume that the watermarking algorithm is unknown to the attackers. Thus, we make the hypothesis that attacker wants to undermine the watermark but to maintain the attacked image with acceptable image quality. Table 4 illustrates the attacker's payoffs f₂(N,M) where N and M are from 1 to 11, respectively, and the best selection for each encoder strategy occurs among different attacker's action.

Table 4 The attacker's payoffs and the best selection

Table 5 demonstrates the equilibrium condition from the encoder's payoffs and the attacker's payoffs under the game-theoretic system security design. With the constraint of attacked watermarked image, the equilibrium condition occurs at the state of (N,M) = (7, 7) for Lena image which is equivalent to WSNR value at 28.10dB image quality under JPEG2000 compression with the compression ratio of 100:5 attack with f₁ value of 0.69. Similar game-theoretic design for Tiffany image is also performed and tabulated in Table 6. With the constraint of attacked watermarked image, the equilibrium condition occurs at the state of (N,M) = (6, 8) for Tiffany image which is equivalent to 35.03dB image quality under 100:5 JPEG2000 compression attack with f₁ value of 0.70.

Table 5 Payoff function value for Lena image under JPEG2000 attack and the best selection of (N, M) is (7, 7) under acceptable image quality constraint

Table 6 Payoff function value for Tiffany image under JPEG2000 attack and the best selection of (N, M) is (6, 8) under acceptable image quality constraint

In Table 7, we tabulate the visual quality performance of Lena and Tiffany images before and after JPEG 2000 compression at compression ratio 100:3. There are three rows for both images. The 'before' row means that the image quality measure values are compared between the original image and the watermarked image. The 'after' row means the values of image quality measure are compared between the original image and the attacked watermarked image. The 'after(wm)' row means the image quality measure values are compared between the watermarked image and the compressed watermarked image (attacked image). From Table 7, the visual image quality measures of MSSIM, VIF, PSNR-HVS-M, and WSNR for ACOCOA are better than those of method [18] and [12]. To further investigate the attack effect of compression, the visual difference can be illustrated in Figure 8 and by the close-up comparison in Figure 9. We observe that the watermark patterns for Figure 8d and 8h are still with sharp edges and the logo watermark can be clearly and easily identified by human eyes for Figure 9f and 9l. Therefore, from the experimental results, we demonstrate the ACOCOA technique that is with flexibility and robustness under game-theoretic architecture. Further studies for other images are also performed and we can see similar results for visual image quality measure values and visual comparison.

Table 7 Performance summaries of watermarked color images before and after JPEG 2000 compression at compression ratio 100:3

Figure 8

The visual quality comparison of original and watermarked images. (a), (e) are original Lena and Tiffany images, respectively. (b), (f) are watermarked images by method [18]. (c), (g) are watermarked images by method [12]. (d), (h) are watermarked images by the ACOCOA technique.

Figure 9

The visual quality comparison of close-ups for Lena and Tiffany images after JPEG 2000 compression. (a), (g) are watermarked images by method [18]. (b), (h) are watermarked images by method [12]. (c), (i) are watermarked images by the ACOCOA technique. (d), (j) are watermarked images by method [18] after JPEG 2000 compression. (e), (k) are watermarked images by method [12] after JPEG 2000 compression. (f), (l) are watermarked images by the ACOCOA technique after JPEG 2000 compression.

4.2. Median filter

Applying the same approaches under proposed game-theoretical architecture, the attacks in StirMark [32] have been thoroughly tested and we have found that the experimental results show similar behavior, which provides the best selection of Nash equilibrium under different attacks. Due to the limited space to tabulate all attacks, we only explain median filter attack here but the scheme can be applied for other attacks.

Here the actions for encoder player are V _i (s₁, ...,s _N ) where s₁, ...s _N are different watermark weightings of 0.0, 0.1, 0.2, ..., 1.0 for β _λ,θ . On the other hand, the actions for attacker player are V _i (s₁, ...,s _M ) where s₁, ...,s _M are equivalent to filtering ratio of no filtering, 1 × 1, 3 × 3, 5 × 5, 7 × 7, 9 × 9, 11 × 11 for total seven states under game-theoretic architecture. In Table 8, it is clear that the image quality measure values using ACOCOA technique perform better than those using method [12] and [18] under median filtering. Therefore, the data support that the proposed method is with flexibility and robustness.

Table 8 Performance summaries of watermarked color images before and after 7 × 7 median filtering

4.3. Image recovery and watermark removal attack

To further examine ACOCOA's robustness, we have implemented the method of watermark removal attack [33] for comparison. Figure 10 illustrates the results of the image recovery attack by method [12, 18] and ACOCOA. In Figure 10, the logo pattern by method [18] is completely removed but the contours of logo pattern by method [12] and ACOCOA still exist. By using ACOCOA with game-theoretic architecture, we can easily find the best parameters for visible watermarking technique. In summary, the proposed technique can resolve the issue for watermark encoder to obtain the best watermarking strategy under attacks.

Figure 10

The visual quality comparison of close-ups between watermarked images and the watermarked images after image recovering. (a) is the watermarked image by method [18]. (b) is the watermarked image by method [12]. (c) is the watermarked image by ACOCOA technique. (d) is the watermarked image by method [18] after image recovering. (e) is the watermarked image by method [12] after image recovering. (f) is the watermarked image by the ACOCOA technique after image recovering.

4.4. Discussions

There are several issues that the authors would like to address in this session on game-theoretic architecture for ACOCOA technique.

• (1) Multiple equilibrium conditions

In theory, it is possible to have multiple equilibrium conditions under Nash equilibrium explanation. To clarify such assumption, experimental results of best selections for Peppers, Baboon, and Splash under JPEG2000 attack are listed in Table 9 and it is true that there exist multiple equilibrium conditions. Take Peppers as an example, (7, 7) means weighting factor is 0.6 and compression ratio is 100:5 since there are 11 different weighting actions of 0.0, 0.1, 0.2,..., 1.0 for encoder player and 11 states for compression attacks. Similarly, (6, 8) means weighting factor is 0.5 and compression ratio is 100:4. When the multiple solutions exist in this study, the security concern has higher priority than the image quality. Therefore, the selection of (7, 7) outweighs (6, 8) and the underlined selection is the final choice in Table 9. Consequently, same approach is applied for Baboon and Splash images.

Table 9 Summary of best selections under JPEG2000 attack for Nash equilibrium solution

• (2) New types of attack actions

While the technology improves continuously, there are always new types of attacks for the visible watermarking. Under such situation, the proposed game theoretic architecture is universal and can be used ubiquitously. Therefore, the encoder can simulate the attacker's actions to get the best selection of Nash equilibrium condition.

• (3) Multiple attack actions

It is possible that the watermarked image suffers multiple attacks through the communication channel and the encoder can still apply the game-theoretic architecture to simulate the multiple attacks in order to obtain the best selection among the attacks. For example, the image can be under JPEG2000 compression attack first and then median filtering attack later. Under such scenario, the joint attack will degrade the image further compared with single attack. Accordingly, the attackers should adjust the settings for multiple attacks in order to preserve the acceptable image quality. Otherwise, the image will be requested to resend if the image quality is below the threshold as shown in Figure 2.

• (4) The weighting parameters

The weighting parameters W₁ and W₂ in Equation 8 are 0.4 and 0.6, respectively, in this study which are obtained empirically. Even the encoder can flexibly choose those parameters, more systematical analyses regarding the relationship between image quality and watermark robustness are suggested for future studies. For example, human objective evaluation can be collected for different parameter settings during the use of the game-theoretic security system. Therefore, the sensitivity of parameters between the original image, watermarked image and attacked image can be evaluated by the analysis of variance (ANOVA) technique in order to get the systematic influence values of the correlation coefficients.

• (5) The equal weightings of quality assessment metrics for payoff functions in Equations 8 and 9.

Here, we assume that each quality assessment metric for payoff function has the equal weighting. However, such assumption is adaptable since the quality metrics may play unequal importance for Equations 8 and 9 under the game-theoretic architecture. Since there is no such discussion available during the literature survey, this topic could be further investigated as the further research.

• (6) Selected best parameters for different attacks

Due to the constraints of image quality requirement as shown in Figure 2, the Nash equilibrium will be achieved under attack for best parameters selection based on the game-theoretic architecture. For example, the parameters selected under JPEG 2000 compression are still efficient for other attacks even they may not be the best selection under certain circumstances. If the decision maker wants to obtain a better watermarked image to against the specific attack, the proposed game-theoretic architecture is still the best approach to obtain the most efficient parameters under constraints.

• (7) The computation time for using game-theoretic architecture for ACOCOA watermarking

The computation time for using game-theoretic architecture is determined by each player's strategies/actions and payoffs. The whole complexity should be examined by calculating each individual visual quality metric's computation.

For VIF, the fastest way of computing the determinant of a matrix is actually to use good old Gaussian elimination [34]. The determinant of a triangular matrix is simply the product of the diagonal elements. Every matrix can be reduced to a triangular matrix through elementary row operations, and all of these change the determinant in an easily predictable manner. The complexity of VIF is closely related with Equations A2 and A4 and the total amount of calculation approximately equals to the image size (we can use static array to store the results). Thus, the complexity of variance takes O(n²) computation and the natural logarithm operation also takes roughly O(log n). Hence, the complexity of mutual information between X and its perceptual image Y can be computed as O(n³logn) (O(n • n² • logn)) ≈ O(n² • logn) for n × n image size.

Regarding the complexity of MSSIM, Equation A7 is determined by the global mean, the global variance, the local mean, the local variance, and global covariance. The complexity of global mean, global variance, and global covariance are ≈ O(n²). The complexity of local mean and the local variance is ≈ O(l²), l = 2L + 1 is the window size. In this study, the window size is 8 × 8. Thus, the total amount of calculation approximately equals to the image size and the overall time complexity for MSSIM is no more than O(n²) (O(n² + l²) ≈ O(n²)) since image width n is much larger than l). Consequently, WSNR and correlation have similar operations with the same complexity of O(n²). On the other hand, PSNR-HVS-M utilizes the weighted energy of DCT coefficients of an 8 × 8 image block. By using the fast cosine transform algorithms, the complexity of DCT could be as low as O(n logn) and the total complexity of PSNR-HVS-M will be ≈ O(n² • n logn) = O(n³ log n). In summary, Table 10 tabulates the complexity for each image visual quality metric adopted in this study.

Table 10 The complexity of image visual quality metric for a n × n testing image

From our simulation of JPEG 2000 compression, encoder player has N = 11 strategies and its payoff needs to calculate five different values of image quality (MSSIM, VIF, PSNR-HVS-M, WSNR, and correlation). In the mean time, attack player has M = 11 strategies and its payoff needs to calculate four different values of image quality (MSSIM, VIF, PSNR-HVS-M, and WSNR).

In Table 11, we tabulate the average computation time for each image visual quality metric by using a 512 × 512 testing image. The whole loop of game-theoretic architecture for ACOCOA by considering JPEG2000 attack will take about 1592.463 seconds (26.54 min) under Intel Core2 Quad CPU 2.66GHz, 2G RAM computer. The computation is performed by using Matlab software which can be further optimized by using low level language like C or C⁺⁺ and parallel processing (cloud computing) to speed up the computation.

Table 11 The average computation time for each image visual quality metric

5. Conclusions

The researchers have been working hard to pursue the visible digital watermarking techniques for copyright protection. There are two essential characteristics: first, robustness for common signal processing operations and the second, perceptual translucence of the watermark with acceptable image quality. Since these two issues are correlated closely, how to find the best parameter settings has become a critical factor for the watermarking applications.

In order to resolve these concerns, the ACOCOA technique and a security watermarking system, which is based on game-theoretic approach that provides the best selection for the decision maker, are proposed by studying the effect of transmission power on intensity and perceptual efficiency. The game-theoretic architecture helps us to analyze the watermarking competition game between the encoder and the attacker. In the mean time, it also provides the solution to acquire the best selection between watermark transparency and robustness for digital contents in different strategies/actions with complete information in the dynamic non-cooperative situations.

After thorough simulation and examination, the experimental results demonstrate that the proposed scheme can provide the useful information for the encoder to determine the best watermarking strategy. On the other hand, further investigations of research topics are suggested to get more precise inter-relationship among constituted components of payoff functions for the players. In summary, the proposed game-theoretic technique provides a useful decision methodology for encoder who can make the best selection among choices. Accordingly, our research could help each player to maximize its utility benefits under different situation and resolve the security issue of visual communication.

Appendix

Formulas of image quality measures

Here are the brief descriptions of the image quality measures (IQM) formulas used for payoff function in this study. Interested readers should refer the references for the detailed information.

A.1. VIF [22, 35]

VIF is an image quality assessment approach based on information theory. In reference [22], Sheikh et al. defined the HVS as a typical additive noise channel. An image X is treated as a random signal and sent in at one end. The other end, the brain, receives the visual information Y, which is defined in Equation A1.

$$Y=X+V$$

(A1)

where V is the vision noise and obeys normal distribution with zero mean and ${\sigma }_V^2$.

The mutual information between X and its perceptual image Y can be computed as:

$$I\left(X;Y\right)=log\left(1+\frac{{\sigma }_X^2}{{\sigma }_V^2}\right)$$

(A2)

In contrast to the existing methods, Sheikh et al. built a relationship with the reference X and distorted image X_d with a distortion model.

$$X_{\mathsf{\text{d}}}=kX+Z$$

(A3)

where k is a scalar and Z is the Gaussian noise with zero mean and ${\sigma }_Z^2$. Based on the distortion model, the common information between X and the perceptual image Y_d of the distorted image X_d can be computed as:

$$I\left(X;Y_{\mathsf{\text{d}}}\right)=log\left(1+\frac{k^2{\sigma }_X^2}{{\sigma }_Z^2+{\sigma }_V^2}\right)$$

(A4)

From Equation A4, it can be seen that as the scalar k decreases (blur effect) or noise Z increases (noise, compression effect, or quantization noise), I(X;Y_d) is going to decrease. The visual information fidelity in each frequency band is defined as the ratio between the two mutual information I(X;Y) and I(X;Y)_d. The overall visual information fidelity can be computed as the ratio between two mutual information in all channels.

$$\mathsf{\text{VIF}}=\frac{{\sum }_{j\in \mathsf{\text{channels}}}I\left(X^j;{Y_{\mathsf{\text{d}}}}^j\right)}{{\sum }_{j\in \mathsf{\text{channels}}}I\left(X^j;Y^j\right)}$$

(A5)

A.2. MSSIM [25]

The definition of MSSIM is as following:

$$\mathsf{\text{MSSIM}}\left(X,Y\right)=\frac{1}{M}\sum _{j=1}^M\mathsf{\text{SSIM}}\left(x_j,y_j\right)$$

(A6)

where X and Y are the reference and the distorted images respectively; x _j , y _j are the image contents at the j th local window and M is the number of local windows in the image.

The SSIM metric is calculated on various windows of an image. The measure between two windows of the size N × N, x and y are two nonnegative image signals. The definition of SSIM is as following:

$$\mathsf{\text{SSIM}}\left(x,y\right)=\frac{\left(2{\mu }_x{\mu }_y+C_1\right)\left(2{\sigma }_{xy}+C_2\right)}{\left({\mu }_x^2+{\mu }_y^2+C_1\right)\left({\sigma }_x^2+{\sigma }_y^2+C_2\right)}$$

(A7)

with

μ _x the average of x; μ _y the average of y;

${\sigma }_x^2$ the variance of x;${\sigma }_y^2$ the variance of y;

σ _xy the covariance of x and y;

C₁ and C₂ are two variables to stabilize the division with weak denominator. Typically, it is calculated on window-sizes of 8 × 8.

A.3. PSNR-HVS-M [26, 27]

In reference [27], authors denote a weighted energy of DCT coefficients of an image block 8 × 8 as E_w (X):

$$E_{\mathsf{\text{w}}}\left(X\right)=\sum _{i=0}^7\sum _{j=0}^7{X_{ij}}^2{C}_{ij}$$

(A8)

where X _ij is a DCT coefficient with indices i,j, C _ij is a correcting factor determined by the CSF.

The DCT coefficients X and Y are visually undistinguished if E_w (X - Y) < max (E_w(X)/16, E_w (Y)/16), where E_w (X)/16 is a masking effect E_m of DCT coefficients X (normalizing factor 16 has been selected experimentally).

Reducing of the masking effect due to an edge presence in the analyzed image block, they propose to reduce a masking effect for a block D proportionally to the local variances V(.) in blocks D₁, D₂, D₃, D₄ in comparison to the entire block:

$$E_{\mathsf{\text{m}}}\left(D\right)=E_{\mathsf{\text{w}}}\left(D\right)\delta \left(D\right)∕16$$

where $\delta \left(D\right)=\left(V\left(D_1\right)+V\left(D_2\right)+V\left(D_3\right)+V\left(D_4\right)\right)∕4V\left(D\right)$, V(D) is the variance of the pixel values in block D.

Below is a flowchart of PSNR-HVS-M calculation (see Figure 11).

Figure 11

Flowchart of PSNR-HVS-M calculation.

Reduction by value of contrast masking in accordance to the proposed model is carried out in the following manner. First, the maximal masking effect E_max is calculated as max(E_m (X_e), E_m (X_d)) where X_e and X_d are the DCT coefficients of an original image block and a distorted image block, respectively. Then, the visible difference between X_e and X_d is determined as:

$$\begin{array}{lll}\hfill \mathsf{\text{PSNR - HVS - M}}& =X_{\Delta ij}& \hfill \text{(1)}\\ \hfill & =\left\{\begin{array}{c}{X}_{\mathsf{\text{e}}ij}-X_{\mathsf{\text{d}}ij},i=0,j=0\\ 0,\left|X_{\mathsf{\text{e}}ij}-X_{\mathsf{\text{d}}ij}\right|\le E_{\mathsf{\text{norm}}}∕C_{ij}\\ X_{\mathsf{\text{e}}ij}-X_{\mathsf{\text{d}}ij}-E_{\mathsf{\text{norm}}}∕C_{ij},X_{\mathsf{\text{e}}ij}-X_{\mathsf{\text{d}}ij}>E_{\mathsf{\text{norm}}}∕C_{ij}\\ X_{\mathsf{\text{e}}ij}-X_{\mathsf{\text{d}}ij}+E_{\mathsf{\text{norm}}}∕C_{ij},\mathsf{\text{otherwise}}\end{array}\right.& \hfill \text{(2)}\\ \hfill \text{(3)}\end{array}$$

(A9)

where E_norm is $\sqrt{E_{max}∕64}$.

A.4. WSNR [28]

The CSF was used as a weighting function for noise measurement and the error measurement criterion is the WSNR (weighted SNR):

$$\mathsf{\text{WSNR}}=\mathsf{\text{10 }}{log}_{10}\frac{{\sum _{n=1}^N\left[x_n*c\left(x_n\right)\right]}^2}{{\sum _{n=1}^N\left[\left(\left(x_n\right)-\left(y_n\right)\right)*c\left(x_n\right)\right]}^2}$$

(A10)

where x _n and y _n denote the original image and the noisy image. * denotes linear convolution and c(x _n ) is CSF in the spatial domain.