Defining the properties of watermarking plays an important role in the systematic development of various schemes. For example, in developing a new scheme, the watermarking objectives determine a set of criteria (as discussed in section 1). Each criterion can be expressed in terms of the minimum requirements for a relevant watermarking property. In the design phase, those requirements help characterize the scheme (e.g., by setting constraints for the construction of watermarking functions). In the evaluation phase, measuring (with a suite of tests) how those requirements are fulfilled gives merit to the scheme. The relative importance of each property, thus, can be determined based on the application requirements. This also means that the interpretation and significance of watermarking properties can vary with the application. These properties, in practice, can be interpreted in terms of the inputs and outputs of watermarking components, use of keys, etc. They can also be mutually dependent, which requires a trade-off among the improvements in the properties [23] for an application.
In the image watermarking context, a number of defining properties (considering their relative importance) are studied below: perceptual similarity, visibility, blindness, invertibility, robustness, embedding capacity, error probabilities, and security. In the following sections, we formally define these properties using the developed watermarking model (section 3) and show how they can be interpreted and used in a real application scenario. To simplify reading, from now on, the notations are used without explicitly giving their domains. For example, ‘for all a,b,c,⋯’ will be used to mean ‘for all (a,b,c,⋯) ∈ A×B×C⋯’.
4.1 Perceptual similarity
The perceptual similarity (or imperceptibility) is one of the most important properties for the image applications. Since embedding distortion is inevitable, E exploits the (relatively) redundant information of an image intelligently for a minimum of visual artefacts. In almost any image application, therefore, keeping a watermarked image perceptually similar to the original image becomes an important criterion. Perceptual similarity means the perceptual contents of the two images are ‘sufficiently’ similar to each other, (and thus it is mainly studied for the invisible watermarking schemes; the ‘visibility’ property is discussed below). The requirements for this property may vary with the application scenario. In order to ease the problem of dealing with these varying requirements, we now define the perceptual similarity property using a quantitative approach.
Definition 4.1(Perceptual similarity).
Any two images, i1 and i2, are said to be (d,t)perceptually similar, if d
j
(i1,i2)≤t
j
for all similarity measures d
j
∈ d≡{d1,d2,⋯,d
n
} and thresholds t
j
∈ t≡{t1,t2,⋯,t
n
}.
Various measures are used to quantify the requirements for the perceptual similarity. For example, correlation quality (CQ), signal to noise ratio (SNR), peak or weighted SNR (PSNR or WPSNR), mean square error (MSE), structural similarity index (SSIM), mean or weighted SSIM (MSSIM or WSSIM), normalized cross-correlation (NCC), etc. However, no globally agreed and effective measures for visual quality currently exist [24]. In addition, not all the measures give the similar estimation. Therefore, we define perceptual similarity by defining a similarity measure, which is a set of n-suitable measures that help quantify the perceptual distance between two images. Now, we define two images to be perceptually similar (or imperceptible) for an acceptable value returned by all suitable measures defined for similarity.
As an example to use the above definition, we may consider two measures (i.e., n=2): PSNR and MSSIM, for the similarity measure, d such that d1=PSNR and d2=MSSIM. The given thresholds are t1=60 (dB) and t2=0.995. Two images i1 and i2 are said to be perceptually similar if both d1(i1,i2)≥60 and d2(i1,i2)≥0.995 are satisfied.
4.2 Visibility
A visible watermarking scheme deliberately inserts a watermark such that it appears noticeably on the watermarked image to show some necessary information such as company logo, icon, or courtesy. However, in order that the watermark does not become so strongly pronounced that it takes over the main image, the level of visibility can be controlled, for example, by a parameter α. Visible watermarks are important in recognition and support of possessing a digital image. In contrast, an invisible watermark is embedded by keeping the perceptual content of the watermarked images similar to that of the original images to address security problems in different application scenarios. Therefore, there are schemes which are either visible or invisible based on the appearance of watermark on the watermarked images.
Definition 4.2(Visibility).
A watermarking scheme is called visible or perceptible, if E(·) embeds a given watermark, w, into an image, i, such that the w appears at least noticeably in ī. That is, |E
e
(i,w)−i|=α w for all i, w. Here, α is weight factor that controls the degree of visibility.
A watermarking scheme is called invisible or imperceptible, if E(·) embeds w into i such that the ī is perceptually similar to the original image, i. That is E
e
(i,w)≈i for all i, w.
Although the visibility and perceptual similarity properties share some perceptual aspects of a watermarked image, they need not be confused with each other. As stated in Definition 4.1, the perceptual similarity property determines if an original image and its watermarked version remain ‘perceptually’ the same. On the other hand, Definition 4.2 states that a visible watermark appears on a watermarked image with a predefined degree of visibility, α, and thus strictly speaking for the visible watermarking, the watermarked image is not perceptually similar to the original image. Perceptual similarity property is thus studied for the invisible watermarking schemes
An invisible watermarking scheme usually differs from a visible watermarking scheme, not only in the visibility factor, but also in their embedding processes. Invisible embedding of a watermark aims at keeping the perceptual difference (resulting from the embedding distortion) at a ‘minimum’ level such that the watermarked and original images remain perceptually the same. Their perceptual similarity is verified by quantifying the perceptual difference using similarity measures. The commonly used similarity measures do not indicate any subjective quality degradation, rather they quantify the overall perceptual difference either by their local (e.g., block-wise or kernel-based) or global (e.g., whole image based) operations. As a result, the defined perceptual similarity does not directly indicate whether a watermarking scheme is visible or invisible. However, for an invisible watermarking scheme, the quantified perceptual difference between an original image and its watermarked version would naturally be much lower than that for a visible watermarking scheme.
In short, an invisible scheme may be considered a variant of visible watermarking with a ‘negligible’ (i.e., approaching zero) α, and having an additional (and even more strict) perceptual similarity requirement. Visible watermarking is present in a few applications such as video broadcasting. However, recent research is mainly focussed on invisible watermarking with a high perceptual similarity in various image applications [25–41].
4.3 Blindness
Another important watermarking property is blindness that helps characterize a scheme to be blind, non-blind, or semi-blind. The term blindness (or oblivious) is generally used in cryptography to define a detection process independent of any side information. More specifically, blindness is used to define a computational property of information retrieval (e.g., to define the computational independence on the original information or its derivatives to retrieve the required information). Similarly, blindness defines the detection and extraction process in digital watermarking, although there is no complete definition for a watermarking scheme to be blind or non-blind.
As a requirement for blindness, some schemes consider that no original input image and the information derived from the input image should be required, whereas other schemes consider only avoiding the original input requirement during the detection. Although schemes in both categories are often considered as blind, with a more strict blindness requirement, the schemes in the latter category may eventually fail to achieve the overall design requirements in an image application (e.g., image authentication). Additionally, confusion arises when a scheme is defined as semi-blind. Sometimes, it is considered that if the detection and extraction processes can operate objectively without the original image and its derived information, but still require the original watermark, then the scheme can be semi-blind.
Cox et al. [42] informally defined a blind or oblivious watermark detector in such a way that the detector does not require access to the original (i.e., unwatermarked) image, or some information derived from the original image. Otherwise, the detector is called non-blind or informed. However, their definition is not sufficient to realize three different cases associated with the blindness property. We define here (Definition 4.3) watermarking blindness to distinguish the dependency of D(·) and X(·) on any of the original input data that is used in G(·) and E(·), and thereby distinguish three different cases of this watermarking property.
Definition 4.3(Blindness).
A watermarking scheme is called blind ( or oblivious ) if both D(·) and X(·) are independent of the original image, i and watermark, w. Formally, for all images i1,i2 and watermarks w1,w2, hold both
A watermarking scheme is called semi-blind if either one of D(·) and X(·) is independent of i and/or w. Thus, for semi-blind watermarking, for all images i1,i2 and watermarks w1,w2 either
or
Otherwise, a watermarking scheme is called non-blind ( or non-oblivious or informed ) if both of D(·) and X(·) are dependent on i and/or w. Thus, for all images i,i1 and watermarks w,w1, hold both
We note here that strictly speaking, the detection function D(·) and the extraction function X(·) must have all three inputs: ī, i, and w. However, for instances of blind and semi-blind watermarking, some inputs (e.g., i and w) are not used in D(·) and X(·), and thus, they can be optionally omitted.
It can also be noted that the blindness property, as defined in Definition 4.3 in terms of the watermark detection and extraction functions, can also be considered for the watermark generation function. A non-blind (ie, an original image dependent) G can be helpful in resisting copy attacks (that aims at counterfeiting the D(·) for any invalid modifications, or invalid watermarked images; see section 5.1.6 for the definition of copy attack). The blindness for D is also important, where availability of the original image, watermark or other side information at D(·) can thwart watermarking objectives. Blind and non-blind watermarking schemes are sometimes confused with private and public watermarking, respectively. However, we insist on defining a watermarking scheme to be private and public in terms of their keys (as defined in section 3.2) to avoid any confusion.
4.4 Invertibility
Invertibility (or reversibility or losslessness) is a computational property of watermarking. The meaning of this property is quite intuitive; however, we expect that defining invertibility in the current context would help realize its mutual relation with other properties. In an image application, invertibility is expected to restore any watermarked images to their original versions, where no embedding distortion is allowed in the original image. Such a watermarking criterion motivates construction of an invertible E that helps D(·) to reproduce an original image from the watermarked image [30, 32, 34, 38, 39, 43–60]. Here, we define an invertible watermarking scheme such that it allows inverse computation of E(·) during detection.
Definition 4.4(Invertibility)
A watermarking scheme is invertible ( or reversible or lossless) if the inverse of E is computationally feasible to compute and is used in D to estimate an exact original image, i, from the respective watermarked image, ī. Otherwise, the scheme is called non-invertible watermarking scheme.
From the above definition, if E
e
(i,w)=ī, then for an invertible watermarking scheme, the detection must exist and satisfy . Therefore, such watermarking schemes can be either blind or a semi-blind (according to Definition 4.3). Since, in image applications, an invertible watermarking scheme is mainly designed to reverse the effect of embedding on the original image, the embedding function is only considered to define invertibility of the scheme. However, the concept of an invertible function can also be extended for X, if an invertible G(·) is computationally feasible.
4.5 Robustness
Robustness in watermarking is often confused with its meaning from cryptography [61]. A main reason is probably that watermarking has to consider some spatial or perceptual properties (e.g., perceptual similarity, visibility). Several attempts have been made to informally define the robustness property of watermarking. For example, Piper and Safavi-Naini [62] considered a watermarking scheme as robust if it can successfully detect the watermark in the ‘processed’ images. The strength of this definition depends on how the ‘processed’ image is defined. In contrast, Cox et al. [42] referred to robustness as the ability to detect the watermark after common signal processing techniques. More specifically, robustness can be defined as the degree of resistance of a watermarking scheme to modifications of the host signal due to either common signal processing techniques or operations devised specifically in order to render the watermark undetectable [63]. In summary, watermarking robustness has to deal with (i) defining a set of processing techniques, and (ii) the detection ability for the ‘processed’ images.
We now formalize the concept of watermarking robustness in terms of the processed images and the detection ability. Firstly, a set of processing techniques (i.e., various operations/transforms) is defined below to define a ‘processed’ image for an application. Here, the same set of processing techniques may not be valid for different watermarking applications, and thus a general consideration of the techniques may not be always useful. Secondly, a detection condition is defined as that which determines the detection ability, for the set of ‘processed’ images.
Definition 4.5(Processed image).
A processed image is an image that is not essentially perceptually similar to its original, but a certain amount of distortion, δ is incurred by a processing technique, p ∈ P. That is, if any image, is processed by p then, for the processed image, p(l) the following is true: p(l)=l+δ. Here, P is the set of applicable processing techniques for an application such that , where
is the space of processing techniques.
It is worth noting that, in our earlier work [2, 61], we aimed at avoiding any confusion between the robustness and security properties and considered that a processed image is not perceptually similar to its unprocessed version. That consideration was based on the assumption that only an adversary may want to process a valid watermarked image to achieve the perceptual similarity requirements. However, that assumption is not always valid in practice. For example, a watermarked image can be processed such as by lossless compression and file-format conversion, with the required perceptual similarity property (not only maliciously, but also intentionally as a system requirement). We, therefore, revise our earlier consideration for Definition 4.5 such that a processed image is not necessarily perceptually similar to its unprocessed version. We believe that this revision does not conflict with our earlier intention to avoid the confusion between robustness and security properties
With the Definition 4.5, now we may wish to define the detection condition for the robustness property. Suppose a processing technique, p ∈ P, causes distortion to a watermarked image, ī. As defined in our proposed model, D
d
(·) accepts with the property for all p(ī),i,w|p(ī) ∈ Ī. Here, the pass that returns with and the failure, ⊥ can be used to define two potential variants, robust and fragile respectively, of watermarking schemes for different P. Another variant, semi-fragile watermarking scheme can also be defined considering a suitable subset of P. Thus, we define the robustness property in Definition 4.6 considering detection ability at three different levels
Definition 4.6(Robustness).
A watermarking scheme is defined for the following levels of robustness:
Robust. A watermarking scheme is called robust if for all p ∈ P.
Fragile. A watermarking scheme is called fragile if D
d
(p(ī),i,w)=⊥ for all p ∈ P.
Semi-fragile. A watermarking scheme is called semi-fragile if for all p ∈ P1 and D
d
(p(ī),i,w) = ⊥ for all p ∈ (P∖P1), where P1⊂P.
As stated in Definition 4.6, a successful detection (i.e., D
d
(·)≠⊥) is the basic criterion for a watermarking scheme to be robust to p ∈ P. However, there is no absolute robustness for watermarking, since taking all known/available processing techniques into consideration (for robustness) is not realistic. It is therefore reasonable to identify only the set of applicable processing techniques for the robustness requirements in an application (like knowing the set of potential adversaries for the security requirements in an application, see section 4.8 below). As Definition 4.6 suggests, we also stress that one must have an explicit consideration on P for design and evaluation of a watermarking scheme in a particular application scenario.
When we consider P (the set of applicable processing techniques), we may notice that different processing techniques (e.g., compression, de-noising) have different parameters (e.g., compression ratio, down sampling rate, type and rank of filter). These parameter settings give different strengths to a processing technique. Therefore, it is worth noting that considering a technique, p, means that p is defined with its all required parameter settings. The technique with other settings thus remains outside of P.
4.6 Embedding capacity
Embedding capacity (or simply capacity) is an important, and may be the most-studied, property for watermarking schemes. A lot of studies have reported recently on improving this property maintaining the required perceptual similarity in different ways [30, 32, 38, 39, 50–59]. A number of ways to estimate the steganographic/watermarking embedding capacity by using information theoretic and perceptual model-based methods and detection theory are also present in the literature [64–70]. Capacity estimation is a fundamental problem of steganography [69], where the question is how much data can safely be hidden without being detected? However, in watermarking, the primary constraint for the capacity is its mutual dependence on a few other properties (e.g., perceptual similarity, robustness) rather than the detection problem as in steganography. Therefore, we define watermarking capacity on the basis of perceptual similarity of (i,ī), for which the scheme works objectively (e.g., without a failure).
Definition 4.7(Embedding capacity).
Watermarking embedding capacity for an image, i is the maximum size of any watermark, w=G
g
(i,m,j) for all m and j, to be embedded in i, such that E
e
(i,w)≈i, , and there exists such that .
Definition 4.7 suggests that to know the capacity of a watermarking scheme for an image, one needs to know how many bits can be embedded in the image with achieving the perceptual similarity and error probability (e.g., successful detection) requirements. This capacity estimation method may vary with the type of watermarking schemes. Although several attempts have already been made [64–70] to know the capacity bound as mentioned above, developing a general method for capacity estimation of each type of watermarking schemes could still be interesting. This may also help solve other capacity-related problems like the capacity control [50]
In image applications, embedding capacity is usually expressed as a ratio, bit-per-pixel (bpp). According to Definition 4.7, if the watermarking embedding capacity is n-bit, and the size of watermark is m-bit (i.e., w={1,0}m), then the necessary condition for an invisible watermarking scheme is m<n. This condition suggests that there can be a hidden assumption of recursive embedding in developing an invisible scheme - if the required capacity is not achievable in first run of E(·), the remaining bits can be re-embedded recursively. That assumption may severely affect the performance of a watermarking scheme in practice, and thus needs to be explicitly stated, if applicable.
4.7 Error probability
Error probability is an important property that helps determine the reliability of a watermarking scheme in practice. Some of the important and commonly used measures of error probability are bit error rate (BER), false-positive rate (FPR), false-negative rate (FNR). However, this property is often disregarded in developing a watermarking scheme, assuming a reliable (operating) environment where communication errors are ‘negligible’ and can be managed, for example, by using a suitable error correction code. This assumption is useful to simplify the application scenarios, but for some applications (e.g., proof of ownership), this property needs to be studied explicitly.
For example, BER can be considered to evaluate the performance of the functions D(·) and X(·) in obtaining and respectively. (Here, BER follows its standard definition in communication system.) In our proposed model, we defined D(·) in such a way that the absence of a valid watermark, w in a watermarked image, ī outputs a detection failure. Otherwise, D(·) returns , which indicates that the input image is watermarked. Following this, we define the false positive and false negative for our model below.
Definition 4.8(False positive and false negative)
A watermarking detection in a normal condition is said to be a false positive if D
d
(i,w)≠⊥ for some i. Conversely, a watermarking detection is a false negative if D
d
(ī,i,w)=⊥ for some ī. Here, the normal condition allows the scheme to run with all of its valid inputs, outputs and functions.
Irrespective of application scenarios, ideally, a zero FNR and FPR represents a reliable detection. Particularly, a watermarking scheme can be of no use if a scheme is unable to detect a valid watermark in normal condition of operation. Achieving a zero FNR and FPR in practice, however, may not be realistic for many reasons like communication errors. So, it is reasonable here to define a highly accurate detection for an application scenario in terms of a very low probability (e.g., in the order of 10−6) of detection failure.
However, error probability may be confused with other watermarking properties. Other properties (e.g., security, robustness, perceptual similarity) may also deal with errors, which can be of different types; for example, bit-errors (often termed as distortion) in a valid watermarked/unwatermarked image, which can be incurred maliciously, unintentionally, or as a system requirement, may also cause a detection failure. Further, we note that the function E(·) itself utilize the error signal, e.g., exploiting the redundant bit planes of an image, for embedding. This embedding error can be considered as a system requirement and thus can be addressed in terms of perceptual similarity requirement. Specifically, while error probability measures can be used to determine the system error rate for the reliability of a watermarking scheme, the other perceptual errors (ie, distortion) can be studied in terms of the security, robustness and perceptual similarity properties.
4.8 Security
Security property of watermarking schemes as a whole may be far from easy to conceptualize (and may not be always necessary in practice) [71–73]. Two main possible reasons are (i) application-dependent properties and (ii) the confusion between security and robustness requirements. In practice, different image applications may require different levels of security. Some applications do not need to be secure at all since there is no ultimate benefit in circumvention of watermarking objectives. For example, where a watermark is used only to add value in which they are embedded rather than to restrict uses for some device control applications [42]. Therefore, these types of watermarks do not need to be secure against any hostile attacks, although they still need to be robust against common processing techniques used in those applications. (This is how we defined the robustness property in Definition 4.6.) Although the requirements for robustness and security properties of a watermarking scheme may overlap [61], they need to be considered separately. For security properties, in contrast to robustness, all possible attacks that an adversary may attempt with in a particular scenario are to be studied.
Definition 4.9(Security)
A watermarking scheme is called
–secure if the scheme retains the security against the attack
( ie, if it is ‘hard’ to succeed with the set of adversary actions mounted by the attack
).
An application-specific analytical approach is often considered to study watermarking security [3, 16, 74–80]. In a broad sense, this practice suggests that the security property can be studied for two main types of watermarking schemes: robust and fragile. However, instead of focusing on a specific type of watermarking schemes, in this paper (section 5), we are more interested in studying the general scenarios of a set of possible attacks in an abstract level for image applications. The main idea is to demonstrate how an adversary of different capabilities may win with different conditions. We call this a win condition. Knowing the inputs, outputs and the win conditions would eventually help visualize the possible attacks in an application. (With that visualization, conducting an application-specific security analysis can be easier and more efficient). Here, we consider that identifying the set of attacks in a specific application and defining them in the model are the first steps to defining the watermarking security.