Watermarking in the Age of AI-generated Images

Let me tell you the story of the mysterious land of Digitopia, there lived a great artist named Henry, who was famous for his jaw-dropping digital creations. Suddenly one day, a mysterious AI entity Genai stepped into this town and started producing artwork similar to Henry’s but at a much faster rate. Henry was sad and lost hope as his creations started appearing under Genai’s name until he discovered the power of watermarking.

Watermarking is a way of embedding or hiding an artifact on a given image or text. This allowed Henry to protect his work from GenAI. This invisible artifact preserved his art's integrity and gave Henry a protective shield, preventing his original work from getting lost in the sea of Generative AI images.

The story of Henry and Genai throws light on our current world. Where digital artists coexist with AI-generated art. The increasing amount of AI-generated content necessitates the development of new watermarking techniques to draw a fine line between human creativity and technological advancement, protecting original art and artist from IP theft in the vast realm of the internet.

In this blog, we are first going to understand the full concept of watermarking and then review the paper titled: Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust

NOTE: If you already know about watermarking in general and only want to know about it in the context of AI-generated images/ Tree ring watermarks paper review, skip to the section: Watermarking for AI-generated Images is different

Let’s first understand the concept of watermarking in normal images. There are two types of watermarks: visible and invisible. Both types of watermarks can be removed using several techniques. Simple Photoshop can do the job for visible watermarks, or other AI tools can perform it. If the watermark is visible, it becomes easier to remove it, but what about the invisible ones?

For invisible watermarks, it is slightly more complex to remove them, in this category also, watermarks are divided into two sub-categories, namely reversible and irreversible. In the reversible one, we want to extract the hidden logo to establish ownership of a particular group. With irreversible, we want to identify whether the given image is altered.

1. Reversible Watermarks: These are designed to be easily removable or reversible without causing much change to the original image. They are added in a way that makes their extraction or elimination quite easy while maintaining the quality and integrity of the original image.
2. Irreversible Watermarks: These ones are purposely designed to be difficult or impossible to remove or alter without causing significant change to the original content. Removing such a watermark causes significant changes to the original content. This type of watermark aims to deter unauthorized use or enforce copyright protection.

How do we add Watermark to images?

The spatial and frequency domains are two common ways to add invisible watermarks to a given image.

1. Spatial Domain Watermarking: Spatial domain watermarking involves directly manipulating the image’s pixel value to embed the watermark. This is typically achieved by modifying the color or intensity values of selected pixels in the image. For example, the watermark could be added by altering the pixel values’ least significant bits (LSBs). The changes made in the spatial domain are directly visible in the image.
2. Frequency Domain Watermarking: Frequency domain watermarking involves transforming the image from the spatial domain to the frequency domain using mathematical techniques such as the Discrete Fourier Transform (DFT) or Discrete Wavelet Transform (DWT). In the frequency domain, the image is represented by its frequency components. The watermark is embedded by modifying the coefficients in the frequency domain. Once the watermark is added, the image is transformed back to the spatial domain for visualization.

Why frequency domain watermarking is the preferred choice?

Frequency domain watermarking is often considered better due to its robustness, imperceptibility, and security. The watermark is embedded in the frequency coefficients by transforming an image into the frequency domain using techniques like the Discrete Fourier Transform (DFT) or Discrete Wavelet Transform (DWT). This approach resists common image operations while minimizing the visual impact on the image. Additionally, frequency domain watermarking allows for selective embedding and localization, ensuring the watermark is placed in less perceptually important areas. The inverse transform retrieves the watermarked image, preserving the modifications made in the frequency domain. Overall, frequency domain watermarking offers advantages in terms of preserving image quality, resistance to attacks, and imperceptibility to human perception. The image given below shows how images look in the frequency domain.

Attacking the Images

There are several ways to attack a digital image, here are few common attacks:

1. Crop or Resizing Attacks: Attackers may attempt to remove watermarks by cropping or resizing the image to eliminate the area containing the watermark. By adjusting the image dimensions or removing a portion of the image, they aim to remove or obscure the watermark.
2. Filtering Attacks: Filtering techniques, such as blurring or noise addition, can be employed to degrade the watermark’s visibility or make it more difficult to detect. These attacks aim to distort the watermark by manipulating the image’s pixel values.
3. Copy-Move Attacks: In copy-move attacks, the attacker duplicates a region of the image containing the watermark and pastes it over another area, effectively covering it. This technique exploits the content similarity within the image to remove the watermark.
4. Geometric Attacks: Geometric attacks involve geometric transformations, such as rotation, scaling, or skewing, to alter the image. These transformations could disrupt the watermark pattern and make it harder to extract or identify.
5. Compression Attacks: Watermarks embedded in compressed images may be susceptible to compression attacks. Lossy compression algorithms, in particular, can introduce artifacts that degrade the watermark’s quality or make it less detectable.
6. Collusion Attacks: Collusion attacks combine multiple copies of watermarked content to eliminate or weaken the signal. By averaging or combining the watermarked copies, attackers attempt to cancel out the watermark, making retrieving the original information easier.
7. Brute-Force Attacks: Brute-force attacks involve systematically trying different techniques and algorithms to remove the watermark. Attackers may employ various image processing methods or explore vulnerabilities specific to the watermarking algorithm.

The two common metrics to measure the success of watermarking algorithm are:

Normalized Correlation (NC): Normalized Correlation measures the similarity or correlation between the original and extracted watermarks. It quantifies how well the extracted watermark matches the original watermark.

Bit Error Rate (BER): Bit Error Rate measures the accuracy of the watermark extraction process by quantifying the number of incorrectly detected or altered bits in the extracted watermark compared to the original watermark. A lower BER indicates a higher accuracy and a better quality of the extracted watermark.

Watermarking for AI generated Images is different

Watermarking AI-generated images is particularly challenging due to their dynamic, unique, and often complex nature. Unlike static digital images, AI-generated ones vary greatly in visual elements and are created in response to specific inputs. This complexity can hinder consistent watermark application and may lead to watermarks that are easily distorted or removed during the generation process.

Further complicating the issue is the potential for AI models to be retrained to identify and remove watermarks, a factor that traditional digital watermarking doesn’t have to contend with. Moreover, ensuring that watermarks don’t degrade image quality or aesthetic appeal while being decodable adds another layer of difficulty. Lastly, embedding sensitive information in watermarks might lead to privacy or legal issues. Therefore, effective watermarking of AI-generated images requires strategies that differ significantly from those used for normal digital images.

And lastly, how do we identify that a particular image was captured but not generated by AI?

Can you identify in the above image which is real and which is generated? Surely I can’t. We can’t identify which images are real or generated anymore, which poses a serious threat. This tech has gotten so good that few believe that within the next 5 years, we will have completely AI-generated videos that we won’t be able to tell apart, and the first industry it will take over is pornography. The porn industry is the hub for testing out all new technologies, be it VR, video tagging, or even the subscriber model of YouTube.

Let’s talk about the Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust paper solving the issue of digital authentication in the era of AI-generated images.

What they achieved in this paper, here’s the abstract from the paper:

Unlike existing methods that perform post-hoc modifications to images after sampling, Tree-Ring Watermarking subtly influences the entire sampling process, resulting in a model fingerprint that is invisible to humans. The watermark embeds a pattern into the initial noise vector used for sampling. These patterns are structured in Fourier space so that they are invariant to convolutions, crops, dilations, flips, and rotations. After image generation, the watermark signal is detected by inverting the diffusion process to retrieve the noise vector, which is then checked for the embedded signal. We demonstrate that this technique can be easily applied to arbitrary diffusion models, including text-conditioned Stable Diffusion, as a plug-in with negligible loss in FID.

Now let’s understand in detail what exactly they did and how they did it.

Methodology

The biggest change in their approach is to embed the watermark in the generation process rather than doing it post-generation. This change alone makes this watermarking technique robust and efficient to be scaled for every diffusion based-model.

1. Noise Vector & Diffusion Model: In a diffusion model, each generated image results from a complex transformation applied to a simple random noise vector. When visualized in Fourier space, this noise vector contains many high-frequency components that contribute to the final details of the image.
2. Pattern Imprinting: In the proposed watermarking method, a specific pattern is imprinted into the Fourier space of the noise vector. This pattern is so subtle that it doesn’t perceptibly affect the appearance of the final generated image, but it’s detectable through algorithmic analysis.
3. Modified Distribution: The imprinting process effectively modifies the distribution of generated images. An image drawn from this modified distribution does not carry a watermark in the classical sense (there are no visible post-hoc modifications). Still, the statistical properties of the image have been subtly altered in a consistent way that can be detected by algorithmic analysis.
4. Algorithmic Detection: The watermark detection process involves analyzing the statistical properties of an image and comparing them to the specific pattern imprinted in the Fourier space of the noise vector. If the image were drawn from the modified distribution, the analysis would reveal the watermark’s presence with a high accuracy level.
5. Robustness: The method is designed to be robust against common image transformations like cropping, color jitter, dilation, flips, rotations, or noise. Since the watermark is embedded in the image’s high-frequency components (typically preserved across these transformations), it can still be detected even after significant modifications to the image.
6. Implementation: The Tree-Ring Watermarking method can be integrated into existing diffusion model APIs without requiring additional training or fine-tuning. Only parties in control of the image generation model can detect the watermark.

Understanding the key ring in more detail

This watermarking method introduces a “key” pattern into the Fourier space of the original Gaussian noise array. The Fourier transform offers several properties that can be exploited for this purpose: rotations, translations, and dilations/compressions in pixel space correspond to similar operations in Fourier space, and color jitter corresponds to changing the zero-frequency Fourier mode.

These properties allow the watermark to remain robust against various image transformations. Importantly, the chosen key pattern should resemble Gaussian noise to prevent significant changes in the image distribution that could impact the diffusion model.

Three different types of key patterns are considered:

1. Tree-RingZeros: This pattern uses a circular region in Fourier space to make the watermark invariant to rotations in the image space. An array of zeros forms the key, offering invariance to shifts, crops, and dilations. While robust against manipulations, this key is not Gaussian-like and restricts the usage of multiple keys.

2. Tree-RingRand: This key is drawn from a Gaussian distribution and resembles the original noise array in Fourier space, expecting a minimal impact on generation quality. It allows for multiple keys but lacks invariance to image manipulations.

3. Tree-RingRings: This pattern involves multiple rings with constant values along each ring, drawn from a Gaussian distribution. It is invariant to rotations and provides some invariance to other image transformations while maintaining the Gaussian-like nature of the noise.

Ring patterns, especially in the Tree-RingRings method, ensure robustness to common image manipulations like rotations and maintain the Gaussian-like nature of the noise to prevent significant distribution shifts. This makes it a practical choice for watermarking in this context.

As we can see from the above image, this method adds a watermark that leaves no small noisy artifacts in the image.

One important thing to note here is that this technique slightly modifies the content of the generated image. However, still, it aligns with the given text prompt, and as there is no right or wrong image for a given text prompt, even the modified images are fully acceptable for a given prompt.

How are they detecting watermarks because generated image has none

The paper uses P-values to determine the likelihood that a watermark observed in an image could have occurred naturally by random chance. A P-value is a statistical measure that provides the probability of the observed data (or something more extreme) under a specified statistical model.

The watermark detection process is framed as a statistical hypothesis test in this context. The null hypothesis (H0) states that the entries in the Fourier-transformed image array are drawn from a Gaussian distribution. This assumption is based on the design of the diffusion process, which maps images onto Gaussian noise.

The paper defines a score (η) for each test image, representing a deviation from the expected key pattern in Fourier space. This score follows a noncentral chi-square (χ²) distribution if the null hypothesis is true.

The image is declared to be watermarked if the value of η is too small to occur by random chance likely. The probability of observing a value as small as η is given by the cumulative distribution function (CDF) of the noncentral χ² distribution. This CDF provides the P-value.

A lower P-value means that it’s less likely the observed watermark occurred by random chance, enabling the rejection of the null hypothesis. In other words, the watermark is detected if the P-value is below a chosen threshold. By setting this threshold, one can explicitly control the false positive rate, making false detections statistically unlikely. Therefore, the P-values make the detection results more interpretable and allow for control over the likelihood of false detections.

Conclusion

This paper is a milestone in protecting safety from AI-generated content and making AI-related art verifiable. This is a great attempt to regulate the AI-generated art world and keep the users safe from the malicious use of generated images and videos.

Thanks for your time and patience. Follow me for more such awesome content: https://vishal-ai.medium.com/

References

https://arxiv.org/pdf/2305.20030.pdf