Meet DragGAN: Next-Level AI Image Manipulation
DragGAN could be a paradigm shift in AI-powered content creation
Just a month ago (which seems like an eternity in AI), I wrote an article about GigaGAN highlighting the continued importance of Generative Adversarial Networks (GAN) despite the increasing popularity of diffusion models like Midjourney and Stable Diffusion.
Now we have another research paper introducing an incredible new feature of GAN image generation: the ability to alter images by simply moving their attributes with a mere click and drag of the mouse.
Yes, you heard it right. Let’s look at how DragGAN works and explore the novel possibilities it opens up.
So, what’s GAN?
A Generative Adversarial Network (GAN) is a type of machine learning system that consists of two parts: a generator and a discriminator.
These two parts are trained in a competitive scenario where the generator creates ‘fake’ data (e.g. trying to imitate an image) and tries to fool the discriminator into classifying it as ‘real’.
On the other hand, the discriminator, during training, learns to differentiate real data from fakes. This back-and-forth “competition” enhances the quality of the generator’s outputs over time, which is enabling GANs to produce incredibly realistic synthetic data.
How DragGAN works
DragGAN opens up new possibilities for controlling GANs by allowing any point of a GAN-generated image to be “dragged” to a target point, thus transforming the image. We’re talking about manipulating an image with explicit control over pose, shape, expression, and layout! 🤯
If that wouldn’t be a game-changer already, the paper also introduces a process called “GAN inversion” that allows converting real images into a format that the GAN can comprehend and transform via DragGAN.
How does this work?
GANs learn to represent the data they are trained on within the so-called latent space, a virtual representation of all possible images that the GAN can generate. Each point in an image corresponds to a point in the latent space.
When you select a point in an image and move it, DragGAN finds the corresponding point in the latent space and moves it accordingly. After manipulating points in this latent space, DragGAN translates these points back into actual images. In more technical terms, DragGAN learns a transformation in the latent space that corresponds to the desired movement in the image space.
An exciting new possibility, indeed! You can check out some incredible demo videos on the official DragGAN project page:
https://vcai.mpi-inf.mpg.de/projects/DragGAN/
Original paper:
https://vcai.mpi-inf.mpg.de/projects/DragGAN/data/paper.pdf
➡️ For more information about AI & Creativity, follow me on Twitter or Medium (use my referral link to get full access to all my articles and those of thousands of other writers).
➡️ If you like my content, why not leave a “clap” at the end of this article, so more people can see it?
