AI art | Computer Vision and Pattern Recognition

Facial Editing of Real Videos

Stitch it in Time: GAN

StyleGAN has been widely used to edit realistic images of people, but editing videos poses an additional challenge to maintaining temporal coherency. Moreover, StyleGAN is not usually the best option for editing video because it relies on good quality data that is rare and time-consuming. The authors of this paper propose a new idea that works with face-editing techniques that have been used in StyleGAN, but this time applied to video editing. They base their idea on the latent-editing practices commonly employed with an off-the-shelf, non-temporal StyleGAN model. They also draw on insights from their work and show how they can provide a solid basis for the semantic editing of faces in videos, demonstrating significant improvements over the current state-of-the-art.

@misc{tzaban2022stitch,
      title={Stitch it in Time: GAN-Based Facial Editing of Real Videos},
      author={Rotem Tzaban and Ron Mokady and Rinon Gal and Amit H. Bermano and Daniel Cohen-Or},
      year={2022},
      eprint={2201.08361},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

How come it’s so smooth? Why is there so little jitter?

The answer is simple. The authors have tried something new, and they are doing it by analyzing the individual components of a GAN editing pipeline to find out which ones are consistent. This will save time and lead to a better editing process. Instead of using the wrong techniques, disturbing consistency, or investing a lot of time and effort in restoring it, they examine the different components of a GAN editing pipeline to determine which ones are consistent. Only use those!

Encoders are fluent at the local scale. The generator tuning works well in every setting and maintains a stable alignment. Together they almost perfectly preserve the original video’s consistency.

What does the appearance of this compare to the alternatives? This method can tackle complex scenes with a motion that destabilizes the state-of-the-art.

Computer Vision and Pattern Recognition

Code For the paper — Stitch it in Time: GAN-Based Facial Editing of Real Videos

Code Will be released by 14/02

AI is everywhere 🟠 But the question is, how much do you love it?

• Check out my instagram with new material every week
• If you enjoyed this, follow me on Medium for more
• Want to collaborate? Let’s connect on LinkedIn
• https://linktr.ee/datasculptor
• 3D Machine Learning generated model on sketchfab
• Substack

Why subscribe MLearning.ai?

We are a group of 800+ ML engineers and AI designers working to create intelligent artworks and develop new algorithms for creativity and machine learning. Subscribe to MLearning.ai for updates on our progress, blog posts about AI art, and more opportunities for creative minds like you!

A new perspective. A new idea. A new approach.