Machine Learning Art
End-to-End Framework for Flow-Guided Video Inpainting
try to inpaint your video through colab
Recent video inpainting systems take advantage of optical flow, which captures motion information across frames by propagating pixels along its pathways. The hand-crafted flow-based procedures are applied individually to build the entire inpainting pipeline with these systems. As a result, these procedures are inefficient and significantly rely on interim data from earlier phases. This research presents an End-to-End framework for Flow-Guided Video Inpainting (E2FGVI), which consists of three extensively built trainable modules: flow completion, feature propagation, and content hallucination. The suggested method surpasses state-of-the-art methods in qualitative and quantitative terms, showing promise in terms of efficiency.
- April 2022 — AI art tools update can be found ➡️ HERE ⬅️
Project Page (scroll down)
Video inpainting seeks to fill in the “damaged” sections of video footage with convincing and consistent content. Object removal, video restoration, and video completion are just a few examples of real-world applications. Despite tremendous advances in image inpainting, complicated video settings and degrading video images continue to present hurdles to video inpainting. Doing image inpainting directly on each frame results in temporally inconsistent movies with severe artifacts. High-quality video inpainting must take into account both spatial organization and temporal coherence. Recent advances in Deep Learning have led scientists to search for more efficient methods.
Headlines:
🔵 In compared to SOTA techniques, the suggested E2FGVI demonstrates significant gains on all quantitative criteria.
🔵 Highly efficient: On a Titan XP GPU, the method processes 432x240 films in 0.12 seconds per frame, which is about 15 times quicker than previous flow-based methods. Furthermore, of all the SOTA approaches compared, the method has the lowest FLOPs.
Installation
git clone https://github.com/MCG-NKU/E2FGVI.gitConda Environment and Install Dependencies
conda env create -f environment.yml
conda activate e2fgviConclusion
E2FGVI is an end-to-end trainable flow-based model for video inpainting. The three components
- flow completion,
- feature propagation,
- content hallucination
are intricately built and work together to address various problems in earlier systems. The author’s strategy achieves state-of-the-art quantitative and qualitative performance on two benchmark datasets. Moreover, according to experimental results, it is efficient in inference time and computing complexity.
@inproceedings{liCvpr22vInpainting,
title={Towards An End-to-End Framework for Flow-Guided Video Inpainting},
author={Li, Zhen and Lu, Cheng-Ze and Qin, Jianhua and Guo, Chun-Le and Cheng, Ming-Ming},
booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2022}
}
COLAB:
Project Page:
https://arxiv.org/pdf/2204.02663.pdf
Github:
https://github.com/MCG-NKU/E2FGVI
Keywords: computer vision, video, colab, , machine learning, SOTA, video inpainting, state-of-the-art, E2FGVI,
I invite you to explore the concept of “AI creativity” by reading and learning from the many articles found on 🔵 MLearning.ai 🟠
- Check out my instagram with new material every week
- If you enjoyed this, follow me on Medium for more
- Want to collaborate? Let’s connect on LinkedIn
- https://linktr.ee/datasculptor
- 3D Machine Learning generated model on sketchfab
Data Scientists must think like an artist when finding a solution when creating a piece of code. Artists enjoy working on interesting problems, even if there is no obvious answer.
All our writers (members) receive the opportunity to be promoted on our social media, which increases the popularity of articles published on MLearning.ai