Machine Learning Art

How to edit a NeRF sculpture

Geometry Editing of Neural Radiance Fields

Neural Rendering | Editing 3D Objects

The Neural Radiance Field (NeRF) has shown considerable promise in innovative scene view synthesis. Current NeRF-based approaches, on the other hand, do not allow for user-controlled shape editing in the scene. Several ways of modifying the radiance field according to the user’s requirements have been presented in the past. However, the modifications have been confined to color editing or object movement and rotation.

• May 2022 — AI art tools update can be found ➡️ HERE ⬅️

The authors describe a technique that allows users to control the implicit representation of a scene and synthesizes unique view pictures of the changed scene without retraining the network. They link the target scene’s implicit neural representation and the retrieved explicit mesh representation.

Project Page (scroll down)

To alter the mesh representation of the scene, users can first use well-developed mesh-based deformation algorithms. The approach then uses user changes to bend the camera rays by inserting a tetrahedra mesh as a proxy, resulting in the rendered outcomes of the altered scene. Extensive testing shows that the framework can deliver perfect editing outcomes on both synthetic and real-world data.

Editing a static neural radiance field technique (NeRF)

Users merely need to record multi-view photos to create a NeRF representation, after which they may alter the implicit representation of the scene clearly and intuitively. On the geometry of the scene, which comprises several objects, the approach may conduct user-controlled shape deformation.

Results of deformation transfer

The authors may utilize deformation transfer methods to transfer deformations from the current deformation sequence to the natural captured objects in addition to user-controlled shape deformation. This might lead to some intriguing applications. For example: using a video clip to translate the motions of a human face into a head sculpture.

Shape Morphing

In this research, the authors present the first approach for user-controlled shape morphing of the geometry of a neural radiance field network. By establishing a connection between the explicit mesh representation and the implicit volume representation, their technique may alter the implicit representation using the well-developed triangular mesh deformation method. Users may visualize the editing outcomes from arbitrary views using NeRF’s new view synthesis feature. The approach may be used to alter scene items such as human bodies, animals, and man-made models in common real-world scenarios. Compared to prior NeRF editing approaches, our method offers more freedom and can handle detail editing.

@inproceedings{Yuan22NeRFEditing,
    title={NeRF-Editing: Geometry Editing of Neural Radiance Fields},
    author={Yuan, Yu-Jie and Sun, Yang-Tian and Lai, Yu-Kun and Ma, Yuewen and Jia, Rongfei and Gao, Lin},
    booktitle={Computer Vision and Pattern Recognition (CVPR)},
    year = {2022}
}

Project Page:

https://arxiv.org/pdf/2205.04978.pdf

Code Comming Soon

Keywords: computer vision, Artificial Intelligence, datasets, Machine Learning, AI art, art, digital art, 3D, generative, 3D modeling, NeRF,

I invite you to explore the concept of “AI creativity” by reading and learning from the many articles found on 🔵 MLearning.ai 🟠

• Check out my instagram with new material every week
• If you enjoyed this, follow me on Medium for more
• Want to collaborate? Let’s connect on LinkedIn
• https://linktr.ee/datasculptor
• 3D Machine Learning generated model on sketchfab

Data Scientists must think like an artist when finding a solution when creating a piece of code. Artists enjoy working on interesting problems, even if there is no obvious answer.

All our writers (members) receive the opportunity to be promoted on our social media, which increases the popularity of articles published on MLearning.ai

1. Linkedin (10.5K+ ML-professionals)
2. Twitter (4.8K+ followers)
3. Instagram (2.2K + followers )
4. Sketchfab * — individual vRooML!
5. Facebook
6. Youtube
7. Apple Podcasts
8. Substack

🔵 Submission Suggestions