TEXT-TO-VIDEO
How do I turn text into a video?
Diffusion models generate realistic videos from the text
Most of the available text-to-video tools edit existing movie records into a stock story. This article describes the diffusion models that have been available for several weeks and synthesize new realistic videos based on prompts.
🟠 HQ video generated from text
I recommend you a new alternative method available through Eva Rtology’s Substack, HERE you will find the necessary tool.
Phenaki vs META vs Imagen Video
In September, the researchers described the methods, and the first examples were presented, but unfortunately, the tools were not provided. An alternative approach could be a solution for users today.
Imagen Video was trained on 14 million text-video pairs, 60 million image-text pairs, and the LAION-400M image-text dataset, which is available to the public. This gave it the ability to adapt to a variety of styles. Imagen Video could make videos that looked like watercolors and paintings by Van Gogh. Imagen Video has shown that it understands depth and three-dimensionality. This lets it make videos like drone flythroughs that move around and show objects from different angles without distorting them.
Imagen Video can also make text look good, which is a big improvement over other image-making programs. For example, stable Diffusion and DALL-E 2 have trouble turning prompts like “a logo for ‘Diffusion’ into the type that can be read, but Imagen Video does it quickly, at least based on the paper.
That doesn’t mean that Imagen Video doesn’t have any limits. The clips chose from Imagen Video are sometimes jerky and distorted, with objects that blend together in physically impossible ways and don’t make sense.
“Overall, the text-to-video problem hasn’t been solved yet, and it won’t be long before we can make movies as good as DALL-E 2 or Midjourney,”
To improve this, the Imagen Video team wants to work with the researchers behind Phenaki. Phenaki is a new Google text-to-video system that can turn long, detailed prompts into videos that are more than two minutes long but are of lower quality.
It’s worth learning more about Phenaki to see where the two teams might go if they worked together. Phenaki is more concerned with length and consistency than Imagen Video is. The clips made by Phenaki have the same bugs as those made by Imagen Video, but it’s incredible how closely they match the long, detailed text descriptions that made them.
Try an alternative text-to-video method HERE
I invite you to explore the concept of Machine Learning Art by reading and learning from the many articles found on 🔵 MLearning.ai 🟠
Check out my instagram with new material every week
- If you enjoyed this, follow me on Medium for more
- Want to collaborate? Let’s connect on LinkedIn
- https://linktr.ee/datasculptor
- 3D Machine Learning generated model on sketchfab