Convert Videos to ASCII Movies Art With Python

Learn how to generate ASCII animations from your videos

There are many types of art out there, from painting to music. ASCII art is no different. It consists of representing concepts and pictures using only printable ASCII characters — the ones you find on your keyboard, just to be clear. The simplest form of ASCII art are emoticons such as :-) or /:-(. They can also extend to more complex images, such as this tree made by Crazy Copperhead:

           _ ._  _ , _ ._
         (_ ' ( `  )_  .__)
       ( (  (    )   `)  ) _)
      (__ (_   (_ . _) _) ,__)
          `~~`\ ' . /`~~`
               ;   ;
               /   \
 _____________/_ __ \_____________

Or even to whole real-life photos like this one:

ASCII version of the original image by freestocks on Unsplash

In case you were interested, I’ve written an article on converting images to ASCII art a while back. I suggest you check it out before proceeding, as it may render what I’ll be talking about a bit clearer.

Convert Pictures to ASCII Art

A brief guide on how images work and how the pixels can be represented as ASCII characters — code example included.

towardsdatascience.com

The next step is to create a whole ASCII video, which we’ll be discussing in this article.

Video files and frames

As you might already know, a video file, without considering its audio, is composed of two main parts:

The header, containing metadata such as the file format signature or the frame rate.
The data section, containing information about its frames and pixels.

Every frame is really a matrix (2-dimensional array) of pixels, each represented as a set of channels, used to indicate the pixel’s color and opacity. Common channel configurations are the RGB (Red Green Blue) and RGBA (Red Green Blue Alpha).

ASCII conversion algorithm

For this code example, I’ll be using Python for its simplicity, so that anyone can follow.

First things first, you have to load the video file to convert it to ASCII art.

Once you have access to its content, you have to read its header in order to get some useful metadata, such as the width, height, frame rate (fps), and frame count. This information will be crucial for the next step.

Create a new file to store the converted video. For the sake of clarity, I’ll be referring to such file as a .ascii file, but you can give it whatever extension you like. You then have to write the metadata you gathered before as its header. This step is crucial in order to be able to read and play the ASCII video file later.

Now, get the video frames one by one. For every frame, translate each of its pixels to the corresponding ASCII character. We’ll discuss this process in detail later. Then, store the newly converted frame in the .ascii file. You may also keep every frame in RAM, but that would require an excessive amount of gigabytes of memory. The solution is to store the frames in a file and load them as needed when playing the video.

Now, the video player will then have to load the .ascii file and read its header to get the video’s metadata, which will be used to read the individual frames and display them with the right frame rate.

Lastly, read the frames from the files and print them to the console one by one, taking into account the frame rate. I’ll be discussing this custom print function later.

Converting a frame to ASCII

Now, let’s dive deeper into how frames are represented as ASCII art. For this explanation, I’ll be extending Python with C to write more resource-intensive functions, since Python is far too slow to execute them in a reasonable amount of time. I have omitted the steps specific to extending Python with C functions since it is beyond the scope of this article, but I’ll link the GitHub repo with the full code at the end.

As you may already know, frames are an array of pixels. Each pixel is represented by an array of three values: the color channels for red, green, and blue. Every pixel in the frame has to be translated into an ASCII character to be printed to the console.

Which ASCII character should be used depends on how “intense” the pixel is. Intensity is defined as the percentage of activation of a pixel: so the highest the RGB values, the higher the intensity. For example, a pixel with the RGB values 255, 255, 255 has an intensity of 100%, whereas 0, 0, 0 is 0%.

You then have to choose an ASCII character based on the pixel intensity. The characters are ordered from the one that occupies less space to the bulkiest. For example, “.” is certainly slimmer than “@”, and thus less intense.

With this in mind, you have to iterate over every row in the frame and calculate the intensity and the corresponding ASCII character for every pixel. Since most terminals don’t allow you to reduce the space between the lines and the frame would result stretched, it’s necessary to include a space after every character to fix the aspect ratio.

Playing the video and frame rate

Every video has a set frame rate that makes the flow of frames feel natural. When converting a video to ASCII art, you have to preserve it by storing it in the .ascii file header. It will then be used to alter the duration of every frame when displaying the video.

Since we have already included the newline characters (\n) when converting the frame to ASCII, we don’t have to mind the aspect ratio or frame size: you can just clear the screen and print the whole ASCII string frame to the console.

The tricky part is calculating how should the frame last on the screen before moving to the next. For this reason, you have to measure the time taken to draw the image and subtract it from the default frame duration, which was obtained before by dividing 1 second by the FPS, then by multiplying the result by 1e9 to express it in nanoseconds.

Finally, sleep for the remaining amount of time of the frame duration, if any. In case the frame takes more time to print than the default frame duration, the video just plays at a slower pace.

Below is shown an example of ASCII movie art generated with this tool. Here you can find the original video.

Note that to make the movie fit in your terminal, you may need to reduce your font. Also, I suggest you use a square font for ASCII art. The one I recommend is included in the GitHub repository.

Conclusion

To wrap it up, this has been an explanation of the algorithm to convert a video file to an ASCII art movie with code examples. If you are interested in the full, thoroughly-commented code, check out the project on GitHub.

I hope you enjoyed this article. Thanks for reading!