Convert Images to Tensors in Pytorch and Tensorflow

Learn to transform data natively

As you have known, tensors represents the building blocks for your machine learning project. They are are immutable and can only be created and not updated. Handling tensors is complicated since it is largely dependent on the framework that you used and there is no standardization on how to code it.

As a result, you will often stuck in a situation in which you have to perform custom manipulations such as converting from Pytorch to Tensorflow or changing the dimension of the tensors.

In this tutorial, you will learn to:

• convert images to tensors in both Pytorch and Tensorflow frameworks
• change the dimension order of Pytorch tensor using torch.permute
• change the dimension order of Tensorflow tensor using tf.transpose
• convert tensors from Pytorch to Tensorflow and vice versa

Let’s proceed to the next section and start installing all the necessary Python packages.

Setup

It is highly recommended to create a new virtual environment before you continue with the installation.

Pytorch

Run the following command to install both torch and torchvision packages.

pip install torch torchvision

torchvision is an essential package which provides quite a number of image transformation functions such as resizing and cropping.

Python Imaging Library

In addition, you need to install Python Imaging Library (PIL) which complements torchvision when loading your images. You can install it as follows:

pip install Pillow

Tensorflow

Tensorflow installation is a lot more straightforward now since version 2. Install it with the following command:

pip install tensorflow

Once you are done with the installation, proceed to the next section for the implementation.

Convert Image to Tensorflow Tensor

In this section, you will learn to implement image to tensor conversion code for both Pytorch and Tensorflow framework.

For your information, the typical axis order for an image tensor in Tensorflow is as follows:

shape=(N, H, W, C)

• N — batch size (number of images per batch)
• H — height of the image
• W — width of the image
• C — number of channels (usually uses 3 channels for RGB)

You can easily load an image file (PNG, JPEG, etc.) with the following code:

Change Tensorflow Tensors Dimension

Furthermore, Tensorflow does provide an useful function called tf.transpose to permutes the dimensions of tensor according to the value of perm. By default, perm is set to [n-1…0] where n represent the number of dimension.

Let’s say that you would like to change the shape of tensor from

(224, 224, 3) -> (3, 224, 224)

Simply call the tf.transpose function as follows:

tensor = tf.transpose(tensor, perm=[2, 0, 1])

This comes in handy when performing inference on models that originated from Pytorch (e.g. converted from Pytorch to ONNX to Tensorflow) since the standard structure for image tensor differs between both frameworks.

Convert Image to Pytorch Tensors

On the other hand, the shape for image tensor in Pytorch is slightly different from Tensorflow tensor. It is based on the following torch.Size instead:

torch.Size([N, C, H, W])

• N — batch size (number of images per batch)
• C — number of channels (usually uses 3 channels for RGB)
• H — height of the image
• W — width of the image

There are multiple ways to load image data and one of it is as follows:

• load via Pillow package
• set up transformation function
• apply resize transformation on image
• set up tensor conversion function
• apply tensor conversion function on image

Have a look at the following conversion script:

Unlike Tensorflow which uses the term expand dimension to add a new dimension, Pytorch is based on squeeze and unsqueeze. Squeezing means that you will reduce the dimension by truncating it while unsqueeze will add a new dimension to the corresponding tensor.

Change Pytorch Tensors Dimension

Also, you can easily re-order the dimension of your tensor via torch.permute function. Let’s say that you would like to change the shape as follows:

(3, 224, 224) -> (224, 224, 3)

You should pass in the following input when calling torch.permute function:

torch.permute(tensor, (1, 2, 0))

Convert Tensors between Pytorch and Tensorflow

One of the simplest basic workflow for tensors conversion is as follows:

• convert tensors (A) to numpy array
• convert numpy array to tensors (B)

Pytorch to Tensorflow

Tensors in Pytorch comes with its own built-in function called numpy() which will convert it to numpy array.

py_tensor.numpy()

Then, simply call the tf.convert_to_tensor() function as follows:

import tensorflow as tf

...

tf_tensors = tf.convert_to_tensor(py_tensor.numpy())

Tensorflow to Pytorch

On the other hand, tensors conversion from Tensorflow to Pytorch follows the same pattern since it does have its own numpy() function:

tf_tensors.numpy()

Subsequently, you can call the torch.from_numpy() for tensors conversion:

import torch

...

py_tensors = torch.from_numpy(tf_tensors.numpy())

Conclusion

Let’s recap what you have learned today.

This article started off with a brief explanation on tensors and the difficulties in manipulating tensors.

Next, it moved on to installation process via pip install.

It continued with step-by-step guides on converting image data to Tensorflow tensors as well as how to change the dimension of the tensors based on your own needs.

In addition, it also covered on tensor conversion for Pytorch framework and the corresponding dimension permutation.

Lastly, it highlighted a few code snippets to convert tensors from Tensorflow to Pytorch and vice versa.

Thanks for reading this piece. Have a great day ahead!

References

1. Tensorflow — Introduction to Tensors
2. Tensorflow — tf.io
3. Pytorch — torchvision.transform