Summary

The web content describes the application of Principal Component Analysis (PCA) for image compression, using the MNIST dataset as a practical example to demonstrate the technique and its effectiveness in reducing image dimensions while maintaining quality.

Abstract

The article titled "Image Compression Using Principal Component Analysis (PCA)" provides an in-depth exploration of PCA as a dimensionality reduction technique. It illustrates how PCA can be utilized for image compression by transforming high-dimensional image data into a smaller set of principal components, which retain most of the original data's variability. The MNIST dataset of handwritten digits serves as the case study for this process. The author guides the reader through loading the dataset, preprocessing it by dropping the label column, and applying PCA without prior feature scaling due to the pixel values being on a similar scale. A scree plot is used to determine the optimal number of principal components, balancing image quality and dimensionality reduction. The article demonstrates that using 184 components captures approximately 96% of the variance, resulting in a compressed image that closely resembles the original despite a reduction of 600 dimensions. The conclusion emphasizes the importance of selecting an appropriate number of components to minimize quality loss while achieving significant compression.

Opinions

The author expresses that PCA is a powerful tool for unsupervised learning tasks, particularly for image compression.
It is noted that feature scaling is unnecessary for the MNIST dataset due to the pixel values being on a similar scale, which is an opinion based on the characteristics of the dataset used.
The author suggests that the first 10 principal components do not capture enough variability to produce a clear compressed image, indicating a preference for a higher number of components for better quality.
The article conveys the opinion that a balance must be struck between the number of components used in PCA and the resulting image quality, with a focus on maintaining the integrity of the original image data.
The author concludes that PCA is effective for image compression, as evidenced by the ability to reduce the MNIST dataset's dimensions by 600 while still retaining about 96% of the data's variability and visible content in the compressed image.

Image Compression Using Principal Component Analysis (PCA)

Dimensionality Reduction in Action

Principal Component Analysis (PCA) is a linear dimensionality reduction technique (algorithm) that transform a set of correlated variables (p) into a smaller k (k<p) number of uncorrelated variables called principal components while keeping as much of the variability in the original data as possible.

One of the use cases of PCA is that it can be used for image compression — a technique that minimizes the size in bytes of an image while keeping as much of the quality of the image as possible. In this post, we will discuss that technique by using the MNIST dataset of handwritten digits. After reading this article, you will get hands-on experience in PCA image compression with Python and Scikit-learn.

Let’s get started!

Load the dataset

The MNIST dataset contains the image data of handwritten digits. Since it is in the CSV file format, let’s load it using the Pandas read_csv() function.

import pandas as pd
mnist = pd.read_csv('mnist.csv')
mnist.head()

A part of the MNIST dataset (Image by author)

Each row contains the pixel values of one single image. The pixels which make the image can be considered as dimensions (columns/variables) of the image data. The ‘label’ column contains the values of the digit (0–9). We do not need that column for our analysis because PCA is an unsupervised machine learning task that does not deal with labelled data. So, we can simply drop that column.

mnist.drop(columns='label', inplace=True)
mnist.head()

MNIST data after removing the label column (image by author)

Now, the shape of the dataset is:

mnist.shape
#(60000, 784)

This dataset contains 60,000 images of 28x28 (784) pixels!

Display an image

Let’s display the 2nd image (row) in the MNIST dataset. This image should contain the digit ‘0’ since the label column value of the 2nd row is ‘0’.

Wow! It is the digit 0!

Feature scaling

Since PCA directions are highly sensitive to the scale of the data, we must do feature scaling before applying PCA if the data is not measured on a similar scale.

In the MNIST dataset, the pixel values of each image are ranging from 0 to 255 (similar scale). For example:

#2nd image
print(mnist.iloc[1].min())
print(mnist.iloc[1].max())
#0
#255

Since our data is measured on a similar scale, we do not need to do feature scaling for PCA.

Apply PCA

Choose the right number of dimensions

First, we need to choose the right number of dimensions (i.e., the right number of principal components). For this, we apply PCA with the original number of dimensions (i.e., 784) and create the scree plot to see how well PCA captures the variance of the data.

Explained variance as a function of the number of components (Image by author)

Let’s try using the first 10 components to compress the image. These components do not capture much of the variability in the original data. So, we will not get a clear image.

Compare this with image 1 (original image) that we obtained earlier. This image is not very clear and it is lack of information.

Let’s try using the first 184 components to compress the image. About 96% variability in the original data is captured by the first 184 components. So, this time, we will get a very clear image much similar to the original one.

We can also calculate the explained variance of 184 components:

np.cumsum(pca_184.explained_variance_ratio_ * 100)[-1]
#96.11980535398752

It is 96.1%.

Compare the compressed image with the original one:

This is how we can use PCA for image compression. The image at the left is the original image with 784 dimensions. The image at the right is the compressed image with 184 dimensions. After applying PCA on image data, the dimensionality has been reduced by 600 dimensions while keeping about 96% of the variability in the original image data! By comparing these two images, you can see that there is a slight image quality loss, but the content of the compressed image is still visible!

Summary

Today, we have discussed how we can use PCA for image compression. When the number of dimensions or components increases, the image quality loss decreases. We should always try to keep an optimum number of components that balance the explained variability and the image quality.

The inverse_transform() method of the pca object is used to decompress the reduced dataset back to 784 dimensions. This is very useful for visualizing the compressed image!

Thanks for reading!

This tutorial was designed and created by Rukshan Pramoditha, the Author of Data Science 365 Blog.

Read my other articles at https://rukshanpramoditha.medium.com

2021–04–12