Summary

The website presents a one-class Convolutional Neural Network (CNN) architecture in Keras that combines deep feature extraction with one-class classification in a single step.

Abstract

The article discusses the implementation of a One-Class Convolutional Neural Network (OC-CNN) in Keras, which integrates deep feature extraction with one-class classification. This approach, inspired by a paper by Poojan Oza and Vishal M. Patel, represents an advancement over traditional multi-step methods that use OC-SVM for outlier detection. The OC-CNN utilizes any pre-trained deep learning model as a feature extractor, followed by a multi-layer perceptron to classify the embedded representations as either belonging to the target class (label 1) or noise (label 0). The novelty lies in the method's ability to train on a single class of data by augmenting the dataset with zero-centered Gaussian noise, effectively creating a binary classification problem. The experiment section details how the model can be trained to distinguish between cats and non-cats (or dogs) using a pre-trained VGG network, achieving an accuracy of around 85% in one-class learning. The author emphasizes the versatility and potential cost savings of this approach, which can be applied to various fields by leveraging transfer learning.

Opinions

The author views traditional one-class classification methods like OC-SVM as less effective compared to the proposed OC-CNN approach.
The article suggests that the OC-CNN method is flexible and powerful, as it can use any pre-trained model for feature extraction.
The author is optimistic about the potential of OC-CNNs to reduce the need for extensive labeled datasets, which can be costly and time-consuming to obtain.
The use of Gaussian noise to create a synthetic 'noise class' is presented as a key innovation in the field of one-class classification.
The author believes that the performance of the OC-CNN, with an accuracy of around 85%, is impressive for a one-class learning task.
The article encourages readers to explore the author's GitHub repository for further insights and to stay connected via LinkedIn.

One-Class Neural Network in Keras

Develop a Convolutional VGG for one-class classification tasks

Unsupervised learning, applied in one-class classification, aims to discover rules to separate normal and abnormal data in the absence of labels. One-Class SVM (OC-SVM) is a common unsupervised approach to detect outliers. It considers all the data points as positively labeled instances and builds around them a smooth boundary to detect ‘strange’ samples.

Recently, various approaches based on feature extraction models appear to be a valid instrument to use with OC-SVM. Following the amazing success of deep neural networks as feature extractors, different methods that exploit feature extraction, using deep-learning, and OC-SVM were introduced as multi-step one-class procedures.

In this post, we present a One-Class Convolutional Neural Network architecture (as introduced here) that merges the power of deep networks to extract meaningful data representations along with the one-class objective, all in one-step.

THE MODEL

The structure of the proposed architecture is composed of two parts: a feature extractor and a multi-layer perceptron. A fundamental aspect of this approach is that any pre-trained deep-learning model can be used as the base network for feature extraction. The most common choices regard the adoption of standard network modules for feature extraction, for example, VGG, ResNet, or Inception can be good alternatives for this specific task. There are no fixed rules regarding the points where to cut the network in order to produce a meaningful feature representation. The multilayer perceptron part is located at the end, it receives the embedded representations and tries to classify them into 0 or 1. Here, 1 means that the input sample belongs to the real target class while 0 means that the input sample belongs to the noisy class. As before, the choice of a structure for this part is not fixed. It can be manipulated and tuned in order to achieve better performances.

Between the feature extractor module and the final multilayer perceptron network happens the magic. Exactly there we can find the core idea of the whole procedure, which permits us to join the feature extraction with the classification, all in one step and having at disposal only one set of label. The embedded data samples are ‘corrupted’ adding some zero-centered gaussian noise. These modified samples are then concatenated batch-wise with their original couples. The result of this approach is that we have a duplicated batch of images formed by original samples (class 1) and corrupted ones (class 0). Our aim is that our classification layers can understand this difference and discriminate real images from all the rest.

Schema of the proposed approach. Source HERE

EXPERIMENT

We try to replicate the above workflow using the versatility of Tensorflow and Keras. First of all, we need a one-class classification problem. All classification tasks can be seen as a one-class problem. We can simply select a label of our interest and train a model to recognize it versus the rest, and this is exactly what we do.

The cat-vs-dog task can sound good. This is because we can exploit the power of transfer learning using some standard and pre-trained deep-learning models.

We start choosing a label of our interest, let’s say ‘cats’. We take all and only the images of cats in our train data. The dogs are retrieved during test time. At cats, we assign the label 1. The label 0 is automatically created during training and consists of some random gaussian images. The original batch input of only cats is so duplicated and is now formed by labeled data and random signals. The real images are feed inside the feature extractor network. In our case, it consists of a pre-trained VGG. The VGG creates a meaningful embedded representation of our real images that are concatenated back with the random images. At this point, they pass inside a multilayer perceptron neural network in which weights are trainable. At the end, we have at disposal a full architecture that can recognize cats and that is trained only on cats! The inference of the model can be computed as always. We deactivate the part of the network responsible to produce random embedding representation and maintain only the feature extractor and the multilayer perceptron. We test the procedure passing to the network cats’ and dogs’ images. If successful, dogs must be classified as noise (equivalent to the label 0).

To get reliable results, the fitting procedure must be applied to all the classes at our disposal. In our example, we fit a model only with ‘cats’ and then we fit a model only with ‘dogs’. The total performance is obtained as an average of all the single tasks’ performances. We obtain a final accuracy of around 85% which is very good for a one-class learning procedure!

SUMMARY

In this post, we introduced an architecture for training end-to-end on a one-class classification task. This approach is applicable in every field leveraging pre-trained transfer learning representations. It can be also a good resource to apply in every scenario where retrieving labeled data can be cost-intensive.

CHECK MY GITHUB REPO

Keep in touch: Linkedin

REFERENCES

One-Class Convolutional Neural Network. Poojan Oza, Student Member, IEEE, and Vishal M. Patel, Senior Member, IEEE