The simplest way to train a Neural Network in Python

PyTorch and TensorFlow aren’t the only Deep Learning frameworks in Python. There’s another library similar to scikit-learn.

scikit-learn is my first choice when it comes to classic Machine Learning algorithms in Python. It has many algorithms, supports sparse datasets, is fast and has many utility functions, like cross-validation, grid search, etc.

When it comes to advanced modeling, scikit-learn many times falls shorts. If you need Boosting, Neural Networks or t-SNE, it’s better to avoid scikit-learn.

scikit-learn has two basic implementations for Neural Nets. There’s MLPClassifier for classification and MLPRegressor for regression.

While MLPClassifier and MLPRegressor have a rich set of arguments, there’s no option to customize layers of a Neural Network (beyond setting the number of hidden units for each layer) and there’s no GPU support.

Meet scikit-neuralnetwork

scikit-neuralnetwork addresses the issues with scikit-learn mentioned above. While there are already superior libraries available like PyTorch or Tensorflow, scikit-neuralnetwork may be a good choice for those coming from a scikit-learn ecosystem.

From developers of scikit-neuralnetwork:

scikit-neuralnetwork is a deep neural network implementation without the learning cliff! This library implements multi-layer perceptrons as a wrapper for the powerful pylearn2 library that’s compatible with scikit-learn for a more user-friendly and Pythonic interface.

Install scikit-neuralnetwork

To install scikit-neuralnetwork (sknn) is as simple as installing any other Python package:

pip install scikit-neuralnetwork

Custom Neural Nets

Let’s define X_train and y_train from the Iris dataset to run the examples below:

from sklearn.datasets import load_iris

data = load_iris()
X_train = data['data']
y_train = data["target"]

sknn offers a simple way to make a custom Neural Net. scikit-learn users will feel at home with a familiar API:

from sknn.mlp import Classifier, Layer

nn = Classifier(
    layers=[
        Layer("Maxout", units=100, pieces=2),
        Layer("Softmax")],
    learning_rate=0.001,
    n_iter=25)
nn.fit(X_train, y_train)

X_train, y_train variables are numpy arrays, so you can directly replace your scikit-learn model with a Neural Net from sknn. It even supports sparse datasets.

Convolutional Neural Nets

sknn has support for Convolutional Neural Nets. Finally, you will be able to achieve a state-of-the-art score on MNIST in a scikit-learn ecosystem.

from sknn.mlp import Classifier, Convolution, Layer

nn = Classifier(
    layers=[
        Convolution("Rectifier", channels=8, kernel_shape=(3,3)),
        Layer("Softmax")],
    learning_rate=0.02,
    n_iter=5)
nn.fit(X_train, y_train)

Recurrent Neural Nets

What about RNNs, like Long Short Term Memory (LTSM) or Gated Recurrent Unit (GRU)? RNNs are usually used for modeling sequences, like time series or textual data.

By going through the documentation, it seems there isn’t direct support for RNNs. There is support for native and custom layers, which should make the implementation of RNN possible.

From documentation:

You can use this feature to implement recurrent layers like LSTM or GRU, and any other features not directly supported. Keep in mind that this may affect compatibility in future releases, and also may expose edge cases in the code (e.g. serialization, determinism).

If you plan to work with RNNs I would recommend learning PyTorch or TensorFlow. If you need a quick start guide, I wrote an article about it a while ago:

Pipelines

scikit-learn has pipelines, which wrap feature transformation with modeling into a single pipeline.

Pipelines reduce the chance of overfitting and generally reduce the chances of various mistakes. They are also very useful when making cross-validation or grid search.

Many Machine Learning libraries don’t support scikit-learn pipelines so we need to implement it by ourselves. The great thing about scikit-neuralnetwork is that it fully supports scikit-learn pipelines.

Below is an example of a pipeline that scales features and trains a simple Neural Net.

from sknn.mlp import Classifier, Layer

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import MinMaxScaler

pipeline = Pipeline([
        ('min/max scaler', MinMaxScaler(feature_range=(0.0, 1.0))),
        ('neural network', Classifier(layers=[Layer("Softmax")], n_iter=25))])
pipeline.fit(X_train, y_train)

Check the Churn prediction article for the advanced usage of a pipeline. I show how to process categorical and numerical features separately:

GPU support

Unlike scikit-learn, scikit-neuralnetwork has support for GPUs as it is based on the Lasagne library. Note, GPU support requires an NVIDIA GPU with CUDA support. If you have a MacBook you most probably have a Radeon GPU, which doesn’t support CUDA.

From Lasagne documentation:

Thanks to Theano, Lasagne transparently supports training your networks on a GPU, which may be 10 to 50 times faster than training them on a CPU. Currently, this requires an NVIDIA GPU with CUDA support, and some additional software for Theano to use it.

To use GPU Backend you just need to import:

# Use the GPU in 32-bit mode, falling back otherwise.

from sknn.platform import gpu32

From documentation:

WARNING: This will only work if your program has not yet imported the theano module, due to the way that library is designed. If THEANO_FLAGS are set on the command-line, they are not overridden.

Conclusion

If you’re well versed with sklearn library, then scikit-neuralnetwork may be a good starting point to get familiar with Deep Learning.

scikit-neuralnetwork is also useful when we need a Neural Network that works as a drop-in replacement for a sklearn algorithm.

I would recommend learning PyTorch if you’re just starting with Deep Learning or you plan to be in the field for the long term.

Let’s connect

Talk: Book a call Socials: YouTube 🎥 | LinkedIn | Twitter Code: GitHub