Building a Basic Perceptron Model with Python | Towards AI

A Basic Perceptron Model Using Least Squares Method

Just like the billions of neurons that make up the human nervous system, the perceptron is the basic unit of artificial intelligence. Every thought, action, emotion or decision that we make reflect the activities of the nervous system which is a master system that controls and communicates with every part of your body. Biological intelligence relies on this complex mechanism of billions of neurons organized in different layers that communicate with one another through electrical and chemical signals.

To understand how biological intelligence is produced, it's important to understand how the basic building block called neuron functions.

The biological neuron has 3 main functions:

1. Sensory input. The neuron uses its dendrites (receptive regions) to monitor changes occurring both inside and outside the body. The information gathered is called sensory input.
2. Integration: The cell body system processes and interprets sensory input and decides what should be done at each moment, a process called integration. If sensory input is below a certain threshold, the sensory signal is not activated.
3. Motor output. If the sensory input signal is above a certain threshold, the neuron produces an output that is transmitted via synaptic gaps by neurotransmitters. The neurotransmitters will either excite or inhibit a nearby neuron.

Similar to biological intelligence, artificial intelligence is produced by a complex network of basic building blocks called perceptron. The perceptron functions using the same principle as a neuron:

1. Input
2. Integration
3. Output

We shall focus here on how to build a basic perceptron model using python. This knowledge is fundamental for understanding more advanced models such as neural networks, which are complex systems of thousands and billions of perceptrons, with the capability of producing artificially intelligent systems such as self-driving cars.

Basic Perceptron Model

Python’s sklearn package contains several classifiers such as the Perceptron, SupportVectorClassifier, LogisticRegressionClassifer, DecisionTreeClassifier, RandomForestClassifier, and KNN classifier. While it is important to use these ready-made machine learning algorithms, every beginner in the field must master the basics of how these algorithms work. A good place to start your journey into neural networks and deep learning models is by considering the perceptron.

In this example, we build a simple perceptron model in which the learning weights are calculated using the least-squares method.

The perceptron model has the following four main steps:

1. Training
2. Activation
3. Quantization
4. Prediction

X represents the attribute or predictor matrix, and y represents the class. We shall illustrate our model using the Iris dataset. The dataset contains the following attribute information:

1. sepal length in cm
2. sepal width in cm
3. petal length in cm
4. petal width in cm

The three classes are

1. Iris Setosa
2. Iris Versicolor
3. Iris Virginica

For simplicity, we perform binary classification. We use the two flower classes Setosa and Versicolor for practical reasons. However, the perceptron algorithm can be extended to multi-class classification — for example, through the One-vs.-All technique.

Model Implementation Using Python

This code applies the perceptron classification algorithm to the iris dataset. The weights used for computing the activation function are calculated using the least-squares method. This method is different from Rosenblatt’s original perceptron rule where the weights are calculated recursively. For more information about the implementation of Rosenblatt’s perceptron algorithm, see the following book: “Python Machine Learning” by Sebastian Raschka.

Import Necessary Libraries

import pandas as pd

import numpy as np

import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split

Define Perceptron Classifier Object

class Perceptron(object):
    """Perceptron classifier using least-square method to calculate weights.
      
    Attributes
    -----------
    w : 1d-array
        Weights after fitting.
    """   
    def fit(self, X, y):
        """Fit training data.
        
        Parameters
        ----------
        X : {array-like}, shape = [n_samples, n_features]
        Training vectors, where n_samples is the number of samples and n_features is the number of features.
        y : array-like, shape = [n_samples]
        Target values.
        Returns
        -------
        self : object
        """
        self.w = np.dot(np.linalg.inv(np.dot(X.T,X)),np.dot(X.T,y))
        return self
    
    def predict(self, X):
        """Return class label after unit step"""
        return np.where(np.dot(X,self.w) >= 0.0, 1, -1)

Import Iris Dataset

df = pd.read_csv('iris.data.csv', header=None)print(df.tail())
y = df.iloc[0:100, 4].values
y = np.where(y == 'Iris-setosa', -1, 1)
X = df.iloc[0:100, 0:4].values

plt.scatter(X[:50, 0], X[:50, 2],color='red', marker='o', label='setosa')

plt.scatter(X[50:100, 0], X[50:100, 2],color='blue', marker='x', label='versicolor')

plt.xlabel('sepal length (cm)')
plt.ylabel('petal length (cm)')
plt.legend(loc='upper left')
plt.show()

Training, Testing, and Evaluation

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=21, stratify=y)
ppn = Perceptron()
ppn.fit(X_train,y_train)
y_pred=ppn.predict(X_test)
accuracy = 100*np.sum(y_pred==y_test)/len(y_test)
print("accuracy of the model:= " + str(accuracy))

In summary, we have demonstrated how a basic perceptron model can be built in python using the least-squares method for calculating weights that are then used for calculating the activation function. The perceptron model is the basic building block for more advanced neural network systems. Every beginner in the field of deep learning and artificial intelligence should master the basics of the perceptron model.

The code and dataset for this article can be downloaded from this Github repository: https://github.com/bot13956/perceptron_classifier.