Neural Network Concrete Examples

Let’s Make These Neural Networks Together

This article explains how we can manually implement a neural network which should also help us understand how neural networks work.

If you want to understand the basics of neural networks then please do read this article:

The best way to understand the concept of neural networks is to implement one and work on the practical exercises which I will demonstrate in this article.

Practical Exercises

1. I will be presenting a scenario along with its description.
2. Then we’ll configure the neural network together
3. Finally, we’ll use the TensorFlow library to implement a neural network together.

TensorFlow is an open source library. It can perform a number of functions on graphs. One of its strengths is easily letting the scientists implement neural networks. It can run on CPUs and GPUs.

Scenario: Compute OR operator neural network

Explanation:

• There are two inputs: X and Y which can produce an output Z

We are going to choose the value of bias and weights of X and Y such that:

• When either X or Y is 1 then the value of Z is 1, otherwise it is 0.

We could come up with following neural network configuration:

1. One input layer with two neurons. We are going to pass in the value of X to the first neuron and the value of Y to the second neuron in the input layer.
2. One output layer with one neuron. It will return the value for Z
3. The activation function for the output layer will be Sigmoid.
4. As a result:

• Z = 0 if the output layer produces a value lower than 0.5
• Z = 1 if the output layer produces a value greater or equal to 0.5

After trial and error, this is the configuration I have come up with:

• Bias = -1
• Weight of X = 2
• Weight of Y = 2

Let’s Check:

As an instance, when X = 1 and Y = 0 Then the neuron will compute:

Intermediate Z = 1*2 + 0*2 + (-1) = 2–1 = 1

Z = Sigmoid(1) = 1

It computes OR function with 100% accuracy. This neural network can now simulate an OR gate. Here are the results:

It’s important to note that this logic can now be used to predict and answer business questions.

Business OR scenario

As an instance, if we were working in a credit risk department of a bank then we can use a neural network to predict the credit exposure events. For example, given the counterparty rating decreases OR the counterparty misses its payment, we are going to get an impact on the exposure. The impact can be computed using an OR neural network.

2 Scenario: Compute AND operator neural network

Explanation:

• There are two inputs: X and Y which produce an output Z

We are going to choose the value of bias and weights of X and Y such that:

• When both X and Y is 1 then the value of Z is 1, otherwise it is 0.

We could come up with following neural network configuration:

1. One input layer with two neurons. We are going to pass in the value of X to the first neuron and the value of Y to the second neuron.
2. One output layer with one neuron. It will return the value for Z
3. The activation function for the output layer will be Sigmoid.
4. As a result:

• Z = 0 if the output layer produces a value lower than 0.5
• Z = 1 if the output layer produces a value greater or equal to 0.5

After trial and error, this is the configuration I have come up with:

• Bias = -3
• Weight of X = 2
• Weight of Y = 2

Let’s Check:

As an instance, when X = 1 and Y = 0 Then the neuron will compute:

Intermediate Z = 1*2 + 0*2 + (-3) = 2–3 = -1

Z = Sigmoid(-1) = 0

It produces an AND function with 100% accuracy. This neural network can now simulate an AND gate. Note, the only difference between an OR and AND neural network is the chosen value of Bias.

Bias plays an important role in predictions.

Have a look at detailed results:

It’s important to note that this logic can now be used to predict and answer business questions.

Business AND scenario

As an instance, if we were working on a trading floor of a bank, we can use an AND neural network to predict whether a stock should be bought. For example, given the stock has good profitability AND the historical profitability is good too then we will buy the stock. This can be achieved by using an AND neural network.

A number of neural networks can be combined together to produce a complex neural network.

3 Scenario: Let’s get ourselves familiar with the practice of implementing a neural network.

Build a neural network with following configuration:

• 2 hidden layers, 32 neurons each with RELU activation function
• Output layer should have 5 neurons using Softmax activation function
• Adam Optimizer with loss function of mean square error and learning rate as 0.01
• 5 epocs and batch size of 100

import tensorflow as tf
from tensorflow import keras

x = layers.Dense(32, activation='relu')(inputs)
x = layers.Dense(32, activation='relu')(x)
predictions = layers.Dense(5, activation='softmax')(x)

model = tf.keras.Model(inputs=inputs, outputs=predictions)

learning_rate = 0.01

# Configure a model for mean-squared error regression.
model.compile(optimizer=tf.train.AdamOptimizer(learning_rate),
              loss='mse') # mean squared error) 
model.fit(data, labels, epochs=5, batch_size=100)

If you want to understand what weights and bias are then please read:

If you want to understand what neural network layers are then please read:

If you want to understand how neural network neurons work then please read:

Summary

This article presented how the configuration of a neural networks works. It will make it transparent for us to understand how neural networks work.

Hope it helps.