Day 45: 60 days of Data Science and Machine Learning Series
Recurrent Neural Network…
Welcome back peeps. In this post we are going to learn the basics of Recurrent Neural Networks through a project.
Recurrent Neural Network, created in the 1980’s, is a state of the art algorithm for dealing with sequential data by using internal memory to remember important things about the input RNN’s received to precisely predict what’s coming next. RNN’s are popularly used in language translation, natural language processing (nlp), speech recognition, captioning etc.
Some of the other best Series —
100 days : Your Data Science and Machine Learning Degree Series with projects
Complete Data Visualization and Pre-processing Series with projects
Projects Videos —
All the projects, data structures, SQL, algorithms, system design, Data Science and ML , Data Analytics, Data Engineering, , Implemented Data Science and ML projects, Implemented Data Engineering Projects, Implemented Deep Learning Projects, Implemented Machine Learning Ops Projects, Implemented Time Series Analysis and Forecasting Projects, Implemented Applied Machine Learning Projects, Implemented Tensorflow and Keras Projects, Implemented PyTorch Projects, Implemented Scikit Learn Projects, Implemented Big Data Projects, Implemented Cloud Machine Learning Projects, Implemented Neural Networks Projects, Implemented OpenCV Projects,Complete ML Research Papers Summarized, Implemented Data Analytics projects, Implemented Data Visualization Projects, Implemented Data Mining Projects, Implemented Natural Leaning Processing Projects, MLOps and Deep Learning, Applied Machine Learning with Projects Series, PyTorch with Projects Series, Tensorflow and Keras with Projects Series, Scikit Learn Series with Projects, Time Series Analysis and Forecasting with Projects Series, ML System Design Case Studies Series videos will be published on our youtube channel ( just launched).
Subscribe today!
Tech Newsletter —
If you are interested, you can join my newsletter through which I send tech interview tips, techniques, patterns, hacks — Software Development, ML, Data Science, Startups and Technology projects to more than 30K readers. You can subscribe to Tech Brew :
In this project we are going to create, train, and evaluate a recurrent neural network (RNN) in Keras.
Let’s dive in!
Import all the necessary libraries
import numpy as npfrom tensorflow.keras.models import Sequential
from tensorflow.keras.layers import TimeDistributed, Dense, Dropout, SimpleRNN, RepeatVector
from tensorflow.keras.callbacks import EarlyStopping, LambdaCallbackfrom termcolor import coloredGenerate Data
all_chars='0123456789+'
num_features = len(all_chars)
print('no of features:', num_features)
char_to_index= dict((c,i) for i,c in enumerate(all_chars))
index_to_char= dict((i,c) for i, c in enumerate(all_chars))Output —
no of features: 11def generate_data():
first = np.random.randint(0,100)
second = np.random.randint(0,100)
example = str(first)+ '+' + str(second)
label = str(first+second)
return example, labelgenerate_date()Output —
('52+3', '55')Create the Model
hidden_units=128
max_time_steps=5model = Sequential([
SimpleRNN(hidden_units,input_shape=(None,num_features)),
RepeatVector(max_time_steps),
SimpleRNN(hidden_units,return_sequences=True),
TimeDistributed(Dense(num_features,activation='softmax'))
]
)model.compile(
loss='categorical_crossentropy',
optimizer = 'adam',
metrics=['accuracy']
)
model.summary()Output —
Layer (type) Output Shape Param #
=================================================================
simple_rnn_4 (SimpleRNN) (None, 128) 17920
_________________________________________________________________
repeat_vector_2 (RepeatVecto (None, 5, 128) 0
_________________________________________________________________
simple_rnn_5 (SimpleRNN) (None, 5, 128) 32896
_________________________________________________________________
time_distributed_2 (TimeDist (None, 5, 11) 1419
=================================================================
Total params: 52,235
Trainable params: 52,235
Non-trainable params: 0
_________________________________________________________________Vectorize and Devectorize data
def vectorize_example(example,label):
x=np.zeros((max_time_steps,num_features))
y=np.zeros((max_time_steps,num_features))
diff_x = max_time_steps - len(example)
diff_y = max_time_steps - len(label)
for i,c in enumerate(example):
x[i+diff_x,char_to_index[c]] =1
for i in range(diff_x):
x[i,char_to_index['0']] = 1
for i,c in enumerate(label):
y[i+diff_y,char_to_index[c]] =1
for i in range(diff_y):
y[i,char_to_index['0']] = 1
return x,ye, l = generate_data()
print(e,l)
x,y= vectorize_example(e,l)
print(x.shape,y.shape)Output —
26+32 58
(5, 11) (5, 11)def devectorize_example(example):
result = [index_to_char[np.argmax(vec)] for i,vec in enumerate(example)]
return ''.join(result)
devectorize_example(x)Output —
'26+32'devectorize_example(y)Output —
'00058'Create Dataset
def create_dataset(num_examples=2000):
x=np.zeros((num_examples,max_time_steps,num_features))
y=np.zeros((num_examples,max_time_steps,num_features))
for i in range(num_examples):
e,l = generate_data()
e_v, l_v = vectorize_example(e,l)
x[i] = e_v
y[i] = l_v
return x,y
x,y = create_dataset()
print(x.shape,y.shape)Output —
(2000, 5, 11) (2000, 5, 11)devectorize_example(x[0])
devectorize_example(y[0])Output —
'38+68'
'00106'Training the Model
l_cb=LambdaCallback(
on_epoch_end = lambda e,l: print('{:.2f}'.format(l['val_acc']),end=' _ ')
)
es_cb=EarlyStopping(monitor='val_loss',patience=10)
model.fit(x,y,epochs=500,batch_size=256,validation_split=0.2,
verbose=False,callbacks=[es_cb,l_cb])Output —
0.58 _ 0.58 _ 0.61 _ 0.61 _ 0.62 _ 0.62 _ 0.63 _ 0.63 _ 0.63 _ 0.64 _ 0.64 _ 0.66 _ 0.64 _ 0.65 _ 0.66 _ 0.65 _ 0.67 _ 0.68 _ 0.67 _ 0.69 _ 0.68 _ 0.69 _ 0.70 _ 0.70 _ 0.71 _ 0.71 _ 0.72 _ 0.72 _ 0.71 _ 0.73 _ 0.73 _ 0.71 _ 0.74 _ 0.75 _ 0.73 _ 0.75 _ 0.75 _ 0.75 _ 0.76 _ 0.76 _ 0.76 _ 0.75 _ 0.77 _ 0.77 _ 0.77 _ 0.77 _ 0.78 _ 0.77 _ 0.78 _ 0.78 _ 0.78 _ 0.79 _ 0.79 _ 0.80 _ 0.80 _ 0.79 _ 0.82 _ 0.82 _ 0.83 _ 0.82 _ 0.82 _ 0.84 _ 0.84 _ 0.84 _ 0.85 _ 0.85 _ 0.85 _ 0.86 _ 0.85 _ 0.86 _ 0.86 _ 0.87 _ 0.88 _ 0.88 _ 0.88 _ 0.88 _ 0.89 _ 0.89 _ 0.90 _ 0.90 _ 0.90 _ 0.90 _ 0.89 _ 0.90 _ 0.90 _ 0.90 _ 0.90 _ 0.92 _ 0.91 _ 0.91 _ 0.92 _ 0.92 _ 0.92 _ 0.91 _ 0.92 _ 0.92 _ 0.92 _ 0.92 _ 0.92 _ 0.92 _ 0.93 _ 0.92 _ 0.93 _ 0.92 _ 0.93 _ 0.94 _ 0.93 _ 0.93 _ 0.93 _ 0.93 _ 0.93 _ 0.94 _ 0.94 _ 0.94 _ 0.94 _ 0.94 _ 0.94 _ 0.94 _ 0.95 _ 0.94 _ 0.95 _ 0.95 _ 0.94 _ 0.94 _ 0.94 _ 0.95 _ 0.94 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _ 0.95 _x_test,y_test = create_dataset(10)
preds = model.predict(x_test)for i,pred in enumerate(preds):
y=devectorize_example(y_test[i])
y_hat = devectorize_example(pred)
col='green'
if y!= y_hat:
col='red'
out='Input: '+ devectorize_example(x_test[i])+' Out: ' +y+'Pred:' +y_hat
print(colored(out,col))Output —
Input: 82+54 Out: 00136 Pred:00136
Input: 60+81 Out: 00141 Pred:00141
Input: 15+99 Out: 00114 Pred:00114
Input: 00+10 Out: 00010 Pred:00012
Input: 090+1 Out: 00091 Pred:00090
Input: 20+24 Out: 00044 Pred:00044
Input: 55+29 Out: 00084 Pred:00084
Input: 36+47 Out: 00083 Pred:00083
Input: 10+12 Out: 00022 Pred:00022
Input: 71+56 Out: 00127 Pred:00127Cheat sheet for RNN : https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks
Learnings —
How to create, train, and evaluate a recurrent neural network (RNN) in Keras.
Day 46: Coming soon!
Follow and Stay tuned. Keep coding :)
For other projects, tune to —
Build Machine Learning Pipelines( With Code)
Recurrent Neural Network with Keras
Clustering Geolocation Data in Python using DBSCAN and K-Means
Facial Expression Recognition using Keras
Hyperparameter Tuning with Keras Tuner
Custom Layers in Keras
That’s it fellas. Peace out and keep coding :)
Stay Tuned and of-course let me end this post with a quote by Steve Jobs ;)
“Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work. And the only way to do great work is to love what you do. If you haven’t found it yet, keep looking. Don’t settle. As with all matters of the heart, you’ll know when you find it.”
