Free AI web copilot to create summaries, insights and extended knowledge, download it at here

14142

Abstract

<div> <h2>Machine learning education | TensorFlow</h2> <div><h3>When beginning your educational path, it’s important to first understand how to learn ML. We’ve broken the learning…</h3></div> <div><p>www.tensorflow.org</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*te8THhULDJ2pu04t)"></div> </div> </div> </a> </div><p id="d43b">In this project we are going to learn how to use Tensorflow to perform natural language processing tasks with a project. The data for this project can be found here —</p><div id="a394" class="link-block"> <a href="https://www.kaggle.com/c/tweet-emotion-detection/data"> <div> <div> <h2>Tweet Emotion Detection</h2> <div><h3>Your task is to predict which one of four emotions is expressed by a tweet.</h3></div> <div><p>www.kaggle.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*xa85bgTi4OAYOKn-)"></div> </div> </div> </a> </div><h2 id="3cf7">Import necessary libraries</h2><div id="bd1d"><pre><span class="hljs-keyword">import</span> tensorflow <span class="hljs-keyword">as</span> tf <span class="hljs-keyword">import</span> numpy <span class="hljs-keyword">as</span> np <span class="hljs-keyword">import</span> matplotlib.pyplot <span class="hljs-keyword">as</span> plt %matplotlib inline</pre></div><div id="42f1"><pre><span class="hljs-keyword">import</span> nlp <span class="hljs-keyword">import</span> random <span class="hljs-title">from</span> tensorflow.keras.preprocessing.text <span class="hljs-keyword">import</span> Tokenizer <span class="hljs-title">from</span> tensorflow.keras.preprocessing.sequence <span class="hljs-keyword">import</span> pad_sequences</pre></div><div id="594f"><pre>def <span class="hljs-built_in">show_history</span>(h): epochs_trained = <span class="hljs-built_in">len</span>(h.history[<span class="hljs-string">'loss'</span>]) plt.<span class="hljs-built_in">figure</span>(figsize=(<span class="hljs-number">16</span>, <span class="hljs-number">6</span>))</pre></div><div id="dc60"><pre>plt<span class="hljs-selector-class">.subplot</span>(<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">1</span>) plt<span class="hljs-selector-class">.plot</span>(<span class="hljs-built_in">range</span>(<span class="hljs-number">0</span>, epochs_trained), h<span class="hljs-selector-class">.history</span><span class="hljs-selector-class">.get</span>(<span class="hljs-string">'accuracy'</span>), label=<span class="hljs-string">'Training'</span>) plt<span class="hljs-selector-class">.plot</span>(<span class="hljs-built_in">range</span>(<span class="hljs-number">0</span>, epochs_trained), h<span class="hljs-selector-class">.history</span><span class="hljs-selector-class">.get</span>(<span class="hljs-string">'val_accuracy'</span>), label=<span class="hljs-string">'Validation'</span>) plt<span class="hljs-selector-class">.ylim</span>(<span class="hljs-selector-attr">[0., 1.]</span>) plt<span class="hljs-selector-class">.xlabel</span>(<span class="hljs-string">'Epochs'</span>) plt<span class="hljs-selector-class">.ylabel</span>(<span class="hljs-string">'Accuracy'</span>) plt<span class="hljs-selector-class">.legend</span>()</pre></div><div id="5271"><pre>plt<span class="hljs-selector-class">.subplot</span>(<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">2</span>) plt<span class="hljs-selector-class">.plot</span>(<span class="hljs-built_in">range</span>(<span class="hljs-number">0</span>, epochs_trained), h<span class="hljs-selector-class">.history</span><span class="hljs-selector-class">.get</span>(<span class="hljs-string">'loss'</span>), label=<span class="hljs-string">'Training'</span>) plt<span class="hljs-selector-class">.plot</span>(<span class="hljs-built_in">range</span>(<span class="hljs-number">0</span>, epochs_trained), h<span class="hljs-selector-class">.history</span><span class="hljs-selector-class">.get</span>(<span class="hljs-string">'val_loss'</span>), label=<span class="hljs-string">'Validation'</span>) plt<span class="hljs-selector-class">.xlabel</span>(<span class="hljs-string">'Epochs'</span>) plt<span class="hljs-selector-class">.ylabel</span>(<span class="hljs-string">'Loss'</span>) plt<span class="hljs-selector-class">.legend</span>() plt<span class="hljs-selector-class">.show</span>()</pre></div><div id="c6f5"><pre><span class="hljs-keyword">def</span> <span class="hljs-title function_">show_confusion_matrix</span>(<span class="hljs-params">y_true, y_pred, classes</span>): <span class="hljs-keyword">from</span> sklearn.metrics <span class="hljs-keyword">import</span> confusion_matrix

cm = confusion_matrix(y_true, y_pred, normalize=<span class="hljs-string">'true'</span>)</pre></div><div id="9822"><pre>plt<span class="hljs-selector-class">.figure</span>(figsize=(<span class="hljs-number">8</span>, <span class="hljs-number">8</span>))
sp = plt<span class="hljs-selector-class">.subplot</span>(<span class="hljs-number">1</span>, <span class="hljs-number">1</span>, <span class="hljs-number">1</span>)
ctx = sp<span class="hljs-selector-class">.matshow</span>(cm)
plt<span class="hljs-selector-class">.xticks</span>(<span class="hljs-built_in">list</span>(<span class="hljs-built_in">range</span>(<span class="hljs-number">0</span>, <span class="hljs-number">6</span>)), labels=classes)
plt<span class="hljs-selector-class">.yticks</span>(<span class="hljs-built_in">list</span>(<span class="hljs-built_in">range</span>(<span class="hljs-number">0</span>, <span class="hljs-number">6</span>)), labels=classes)
plt<span class="hljs-selector-class">.colorbar</span>(ctx)
plt<span class="hljs-selector-class">.show</span>()</pre></div><h2 id="e3d6">Importing Data</h2><div id="ae19"><pre>dataset = nlp.load_dataset(<span class="hljs-string">'emotion'</span>)

train = dataset[<span class="hljs-string">'train'</span>] val = dataset[<span class="hljs-string">'validation'</span>] test = dataset[<span class="hljs-string">'test'</span>] def gt(data): tweets = [x[<span class="hljs-string">'text'</span>] for x in data] labels = [x[<span class="hljs-string">'label'</span>] for x in data] return tweets, labels tweets, labels = gt(train) tweets[<span class="hljs-number">2</span>],labels[<span class="hljs-number">2</span>]</pre></div><p id="0c43">Output —</p><div id="e1a9"><pre>(<span class="hljs-symbol">'im</span> grabbing a minute to post i feel greedy wrong', <span class="hljs-symbol">'anger</span>')</pre></div><h2 id="308b">Tokenizer</h2><div id="f58d"><pre>tokenizer = <span class="hljs-built_in">Tokenizer</span>(num_words=<span class="hljs-number">10000</span>, oov_token =<span class="hljs-string">'<UNK>'</span>) tokenizer<span class="hljs-selector-class">.fit_on_texts</span>(tweets) tokenizer<span class="hljs-selector-class">.texts_to_sequences</span>(<span class="hljs-selector-attr">[tweets[1]</span>])</pre></div><div id="a63b"><pre><span class="hljs-attribute">tweets</span>[<span class="hljs-number">2</span>]</pre></div><p id="6ed6">Output —</p><div id="6ec8"><pre><span class="hljs-comment">'im grabbing a minute to post i feel greedy wrong'</span></pre></div><h2 id="902c">Padding and Truncating Sequences</h2><div id="29bf"><pre>lengths = <span class="hljs-selector-attr">[len(t.split(<span class="hljs-string">' '</span>)) for t in tweets]</span> plt<span class="hljs-selector-class">.hist</span>(lengths,bins = <span class="hljs-built_in">len</span>(<span class="hljs-built_in">set</span>(lengths))) plt<span class="hljs-selector-class">.show</span>()</pre></div><p id="3337">Output —</p><figure id="b90e"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*ArPxLTn_OMZ7fjiAtUd04g.png"><figcaption></figcaption></figure><div id="2dd5"><pre>maxlen = <span class="hljs-number">50</span> def gs(tokenizer,tweets): seq = tokenizer.texts_to_sequences(tweets) padded = pad_sequences(seq,truncating=<span class="hljs-string">'post'</span>,padding = <span class="hljs-string">'post'</span>,maxlen=maxlen) <span class="hljs-keyword">return</span> padded padded_train_seq = gs(tokenizer,tweets) padded_train_se<span class="hljs-string">q[0]</span></pre></div><p id="9b07">Output —</p><div id="c9ac"><pre><span class="hljs-attribute">array</span>([ <span class="hljs-number">2</span>, <span class="hljs-number">139</span>, <span class="hljs-number">3</span>, <span class="hljs-number">679</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-attribute">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-attribute">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>, <span class="hljs-number">0</span>], dtype=int32)</pre></div><h2 id="3e9b">Preparing Labels</h2><div id="bc89"><pre>classes = <span class="hljs-built_in">set</span>(labels) plt.hist(labels, <span class="hljs-attribute">bins</span>=11) plt.show()</pre></div><p id="17a6">Output —</p><figure id="166e"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*5xShGHYc4BmQJUiMFhTtpA.png"><figcaption></figcaption></figure><div id="731d"><pre>class_to_index = <span class="hljs-built_in">dict</span>((c,i) <span class="hljs-keyword">for</span> <span class="hljs-selector-tag">i</span>,c <span class="hljs-keyword">in</span> <span class="hljs-built_in">enumerate</span>(classes)) index_to_class = <span class="hljs-built_in">dict</span>((v,k) <span class="hljs-keyword">for</span> k,v <span class="hljs-keyword">in</span> class_to_index<span class="hljs-selector-class">.items</span>()) <span class="hljs-function"><span class="hljs-title">print</span><span class="hljs-params">(class_to_index)</span></span> <span class="hljs-function"><span class="hljs-title">print</span><span class="hljs-params">(index_to_class)</span></span> names_to_ids = lambda labels: np<span class="hljs-selector-class">.array</span>(<span class="hljs-selector-attr">[class_to_index.get(x) for x in labels]</span>) train_labels = <span class="hljs-built_in">names_to_ids</span>(labels)</pre></div><p id="7e01">Output —</p><div id="4a80"><pre>{'joy': <span class="hljs-number">0</span>, 'fear': <span class="hljs-number">1</span>, 'love': <span class="hljs-number">2</span>, 'surprise': <span class="hljs-number">3</span>, 'anger': <span class="hljs-number">4</span>, 'sadness': <span class="hljs-number">5</span>} {<span class="hljs-number">0</span>: 'joy', <span class="hljs-number">1</span>: 'fear', <span class="hljs-number">2</span>: 'love', <span class="hljs-number">3</span>: 'surprise', <span class="hljs-number">4</span>: 'anger', <span class="hljs-number">5</span>: 'sadness'}</pre></div><h2 id="e7db">Creating and Training RNN Model</h2><div id="cbb1"><pre>model = <span class="hljs-keyword">tf</span>.keras.models.Sequential([ <span class="hljs-keyword">tf</span>.keras.layers.Embedding(<span class="hljs-number">10000</span>,<span class="hljs-number">16</span>,input_length= maxlen), <span class="hljs-keyword">tf</span>.keras.layers.Bidirectional(<span class="hljs-keyword">tf</span>.keras.layers.LSTM(<span class="hljs-number">20</span>,return_sequences=True)), <span class="hljs-keyword">tf</span>.keras.layers.Bidirectional(<span class="hljs-keyword">tf</span>.keras.layers.LSTM(<span class="hljs-number">20</span>)), <span class="hljs-keyword">tf</span>.keras.layers.Dense(<span class="hljs-number">6</span>,activation=<span class="hljs-string">'softmax'</span>) ])</pre></div><div id="338a"><pre>model<span class="hljs-selector-class">.compile</span>(

   loss = <span class="hljs-string">'sparse_categorical_crossentropy'</span>,
   optimizer = <span class="hljs-string">'adam'</span>,
   metrics = <span class="hljs-selector-attr">[<span class="hljs-string">'accuracy'</span>]</span></pre></div><div id="e216"><pre>)</pre></div><div id="dd7e"><pre><span class="hljs-keyword">model</span>.summary()</pre></div><p id="44c9">Output —</p><div id="e02c"><pre>Model: "sequential"

<span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong">____</

Options

span><span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong"></span><span class="hljs-strong">___</span> <span class="hljs-section"> Layer (type) Output Shape Param #
=================================================================</span> embedding (Embedding) (None, 50, 16) 160000
<span class="hljs-code">
bidirectional (Bidirectiona (None, 50, 40) 5920
l)

bidirectional_1 (Bidirectio (None, 40) 9760
nal)

dense (Dense) (None, 6) 246

================================================================= Total params: 175,926 Trainable params: 175,926 Non-trainable params: 0 _________________________________________________________________</span></pre></div><p id="0a06">Training —</p><div id="c7ae"><pre>vt, vl = gt(val) <span class="hljs-keyword">vs</span> = gs(tokenizer,vt) vl = names_to_ids(vl) h = model.fit(

   padded_train_seq, train_labels,
   validation_data = (<span class="hljs-keyword">vs</span>,vl),
   epochs = <span class="hljs-number">20</span>,
   callbacks = [
                <span class="hljs-keyword">tf</span>.keras.callbacks.EarlyStopping(monitor =<span class="hljs-string">'val_accuracy'</span>,patience = <span class="hljs-number">2</span>)
   ]</pre></div><div id="6ed2"><pre>)</pre></div><p id="40a2">Output —</p><div id="abf9"><pre>Epoch <span class="hljs-number">1</span>/<span class="hljs-number">20</span>

<span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 50s 78ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">1.3148</span> - accuracy: <span class="hljs-number">0.4819</span> - val_loss: <span class="hljs-number">0.8473</span> - val_accuracy: <span class="hljs-number">0.6855</span> Epoch <span class="hljs-number">2</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 76ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.6071</span> - accuracy: <span class="hljs-number">0.7489</span> - val_loss: <span class="hljs-number">0.5698</span> - val_accuracy: <span class="hljs-number">0.7660</span> Epoch <span class="hljs-number">3</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 76ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.3830</span> - accuracy: <span class="hljs-number">0.8587</span> - val_loss: <span class="hljs-number">0.4630</span> - val_accuracy: <span class="hljs-number">0.8545</span> Epoch <span class="hljs-number">4</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 76ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.2383</span> - accuracy: <span class="hljs-number">0.9237</span> - val_loss: <span class="hljs-number">0.3943</span> - val_accuracy: <span class="hljs-number">0.8735</span> Epoch <span class="hljs-number">5</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 75ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.1643</span> - accuracy: <span class="hljs-number">0.9469</span> - val_loss: <span class="hljs-number">0.4174</span> - val_accuracy: <span class="hljs-number">0.8750</span> Epoch <span class="hljs-number">6</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 75ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.1266</span> - accuracy: <span class="hljs-number">0.9587</span> - val_loss: <span class="hljs-number">0.3923</span> - val_accuracy: <span class="hljs-number">0.8760</span> Epoch <span class="hljs-number">7</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 37s 75ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.0960</span> - accuracy: <span class="hljs-number">0.9695</span> - val_loss: <span class="hljs-number">0.4180</span> - val_accuracy: <span class="hljs-number">0.8800</span> Epoch <span class="hljs-number">8</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 75ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.0866</span> - accuracy: <span class="hljs-number">0.9726</span> - val_loss: <span class="hljs-number">0.3984</span> - val_accuracy: <span class="hljs-number">0.8815</span> Epoch <span class="hljs-number">9</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 75ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.0906</span> - accuracy: <span class="hljs-number">0.9725</span> - val_loss: <span class="hljs-number">0.4054</span> - val_accuracy: <span class="hljs-number">0.8855</span> Epoch <span class="hljs-number">10</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 76ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.0652</span> - accuracy: <span class="hljs-number">0.9794</span> - val_loss: <span class="hljs-number">0.4151</span> - val_accuracy: <span class="hljs-number">0.8850</span> Epoch <span class="hljs-number">11</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 75ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.0797</span> - accuracy: <span class="hljs-number">0.9744</span> - val_loss: <span class="hljs-number">0.3693</span> - val_accuracy: <span class="hljs-number">0.8945</span> Epoch <span class="hljs-number">12</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 75ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.0586</span> - accuracy: <span class="hljs-number">0.9822</span> - val_loss: <span class="hljs-number">0.3849</span> - val_accuracy: <span class="hljs-number">0.8965</span> Epoch <span class="hljs-number">13</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 75ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.0402</span> - accuracy: <span class="hljs-number">0.9869</span> - val_loss: <span class="hljs-number">0.3834</span> - val_accuracy: <span class="hljs-number">0.9005</span> Epoch <span class="hljs-number">14</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 75ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.0355</span> - accuracy: <span class="hljs-number">0.9877</span> - val_loss: <span class="hljs-number">0.4057</span> - val_accuracy: <span class="hljs-number">0.8965</span> Epoch <span class="hljs-number">15</span>/<span class="hljs-number">20</span> <span class="hljs-number">500</span><span class="hljs-regexp">/500 [==============================] - 38s 75ms/</span><span class="hljs-keyword">step</span> - loss: <span class="hljs-number">0.0345</span> - accuracy: <span class="hljs-number">0.9882</span> - val_loss: <span class="hljs-number">0.4259</span> - val_accuracy: <span class="hljs-number">0.8935</span></pre></div><h2 id="fc0b">Model Evaluation</h2><div id="a1a8"><pre><span class="hljs-function"><span class="hljs-title">show_history</span><span class="hljs-params">(h)</span></span></pre></div><p id="2bdb">Output —</p><figure id="505b"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*zj8zypzEdXxYvlel8Uz0cw.png"><figcaption></figcaption></figure><p id="c4c2"><b><i>Learnings —</i></b></p><p id="a436">How to use NLP, tokenizer, deal with Sequences in TensorFlow and create and train a Recurrent Neural Network.</p><p id="90be"><b><i>Day 43: Coming soon!</i></b></p><p id="a039">Follow and Stay tuned. Keep coding :)</p><h1 id="a69d">For other projects, tune to —</h1><p id="b31f"><b>Build Machine Learning Pipelines( With Code)</b></p><div id="5b37" class="link-block"> <a href="https://medium.datadriveninvestor.com/build-machine-learning-pipelines-with-code-part-1-bd3ed7152124"> <div> <div> <h2>Build Machine Learning Pipelines( With Code) — Part 1</h2> <div><h3>Complete implementation…</h3></div> <div><p>medium.datadriveninvestor.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*KdToBD8RDMBH4jXM.png)"></div> </div> </div> </a> </div><p id="946c"><b>Recurrent Neural Network with Keras</b></p><div id="607d" class="link-block"> <a href="https://medium.datadriveninvestor.com/recurrent-neural-network-with-keras-b5b5f6fe5187"> <div> <div> <h2>Recurrent Neural Network with Keras</h2> <div><h3>Project Implementation and cheatsheet…</h3></div> <div><p>medium.datadriveninvestor.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*xs3Dya3qQBx6IU7C.png)"></div> </div> </div> </a> </div><p id="56e1"><b>Clustering Geolocation Data in Python using DBSCAN and K-Means</b></p><div id="2b3e" class="link-block"> <a href="https://medium.datadriveninvestor.com/clustering-geolocation-data-in-python-using-dbscan-and-k-means-3705d9f44522"> <div> <div> <h2>Clustering Geolocation Data in Python using DBSCAN and K-Means</h2> <div><h3>Project Implementation…</h3></div> <div><p>medium.datadriveninvestor.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*0uPCZnohdaPCO4NN.png)"></div> </div> </div> </a> </div><p id="a29c"><b>Facial Expression Recognition using Keras</b></p><div id="ccaa" class="link-block"> <a href="https://medium.datadriveninvestor.com/facial-expression-recognition-using-keras-cbdd661a0a54"> <div> <div> <h2>Facial Expression Recognition using Keras</h2> <div><h3>Project Implementation…</h3></div> <div><p>medium.datadriveninvestor.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*CGch7hzdjg1fpgKy.jpg)"></div> </div> </div> </a> </div><p id="0db7"><b>Hyperparameter Tuning with Keras Tuner</b></p><div id="6dff" class="link-block"> <a href="https://medium.datadriveninvestor.com/hyperparameter-tuning-with-keras-tuner-3a609d3fd85b"> <div> <div> <h2>Hyperparameter Tuning with Keras Tuner</h2> <div><h3>Project Implementation….</h3></div> <div><p>medium.datadriveninvestor.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*jlaEz8AZaptNWHEr.png)"></div> </div> </div> </a> </div><p id="fed8"><b>Custom Layers in Keras</b></p><div id="e4fd" class="link-block"> <a href="https://medium.datadriveninvestor.com/custom-layers-in-keras-de5f793217aa"> <div> <div> <h2>Custom Layers in Keras</h2> <div><h3>Code implementation …</h3></div> <div><p>medium.datadriveninvestor.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*1IH67KJadqeqeO01.png)"></div> </div> </div> </a> </div><p id="2ea9"><b><i>That’s it fellas. Peace out and keep coding :)</i></b></p><p id="ec55">Stay Tuned and of-course let me end this post with a quote by Steve Jobs ;)</p><p id="5004" type="7">“Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work. And the only way to do great work is to love what you do. If you haven’t found it yet, keep looking. Don’t settle. As with all matters of the heart, you’ll know when you find it.”</p></article></body>

Day 42: 60 days of Data Science and Machine Learning Series

RNN and Tensorflow with a project ..

How RNN works ( Pic credits : Research Gate)

Welcome back peeps. In this post we are going to understand the basics of Tensorflow with a project.

Recurrent Neural Network, created in the 1980’s, is a state of the art algorithm for dealing with sequential data by using internal memory to remember important things about the input RNN’s received to precisely predict what’s coming next. RNN’s are popularly used in language translation, natural language processing (nlp), speech recognition, captioning etc.

RNN vs Feed Forward Network ( Pic credits : IBM)

Some of the other best Series —

30 Days of Natural Language Processing ( NLP) Series

30 days of Data Engineering with projects Series

60 days of Data Science and ML Series with projects

100 days : Your Data Science and Machine Learning Degree Series with projects

23 Data Science Techniques You Should Know

Tech Interview Series — Curated List of coding questions

Complete System Design with most popular Questions Series

Complete Data Visualization and Pre-processing Series with projects

Complete Python Series with Projects

Complete Advanced Python Series with Projects

Kaggle Best Notebooks that will teach you the most

Complete Developers Guide to Git

All the Data Science and Machine Learning Resources

210 Machine Learning Projects

30 days of Machine Learning Ops

Projects Videos —

Subscribe today!

Ignito

Excited to share that we have launched our Youtube channel — Ignito to cover all the projects and coding exercise for …

www.youtube.com

Tech Newsletter —

If you are interested, you can join my newsletter through which I send tech interview tips, techniques, patterns, hacks — Software Development, ML, Data Science, Startups and Technology projects to more than 30K readers. You can subscribe to Tech Brew :

Ignito

Data Science, ML, AI and more… Click to read Ignito, by Naina Chaturvedi, a Substack publication. Launched 7 months…

naina0405.substack.com

Tensorflow is an open source platform for machine learning and deep learning developed by Google Brain Team and written in C++, Python, and CUDA created for large numerical computations and deep learning. It ingests the data in the form of tensors which are nothing but multi-dimensional arrays of higher dimensions to handle large amounts of data. It works on the data flow graphs that have nodes and edges and supports both CPUs and GPUs. It works by preprocessing the data, building the model, training and estimating the model.

A good reference to Tensorflow ( used in this project as well ) —

Machine learning education | TensorFlow

When beginning your educational path, it’s important to first understand how to learn ML. We’ve broken the learning…

www.tensorflow.org

In this project we are going to learn how to use Tensorflow to perform natural language processing tasks with a project. The data for this project can be found here —

Tweet Emotion Detection

Your task is to predict which one of four emotions is expressed by a tweet.

www.kaggle.com

Import necessary libraries

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

import nlp
import random
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences

def show_history(h):
    epochs_trained = len(h.history['loss'])
    plt.figure(figsize=(16, 6))

plt.subplot(1, 2, 1)
    plt.plot(range(0, epochs_trained), h.history.get('accuracy'), label='Training')
    plt.plot(range(0, epochs_trained), h.history.get('val_accuracy'), label='Validation')
    plt.ylim([0., 1.])
    plt.xlabel('Epochs')
    plt.ylabel('Accuracy')
    plt.legend()

plt.subplot(1, 2, 2)
    plt.plot(range(0, epochs_trained), h.history.get('loss'), label='Training')
    plt.plot(range(0, epochs_trained), h.history.get('val_loss'), label='Validation')
    plt.xlabel('Epochs')
    plt.ylabel('Loss')
    plt.legend()
    plt.show()

def show_confusion_matrix(y_true, y_pred, classes):
    from sklearn.metrics import confusion_matrix
    
    cm = confusion_matrix(y_true, y_pred, normalize='true')

plt.figure(figsize=(8, 8))
    sp = plt.subplot(1, 1, 1)
    ctx = sp.matshow(cm)
    plt.xticks(list(range(0, 6)), labels=classes)
    plt.yticks(list(range(0, 6)), labels=classes)
    plt.colorbar(ctx)
    plt.show()

Importing Data

dataset = nlp.load_dataset('emotion')
train = dataset['train']
val = dataset['validation']
test = dataset['test']
def gt(data):
  tweets = [x['text'] for x in data]
  labels = [x['label'] for x in data]
  return tweets, labels
tweets, labels = gt(train)
tweets[2],labels[2]

Output —

('im grabbing a minute to post i feel greedy wrong', 'anger')

Tokenizer

tokenizer = Tokenizer(num_words=10000, oov_token ='<UNK>')
tokenizer.fit_on_texts(tweets)
tokenizer.texts_to_sequences([tweets[1]])

tweets[2]

Output —

'im grabbing a minute to post i feel greedy wrong'

Padding and Truncating Sequences

lengths = [len(t.split(' ')) for t in tweets]
plt.hist(lengths,bins = len(set(lengths)))
plt.show()

Output —

maxlen = 50
def gs(tokenizer,tweets):
  seq = tokenizer.texts_to_sequences(tweets)
  padded = pad_sequences(seq,truncating='post',padding = 'post',maxlen=maxlen)
  return padded
padded_train_seq = gs(tokenizer,tweets)
padded_train_seq[0]

Output —

array([  2, 139,   3, 679,   0,   0,   0,   0,   0,   0,   0,   0,   0, 0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0], dtype=int32)

Preparing Labels

classes = set(labels)
plt.hist(labels, bins=11)
plt.show()

Output —

class_to_index = dict((c,i) for i,c in enumerate(classes))
index_to_class = dict((v,k) for k,v in class_to_index.items())
print(class_to_index)
print(index_to_class)
names_to_ids = lambda labels: np.array([class_to_index.get(x) for x in labels])
train_labels = names_to_ids(labels)

Output —

{'joy': 0, 'fear': 1, 'love': 2, 'surprise': 3, 'anger': 4, 'sadness': 5}
{0: 'joy', 1: 'fear', 2: 'love', 3: 'surprise', 4: 'anger', 5: 'sadness'}

Creating and Training RNN Model

model = tf.keras.models.Sequential([
                                    tf.keras.layers.Embedding(10000,16,input_length= maxlen),
                                    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(20,return_sequences=True)),
                                    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(20)),
                                    tf.keras.layers.Dense(6,activation='softmax')
])

model.compile(
    
       loss = 'sparse_categorical_crossentropy',
       optimizer = 'adam',
       metrics = ['accuracy']

model.summary()

Output —

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding (Embedding)       (None, 50, 16)            160000    
                                                                 
 bidirectional (Bidirectiona  (None, 50, 40)           5920      
 l)                                                              
                                                                 
 bidirectional_1 (Bidirectio  (None, 40)               9760      
 nal)                                                            
                                                                 
 dense (Dense)               (None, 6)                 246       
                                                                 
=================================================================
Total params: 175,926
Trainable params: 175,926
Non-trainable params: 0
_________________________________________________________________

Training —

vt, vl = gt(val)
vs = gs(tokenizer,vt)
vl = names_to_ids(vl)
h = model.fit(
    
       padded_train_seq, train_labels,
       validation_data = (vs,vl),
       epochs = 20,
       callbacks = [
                    tf.keras.callbacks.EarlyStopping(monitor ='val_accuracy',patience = 2)
       ]

Output —

Epoch 1/20
500/500 [==============================] - 50s 78ms/step - loss: 1.3148 - accuracy: 0.4819 - val_loss: 0.8473 - val_accuracy: 0.6855
Epoch 2/20
500/500 [==============================] - 38s 76ms/step - loss: 0.6071 - accuracy: 0.7489 - val_loss: 0.5698 - val_accuracy: 0.7660
Epoch 3/20
500/500 [==============================] - 38s 76ms/step - loss: 0.3830 - accuracy: 0.8587 - val_loss: 0.4630 - val_accuracy: 0.8545
Epoch 4/20
500/500 [==============================] - 38s 76ms/step - loss: 0.2383 - accuracy: 0.9237 - val_loss: 0.3943 - val_accuracy: 0.8735
Epoch 5/20
500/500 [==============================] - 38s 75ms/step - loss: 0.1643 - accuracy: 0.9469 - val_loss: 0.4174 - val_accuracy: 0.8750
Epoch 6/20
500/500 [==============================] - 38s 75ms/step - loss: 0.1266 - accuracy: 0.9587 - val_loss: 0.3923 - val_accuracy: 0.8760
Epoch 7/20
500/500 [==============================] - 37s 75ms/step - loss: 0.0960 - accuracy: 0.9695 - val_loss: 0.4180 - val_accuracy: 0.8800
Epoch 8/20
500/500 [==============================] - 38s 75ms/step - loss: 0.0866 - accuracy: 0.9726 - val_loss: 0.3984 - val_accuracy: 0.8815
Epoch 9/20
500/500 [==============================] - 38s 75ms/step - loss: 0.0906 - accuracy: 0.9725 - val_loss: 0.4054 - val_accuracy: 0.8855
Epoch 10/20
500/500 [==============================] - 38s 76ms/step - loss: 0.0652 - accuracy: 0.9794 - val_loss: 0.4151 - val_accuracy: 0.8850
Epoch 11/20
500/500 [==============================] - 38s 75ms/step - loss: 0.0797 - accuracy: 0.9744 - val_loss: 0.3693 - val_accuracy: 0.8945
Epoch 12/20
500/500 [==============================] - 38s 75ms/step - loss: 0.0586 - accuracy: 0.9822 - val_loss: 0.3849 - val_accuracy: 0.8965
Epoch 13/20
500/500 [==============================] - 38s 75ms/step - loss: 0.0402 - accuracy: 0.9869 - val_loss: 0.3834 - val_accuracy: 0.9005
Epoch 14/20
500/500 [==============================] - 38s 75ms/step - loss: 0.0355 - accuracy: 0.9877 - val_loss: 0.4057 - val_accuracy: 0.8965
Epoch 15/20
500/500 [==============================] - 38s 75ms/step - loss: 0.0345 - accuracy: 0.9882 - val_loss: 0.4259 - val_accuracy: 0.8935

Model Evaluation

show_history(h)

Output —

Learnings —

How to use NLP, tokenizer, deal with Sequences in TensorFlow and create and train a Recurrent Neural Network.

Day 43: Coming soon!

Follow and Stay tuned. Keep coding :)

For other projects, tune to —

Build Machine Learning Pipelines( With Code)

Build Machine Learning Pipelines( With Code) — Part 1

Complete implementation…

medium.datadriveninvestor.com

Recurrent Neural Network with Keras

Recurrent Neural Network with Keras

Project Implementation and cheatsheet…

medium.datadriveninvestor.com

Clustering Geolocation Data in Python using DBSCAN and K-Means

Clustering Geolocation Data in Python using DBSCAN and K-Means

Project Implementation…

medium.datadriveninvestor.com

Facial Expression Recognition using Keras

Facial Expression Recognition using Keras

Project Implementation…

medium.datadriveninvestor.com

Hyperparameter Tuning with Keras Tuner

Hyperparameter Tuning with Keras Tuner

Project Implementation….

medium.datadriveninvestor.com

Custom Layers in Keras

Custom Layers in Keras

Code implementation …

medium.datadriveninvestor.com

That’s it fellas. Peace out and keep coding :)

Stay Tuned and of-course let me end this post with a quote by Steve Jobs ;)

“Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work. And the only way to do great work is to love what you do. If you haven’t found it yet, keep looking. Don’t settle. As with all matters of the heart, you’ll know when you find it.”