Free AI web copilot to create summaries, insights and extended knowledge, download it at here
2796
Abstract
quote><p id="3487">Run the following script with <code>root </code>priviledges:</p><div id="ca20"><pre>curl -fsSL https://ollama.com/install.sh | sh</pre></div><p id="5309">Once ollama is setup, open your terminal (in both <i>Windows </i>or <i>Linux</i>) and type the following command:</p><div id="9f60"><pre>ollama pull llama3</pre></div><figure id="5b90"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*_W7f8UlwxvS_rUo-0TR0rA.png"><figcaption>This pulling of the files will take some time (for me it took approx 5min)</figcaption></figure><p id="b703">Open your preffered python editor and create a folder with the name <code><i>local_llm</i></code><i>.</i></p><p id="f823">Inside this folder create a <code><b>requirements.txt</b></code><b> </b>file and paste the follwoing content.</p><div id="d102"><pre><span class="hljs-attr">ollama</span>==<span class="hljs-number">0.1</span>.<span class="hljs-number">8</span>
<span class="hljs-attr">streamlit</span>==<span class="hljs-number">1.33</span>.<span class="hljs-number">0</span></pre></div><p id="ce14">Now, let’s create a file named <code>app.py</code> and add the following code:</p><div id="5ef4"><pre><span class="hljs-keyword">import</span> streamlit <span class="hljs-keyword">as</span> st
<span class="hljs-keyword">import</span> ollama</pre></div><p id="3942">Next, we’ll create the Streamlit app. We’ll start by setting the title and initializing the message history:</p><div id="3676"><pre>st.title(<span class="hljs-string">"💬 Local LLMBot"</span>)
<span class="hljs-keyword">if</span> <span class="hljs-string">"messages"</span> <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> st.session_state:
st.session_state[<span class="hljs-string">"messages"</span>] = [{<span class="hljs-string">"role"</span>: <span class="hljs-string">"assistant"</span>, <span class="hljs-string">"content"</span>: <span class="hljs-string">"How can I help you?"</span>}]</pre></div><p id="a189">Now, let’s display the chat history by iterating through the messages:</p><div id="5df4"><pre><span class="hljs-keyword">for</span> msg <span class="hljs-keyword">in</span> st.session_state.messages:
<span class="hljs-keyword">if</span> msg[<span class="hljs-string">"role"</span>] == <span class="hljs-string">"user"</span>:
st.chat_message(msg[<span class="hljs-string">"role"</span>], avatar=<span class="hljs-string">"🧑💻"</span>).write(msg[<span class="hljs-string">"content"</span>])
<span class="hljs-keyword">else</span>:
st.chat_message(msg[<span class="hljs-string">"role"</span>], avatar=<span class="hljs-string">"🤖"</span>).write(msg[<span class="hljs-string">"content"</span>])</pre></div><p id="c986"><b><i>Creating the Response Generator</i></b></p><p id="f311"
Options
To generate responses using Ollama, we’ll create a function called <code>generate_response()</code>. This function will use the <code>ollama.chat()</code> function to generate responses in a streaming manner:</p><div id="41f7"><pre><span class="hljs-keyword">def</span> <span class="hljs-title function_">generate_response</span>():
response = ollama.chat(model=<span class="hljs-string">'llama3'</span>, stream=<span class="hljs-literal">True</span>, messages=st.session_state.messages)
<span class="hljs-keyword">for</span> partial_resp <span class="hljs-keyword">in</span> response:
token = partial_resp[<span class="hljs-string">"message"</span>][<span class="hljs-string">"content"</span>]
st.session_state[<span class="hljs-string">"full_message"</span>] += token
<span class="hljs-keyword">yield</span> token</pre></div><p id="4d2e"><b><i>Adding User Input and Generating Response</i></b><i>s</i></p><p id="2a90">Lastly, we’ll add a chat input field using Streamlit and create a function to handle the user’s input. When the user enters a message, we’ll append it to the messages list and display it in the chat window. We’ll then clear the <code>full_message</code> state variable and start streaming the response:</p><div id="d199"><pre><span class="hljs-keyword">if</span> prompt := st.chat_input():
st.session_state.messages.append({<span class="hljs-string">"role"</span>: <span class="hljs-string">"user"</span>, <span class="hljs-string">"content"</span>: prompt})
st.chat_message(<span class="hljs-string">"user"</span>, avatar=<span class="hljs-string">"🧑💻"</span>).write(prompt)
st.session_state[<span class="hljs-string">"full_message"</span>] = <span class="hljs-string">""</span>
st.chat_message(<span class="hljs-string">"assistant"</span>, avatar=<span class="hljs-string">"🤖"</span>).write_stream(generate_response)
st.session_state.messages.append({<span class="hljs-string">"role"</span>: <span class="hljs-string">"assistant"</span>, <span class="hljs-string">"content"</span>: st.session_state[<span class="hljs-string">"full_message"</span>]})</pre></div><p id="a4eb"><b><i>Running the App</i></b></p><p id="6673">To run the app, simply execute the following command in your terminal:</p><div id="4de5"><pre>streamlit run app.py</pre></div><h2 id="4e58">Conclusion</h2><p id="b825">In this article, we’ve learned how to create a local chatbot using <i>Ollama </i>and <i>Streamlit</i>. By combining these powerful tools, you can build your own chatbot with a user-friendly interface, making it easy to interact with the ‘<i>llama3</i>’ model. This is just the beginning, and you can further customize and expand the functionality of your chatbot by integrating additional features and models.</p></article></body>
