How to ‘stream’ output in ChatGPT style while using openAI Completion method
Live streaming of completion response wihle using openai.Completion.create method
👨🏾💻 GitHub ⭐️| 🐦 Twitter | 📹 YouTube | ☕️ BuyMeaCoffee | Ko-fi💜
TLDR: In this article, we’ll explore how to build an AI assistant using Streamlit and OpenAI, and how to use the stream argument to change the mode of operation from batch to streaming.
Introduction
Streamlit is a web-based framework for building data-driven applications. It’s designed to be simple to use and requires minimal code, making it a great choice for building quick prototypes or small applications. On the other hand, OpenAI is a leading AI research institute that has developed some of the most advanced language models in the world. With OpenAI’s API, we can use its language models to generate natural language text, answer questions, and more.
In this example, we will use the OpenAI API to create a text completion model. Our AI assistant will take in a prompt, generate a response, and display the result in a live streaming format within our Streamlit front-end framework.
You can refer to the video as well, which will walk you through step-by-step
The Code
In this post I won’t be going through the streamlit syntax , since they are pretty self explantory. However, I would highly encourage to check out my previous blog posts (in the reference section cited ) or video tutorials which has explained those parts in details.
Let’s start with the intsallation of the dependencies,
pip install streamlit openai streamlit-pills
import openai
import streamlit as st
from streamlit_pills import pills
openai.api_key = st.secrets['api_secret']
st.subheader("AI Assistant : Streamlit + OpenAI: `stream` *argument*")
# You can also use radio buttons instead
selected = pills("", ["NO Streaming", "Streaming"], ["🎈", "🌈"])
user_input = st.text_input("You: ",placeholder = "Ask me anything ...", key="input")
if st.button("Submit", type="primary"):
st.markdown("----")
res_box = st.empty()
if selected == "Streaming":
report = []
# Looping over the response
for resp in openai.Completion.create(model='text-davinci-003',
prompt=user_input,
max_tokens=120,
temperature = 0.5,
stream = True):
# join method to concatenate the elements of the list
# into a single string,
# then strip out any empty strings
report.append(resp.choices[0].text)
result = "".join(report).strip()
result = result.replace("\n", "")
res_box.markdown(f'*{result}*')
else:
completions = openai.Completion.create(model='text-davinci-003',
prompt=user_input,
max_tokens=120,
temperature = 0.5,
stream = False)
result = completions.choices[0].text
res_box.write(result)
st.markdown("----")Follow the commented section of the code for bette understanding. The stream argument in the code is a crucial aspect that sets this AI assistant apart. The argument is boolean in naure and provides the flexibility to choose between receiving the AI response in a streaming fashion or as a single response. When stream = True, the AI response is broken down into smaller parts and displayed as each part arrives. This feature creates a near real-time experience for the user, as they can see the AI's response being built piece by piece. On the other hand, when stream = False, the AI response is returned as a single string and displayed in one go.
Check the video in the tweet section below, it seems to work pretty smooth. A chatGPT vibe ? which we all adore in recent days, isn’t it ?
By providing users with the choice between a streaming or single response, the code snippet demonstrates the versatility of OpenAI’s GPT-3 and the Streamlit library. This feature makes the AI assistant more user-friendly and provides a better overall user experience.
Bonus : The tutorial video mentioned — demonstrates on how to adapt this stream feature in Google Collab / Jupyter Notebook / Terminal — make sure to refer to the video for such usecases.
Conclusion
In conclusion, the code snippet demonstrated in this blog post showcases the power of OpenAI’s GPT-3 and the Streamlit library, and how these technologies can be used to build a versatile and user-friendly AI assistant.
👨🏾💻 GitHub ⭐️| 🐦 Twitter | 📹 YouTube | ☕️ BuyMeaCoffee | Ko-fi💜
Hi there ! I’m always on the lookout for sponsorship, affiliate links and writing/coding gigs to keep broadening my online contents. Any support, feedback and suggestions is very much appreciated ! Interested ? Drop an email here : [email protected]
References
- Open AI document
- This video was inspired by the comment of “Ron Berito” here over this Medium blog post — https://readmedium.com/build-your-own-chatbot-with-openai-gpt-3-and-streamlit-6f1330876846
- Summarizing Scientific Articles with OpenAI ✨ and Streamlit — https://readmedium.com/summarizing-scientific-articles-with-openai-and-streamlit-fdee12aa1a2b?source=rss-bf79cad6afa1------2
- Build Your Own Chatbot with openAI GPT-3 and Streamlit — https://readmedium.com/build-your-own-chatbot-with-openai-gpt-3-and-streamlit-6f1330876846?source=rss-bf79cad6afa1------2
- ChatGPT helped me to built this Data Science Web App using Streamlit-Python — https://readmedium.com/chatgpt-build-this-data-science-web-app-using-streamlit-python-25acca3cecd4?source=rss-bf79cad6afa1------2
