Data Science, Programming

Analysing Whatsapp Group Chats using StreamLit— Part II

The journey of my application from a notebook to a web application using Streamlit and Heroku.

Don’t be satisfied with “almost” completing a task. Continue until you complete it.

In data science, it is important to work on real-time projects. It is equally important to share your work with the world which ensures that we receive constant feedback and can enhance the performance of our application. Jupyter Notebooks, Google Colab links are a good way to share our work. But in most cases, a client or the end-user is a person with minimal technical knowledge. How do you share your work with them?

A Web-App comes to the rescue! We can embed our analysis into a Web Application and share it with the outside world. Hence the load on the client reduces and it makes their work easier. Do read my previous article, to understand how we performed the data collection, cleaning, transforming, and visualization parts. In this article, I will talk about how I created a web application having minimal code using Streamlit.

What is Streamlit?

Streamlit is an open-source app framework with which we can create performant and beautiful web apps for Machine Learning and Data Science. Everything is written in pure python. Let us begin!

1. Make sure that you have Python 3.6 or greater installed.
2. Install Streamlit using PIP:

pip install streamlit

Run the following command.

streamlit hello

That’s it! In the next few seconds, the sample app will open in a new tab in your default browser.

A default layout that looks simple and neat can be seen. To create our app let us create a new python script Whatsapp_Analysis.py and start writing the code! Firstly we will import the required dependencies. Streamlit provides us some utility functions like st.text() , st.sidebar.text() , st.markdown(), etc. We can embed HTML code in st.markdown() which ensures we can modify the UI.

The output —

Surprisingly, 70% of the UI is completed with the above 10 lines of code. The above code is responsive too and when viewed on the mobile -

Making web apps responsive is the biggest issue a front developer can face, but luckily the framework handles this in our case. Now we need to load the exported chat file and run the analysis.

Data flow

Streamlit's architecture allows us to write apps like we write python scripts. To render any update on the screen Streamlit runs the python script from top to bottom. Whenever we interact with any button on the screen or give any input, Streamlit runs the whole python script from top to bottom.

You may think this is a tedious process for longer scripts. Yes, it is! but this is where Streamlit does some heavy lifting for us behind the scenes. A big player st.cache() decorator comes into the picture. This allows the app to skip certain costly computations (whose outputs are not changed)when the app reruns.

Widgets

When you’ve got the data into the state that you want to explore, you can add in widgets like st.slider(), st.button() or st.selectbox(). It’s really straightforward — just treat widgets as variables:

In our chat analysis app, we will be using two widgets. One to upload the chat file and another to filter the stats based on user input.

uploaded_file = st.file_uploader("Upload Your Whatsapp Chat.(.txt file only!)", type="txt")

This will create a file uploader and will only accept files of type .txt .The first argument is the heading of this file uploader.

Clicking on browse files will direct us to the local system and we must upload a .txt file.

The .txt file is a text file that consists of the exported WhatsApp chat. As discussed in my previous article we will parse the text, split it into Date, Time, Author, Message categories, and create a data frame. We will also clean the data frame by removing the missing values and also perform appropriate type conversions (ie. convert date string to date-time format, etc.). After this, we can filter the data frame as required and create plots.

Since I used Plotly for visualization, we can create plots using the function st.plotly_chart(fig) to create Plotly figures. For example, to create an emoji distribution chart we will write a function visualize_emoji() that will create a Plotly figure by filtering the data frame and returns the figure.

Now we can draw the plot using the following code.

st.subheader("**%s's emoji distribution 😂**"% name)
st.text("Hover on Chart to see details.")
st.plotly_chart(visualize_emoji(data),use_container_width=True)

Here name indicates the individual name or group.

Output -

Similarly, all the plots mentioned in the previous article are created. Another feature included in the web app is the ability to filter stats based on users. We provide a dropdown and ask the user to select a particular user whose individual stats will be displayed.

To create a dropdown we will use the st.selectbox() command. We will pass a list consisting of unique authors and an option “all”(for the whole group stats) to the st.selectbox() command. The user-selected option is stored in a variable called option and this variable is used to filter the stats in the data frame.

authorlist = list(data.Author.unique())
authorlist.insert(0,'All')
st.subheader("**Who's Stats do you want to see?**")
option = st.selectbox("", authorlist)

st.cache()

As I already mentioned, whenever we give a new input Streamlit runs the python script from top to bottom. In our case when we select one user from the dropdown, Streamlit runs the whole script which includes opening the file, parsing the file, creating the data frame, and applying filtering.

This is not an optimal way, because filtering requires only the data frame and parsing the data every time, and creating the data frame again when we click on each user is not appropriate. Hence we will use the st.cache() command along with the load_data() function. This will ensure that the second time load_data() function is run, it’s results will be directly fetched from the cache instead of re-running the function. This increases the speed of the application and enhances performance.

@st.cache(allow_output_mutation=True)
# This function parses the text file and creates the data frame.
def load_data(uploaded_file):
         .....function body

Deployment using Heroku

Heroku is a platform as a service (PaaS) that enables developers to build, run, and operate applications entirely in the cloud. We will deploy our web app to Heroku. This is a short and simple process which is very well explained in this article.

You can visit the fully running web application by clicking on the below link. Whatsapp-Group-Chat-Analyzer.

Conclusion

In this article, I tried to explain the creation of a web app for a data science project using Streamlit. It is very important for data analysts to showcase their results and present them in a simple way to both technical/non-technical users. I hope you learned something new today!

I hope you took home some new concepts today! If you would like to get in touch, connect with me on LinkedIn.