Data Analysis with ChatGPT

In the age of big data and machine learning, data analysis has become a key skill for numerous sectors — from business and finance to academia and science. But what if we could leverage the power of artificial intelligence to make this process even more effective and easier? Enter Chat GPT, an advanced language model developed by OpenAI, ready to supercharge your data analysis efforts.

What is Chat GPT?

I know this is not a news but Chat GPT is built on the GPT-4 architecture, and it’s a state-of-the-art language model developed by OpenAI. It’s essentially an artificial intelligence that’s been trained on a diverse range of internet text. What sets Chat GPT apart is its impressive ability to generate human-like text responses, making it an incredibly versatile tool, capable of fulfilling a wide array of tasks, from drafting emails and writing code, to tutoring in a variety of subjects, and yes, even analyzing data.

How Can Chat GPT Aid in Data Analysis?

1. Data Cleaning: Chat GPT can assist with data cleaning tasks, such as identifying and filling in missing values, removing duplicate entries, and converting data types.
2. Data Exploration: With its ability to understand and generate human-like text, Chat GPT can answer questions about your dataset and provide summaries of your data, making the exploration phase more efficient.
3. Data Visualization: Chat GPT can generate code for data visualization using popular libraries such as matplotlib and seaborn.
4. Statistical Analysis: From descriptive statistics to hypothesis testing, Chat GPT can provide insights and conduct statistical analysis on your dataset.
5. Predictive Modeling: Chat GPT can help in the creation of machine learning models, from data preprocessing, feature selection, model training, optimization, to interpretation of results.

With the installation of the packages below, you can get started:

pip install pandas numpy pandasai

Demo Dataset

In this section, we will be creating a demo dataset that can be turbo boosted with Artificial Intelligence.

import pandas as pd
import numpy as np
from random import choices

# Set a seed for reproducibility
np.random.seed(0)

# Number of records
num_records = 1000

# Number of outlets and products
num_outlets = 10
num_products = 5

# Generate dates for 3 months
dates = pd.date_range(start='2023-01-01', end='2023-03-31').tolist()
dates = choices(dates, k=num_records)

# Generate outlet IDs
outlets = ['Outlet_' + str(i+1) for i in range(num_outlets)]
outlets = choices(outlets, k=num_records)

# Generate product names
products = ['Product_' + str(i+1) for i in range(num_products)]
products = choices(products, k=num_records)

# Generate units sold (assume a random number between 1 and 100)
units_sold = np.random.randint(1, 100, num_records)

# Generate price per unit (assume a random price between $10 and $200)
price_per_unit = np.random.uniform(10, 200, num_records)

# Calculate total sales
total_sales = units_sold * price_per_unit

# Create the dataframe
df = pd.DataFrame({
    'Date': dates,
    'Outlet': outlets,
    'Product': products,
    'Units_Sold': units_sold,
    'Price_Per_Unit': price_per_unit,
    'Total_Sales': total_sales
})

df

AI Integration

To leverage the powers of Chat-GPT and GPT-4, you will be to get your API key from OpenAI, create a new Secret Key and replace the YOUR OPENAI API KEY GOES HERE with the secret key.

from pandasai import PandasAI
from pandasai.llm.openai import OpenAI

llm = OpenAI(api_token="YOUR OPENAI API KEY GOES HERE")
pandas_ai = PandasAI(llm)

When you run commands, such as

pandas_ai.run(df, "Which product has the highest total_sales?")

#output
'Product_3'

Data Cleansing

pandas_ai.run(df, "Format Price_Per_Unit and Total_Sales to 2 decimal places")

Plotting Charts

Github Code: here

Pandas Documentation: here

PandasAI Documentation: here

In Plain English

Thank you for being a part of our community! Before you go:

• Be sure to clap and follow the writer! 👏
• You can find even more content at PlainEnglish.io 🚀
• Sign up for our free weekly newsletter. 🗞️
• Follow us on Twitter(X), LinkedIn, YouTube, and Discord.