Three ChatGPT Principles For Better Code Generation

An illustrative experiment on how to get more out of ChatGPT

Artificial Intelligence (AI)-based coding assistants show unprecedented performance, and the way code is written is completely transformed. This article shares three observations with the latest ChatGPT (GPT-4) for code generation and how to improve the results. Ultimately, by highlighting the potential limitations of ChatGPT, you can improve code generation.

Introduction: AI accelerated coding

When thinking about AI accelerated coding, one of the most remarkable milestones was GitHub Copilot, which was announced on June 29, 2021 and officially released in October of the same year.

GitHub Copilot uses OpenAI’s Codex model, a descendent of GPT-3. It was immediately clear how transformative this tool was going to be. You would start typing code, and it would give you a suggestion on how to continue; even more, you could start with just some documentation on what the code should do, and then you could watch how a suggestion comes. Autocomplete on Steroids — which dramatically speeds up coding. GitHub claims it accelerates coding by 55%, increases the likelihood of you completing the task, and ultimately makes you happier. In fact, sometimes the code suggestions are so eerily good and spot on that you are left speechless on how it can know that you wanted to code this way without even finishing your prompt.

Aside from this remarkable technical milestone, it also sparked quite a bit of controversy, as the AI was trained on publicly available repos. This raises all sorts of questions — and AI-generated code in general — regarding ownership and copyright. These issues aside, to me, as an advocate of open source and making your code publically available, I liked the idea. If you make your project public, and even if your code is not actively used, it can help others to write better code — and in this case, by training an AI. The latest version GitHub Copilot X, announced on March 22, 2023, is not released yet, but there is a lot we can expect.

So while many had used AI-assisted code generation already, it was not until ChatGPT that it took off for the masses. Here, you could clearly have a conversation about what you want to have and subsequently improve the output.

Now let’s dive deeper and go into the details of writing better prompts for ChatGPT. All of the examples below were generated with GPT-4.

Observation 1: Don’t start with zero but evaluate options

With ChatGPT at your disposal, you might be tempted to approach a problem naively: To forget everything and ask again how ChatGPT would solve the problem. As it has access to a vast amount of knowledge, it might know a better way to solve a problem than you. However, getting a tech stack for a problem might not be the best example, and you, as an experienced programmer, might have an edge. Consider the following prompt:

I want to build a simple ML website where users can upload images, a ML algorithm is used and a result is displayed. What language should I use for that and what libraries? I want a basic prototype as fast as possible.

ChatGPT suggested the following:

Frontend: HTML, CSS, and JavaScript for building the user interface. You can use a frontend framework like Bootstrap to speed up the process.

Backend: Python is a popular language for machine learning and has many libraries that can help you get started quickly. For the backend, you can use a web framework like Flask or Django. Flask is lighter and easier to set up, making it a good choice for a quick prototype.

ML Libraries: For image processing and ML model implementation, you can use popular Python libraries like TensorFlow, Keras, or PyTorch. For basic image manipulation and preprocessing, you can use libraries like Pillow or OpenCV.

While this advice is a good start, it is a full tech stack separated into front- and backend. It seems to have overlooked the as fast as possible part of the prompt, and you would be missing out on libraries that are specifically designed for fast prototyping, like gradio or streamlit. Interestingly, when you ask ChatGPT about them, it will confirm that these libraries would be a good fit for the task.

Now you might suggest that the initial prompt is not ideal. But wait: We can ask ChatGPT to refine the prompt so that it should be better in finding the packages I suggested — However, when testing this, this didn’t work, and they still were not listed.

You can also ask it how to refine the prompt so that such packages show up — but without success: They will not get listed; when I tested, it even said that such libraries don’t exist. I also tried it with WebChatGPT, which can search the web using the Chrome Plugin, with no change in outcome.

My recommendation is, therefore, that you don’t ask naively but rather find showcases for similar problems, and then use this and ask ChatGPT if this could be a good solution for the task. To find good showcases, medium.com is actually a goldmine. If you prepend medium and use the same prompt for a Google search, one of the first search results is this article about deploying ML models by Abhay Parashar, which would be exactly what you are looking for. This can also serve as a cautionary tale for our managers, that will be blindly setting requirements based on ChatGPT output.

I encountered this kind of behavior multiple times — e.g., another example is asking: “I want to build a large scale image processing pipeline on AWS. What services should I use for that?”. This will list lambda functions and not AWS batch, which is probably a much better-fitting service. Lambda functions can’t run longer than 15 minutes, so they are probably not the natural choice for heavier workloads. Again, a Google Search with the same prompt would get you straight to this article showing how it’s done. We can therefore draw the following conclusion:

Conclusion 1: Don’t start with zero but find a similar working solution and propose this for ChatGPT and ask for feedback.

Observation 2: Be up to date by providing additional input

Now, we have found some good reference projects and want to translate this into our problem. Suppose we want to build a streamlit application, and this time, we want to upload a *.csv file and want to make it editable. We know what packages already, so we write the following prompt:

Can you give me code for a streamlit application that loads a csv file as a pandas dataframe and displays it? Additionally, I want the dataframe to be editable.

You’ll get the following code:

import streamlit as st
import pandas as pd
from streamlit_aggrid import AgGrid, GridUpdateMode, DataReturnMode

st.title("CSV File Loader & Editor")

uploaded_file = st.file_uploader("Upload a CSV file", type="csv")

if uploaded_file is not None:
    # Load the CSV file as a Pandas DataFrame
    df = pd.read_csv(uploaded_file)
    st.subheader("Original DataFrame")
    st.write(df)

    st.subheader("Editable DataFrame")

    # Configure ag-Grid options
    grid_options = {
        "defaultColDef": {
            "editable": True,
            "sortable": True,
            "filter": True,
            "resizable": True,
        },
    }

    # Display the editable DataFrame using ag-Grid
    response = AgGrid(
        df,
        gridOptions=grid_options,
        height=600,
        width="100%",
        data_return_mode=DataReturnMode.FILTERED_AND_SORTED,
        update_mode=GridUpdateMode.VALUE_CHANGED,
        fit_columns_on_grid_load=True,
        allow_unsafe_jscode=True,
    )

    st.subheader("Edited DataFrame")
    st.write(response["data"])

This code — in principle, doesn’t look too bad. It doesn’t run as is, but the fundamental idea makes sense. Streamlit natively didn’t support editable dataframes, and this was the go-to workaround, by using streamlit-aggrid. While this was the go-to in the past, streamlit recently, on February 28, introduced a feature for editable dataframes, and the solution is not using this. Again, we are using GPT-4, which is from March 23. It, therefore, could have known the feature — but was that so? If we ask it to use st.experimental_data_editor, we get this:

import streamlit as st
import pandas as pd

st.title("CSV File Loader & Editor")

uploaded_file = st.file_uploader("Upload a CSV file", type="csv")

if uploaded_file is not None:
    # Load the CSV file as a Pandas DataFrame
    df = pd.read_csv(uploaded_file)
    st.subheader("Original DataFrame")
    st.write(df)

    st.subheader("Editable DataFrame")
    
    # Display the editable DataFrame using st.experimental_data_editor
    edited_df = st.experimental_data_editor(df)
    st.subheader("Edited DataFrame")
    st.write(edited_df)

So clearly, ChatGPT knows that this feature exists. Even better: This code works right away and uses the new feature. Again, I asked ChatGPT to optimize my prompt so that it would go directly for this, and again the new prompt did not improve the outcome. Supplementing ChatGPT with this kind of information means, of course, that you will need to read the release notes of packages so that you know about the latest features. In summary, and similar to Observation 1, we could nudge ChatGPT in the “right” direction by pointing to additional context.

Conclusion 2: If you are using certain packages, suggest new functionality based on the latest releases to not miss out on them.

Observation 3: Iterate and Refine

For the last example, we refined the output by suggesting additional context. Generally, it is very good to follow up and run several iterations for a better solution. Let’s illustrate this with a small example: Suppose you want to write some signal processing software that needs to calculate the cross-correlation of two signals. Essentially, cross-correlation is a measure of the similarity between two series. We start our prompt as follows:

Write a Python function to calculate the cross-correlation of two signals.

import numpy as np

def cross_correlation(signal1, signal2):
    """
    Calculate the cross-correlation of two signals.

    Args:
    signal1 (list or numpy array): The first signal.
    signal2 (list or numpy array): The second signal.

    Returns:
    numpy array: The cross-correlation of the input signals.
    """
    # Convert the input signals to numpy arrays if they are not already
    signal1 = np.array(signal1)
    signal2 = np.array(signal2)

    # Ensure that the signals have the same length
    if signal1.size != signal2.size:
        raise ValueError("Input signals must have the same length")

    # Calculate the cross-correlation using numpy's correlate function
    cross_corr = np.correlate(signal1, signal2, mode='full')

    return cross_corr

# Example usage:
signal1 = [1, 2, 3, 4]
signal2 = [4, 3, 2, 1]

result = cross_correlation(signal1, signal2)
print(result)

This function does the trick, is well documented, and even would catch an error when having arrays of different lengths. However, it is probably not very fast. If you have read the Wikipedia article about correlation, it states that you can have a faster version by using a “Fast Fourier Transform” (FFT) approach. We continue by asking ChatGPT the following:

Can you make it faster?

import numpy as np

def cross_correlation_fft(signal1, signal2):
    """
    Calculate the cross-correlation of two signals using the FFT-based approach.

    Args:
    signal1 (list or numpy array): The first signal.
    signal2 (list or numpy array): The second signal.

    Returns:
    numpy array: The cross-correlation of the input signals.
    """
    # Convert the input signals to numpy arrays if they are not already
    signal1 = np.array(signal1)
    signal2 = np.array(signal2)

    # Ensure that the signals have the same length
    if signal1.size != signal2.size:
        raise ValueError("Input signals must have the same length")

    # Calculate the FFT of both signals and the complex conjugate of the second signal's FFT
    fft_signal1 = np.fft.fft(signal1)
    fft_signal2 = np.fft.fft(signal2).conj()

    # Multiply the FFT of the first signal and the complex conjugate of the FFT of the second signal
    product = fft_signal1 * fft_signal2

    # Calculate the inverse FFT of the product
    cross_corr = np.fft.ifft(product)

    # Make sure the result is real and return it as a numpy array
    return np.real(cross_corr)

# Example usage:
signal1 = [1, 2, 3, 4]
signal2 = [4, 3, 2, 1]

result = cross_correlation_fft(signal1, signal2)
print(result)

And voila, we get a solution that uses FFT. However, the code is essentially a re-implementation and very long, so we follow up with the following prompt:

Is there an easier way to do this?

import numpy as np
from scipy.signal import correlate

def cross_correlation_scipy(signal1, signal2):
    """
    Calculate the cross-correlation of two signals using scipy's correlate function.

    Args:
    signal1 (list or numpy array): The first signal.
    signal2 (list or numpy array): The second signal.

    Returns:
    numpy array: The cross-correlation of the input signals.
    """
    # Convert the input signals to numpy arrays if they are not already
    signal1 = np.array(signal1)
    signal2 = np.array(signal2)

    # Ensure that the signals have the same length
    if signal1.size != signal2.size:
        raise ValueError("Input signals must have the same length")

    # Calculate the cross-correlation using scipy's correlate function
    cross_corr = correlate(signal1, signal2, mode='full', method='auto')

    return cross_corr

# Example usage:
signal1 = [1, 2, 3, 4]
signal2 = [4, 3, 2, 1]

result = cross_correlation_scipy(signal1, signal2)
print(result)

We get the solution using the scipy — package, and this is probably the best solution so far — just by asking to make it faster and easier. For completeness, instead of using ChatGPT, you could have also just read the NumPy documentation for the np.correlate, where this is also documented:

de>numpy.correlate may perform slowly in large arrays (i.e. n = 1e5) because it does not use the FFT to compute the convolution; in that case, de>scipy.signal.correlate might be preferable.

Similarly, you can use certain keywords to optimize your code in certain directions. Here is a (GPT-4)-generated table that you can use for this kind of refinement:

Conclusion 3: Use keywords to systematically refine the code output with prompts.

Note that will it might be tempting to put this into one giant prompt along the lines of:

Write a Python function to calculate the cross-correlation of two signals. Optimize performance. Reduce memory usage. Improve readability. Simplify.

I have found that I didn’t get to the scipy-solution as before, even after prompting to re-check to optimize the requirements, and I found it better to optimize one criterion at a time.

Outlook

We observed three different kinds of behaviors and drew conclusions on how we can improve code output for ChatGPT. Hopefully, I could show that while ChatGPT can provide good initial solutions, they will need refinement. Interestingly, sometimes the information is readily available, e.g., if you would have just read the documentation. Conversely, it would take some iterations with ChatGPT to get there on its own.

Overall, this somewhat shifts the focus from you writing the code to reviewing it. And for AI-generated solutions that are very complex, it will be much more difficult to process and understand. For now, my closest analogy to this shift is the one of learning a language: If you are not proficient, you can probably get a long way using translation tools and knowing a couple of words. But once there is something more complex and you are not proficient, you will be completely lost. I recently encountered this meme that illustrates the shift in roles from writing to reviewing:

Some of the steps involved — e.g., reading release notes and browsing medium articles can probably be automated soon; we see some tools in the direction on the horizon, such as the web browsing plugin or automatic agents like Auto-GPT.

I personally enjoy ChatGPT for small scripts and generating structures for larger projects. In the day-to-day business, I find GitHub Copilot the most useful tool and can’t wait for the update.

Do you have additional tips that you found useful when using ChatGPT for coding? Feel free to leave a suggestion in the comments.

Stay updated with the latest news and updates in the creative AI space — follow the Generative AI publication.