avatarAngelica Lo Duca

Summary

The web content provides guidance on improving ChatGPT outputs by configuring key parameters such as temperature, Top P, frequency penalty, and presence penalty through practical examples and scenarios.

Abstract

The article delves into the nuances of fine-tuning ChatGPT's generative AI capabilities by adjusting specific configuration parameters. It draws insights from David Clinton's book "The Complete Obsolete Guide to Generative AI," explaining how parameters like temperature, Top P, frequency penalty, and presence penalty can influence the creativity and determinism of ChatGPT's responses. The author illustrates these concepts with an example related to population growth data, demonstrating how different settings can yield varied outputs from ChatGPT. By experimenting with these parameters, users can tailor the AI's output to be more conservative or more creative, and to control for repetition and diversity in the text generated.

Opinions

  • The author believes that understanding and configuring the main parameters of an AI model, such as ChatGPT, is crucial for adapting its outputs to specific user needs.
  • The author suggests that using the free version of ChatGPT can be beneficial for practical tests and learning how to apply theoretical knowledge from books like Clinton's.
  • The article conveys the opinion that combining AI-generated text with human oversight can lead to more effective data storytelling, as shown in the author's own book "Data Storytelling with Generative AI: Using Python and Altair."
  • The author emphasizes the importance of experimenting with different parameter settings to achieve the desired balance between deterministic and creative outputs in AI-generated text.
  • It is implied that readers interested in generative AI should also explore related literature and resources to enhance their understanding and practical skills.

ChatGPT, Generative AI

How to Improve Your ChatGPT Outputs Using Configuration Parameters

A focus on configuring the temperature, the Top P, the frequency penalty, and the presence penalty directly in your ChatGPT prompts

Photo by Growtika on Unsplash

I’ve recently been reading a very interesting book by David Clinton, entitled The Complete Obsolete Guide to Generative AI, published by Manning Publications. In the second chapter, the author describes what the main parameters of an AI model are and how to configure them to adapt to their needs. The configuration parameters include temperature, Top P value, frequency penalty, and presence penalty.

Configuring these parameters could help ChatGPT understand the type of output you want to produce. For example, you could set some parameters if you want ChatGPT to generate a more deterministic output (strictly related to the input). Instead, if you want ChatGPT to be more creative while generating the output, you could set other parameters.

To understand how the output type works, David Clinton gives an example related to weather in his book. A more deterministic output could be: Today, it’s sunny, and the temperature is 30°C. A more creative output could be: Today, it’s sunny, and you may have a walk.

Driven by the descriptions I found in the book, I tried to implement some examples myself to see how to apply what was written in the book in a practical case. I hope you, too, can benefit from this article.

For those familiar with the OpenAI API, the configuration parameters can be passed directly as arguments in the API functions. Instead, using them in the ChatGPT web interface is not immediate. In this article, I’ll show you how to include these parameters directly in the ChatGPT prompt.

To do practical tests, we will use an imaginary scenario and the free version of ChatGPT, which you can find at this link. I remind you that before using ChatGPT, you must create an account and read the platform’s usage policies.

In this article, we’ll cover:

  • Scenario Setup
  • Temperature
  • Top P value
  • Frequency penalty
  • Presence penalty

Scenario Setup

As a scenario, we will use the use case described in the third chapter of my book, Data Storytelling with Generative AI using Python and Altair. If you are interested, you can download the Python code relating to this scenario for free from the book’s GitHub repository, available at this link.

Imagine that you want to generate the description associated with the following chart:

Image by Author

The chart describes a comparison between the population growth in North America (NA) and the average value for the rest of the world. The chart shows the difference between each year and the baseline, 1960. There is a gap of 367 million. You want ChatGPT to generate a description for your chart.

We’ll use the following baseline prompt and then adapt it to the various configurations:

Write a commentary of 200 characters for the following scenario: a comparison between the population growth in North America and the rest of the world. There is a gap of 367 million in 2020, with North America having the lower value.

Temperature

Temperature defines the amount of randomness used while generating the output. It enables the user to adjust the level of creativity and unpredictability of the output. Temperature ranges from 0 (highly structured and conservative output) to 2 (highly creative and unpredictable output), and the default value is 1.

Let’s try different temperature values for our scenario.

Temperature = 0 (conservative)

Write the following prompt:

Write a commentary of 200 characters for the following scenario using a temperature = 0: a comparison between the population growth in North America and the rest of the world. There is a gap of 367 million in 2020, with North America having the lower value.

The following figure shows a possible output produced by ChatGPT:

Image by Author using the ChatGPT interface

The produced output is very similar to the original text, meaning that the text is very conservative.

Temperature = 1 (medium)

Write the following prompt:

Write a commentary of 200 characters for the following scenario using a temperature = 1: a comparison between the population growth in North America and the rest of the world. There is a gap of 367 million in 2020, with North America having the lower value.

The following figure shows a possible output produced by ChatGPT:

Image by Author using the ChatGPT interface

ChatGPT has introduced something new compared to the original text, including the factors that have contributed to this contrast.

Temperature = 2 (high creativity)

Write the following prompt:

Write a commentary of 200 characters for the following scenario using a temperature = 2: a comparison between the population growth in North America and the rest of the world. There is a gap of 367 million in 2020, with North America having the lower value.

The following figure shows a possible output produced by ChatGPT:

Image by Author using the ChatGPT interface

The level of creativity here is high because it includes aspects related to society (societal dynamics).

Top P

Top P is also known as nucleus sampling or penalty-free sampling. It helps to control the diversity of the generated text. Use this technique if you want to generate responses that don’t completely deviate from the topic. The range is between 0 and 1. A higher Top P makes the output more diverse, while a lower value makes the model more deterministic. The default value is 1.

Let’s try different Top P values for our scenario.

Top P = 0.1 (more deterministic)

Write the following prompt:

Write a commentary of 200 characters for the following scenario using a Top P = 0.1: a comparison between the population growth in North America and the rest of the world. There is a gap of 367 million in 2020, with North America having the lower value.

The following figure shows a possible output produced by ChatGPT:

Image by Author using the ChatGPT interface

The produced output is very similar to the original input prompt.

Top P = 0.9 (more diverse)

Write the following prompt:

Write a commentary of 200 characters for the following scenario using a Top P = 0.9: a comparison between the population growth in North America and the rest of the world. There is a gap of 367 million in 2020, with North America having the lower value.

The following figure shows a possible output produced by ChatGPT:

Image by Author using the ChatGPT interface

Compared to the original prompt, the text is diverse since it contains different words.

Frequency Penalty

Frequency penalty reduces redundant output. This enables you to control the trade-off between generating diverse responses and avoiding repetitive patterns. The value can range from -2.0 (high repetition) and 2.0 (low repetition).

Let’s try different Frequency Penalty values for our scenario.

Frequency Penalty = -2 (high repetition)

Write the following prompt:

Write a commentary of 200 characters for the following scenario using a Frequency Penalty = -2: a comparison between the population growth in North America and the rest of the world. There is a gap of 367 million in 2020, with North America having the lower value.

The following figure shows a possible output produced by ChatGPT:

Image by Author using the ChatGPT interface

The text is very similar to the input prompt.

Frequency Penalty = 0 (neutral)

Write the following prompt:

Write a commentary of 200 characters for the following scenario using a Frequency Penalty = 0: a comparison between the population growth in North America and the rest of the world. There is a gap of 367 million in 2020, with North America having the lower value.

The following figure shows a possible output produced by ChatGPT:

Image by Author using the ChatGPT interface

In this case, the text introduces something novel compared to the original text.

Frequency Penalty = 2 (low repetition)

Write the following prompt:

Write a commentary of 200 characters for the following scenario using a Frequency Penalty = 2: a comparison between the population growth in North America and the rest of the world. There is a gap of 367 million in 2020, with North America having the lower value.

The following figure shows a possible output produced by ChatGPT:

Image by Author using the ChatGPT interface

The text is different from the original prompt and introduces many elements of novelty.

Presence Penalty

Presence Penalty controls the repetition of phrases and words in generated texts. It prevents the model from repeating the same phrasing or words too often in the generated output. The presence penalty ranges from -2 (more flexible while generating text) to 2 (strong discouragement in repetition).

It may seem that frequency and presence penalties may overlap. However, they represent different concepts.

In his book, The Complete Obsolete Guide to Generative AI, David Clinton explains the difference:

The frequency penalty […] discourages the model from repeating the same words or phrases too frequently within the generated text. The presence penalty […] encourages the model to generate words that were not in the input.

Let’s try different Presence Penalty values for our scenario.

Presence Penalty = -2 (more flexible)

Write the following prompt:

Write a commentary of 200 characters for the following scenario using a Presence Penalty = -2: a comparison between the population growth in North America and the rest of the world. There is a gap of 367 million in 2020, with North America having the lower value.

The following figure shows a possible output produced by ChatGPT:

Image by Author using the ChatGPT interface

Presence Penalty = 2 (strong discouragement in repetition)

Write the following prompt:

Write a commentary of 200 characters for the following scenario using a Presence Penalty = 2: a comparison between the population growth in North America and the rest of the world. There is a gap of 367 million in 2020, with North America having the lower value.

The following figure shows a possible output produced by ChatGPT:

Image by Author using the ChatGPT interface

A high presence penalty value encourages the model to generate new unseen words.

Summary

Congratulations! You have just learned how to use the temperature, Top P value, frequency penalty, and presence penalty in your ChatGPT prompt!

Keep in mind to set one parameter each time; otherwise, the model could get confused.

If you’ve read this far, I’m pretty satisfied for today. Thanks, and see you next time :)

Resources

You may also be interested in…

ChatGPT
Generative Ai Tools
Prompt Engineering
Data Science
Artificial Intelligence
Recommended from ReadMedium