CoT and Re2 : How to Significantly Improve Prompting with ChatGPT and LLMs

Summary

The website content discusses techniques to enhance the performance of ChatGPT and other language models (LLMs) through improved prompting, specifically using Chain of Thought (CoT) and Re-Reading (Re2).

Abstract

The article titled "CoT and Re2: How to Significantly Improve Prompting with ChatGPT and LLMs" explains the impact of prompting techniques on the quality of responses from language models like ChatGPT. It introduces the Chain of Thought (CoT) method, which guides the model through a problem step-by-step, improving accuracy and problem-solving capabilities, though it may increase response length and operational costs. Variants like Concise Chain of Thought (CCoT) aim to optimize the number of tokens in responses. Additionally, the article presents the Re-Reading (Re2) technique, which involves repeating the prompt to enhance comprehension, and it can be used in conjunction with or independently of CoT. The author suggests that combining these techniques can lead to more accurate results and emphasizes the growing importance of English as a "programming language" in the context of LLMs.

Opinions

The author believes that prompt engineering is crucial for improving the output of LLMs.
The author finds it interesting and somewhat amusing that the Re2 technique mirrors the advice often given by high school teachers to read questions twice.
The author posits that English, with the right prompt engineering, is becoming a primary tool for programming language models.
The author encourages readers to experiment with these techniques and share their results.
The author suggests that the combination of CoT and Re2 can yield better outcomes than using either technique alone.
The author is optimistic about the future of LLMs and the role of natural language in their development and application.

CoT or Chain of Thought

One of the techniques to improve the accuracy of the answers is to use CoT or Chain of Thought, by simply adding to your prompt “Detail your reasoning” or “think step by step”.

Indeed, CoT, or Chain of Thought, is a method used in LLM to guide them through a problem in a detailed, step-by-step process. This approach helps the models to more accurately find solutions to various challenges. When breaking down a problem into smaller, manageable steps, the Chain of Thought method significantly enhances the model’s ability to solve complex tasks. This technique comes in different forms, each with its unique advantages and disadvantages.

The “Zero-shot” version prompts the model to “think step-by-step” without prior examples,

The “Few-shot” version offers examples of problems along with their solutions, demonstrating the step-by-step thought process.

Implementing the Chain of Thought has been proven to improve the model’s performance by as much as 80% in certain cases. However, this improvement in accuracy and problem-solving capability often leads to longer responses, which in turn can increase the operational costs associated with using the model due to the extended length of the responses (and hence, the number of tokens).

Multiple other variants of CoT has been tested like CCoT or Concise Chain of Thought, which improves the reasoning (or accuracy) along with optimizing by around 50% the number of tokens in the response. You can check in more details here about CCoT.

Re2 or Re-Reading

Now, here is an interesting another new and complementary technique that seems to improve the accuracy of the results, according to a new paper, as well as some tests I made. It is called Re2 or Re-Reading. Basically, when providing a prompt, you should just re-put that prompt again, preceded by “Read the question again” or “Read again”.

Re2 concentrates on the input, processing questions twice to deepen the model’s comprehension. This is interesting but also funny, and makes me remember highschool teachers telling us to read twice before answering. I would even argue that it shows to what extend LLM “mimic” human inference and reasoning, even if this is not 100% accurate.

Here is an example on how to use it :

Prompting example using CoT and Re2 — From this paper

Of course this adds more tokens into the context window, but still, for relatively short prompts, it will get you better result! Feel free to share your tests in the comments.

Re2 showed better result when combined with CoT, but also without combination with CoT.

Closing thoughts

Keeping it very short and straight to the point this time, I hope you appreciated reading this quick article on how you can improve your prompting.

I think, as we continue moving forward on the LLM space, English (or Natural Language) will become the hottest programming language. Indeed. With the right prompt engineering, using the right words and sentences, you can significantly improve the output for your LLM use cases.

If you found this article useful, please clap and share your thoughts.

I regularly write about AI and Data, feel free to follow me :