Prompt Engineering via Prompt Patterns — Question Refinement Pattern

The article is part of series: Prompt Engineering via Prompt Patterns

You must be expert in something in your life, be it banking, finance, cooking, studies, monopoly well anything really 😊. Just reflect back and think of the time you were entering that field and asking questions about that topic from your mentor back then. You might laugh at some of those today.

You can switch to video version of this article

When asking questions about something you are not intimately familiar with, the questions lack the depth that comes with experience. You don’t even know what you don’t know. The problem does not go away with age, or in some cases not even with experience. You are always striving for something at an increasingly higher level where new information and technologies creep in.

In a session with ChatGPT, the chances of ChatGPT having more knowledge about the relevant field than you are always much higher. You would never get the chance or time to read, absorb and process public information available on the internet. So we implicitly know that any question we ask ChatGPT would be based off limited information that we possess. So why not ask ChatGPT to refine the question and add any missing detail, and come up with a better version of the question than you initially offered. Many smart people like yourself have done that successfully to solve problems and hence that technique is now a prompt pattern with a name, “The question refinement pattern”.

Using this pattern, ChatGPT or rather any LLM (Large language model) would aid the user in finding the right question to ask to arrive at an accurate answer, that too while avoiding trial and error prompting.

Additionally, if user is asking a question, chances are they are not domain experts and hence can’t phrase the right question, and potentially unaware of additional information helpful in phrasing the question. During a regular session, LLM generally asks you additional information, and sometimes notifies you of important assumptions it made to come up with the response. With this pattern, and its associated instructions, LLM may directly add that information and assumptions directly in the prompt, and then come up with a better answer to a better question taking limiting guesswork and hidden assumptions.

This is not magic. LLMs, like google, know what other questions are generally associated with the type of question you are asking, probably coz you may not be the first one to do so, and thus are able to augment that ‘related content’ to your original question. On your part, you are leveraging the understanding of LLM to best utilize its processed information to get the better version of answer than you would have if you had relied on your own initial question.

The pattern is used with phrases as simple as “Within scope X, suggest a better version of the question to use instead” and optionally “prompt me if I would like to use the better version instead”.

The use of scope is important to convey the desired outcome you want to achieve by providing contextual information. Otherwise the instruction becomes vague and can be interpreted as a rewording request of question to make it say ‘a better English version’. Not something you are looking for right? But come to think of it, without going into any examples for this pattern, we just came up with an example of use just using the key phrase. See the initial prompt without scope information is vague if used in any conversation even those with humans and can be misinterpreted to be a request to improve grammer. The answer would surely be disappointing, and hence a better version of the question explaining the context, intent and scope of question would greatly help.

Another example can be of a question, “should I buy a hybrid or electric car” and again you can see its not a good question without contextual information like what is your financial situation, do you already have car, how far do you travel each day, where do you live, and most importantly what your goals are to switch to hybrid i.e. are you an environmentally concerned person or want to save on fuel costs. Point being, there are so many ways to rephrase or rather improve this question to get a response that is actually helpful.

There is an interesting side effect to this pattern that it lets you know what kind of direction ChatGPT intends to take your question in by making its current intention visible. Referring to examples in previous section, the question can be interpreted and answered in a lot of ways, not all of these ways are where you want to go with this conversation. ChatGPT or LLM would improve the question to an extent, but that would be enough for you to see which aspects it is trying to emphasize and where providing more information in your prompt would help steer the conversation in intended direction. So you can either accept the improved answer as is, or come up with an even more improved answer based on improved answer you got from LLM. The pattern not only helps leverage LLM processed knowledge, but also helps you figure which additional information from you would lead to a better answer or decision.

Lets take one more example to go over a rather undesirable side effect of this pattern. When you use this pattern, LLM might ask you clarifying questions to narrow down the scope or assumptions to a more concrete level. The consequence being you might loose the big picture. Taking another car example that “what car should I buy”, it might ask you whether you already own a car, and you respond with not just yes but also tell the make and model of the car. Now LLM might add that to context and refine question but makes an in built assumption that you like cars of certain engine and body type and brand and say improve the question in a way that electric or hybrid cars go out of conversation entirely. You can go around this issue by explicit instruction like ‘do not scope my questions to specific brands or engine types’

Another technique is to combine this pattern with another patterns and instruct it say ‘whenever I ask a question, ask four additional questions that would help you produce a better version of my original question. Then use my answers to suggest a better version of my original question’

I hope question refinement pattern makes sense to you know and you would be able to use it. If you feel the article helped improve your understanding on LLM usage, please like/share, and subscribe on our YouTube channel to be notified of upcoming content. Thank you and good bye.

Next: Prompt Engineering via Prompt Patterns — Audience Persona Pattern