Prompt Engineering via Prompt Patterns— Few Shots Prompting

The article is part of series: Prompt Engineering via Prompt Patterns

You can switch to video version of this article

A few shots are often good enough to teach anyone a lesson, one they can never forget. While this sinister technique works great for humans, turns out it is equally effective on large language models as well. Slight difference being that you are not making an example of the model, rather you are providing examples to the model to follow along. This is what this pattern is, teaching the large language model via examples. Give it some examples, and good responses, teaching it what a good output and formatting looks like and hoping it would pick on the pattern and follow along.

Why few shots prompting? Well because a single example is not always enough. If there is, there also exists something called ‘one shot prompting’ and even ‘zero shot prompting’.

Before going into details of these terms, it is important to understand what we are trying to accomplish here. We are trying to provide LLM with some examples and then we expect that it would follow our examples in subsequent prompts and respond with a similar pattern as set by our examples

Now, totally depending on the context of usage, LLM is for most cases pre trained on many examples, specially if a ‘noun’ exists for such pattern. An example of that can be cooking recipes or diet plans. If you ask for one from LLM, it would respond with the standard norm of a recipe i.e. ingredients followed by steps. For this example, you don’t need to train it at all using examples, it is pre trained. Ask for a great omlete recipe and you would get it, structured as it should be, with zero examples needed. This is called ‘zero shot prompting’.

Then comes ‘one shot prompting’, where you provide just one example and LLM picks up on that. The simplest example can be a prompt like Input: Add 1+1 Output: The answer to 1+1 is 2, now write Input: 3+3 in next prompt and see one shot prompting in action with model responding with, The answer to 3+3 is 6. This is generally helpful if you need output in a custom format and/or your example has static and dynamic portions. The model senses how to format the output as per your desired format and follows along. Another example can be of sentiment analysis where example like, ‘the delivered package turned out to be totally different from the picture and did not fit either’, sentiment: negative, followed by next prompt “I really liked the tie I ordered from your website”, Sentiment: would receive a response “Positive” from LLM.

Yes finally we are going to see examples of ‘few shot prompting’. Before we go for the example, you might have picked up on the pattern here yourself. The difference between zero, one and few shots prompting is how much you are trying to leverage existing training of the large language model starting with total dependence to no dependence on existing training respectively.

Let’s start by the simplest one

Fruit: Banana

Taste: Sweet

Fruit: Lemon

Taste: Sour

Fruit: Avocado

Taste: Sweet

Fruit: Orange

Taste: Sour

Fruit: Mango

Taste:

And Most probably you would be rewarded with the answer ‘Sweet’.

Now any further prompts with just statements like above and prefixed with Fruit: would be responded with Taste: followed by sweet or sour for the fruit in your input statement. LLMs like ChatGPT would continue to play along in subsequent prompts as they have learnt the pattern.

Models try to respond with most probable next words, they are learning patterns. Here it learnt the pattern using examples you provided to it. If you provide Taste: in the prompt, it is going to skip that in response and just produce the type as the response. It is pattern following at its best and you can take it forward as per your choosing. This also gives you good insight as to how it learnt language itself, where, during its training, it picked up on the repeating patterns in training data and learnt to respond accordingly. That is why large language models can speak and translate into languages they are not even explicitly trained on. Here it just did the same with your provided examples since it is trained to do so by default.

Previous example was a really simple one where we just picked on the taste of the fruit. Let’s take it one notch up and try something more interesting. Let’s try doing few shots prompting by providing it some situations and then some actions as a response.

Situation: There is a short circuit happening in switchboard of the classroom.

Action:

1. Call the mains control room to cut off power supply.

2. Ask students to evacuate the class room.

3. Get the fire extinguisher for electrical fires and try to extinguish the fire.

4. Ask someone to call fire brigade.

Situation: Student gets injured during soccer practice

Action:

1. Immediately call the in-house doctor to assess the injury.

2. After consulting with the doctor, give parents a call for their consent if some medication needs to be administered.

3. If doctor advises so, ask them to come and pick their kid from school.

Situation: Class is in progress and an earthquake hits.

Action:

With this example, we are asking the LLM to provide action to take in the given situation. We have provided it few shot examples of various crisis situations at school and actions that best handle the situation as per your school’s crisis management plan. Now you present it with a unique situation and ask for action. Now it should come up with an intelligent, insightful action plan formatted as a bulleted list like the examples you provided. But the important bit is, it is going to tap into its training to come up with a brilliant answer.

Now using few shot examples, you can either come up with a very detailed what if analysis per situation, or just a few lines plan, or for some other examples, a verb or short actionable statements. It would use its knowledge to answer, but would look up to your examples to format the output as well as decide how detailed the output needs to be. Do admire the helpfulness of LLMs and patterns like few shot examples to come up with plans like above which is a big advantage of this pattern in particular.

The fun doesn’t just end here. Combining your few shot examples with generative AI capability of language model gives a very interesting ability i.e. to expand on the examples you have been providing. This is useful to perform a great ‘what if’ analysis of the situation where you have provided it examples of actions taken in certain situations, and then expand on your examples with help of language model to generate data on situations, as well as actions required to be taken in such situations.

Let’s say after providing a few examples like the situation/action prompts earlier, now if you instruct it to ‘generate more examples’ followed by Situation:, it is going to generate a bunch of unique situations for the given context i.e. crisis situations faced in school, formatted according to your example data, and provide actions to take in such situations in the same breath.

This can be creatively used in a ton of situations where you want to generate statistical filler data, or how to generate product descriptions, or creating questions for students in your class, or use it in a lot of content generation, crisis management or other kind of planning scenarios. If you are say tasked with creating crisis management plans for school and you are trying to decide action items for your team for certain situations, few shot examples is a great way to generate a crisis response policy document.

The technique works because the tool already knows how to produce the next series of words so it is ready to pick up on the pattern that you are providing and start generating output in the format and structure that you want it to. It also has gone over loads of FAQs, policy documents, manuals, to do lists, scripts etc as part of its training and hence is ready to contribute to your few examples with a rich meaningful input.

Few shot examples need not be limited to question or answer, or situation and action, they can be expanded to incorporate ‘intermediate steps’ to breakdown the problem at hand into step by step instructions or add statements that clarify the decision making process that would lead to the next step.

Few shot examples are kind of a script to generate further content with and ChatGPT follows the script brilliantly filling it with its own novel content, but keeping the original formatting and structure of conversation intact. A non LLM example would be mission impossible movie series where the script for later additions seems to have been generated using a large language model using few shot examples of earlier movies. Apologies to those who haven’t followed that very popular series, but for those who did, they would totally get what I am saying here.

The pattern works best if we keep the template short and to the point, or in other words, less confusing. Short, straightforward simple examples work best since there is less chance of confusion and LLM can quickly pick up on the important bits of aspects you are trying to give weightage to.

The beauty of this pattern is that you don’t have to finger hold the LLM. In fact, you can give it a few examples and then tap into its immense knowledge base to expand on the topic, learning something while teaching it. It would pick up on the semantics of your examples and add valuable info on top of it. Most of the times you just need to focus on using examples to keep its output formatted in a certain way, while it starts picking up on the context of these examples itself and keeps expanding it further.

I came across a very interesting example where few shot prompting totally fails. Lets say we say

Input: Brick

Output: hard

Input: Pillow

Output: Soft

Input: Car

Output:

Now what do you expect the output to be? Well be ready to be disappointed if you were expecting hard, as it is going to say Fast. Why? Because it picked up on the defining attribute of the input. It would produce hard or soft if you instruct it to confine its output to be these two words, but see examples can convey different meaning than the one you were thinking about. Here car is not generally associated with hard or soft, while brick and pillow are, so it tried to reason that we are trying to output the most defining aspect of the intput, and hence it generated ‘fast’.

Examples need to have good context and detail and this helps it figure out how it needs to think of the examples. The prefixes and formatting you specify in examples need to be meaningful and give additional context to the model to work with.

The pattern never stops you or asks you to start with the example directly. You are free to specify the rules and end goal in initial prompt telling the model what your end goal in mind is. This is specially true if the examples are vague and rules might be hard to extract from the limited set of examples you are providing.

LLM would reason similar to how humans do, so if some examples can confuse certain humans, they can confuse LLM as well. You may need to test and fine tune the output if it deviates from your intended trajectory just like it did in our example. You should get it right with a few iterations. You can checkout iterative prompting video on the channel to learn more about that technique.

This is it for a somewhat lengthy article on few shots prompting. Please clap and share if you found it useful. You can consider subscribing to our YouTube channel as well. Thank you!!!.

Next: Prompt Engineering — Chain of Thought Prompting