Prompt Engineering via Prompt Patterns — ReAct Prompting
The article is part of series: Prompt Engineering via Prompt Patterns
You can switch to video version of this article
So far we have explored tons of ChatGPT features in our ChatGPT features series and how to creatively use the tool using prompt engineering series, but we have used the tool primarily as a chatbot and relied on either pre training or custom training that we did ourselves to unlock some features. Although this unlocks a world of possibilities, the tool is chained, constrained or whatever is the best word to describe it to use its current knowledge and respond to you with text.
Now up until recently, there used to be a cutoff date for training the model, so if you ask it some question regarding an event that occurred past that date, it would simply be unable to answer. Similarly, the tool has been trained on the public internet, which makes up its knowledge base, but what if you need to find some details not present on public internet, but on your internal network, like whether there is a non smoking room available on such and such dates in a particular hotel, it is again unable to service your query. And finally, what if you want it to take some action on your behalf, lets say book the hotel room on given dates, it is utterly useless in current form as it is unable to interact with any external entity. It can only suggest what you need to do to book the room yourself. So suffocating and what a waste of potential right?
Hollywood has long normalized us to the idea of AI being able to act, so for most of you watching this article, this won’t be a novel concept. However, in practical life, this approach is full of potholes. When we come to envision the idea of AI acting, our first instinct is to visualize an evil robot trying to pull out a gun and try shooting the hero, or out of fantasy world a robot trying to clean a room. But before going that far, lets quickly evaluate the most primitive form of action i.e. trying to find information the model does not possess.
Now just pause for a second and think, if I ask you a question which you don’t know the answer to, but you are obligated to give me a good answer, what would you do, say your boss sent you an email to ask if any train leaves to nearby city early morning so he can be there in time to see a popular tourist attraction. Your options are either to just reply with ‘No’ and click send, but if you are a sane, reasonable member of his team and respect him as your boss, you probably won’t do so. Now this situation is not so different for you the employee, and a chatbot like ChatGPT trying to respond to your prompt. Both of you do not know the answer. Now what would you do? And how? And most importantly, what would be the process you would follow to give a more informative reply than a simple ‘sorry no idea!!!’. Please pause and think. Then resume.
Well since you are a smart person, I am sure you would try to find the train schedule online. That was easy. You would google to find the train operator website, pull out the schedule, find the train, and then going one step further, you would check if there are seats available on that train. Easy right?
But would it be as easy for your grandmother who never learnt to use internet? Probably not.
Unfortunately, same is true for large language models, which have never been trained to use the internet. But they know how to use it, you would argue, and you would be right, but being able to walk up to your computer, opening browser and starting to google is much easier for you than it is for a machine learning model right?
But that can change with your help. You can train the model to ask for your help whenever it needs some information, you query the information online, pump the answer back to the model, and Bob’s your uncle.
The example we took was really simple and you could do it yourself, but if it was a bit complex where you could leverage model’s in built knowledge like say the question from boss was I want to reach City X to see tourist attraction Y, what time should I leave home to be able to be done with my sightseeing tour by afternoon, would require a more complex search for traffic routes and congestion, train schedules, timings for tourist attraction and busy hours and what not, now LLM must be aware of most of these since people complain a lot online and one of the entities that read such nagging complaints with interest are the language model crawlers. Also it would be much easier for LLM to calculate the departure time than yourself.
It would need help querying the latest train and attraction schedule though, but, and here is the important bit, it should be able to come up with what to query on its own. Alright you agree to help the poor model, but wait, have you heard the camel and the tent story in which the camel shivering in the cold asks its owner to just let his head in the warm tent? Then the paws, then legs and eventually the camel is fully inside and asks its owner to move out since there is no space for two bodies in a little tent? Well, coming back to the LLM, if you are helping in searching for information, you would end up helping out with train schedules, querying busy hours for attraction, and eventually making the booking as well as sending email to your boss with departure and booking information with no guarantee he would be happy enough to give you a raise.
Before going into details of how the prompt would look like, lets dig a bit deeper into how that whole process would unfold. Alright so far we have established that for most situations, a LLM like ChatGPT should be able to figure out a chain of thought, reasoning or an action plan for what needs to be done to reach the end goal. It has figured out the ‘what’ and all that is missing is ‘how’. To solve the given problem entirely on its own, the model needs access to a bunch of services including search or wikipedia, calculator, train schedule and booking system and probably an email or sms service. But since it is a model and still needs your help, its plan would need to look like a thought or plan, followed by a action request to you for external information, after which the model would pause and wait for your input, you would run the request say a web query, and then input the results back to the model say via prompt, and model would resume its action plan until it is done or requires another action request to be processed. Finally it would arrive at an answer and compose an email to be sent, which you would do on its behalf but the subject, body text etc would be provided by the model.
Isn’t that a win win for both of you? You can debate who did the heavy lifting here since you still had to perform actions like web query and putting results back to the model, but can’t deny that it saved you considerable effort. You leveraged the model’s built in training to figure out the action plan, intermediate steps or things to do, and since you are not an expert in commute calculation, the model would have taken several factors into consideration that you might not even be aware of, or for sake of your big ego lets say you might have missed those by mistake.
Now on part of ChatGPT, it had to do the hard work of coming up with an action plan using the pre training and knowledge it has, and incorporating all the required factors into consideration, road congestion, train schedule, tourist attraction time table, day of the week, weather, queue waiting times and what not. If you have read our article on Chain Of Thought prompting, it would come up with the entire chain of thought i.e. reasoning paths to calculate departure time, and it would also come up with associated acts to perform, and by acts, I mean requests to you for external actions to perform with associated feedback like putting search results back into the prompt. Finally, if you want to ensure the answer is based on multiple reasoning paths, you would ask model to perform Chain of thought — self consistency technique for which we also have an article, and it would use solve the same problem in multiple ways, which also means more acts, but you would have greater satisfaction that departure time you are sending your boss to be more accurate, realistic and grounded in facts.
In case you are wondering when will I introduce the ReAct pattern this article is about, good news is I have partially introduced it already. Human intelligence is the ability to seamlessly combine task oriented actions with verbal reasoning, a kind of inner dialog with yourself as to how best to approach a specific task. The research paper that introduced ReAct pattern used the example of cooking a dish in kitchen, where you have a task at hand, we reason in language to track progress, handle exceptions or adjust the plan according to situation, and realize what external information is needed (like search internet for preparing dough). We may also act like open fridge and check ingredients to support reasoning and answer questions like what dish can I make with available ingredients. Synergy between acting and reasoning allows humans to learn new tasks quickly and perform robust decision making or reasoning even under previously unseen circumstances or facing information uncertainties. The last point is what makes ReAct prompt engineering technique so powerful, that model is able to ‘react’ or adjust to novel situations or exceptions and adjust the plan according to new information.
You can consider LLM to be like a brain in a box which can think independently, has all the knowledge to complete the task, but no organs to accomplish the task. To visualize, consider you can think but have no body. How would you prepare the meal now? You can think what you need to do and the steps, but how would you know if the water is hot, or you don’t know how to prepare dough? What would you do? Well you would still imagine stuff aka hallucinate about it. Now consider we do give it a bunch of sensors and motor controlled limbs to operate. Now you would adjust your plan accordingly and use sensor inputs to guide motor actions. Your plan still needs to figure which motor or sensor to use at what time. This would create a think, act, observe sequence interleaved with your chain of thought, reasoning path, or action plan. This is what ReAct pattern is.
The paper provides a beautiful comparison between think only, act only and ReAct (reason plus act) approach with two different examples. Please pause and have a look at these and admire how the reason plus act approach beautifully approaches the problem at hand
WE have seen chain of thought reasoning already for arithmetic, commonsense and symbolic reasoning tasks, but it is limited to thoughts and void of real world or external world information which limits its ability to reason reactively or update its knowledge. This leads to fact hallucinations and error propagation over reasoning process
React fixes that, ti a general paradigm to combine rasoning and acing with language models for solving diverse language reasoning and decision making tasks. It prompts llm to generate both verbal reasoning traces and actions pertaining to a task in an interleaved manner allowing odel to perform dynamic reasoning to create, maintain and adjust high level plans for acting (reason sto act) while also interact with external environments (like Wikipedia) to incorporate additiona information into reasoning (act to reason)
Most effective approach is combining ReAct with CoT allowing use of both internal knowledge and externally obtained information during reasoning.
The best thing about ReAct is its intuitive and very easy to design where humans can lay down their thoughts in language on top of their actions taken. It is general and flexible and applicable to a variety of scenarios with varying reasoning needs including but not limited to QA, fact verification, text game and web navigation. It is performant and robust and lasty, it is human aligned and controllable since it uses opaque or visible reasoning which can be interpreted and adjusted as necessary.
Now that we are familiar with ReAct, Lets go back to our earlier example where you were helping out LLM for external operations. Let’s say we want to move you out of the picture and automate the process.
We do so by prompting the LLM say ChatGPT with following addition to original prompt
if you are
asked a question that either requires knowledge that you don’t currently possess or events that happened after training cutoff date
or you need to send an email
or you need to query schedule of trains today,
or you need to book a seat on a train going from source city to destination city,
you will do the following
You will run in a loop of Thought, Request, PAUSE, Result and Answer
Use thought to describe your thoughts about the question you have been asked
Use Request to choose one of the external systems available to you and compose a Request, then return PAUSE
I will run the request for you and prompt with the results of executing the request
Answer will be your final answer based on the request
Your available external systems are
schedule: e.g. schedule Src>>Dest
Returns a list of trains going from source city to destination city today
wikipedia: e.g. Wikipedia: GPT
Returns a summary from searching Wikipedia
Booking: e.g. booking:<
Books a seat in train going from Src to Dest city at given time. Returns booking id
Email: e.g. email: {to:
Sends an email to recipient with given subject and body. Returns sent or failed
Now you can prompt the LLM or ChatGPT with the same question about figuring out time for departure with added instructions to book a seat and send email to your boss. This gives you an excellent dry run of when and how it would use external tools for problem solving. Of course it is relying on you for such operations so you can feed it back fake data like a made up train schedule or booking ids, but you would be able to see how it uses ReAct to creatively use these tools. Note also that if you ask it a question that does not require use of external tools like Wikipedia as it can answer with its existing knowledge, it would not request you to run unnecessary queries.
Additionally note that instead of prompt like above, you can also teach or train ChatGPT by examples where we first give it an example of how to think action result for the task where it is taught to use external tools to find information it does not have. Then you give it a similar task and prompt it to think. And it should follow along.
Note in result, it would give disclaimers that it is a language model unable to take actual actions so relying on hypothetical results for now, but it can be taught to use real tools too. It would pause, and then wait for results to be fed in.
It’s the thought that counts. Important bit is, it is applying its training to figure ‘when’ to use external tools to get information it can’t otherwise. It is problem solving the AI way where it figures out the best course of actions or a plan to execute to get the task done, using the available external options it has to accomplish the task and filling the gaps.
The approach, called ReAct, also fixes hallucination problem since LLM does not need to ‘fill in’ using bogus information, but has ability to figure the information out using the provided tools.
Here ReAct allows to retrieve information to support reasoning say from Wikipedia, while also use reasoning to target what to retrieve next, demonstrating a synergy of reasoning and acting.
The problem solving process is more factual and grounded where CoT is more accurate in formulating reasoning structure but can easily suffer from hallucinated facts or thoughts. Best to combine both and let model decide when to switch to the other method based on following
When react fails to return an answer within given steps, back off to CoT-SC.
we want to teach LLM how to think through process and understand steps using other tools to get info , taking actions and observing results But aren’t you tired of working for ChatGPT and running its errands already? And thinking why can’t this be automated so I can do something useful with my life?. Well you are in luck and It is time to introduce ChatGPT Plugins. The term plugins in context of ChatGPT exactly means external tools. They are code snippets that enable ChatGPT to interact with external world using their API. Please read more about plugins on openai website. Note it is exactly the opposite of integrating chatgpt via API in your code as plugins are controlled ‘BY’ ChatGPT, whereas you control ChatGPT using its API.
Since the article is already too long, I won’t be going into details of how to integrate plugins and you can read from the link in description where OpenAI describes how to install and activate them. Additionally note that plugins are not available in free version of ChatGPT and are an option for paid version only. At this time, you can use three plugins at a given time. Just like we discussed and quoting directly from the plugins page, “When a user asks a relevant question, the model may choose to invoke an API call from your plugin if it seems relevant”. ChatGPT leverages the description from the Plugin Manifest to decide when a plugin should be used in response to a prompt. But you haven’t watched the ReAct pattern to be dependent on ChatGPT for using a plugin. You can instruct it to use a particular plugin say for web query or hotel booking or making your prompt perfect, which by the way is name of a very popular plugin. You can even write a plugin of your own, again something we will not cover in this article. There is a big and growing marketplace for plugins so please be sure to explore it.
https://openai.com/blog/chatgpt-plugins
There is a big and growing library of plugins available which can be used to accomplish all the tasks that we discussed earlier as hypothetical examples.
LLMs have been employed more and more as a policy model for decision making specially in interactive environments. ReAct also provides interpretable decision traces. Complex tasks with large action spaces require more demonstration to learn well which can easily go beyond the input length limit of in context learning.
Note that this approach is not without its issues, and one should no there are no guarantees that ChatGPT would
Choose the right external tool or plugin when necessary (or not when not necessary)
Will compose the right request
Will process the response from external system correctly
And so on
I hope you would be able to utilize ReAct prompt engineering technique in real life with help of this article. If you liked, please clap/share and subscribe to our YouTube channel to be notified of exciting content like this. Thank you.
Next article: Prompt Engineering via Prompt Patterns— Outline Expander Pattern
