Tutorial : Multi-Agent interactions with Autogen and Gemini — Part 3: Introducing Manual Human Intervention

This part is a continuation of exploring the basics of getting Multi-Agent conversations using the Autogen framework.

The series:

• Part 1 : Introduction and Getting Started
• Part 2 : Terminating Conversations
• Part 3 : Human in the Loop : How to introduce manual human intervention (this post)
• Part 4 : Using local LLM’s
• Part 5 : Using AgentOps : Monitoring, Metrics for your Agent

Introducing our Human and Autonomous Agent interaction

So far in the series, we have covered Autonomous Agents and seen how we can make multiple autonomous agents communicate with each other. We then investigated ways for terminating the conversations, either via maximum number of turns or by looking for a specific text/content in the response from one of the agents and then making that as the condition for terminating the conversation.

Let’s move on to another aspect of working with autonomous agents and that involves us, the human. What if we would like a human in the middle or in some sort of a feedback loop. The picture below indicates what exactly I am trying to hint at.

So we have the following scenario:

1. A human asks the Autonomous Agent to do some work.
2. The Agent fulfills the task and then provides its output back to the human.
3. The human reviews the output received from the Agent. If they are satisfied with the output, they will say “ok” and end the conversation. If they are not satisfied with the output, they might have a default reply (maybe do the task again) or they might put in some additional context/instruction and ask the agent to do the task again.
4. So the conversation goes back and forth and it is completely in the hands or control of the human to end the loop.

Enter Autogen’s HumanProxy

Autogen has nice support for bringing in the human in the loop. It is well documented over here and we shall see a simple example of how to make this possible.

Our example is as follows:

1. We (Human) will ask for a quote from an Autonomous Agent. For e.g. “a famous life quote from a Greek Philosopher”. We will initiate the conversation with a ConversableAgent instance as we saw earlier.
2. The Autonomous Agent will take up this up and provide a quote back.
3. The Autogen framework will then prompt us (human) to see the output and then decide to proceed ahead or terminate or even mention to the Agent that its “ok” and that we are happy with the output.

The code is listed below:

Let’s go through the code:

1. We are instantiating a ConversableAgent i.e. human and you can notice that we are setting the parameter for human_input_mode to ALWAYS and the llm_config for this is None. We are also setting an optional parameter named default_auto_reply to a prompt where if there is no response from the human and they just want to Skip (as we will see in the sample run), this default_auto_reply will be sent to the autonomous agent again.
2. We then instantiate our quote_agent and this should be familiar now. The key thing to note is the Gemini model configuration list is passed to the llm_config parameter. Its system_message parameter indicates what this Agent can do i.e. provide life quotes but this could be something of your interest. The human_input_mode is set to NEVER, which is what we have been doing for Autonomous Agents. Finally, the is_termination_msg is to set to ok, so if the human reviews the quote that it provides and says ok, then conversation can end.
3. Finally, the human initiates the chat with the quote_agent via the initiate_chat method.

Simple isn’t it? I like the consistency in this framework and the ease via which you can configure an Agent to be autonomous or a human proxy.

Take a look at one sample run, where as a human input, I am fine with the first quote provided by the agent and end up telling its ok.

python human-autoagent-interaction.py 
flaml.automl is not available. Please install flaml[automl] to enable AutoML functionalities.
Human (to quote_agent):

Provide me a quote that is from one of the Greek philosophers on Life

--------------------------------------------------------------------------------
quote_agent (to Human):

"The unexamined life is not worth living." - **Socrates** 


--------------------------------------------------------------------------------
Replying as Human. Provide feedback to quote_agent. Press enter to skip and use auto-reply, or type 'exit' to end the conversation: ok
Human (to quote_agent):

ok

--------------------------------------------------------------------------------

In the next run, I will simply press enter to skip as the Autogen framework tells me to do. This means I am not happy with the quote and would like the Quote Agent to try again and give me a different quote. Remember the default_auto_reply parameter that we used for the Human ConversableAgent earlier, which was set to Give me another quote will be sent to the agent and the conversation continues. The sample run for this scenario is shown below:

Human (to quote_agent):

Provide me a quote that is from one of the Greek philosophers on Life

--------------------------------------------------------------------------------
quote_agent (to Human):

"The unexamined life is not worth living." - **Socrates** 


--------------------------------------------------------------------------------
Replying as Human. Provide feedback to quote_agent. Press enter to skip and use auto-reply, or type 'exit' to end the conversation: 

>>>>>>>> NO HUMAN INPUT RECEIVED.

>>>>>>>> USING AUTO REPLY...
Human (to quote_agent):

Give me another quote

--------------------------------------------------------------------------------
quote_agent (to Human):

"Everything has beauty, but not everyone sees it." - **Confucius** 


--------------------------------------------------------------------------------
Replying as Human. Provide feedback to quote_agent. Press enter to skip and use auto-reply, or type 'exit' to end the conversation: 

>>>>>>>> NO HUMAN INPUT RECEIVED.

>>>>>>>> USING AUTO REPLY...
Human (to quote_agent):

Give me another quote

--------------------------------------------------------------------------------
quote_agent (to Human):

"The only true wisdom is in knowing you know nothing." - **Socrates** 


--------------------------------------------------------------------------------
Replying as Human. Provide feedback to quote_agent. Press enter to skip and use auto-reply, or type 'exit' to end the conversation: ok
Human (to quote_agent):

ok

--------------------------------------------------------------------------------

All code for this article is available in the repository below in the app3 folder.

Conclusion

This part highlighted how we can mix humans and autonomous Agents in a conversation using the Autogen framework. Hope this was useful. I plan to look at some Conversation Design Patterns, Local Model Execution in upcoming posts using this framework. Stay tuned.