Summary

The web content provides a tutorial on terminating multi-agent conversations using the Autogen framework, specifically focusing on implementing a termination condition in the CFP Writer agent based on feedback from the CFP Reviewer.

Abstract

This tutorial is the second part of a series exploring multi-agent interactions within the Autogen framework, emphasizing the termination of conversations. It builds upon the initial setup of a multi-agent scenario involving a CFP Writer and a CFP Reviewer, which collaborate to refine a Call for Proposals/Paper (CFP). The article discusses the use of the max_turns parameter to limit conversation rounds and introduces a more dynamic termination method using the is_termination_msg condition. This condition allows the CFP Writer to end the conversation upon receiving a "Looks good" message from the CFP Reviewer, indicating no further significant improvements are needed. The tutorial also references the framework's documentation on chat termination and provides code snippets for implementing the new termination condition. It concludes by suggesting that this method can prevent unnecessary costs from excessive conversation turns and prepares the reader for the next part of the series, which will cover human intervention in agent interactions.

Opinions

The author believes that setting a fixed number of conversation turns (max_turns) is not optimal for all scenarios, as it may lead to unnecessary costs without significant improvements.
The framework designers are acknowledged for providing various methods to terminate conversations, which shows foresight and consideration for practical applications.
The author suggests that a combination of strategies, including both a maximum number of turns and a dynamic termination message, may yield an optimal result in controlling conversation costs and effectiveness.
The article implies that the reader should be cautious about allowing conversations to continue indefinitely until a termination message is received, as this could also lead to high costs.
The author expresses enthusiasm about the minimal code changes required to implement the new termination condition, indicating ease of use and flexibility of the Autogen framework.
The conclusion anticipates the next part of the series with excitement, hinting at the value of incorporating human oversight in multi-agent systems.

Tutorial : Multi-Agent interactions with Autogen and Gemini — Part 2 : Terminating Conversations

This part is a continuation of exploring the basics of getting Multi-Agent conversations going with Autogen framework.

The series:

Part 1 : Introduction and Getting Started
Part 2 : Terminating Conversations (this post)
Part 3 : Human in the Loop : How to introduce manual human intervention

In the first part of the series, we setup a Multi-Agent Scenario that had two agents, a CFP Writer and a CFP Reviwer. Both of them collaborated with each other to improve a Call for Proposals/Paper (CFP) iteratively. The CFP Reviewer will review the proposal and offer suggestions to improve. The CFP Writer will then look at those suggestions and do its best to incorporate that. Behind the scenes in the Autogen framework, we employed Google Gemini models.

The source code for the original interaction is available over here:

autogen-experiments/app1 at main · rominirani/autogen-experiments

Experiments with Autogen Framework (https://github.com/microsoft/autogen) - autogen-experiments/app1 at main ·…

github.com

The key thing to note in this interaction is in the last line of the main file ( multi-agent-interaction.py. I reproduce the key snippet of code over here:

cfp_writer.initiate_chat(
      cfp_reviewer,
      message="Following is a rough idea for which I would like a talk proposal that I can submit: \n" 
               + initial_cfp_generation_reply["content"], max_turns=2)

Notice the parameter max_turns, which has a value of 2. This means that the back and forth conversation will continue for a maximum of 2 turns, before it all ends. Do we need to have it for few more turns? We don’t know that. Should it continue indefinitely or to put that better, for a lot of turns? That’s not a good idea either, since the costs could add up and not provide too much of an improvement incrementally.

Terminating Conversations

What if we could terminate the conversation based on some feedback from the CFP Reviewer? If the CFP Reviewer mentions that there is not much to improve and puts in some text in its response, for e.g. Looks good, that could be a trigger to the framework executor to stop the conversation.

It turns out that the framework designers have given this some good thought and there are various ways to do that as documented over here:

Terminating Conversations Between Agents | AutoGen

Open In Colab

microsoft.github.io

In fact, by using the max_turns parameter, we were indeed one of the ways to terminating the conversation i.e. after a certain number of tries. We are now going to look at another mechanism is_termination_msg in one of the agents i.e. CFP Writer, so that if the Agent sees that message in the response received from the CFP Reviewer , then it can terminate the conversation.

Sounds good? The implementation or incremental change in our code to do that is minimal as we will see next.

Add termination condition to CFP Writer

The first change that we are going to do is to add the is_termination_msg condition to the CFP Writer, so that if the Agent sees the text Looks good in the response from the CFP Reviewer, then it can exit the conversation and there shall be no more turns. The modified code for instantiating the CFP Writer is shown below:

cfp_writer = ConversableAgent(
    name="CFP Writer",
    system_message=SYSTEM_MESSAGE_CFP_WRITER,
    llm_config = {"config_list" : config_list_gemini},
    code_execution_config = False,
    human_input_mode = "NEVER",
    function_map = None,
    is_termination_msg=lambda msg: "looks good" in msg["content"].lower(),
)

The key thing to highlight here is the code given below, which checks for the text looks good in the response from the CFP Reviewer.

is_termination_msg=lambda msg: "looks good" in msg["content"].lower()

Now, that we have modified the CFP Writer, we also need to mention in the System Message for the CFP Reviewer, that if there are no significant improvements, then it should mention looks good. So that’s the next change in the code, where we tweak the System Message, as shown below:

SYSTEM_MESSAGE_CFP_REVIEWER = """
Your task is to review a submission for a technical talk.
... <REST OF THE SYSTEM PROMPT>...

If there are no significant changes, please mention "looks good" at the end of the message.
"""

The final change is to drop the max_turns parameter that we had set to “2” in the previous iteration. In other words, we are letting the conversation continue till the CFP Reviewer is going to mention a “looks good”.

Note: It may not be a good idea to let the conversation continue till the text “looks good” is there in the response from the CFP Reviewer. There could be a significant number of conversations going back and forth at times, which might result in a cost too. So look at multiple strategies and/or a combination of both of these to achieve some sort of an optimal result after a certain number of iterations.

The final code is shown below:

If you would like to check out the repository, here you go:

GitHub - rominirani/autogen-experiments: Experiments with Autogen Framework…

Experiments with Autogen Framework (https://github.com/microsoft/autogen) - GitHub - rominirani/autogen-experiments…

github.com

Conclusion

Hope this part builds upon the previous one and shows a different way of terminating a conversation between two agents in the Autogen framework. In the next part of the series, we will look at how we can design a Human Agent in the loop, such that the Human Agent (us) can interact with various other Autonomous Agents, possibly review their work, ask them to do it again and more. Stay tuned.