Tutorial : Multi-Agent interactions with Autogen and Gemini — Part 5 : AgentOps
Hope you have been enjoying the series on Autogen framework so far. We’ve taken a look at getting started, autonomous agents talking to each other, a human talking to an autonomous agent, then we’ve used both Google Gemini models and a local Gemma model, powered by Ollama, as the LLMs for the interactions.
The series so far:
- Part 1 : Introduction and Getting Started
- Part 2 : Terminating Conversations
- Part 3 : Human in the Loop : How to introduce manual human intervention
- Part 4 : Using local LLM’s
- Part 5 : Using AgentOps : Monitoring, Metrics for your Agent (this post)
Metrics, Monitoring for your Agent
So far, we have just written our code, run it and all is well. But the real world demands that we have good control over what is going on with our Agents. By that, it can mean one or more of the following:
- How responsive are our agents?
- If something goes wrong with the Agent, can we understand and debug what happened?
- Can we see multiple back and forth traces between agents and understand what is going on, what messages are being exchanged?
- What if we wanted to replay a session or two between Agents again?
The above is not an exhaustive list and a monitoring platform should provide a lot more metrics with the ability to possibly export metrics and dashboards to integrate.
AgentOps.ai platform
Enter Agent Monitoring and Metrics, a very crucial requirement of any framework. Ideally, I would have preferred that some of these metrics are built within the framework and there is the Autogen SQLite and File Logger, but I thought of exploring the Observability integration with AgentOps.ai.
The documentation for the integration is available over here.
The steps to integrate with the platform are straightforward:
- Get an account at AgentOps.ai
- Integrate the AgentOps Python library and sprinkle in some integration code.
Step 1 : Get an account at AgentOps.ai and obtain API Keys
Visit AgentOps.ai and sign up for the service. The free tier will do for now.
Once you are logged in, visit the main menu on the left and at the bottom, you will find the API Keys section as shown below:
Click on that, create a new project and you will have the API Key. Copy the API Key for now since we shall use that in our code.
Step 2 : Integrate AgentOps Python library
We are going to create a new project to test out the interaction between two autonomous agents : Quote Writer and Quote Reviewer.
We will modify our existing requirements.txt file to add in the agentops module too as shown below:
utogen-agentchat[llm,gemini]~=0.2
agentopsCreate the Python environment with the above requirements.txt file and you are all set to run the main program as shown below.
The main program is available in the file multi-agent-101.py and the code is shown below:
from autogen import ConversableAgent
import autogen
import agentops
config_list_gemini = autogen.config_list_from_json("model_config.json")
agentops.init(api_key="YOUR AGENTOPS API KEY",
default_tags=["my-multi-agent-101-tag"])
quote_writer = ConversableAgent(
name="Quote Generator",
system_message="Your task is to provide famous quotes",
llm_config = {"config_list" : config_list_gemini},
code_execution_config = False,
human_input_mode = "NEVER",
)
quote_reviewer = ConversableAgent(
name="Joke Reviewer",
system_message="Your task is to review a famous quote and provide your comments briefly in a list.",
llm_config = {"config_list" : config_list_gemini},
code_execution_config = False,
human_input_mode = "NEVER",
)
reply = quote_writer.generate_reply(messages=[{"content": "Give me a quote by Albert Einstein", "role": "user"}])
quote_writer.initiate_chat(quote_reviewer,message=reply['content'], max_turns=2)
agentops.end_session("Success")The code is not very different except for the additional agentops module code. Let’s go through it briefly.
- We are still using the
model_config.jsonfile to initialize the list of Gemini models. This file is the same from some of earlier parts that we have seen. - We initialize the
agentopsmodule via theinitmethod and we are also putting in a tag that we can use in their application to identify our conversation. Note that you will have to use the AgentOps API Key that you got, as instructed in the earlier section. - The
quote_writerandquote_revieweragents are the same. Each of them is an Autonomous agent that is using the ConversableAgent class. One of them generates the quotes and the other reviews them. - We finally get an initial quote and then initiate the chat between the
quote_writerand thequote_reviewer. - We end the
agentopssession at the end.
AgentOps.ai in Action
You can run the application via the following command:
$ python multi-agent-101.pyRun this a few times and then visit https://app.agentops.ai/projects, where you can possibly go to your default project and observe the metrics and more. A few sample screenshots are shown below:
Sample run on the command line:
🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=94154fb7-fe9b-4021-af91-138258199d2f
Quote Generator (to Joke Reviewer):
"Strive not to be a success, but rather to be of value."
--------------------------------------------------------------------------------
Joke Reviewer (to Quote Generator):
Here are some brief comments on the quote "Strive not to be a success, but rather to be of value":
* **Focus on Impact:** This quote encourages a shift in focus from personal achievement to making a positive contribution to others.
* **Intrinsic Motivation:** It suggests that true fulfillment comes from serving a purpose larger than oneself, rather than chasing external validation.
* **Long-Term Value:** Value, unlike fleeting success, can have a lasting impact and leave a legacy beyond individual accomplishment.
* **Defining Value:** The quote doesn't explicitly define "value," leaving room for individual interpretation and finding meaning in their own unique contributions.
* **Practical Application:** While aspirational, the quote can be applied to daily life by considering the impact of actions and decisions on others.
--------------------------------------------------------------------------------
Quote Generator (to Joke Reviewer):
Those are great observations about the quote "Strive not to be a success, but rather to be of value"! You've captured its essence perfectly.
Here are a few additional points to consider:
* **Humility and Service:** The quote promotes a sense of humility, recognizing that true value comes from serving others, not from self-promotion.
* **Authenticity:** It encourages people to pursue their passions and contribute in ways that align with their core values, leading to a more authentic and fulfilling life.
* **Legacy:** The quote implies that leaving a positive mark on the world is more meaningful than simply achieving a certain level of success. It encourages us to think about the long-term impact of our choices.
Overall, this quote offers a powerful message about living a purposeful life that goes beyond personal gain. It inspires us to consider the value we bring to the world and the difference we can make through our actions.
--------------------------------------------------------------------------------
Joke Reviewer (to Quote Generator):
You've added some truly insightful points! I especially love how you highlight the connections between the quote and:
* **Humility and Service:** The quote beautifully underscores that genuine value comes from serving others, not self-aggrandizement.
* **Authenticity:** The pursuit of value encourages us to align our actions with our core values, leading to a more authentic and fulfilling life.
* **Legacy:** The quote beautifully underscores the importance of leaving a positive impact on the world, a legacy that transcends individual achievement.
It's clear this quote resonates deeply, prompting us to consider the impact we have and encouraging us to live with purpose and meaning.
--------------------------------------------------------------------------------
🖇 AgentOps: Session stats - Cost: $0.00 Duration: 1.9s LLMs: 4 Tools: 0 Actions: 20 Errors: 0
🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=94154fb7-fe9b-4021-af91-138258199d2fAgentOps.ai web application
Sample Session Drilldown
Source Code
The source code for this application in available in the app5 folder and is linked here:
Conclusion
Hope this helps you understand the mechanism to incorporate observability into your Agents using the AgentOps.ai platform. In the next part in the series, we will investigate into various communication patterns between agents. Stay tuned.
