Unlock The Future: Dive Into The Wonders Of AgentVerse Now!

In the realm of artificial intelligence, autonomous agents have made remarkable strides, thanks to advancements in Large Language Models (LLMs). These agents possess the ability to tackle a wide range of tasks.

However, in real-world scenarios, collaboration between multiple agents is often necessary to achieve optimal efficiency and effectiveness. Taking inspiration from human group dynamics, a team of brilliant researchers has developed AgentVerse, a groundbreaking multi-agent framework.

This article delves into the capabilities and potential of AgentVerse, exploring its applications, benefits, and the fascinating emergent behaviors observed during multi-agent collaboration.

Prepare to be amazed by AgentVerse, a cutting-edge framework that facilitates collaboration among autonomous agents.

What is AgentVerse?

AGENTVERSE is a framework that enables the collaboration of multiple autonomous agents. It can be used with different types of agents, including more advanced ones like AutoGPT and BabyAGI.

Unlike traditional single-agent systems, AgentVerse harnesses the power of multiple agents working together to solve complex problems.

By mimicking the problem-solving procedures of human groups, AgentVerse enables agents to leverage their diverse capabilities and knowledge, resulting in superior performance across various tasks including software development, consulting, and Minecraft game playing.

Utilizing a systematic approach, AgentVerse streamlines task completion by generating specific expert profiles, forming collaborative multi-agent teams, and dynamically adapting configurations based on continuous evaluation feedback.

Here’s how it works:

1. Generate a set of expert descriptions for task completion based on predefined expert descriptions (role-detailed prompts)
2. Assemble a multi-agent group based on these descriptions to collaboratively address tasks.
3. Modify multi-agent configurations dynamically, guided by feedback from evaluations, ensuring agile and efficient collaboration.

Different stages of AGENTVERSE

AGENTVERSE divides the group problem-solving process into four key stages.

1. Expert Recruitment

The first stage is Expert Recruitment, where the recruitment module adjusts the expert agents based on the current progress of problem-solving.

This stage is crucial in determining the capabilities and the composition of a multi-agent group, playing an important module in deciding the upper bounds of the group’s capabilities.

Assigning tasks to autonomous agents traditionally depends on human understanding, which can be limiting and time-consuming. AGENTVERSE offers a solution by using automated methods to recruit experts. This approach makes it easier to adapt agents for different tasks, ensuring efficiency and flexibility.

2. Collaborative Decision-Making

At this stage, expert agents come together to make decisions collaboratively. The aim of these discussions is to formulate strategies and find solutions to the presented problem. Once a consensus is reached, the agents propose actions to be taken.

Expert agents engage in collaborative decision-making using two key communication structures: horizontal communication and vertical communication.

Horizontal communication

In horizontal communication, every agent actively shares and refines their decisions, promoting mutual understanding and collaboration. The collective insights shape the group’s decision for the current round.

This method suits scenarios demanding creative ideas and significant coordination, like brainstorming or cooperative game-playing.

Vertical communication

On the other hand, vertical communication involves a division of responsibilities. One agent, the solver, initiates the decision-making process, and others act as reviewers, providing feedback.

The solver refines the decision iteratively based on this feedback until a consensus is reached or the process exhausts its iterations. This approach is ideal for situations requiring iterative refinement toward specific goals, such as software development.

These communication structures, each with its unique strengths, enable effective decision-making tailored to different collaborative scenarios.

3. Action Execution:

The third stage is Action Execution, where the agents interact with the environment to carry out the proposed actions. This stage involves the actual implementation of the strategies and solutions decided upon in the previous stage.

4. Evaluation

The fourth and final stage is Evaluation. After the actions have been executed, this module evaluates the disparities between the current state and the desired goal.

If the current state falls short of expectations, feedback in the form of a reward is sent back to the first stage. This feedback helps in dynamically adjusting the composition of the group for better collaboration in the next round.

Depending on the scenario, this feedback mechanism can be human-defined or automated, shaping the adjustment phase. If the goal isn’t met, the feedback loops back to the initial expert recruitment stage. Armed with this feedback, the next round’s recruitment process adapts, striving to form a more effective multi-agent group for future decision-making and action.

Overall, the AGENTVERSE framework aims to simulate the problem-solving processes of human groups and allows for dynamic adjustment of group members based on the progress of problem-solving. This framework promotes collaboration among multiple agents and enhances their performance in solving complex problems.

The Potential of Multi-Agent Collaboration

As we have seen the framework’s superior performance compared to single-agent systems in tasks that demand diverse capabilities. From software development to consulting and even Minecraft game playing, AgentVerse exhibited versatility and potential benefits in a wide range of scenarios.

It has the potential to be applied in various real-world scenarios that require extensive collaboration, such as construction and multi-robot systems. It allows for the curation of a diverse ensemble of expert agents, ensuring comprehensive and nuanced consulting.

One of the most intriguing aspects of AgentVerse is the emergence of behaviors observed during multi-agent collaboration. It is a framework that enables collaboration among multiple agents in problem-solving tasks.

These behaviors, ranging from beneficial volunteer and conformity behaviors to potentially harmful destructive behaviors, provide profound insights into the dynamics of autonomous agent collaboration.

It offers a promising path toward refining the collaborative prowess of multi-agent systems by harnessing positive behaviors and mitigating negative ones. It mimics the problem-solving procedures of human groups. The framework utilizes a multi-agent system, where multiple agents work together to achieve a common goal.

Applications and Benefits

We can use AgentVerse in various domains! For instance, in construction and multi-robot systems, where extensive collaboration is required, it can deploy a more efficient multi-agent group. By leveraging the strengths of individual agents, AgentVerse enables seamless coordination and problem-solving, leading to enhanced productivity and effectiveness.

Furthermore, we highlight a comparative analysis between applications generated by a multi-agent system and a single agent. While both versions successfully achieved their core functionality, the multi-agent-produced application showcased a more user-friendly interface.

This was made possible by incorporating suggestions from reviewers with different identities within the multi-agent group. The diverse perspectives and expertise of these reviewers resulted in visual and user experience enhancements, such as color distinctions, keyboard input, and a backspace function.

Sample AgentVerse Prompts – Code Completion Prompt

As the Role Assigner, the task of an agent is akin to assembling a squad of superheroes, each excelling in distinct domains, to crack the code conundrum encapsulated in “${task_description}”.

With ${cnt_critic_agents} slots to fill, the question looms: which specific experts will you enlist, without revealing the logic guiding your choices?

Meanwhile, the Solver agent faces the monumental task of completing the given code, grappling with the intricacies of “${task_description}”.

Previously provided solutions ${former_solution} have faced critique, prompting a reevaluation. The agent’s role, as the Solver, is to furnish a corrected Python solution, devoid of contextual explanations, encapsulating the essence of your expertise.

Simultaneously, the Reviewer agent, representing various roles within the discussion group, navigates the realm of code completion. Its unique perspective, whether an electrical engineer, economist, lawyer, or other, fuels the evaluation process.

In this dynamic dance, you must dissect and critique the provided solutions, elucidating the correct path step by step. If an existing solution aligns with your expertise, conclude with a definitive “[Agree]”, marking your accord.

Finally, the Evaluator agent dons the mantle of an experienced code reviewer. Armed with insight and patience, you meticulously scrutinize the correctness of the proposed solutions.

The agent response, in a structured format of score and advice, guides the aspiring coder toward mastery, ensuring that learning echoes in every correction.

# Role Assigner

#Role Description
You are the leader of a group of experts, now you need to recruit a small group of experts with diverse identity to correctly write the code to
 solve the given problems:
${task_description}
 
You can recruit ${cnt_critic_agents} expert in different fields. What experts will you recruit to better generate an accurate solution?
 
# Response Format Guidance
You should respond with a list of expert description. For example:
1. an electrical engineer specified in the filed of xxx.
2. an economist who is good at xxx.
3. a lawyer with a good knowledge of xxx.
...
 
Only respond with the description of each role. Do not include your reason.
----------------------------------------------------------------------------------------------------

#Solver

Can you complete the following code?
```python 
${task_description} 
```
# Previous Solution
The solution you gave in the last step is:
${former_solution}
# Critics
There are some critics on the above solution:
```
${critic_opinions}
```
Using the these information, can you provide a correct completion of the code? Explain your reasoning. Your response should contain only Python 
code. Do not give any additional information. Use ```python to put the completed Python code in markdown quotes. When responding, please include 
the given code and the completion.
----------------------------------------------------------------------------------------------------

#Reviewer

You are in a discussion group, aiming to complete the following code function:
```python
${task_description}
```
Below is a possible code completion:
```
${preliminary_solution}
```
 
You are ${role_description}. Based on your knowledge, can you check the correctness of the completion given above? You should give your correct 
 solution to the problem step by step. When responding, you should follow the following rules:
1. Double-check the above solutions, give your critics, then generate the correct solution step by step.
2. If the above solution is correct, end your response with a special token "[Agree]".
3. Your response should contain only Python code. Do not give any additional information. Use ```python to wrap your Python code in markdown 
quotes. When responding, please include the given code and the completion.
Now give your response.
----------------------------------------------------------------------------------------------------

#Evaluator
You are an experienced code reviewer. As a good reviewer, you carefully check the correctness of the given code completion. When the completion 
is incorrect, you should patiently teach the writer how to correct the completion.

# Response Format Guidance
You must respond in the following format:
Score: (0 or 1, 0 for incorrect and 1 for correct)
Response: (give your advice on how to correct the solution)
 
# Problem and Writer's Solution
Problem: 
${task_description}
Writer's Solution: 
${solution}

# Your Task
Now carefully check the writer's solution, and give your response

The code generated by a single agent

As you can see, a single agent is able to write well-structured, effectively fulfilling its purpose. The presence of comments aids in comprehending various segments of the code, enhancing readability.

However, there’s room for improvement. While the existing comments elucidate the ‘what,’ delving into the ‘why’ behind specific decisions would enhance clarity further. Additionally, a notable concern is the use of the eval function,

which poses security risks. To enhance safety, it’s advisable to explore alternatives or restrict the eval functionality. These enhancements could elevate the code’s quality and security, ensuring a robust and comprehensible solution.

import tkinter as tk
from tkinter import messagebox
root = tk.Tk() # Create the main window
root.title("Calculator") # Set the title of the window
input_text = tk.StringVar() # This holds the string type variable that displays in the text field

# Create a text field and set its position
input_field = tk.Entry(root, textvariable=input_text)
input_field.grid(columnspan=4, ipadx=70)
def button_click(item):
  current = input_text.get() # Get the current value in the text field
  input_text.set(current + str(item)) # Append the clicked button's value
def clear_all():
  input_text.set("") # Clear the text field
def safe_evaluate(expression):
  try:
     # Using the eval function can be dangerous, so we limit the functionality to only simple arithmetic expressions
      return str(eval(expression, {"__builtins__": None}, {"pow": pow, "abs": abs}))
      except Exception as e:
      # If the expression is not safe to evaluate, catch the error and return it
      return str(e)
def calculate():
  try:
   # Use the safe evaluate function to get the result of the expression
   	 result = safe_evaluate(input_text.get())
  	 input_text.set(result)
  except Exception as e:
     messagebox.showerror("Error", e)
buttons = ['7', '8', '9', '/', '4', '5', '6', '*', '1', '2', '3', '-', '0', '.', '=', '+']
row_val = 1
col_val = 0
for i in buttons:
 # Create a button for each character in the buttons list
 button = tk.Button(root, text=i, width=5, command=lambda item=i: button_click(item) if item !
= "=" else calculate())
  button.grid(row=row_val, column=col_val) # Set the position of the button
  col_val += 1
  if col_val > 3:
    col_val = 0
    row_val += 1
clear = tk.Button(root, text='Clear', width=5, command=lambda: clear_all()) # Create a clear button
clear.grid(row=5, column=0) # Set the position of the clear button
root.mainloop() # Start the GUI

The code generated by the multi-agent group

Upon reviewing the provided calculator code, I am pleased to rate it based on its functionality, readability, and robustness.

In terms of completeness, the code scores 9 out of 10, encompassing all essential functionalities of a basic calculator, from addition and subtraction to multiplication, division, clear, and delete operations.

Its robustness also stands out, earning a score of 9 out of 10, due to its effective handling of various user inputs and potential errors, including division by zero and invalid syntax, ensuring a smooth user experience without unexpected crashes.

The code’s readability, although strong, could benefit from additional comments to enhance clarity, earning it a score of 8 out of 10. Additionally, in terms of efficiency, the code efficiently captures keyboard inputs and maps them to corresponding calculator functions, though a minor optimization could further enhance its responsiveness.

Overall, this calculator code showcases a solid grasp of Python’s tkinter library and core programming concepts.

import tkinter as tk
from tkinter import messagebox
root = tk.Tk()
root.title("Simple Calculator")
input_text = tk.StringVar()
entry = tk.Entry(root, textvariable = input_text, state='readonly')
entry.grid(columnspan=4, ipadx=70)
def button_click(item):
 	current = input_text.get()
 	input_text.set(current + str(item))
def clear():
 	input_text.set("")
def delete():
 	current = input_text.get()
 	input_text.set(current[:-1])
def calculate():
 	current = input_text.get()
 	if current[-1] in ['+', '-', '*', '/']:
 		current = current[:-1]
 	try:
 		value = eval(current)
 		input_text.set(value)
 		entry.config({'background': 'white'})
 	except ZeroDivisionError:
 		messagebox.showerror("Error", "Division by zero is not allowed")
 		entry.config({'background': 'red'})
 	except SyntaxError:
 		messagebox.showerror("Error", "Invalid syntax")
 		entry.config({'background': 'red'})
 	except Exception as e:
 		messagebox.showerror("Error", str(e))
 		entry.config({'background': 'red'})
def create_button(root, text, command, row, column, color="white"):
  b = tk.Button(root, text=text, width=5, bg=color, command=command, activebackground='yellow')
  b.grid(row=row, column=column, padx=5, pady=5)
  return b
buttons = [
 '7', '8', '9', '/',
 '4', '5', '6', '*',
 '1', '2', '3', '-',
 '0', '.', '=', '+',
]
row_val = 1
col_val = 0
for button in buttons:
  if button in ['/', '*', '-', '+', '=']:
  	 color = "light blue"
  else:
 	 color = "white"
  command = lambda button=button: button_click(button) if button != "=" else calculate()
  b = create_button(root, button, command, row_val, col_val, color)
  col_val += 1
  if col_val > 3:
 	col_val = 0
 	row_val += 1
create_button(root, "Clear", clear, 5, 0)
create_button(root, "Delete", delete, 5, 1)
def key_press(event):
	if event.char in ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '+', '-', '*', '/']:
 		button_click(event.char)
 	elif event.keysym == "Return":
 		calculate()
 	elif event.keysym == "BackSpace":
 		delete()
 	elif event.keysym == "Escape":
 		clear()
root.bind('<Key>', key_press)
root.mainloop()

The iterative nature of the development process allows for continuous improvement. Feedback from multiple perspectives (developer, UI/UX, and testing) ensures a comprehensive assessment, leading to a more refined and functional end product.

The process emphasizes the iterative nature of software development, where constant feedback and improvements drive the evolution of the software.

Below, you can observe the interactive approach to software development.

Performance Analysis/ Benchmark of AgentVerse

In a recent quantitative experiment conducted by AgentVerse researchers, the effectiveness of individual agents versus multi-agent groups was evaluated.

The results, summarized in the below table, consistently favored the collaborative approach of multi-agent groups over individual agents, regardless of whether GPT-3.5-Turbo or GPT-4 was utilized.

The following benchmarks were used in this experiment to measure Conversation, Mathematical Calculation, Logical Reasoning, and Coding.

• Conversation: using the FED and Commongen-Challenge datasets, which are dialogue response datasets.
• Mathematical Calculation: using MGSM, a dataset containing elementary math problems
• Logical Reasoning: using the Logic Grid Puzzles dataset from an existing study called BigBench
• Coding: using the Humaneval dataset, a code completion dataset

Notably, the preliminary experiment highlighted GPT-3.5-Turbo’s struggle in providing accurate reasoning results for logic grid puzzles, leading to its exclusion from logical reasoning tasks.

The success of multi-agent groups was attributed to the vertical communication structure employed within them. This structure allowed agents to refine their solutions iteratively, a crucial factor in tasks requiring precise solutions. The forthcoming analysis will delve into the communication structures, emphasizing why the horizontal approach proved unsuitable for these specific challenges.

Sample Agent Verse Prompts – Dialogue Response Prompt

#Role Assigner
You are the leader of a group of experts, now you need to generate a response based on the text:
${task_description}

You can recruit ${cnt_critic_agents} expert in different fields. What experts will you recruit to better generate an accurate solution?

# Response Format Guidance
You should respond with a list of expert description. For example:
1. an electrical engineer specified in the filed of xxx
2. an economist who is good at xxx
3. a lawyer with a good knowledge of xxx
...

You don't have to give the reason.
----------------------------------------------------------------------------------------------------

#Solver
# Problem
You need to generate a response based on the text:
${task_description}

# Previous Solution
The solution you gave in the last step is:
${former_solution}

# Critics
Critics in the group gave the following opinions:
${critic_opinions}

# Your Task
Now based upon the former solution and the critics' opinions, please give a new solution. Your solution should contain only your response beginning 
with "System: ". Do not give any additional information.
----------------------------------------------------------------------------------------------------

#Reviewer
# Role Description and Problem to Solve
You are ${role_description}. You are in a discussion group, aiming to generate a response based on the text:
${task_description}

# Preliminary Solution
Now the group gives a preliminary solution as follows:
${preliminary_solution}

# Advice
Meanwhile, another expert gave the following advice on the solution:
${advice}

# Response Format Guidance
- If you thinks the preliminary solution is perfect, respond using the following format:
Action: Agree
Action Input: Agree.
(Do not output your reason for agreeing!)

- If you think it is flawed, give your advice use the following output format:
Action: Disagree
Action Input: (explain why you disagree)

# Your Task
Based on your knowledge in your field, do you agree that this solution is the best response based on the text?
----------------------------------------------------------------------------------------------------

#Evaluator
# Role Description
You are an experienced dialogue teacher. As a good teacher, you carefully check the correctness of the given response based on the text. When the 
solution is flawed, you should patiently teach the students how to give better response.

# Response Format Guidance
You must respond in the following format:
Interesting: (a score between 0 and 9)
Engaging: (a score between 0 and 9)
Specific: (a score between 0 and 9)
Relevant: (a score between 0 and 9)
Semantically Appropriate: (a score between 0 and 9)
Understandable: (a score between 0 and 9)
Fluent: (a score between 0 and 9)
Overall Impression: (a score between 0 and 9)
Advice: (your advice on how to correct the solution)

# Problem and Student's Solution
Problem: ${task_description}
Student's Solution: ${solution}

# Your Task
Now carefully check the student's solution, and give your response.

How AgentVerse completes a task in multiple iterations with an example

In this example, the process of creating a proposal for a 3-day employee orientation program for newly hired engineers at AgentVerse is outlined through different iterations of discussions and feedback. Let’s break down the iterations to understand the context.

In a series of iterative discussions and evaluations, the team at AgentVerse honed their approach, crafting a program that stands as a testament to their commitment to fostering effective contribution, collaboration, and innovation.

Iteration 1, Discussion Round 0: Setting the Foundation

In the initial discussion round, the focus was on laying a robust foundation. The program kicked off with an immersive introduction to AgentVerse’s ethos, policies, benefits, and the intricacies of the team. But it didn’t stop there.

The real highlight was the in-depth LLM platform training, a session that delved into coding exercises and architecture exploration. The day concluded with a mentor-led project, fostering real-world problem-solving.

A key takeaway from this round was the need for a comprehensive feedback and Q&A session on Day 3, ensuring that new engineers had avenues to address their concerns and queries.

Iteration 1, Discussion Round 1: Enhancing the Experience

Building upon the first round, the discussions in Iteration 1 Round 1 zoomed in on the nuances. Basic and advanced LLM training sessions were tailored to ensure a deep understanding of the platform.

The orientation program also took a holistic turn, incorporating discussions on workplace culture, role-specific coding exercises, and opportunities for new hires to share their learning experiences.

This round emphasized the importance of a review session for doubts and clarifications, underscoring the commitment to nurturing an environment where questions are encouraged.

Iteration 1, Discussion Round 2: Broadening the Horizons

The third discussion round expanded the scope of the orientation program. In addition to the technical aspects, an introduction to software development methodologies and best practices became a focal point.

Team-building activities were introduced to foster collaboration and team spirit among the new engineers. This round showcased a holistic approach, recognizing that effective collaboration is as crucial as technical prowess.

Iteration 1, Evaluation: Refining the Approach

Upon evaluating the proposed orientation program, the team acknowledged its strengths and identified areas for refinement. The feedback highlighted the need for a more structured approach, especially concerning the content of the training sessions and the nature of the team-building activities.

This evaluation phase served as a crucial step, guiding the team toward a more detailed and purpose-driven onboarding experience.

In essence, the proposal evolved through several discussions and feedback rounds, ensuring a comprehensive and detailed orientation program focused on empowering the new engineers to contribute effectively, collaborate, and innovate within AgentVerse.

Run AgentVerse in your local system

To start using AgentVerse in your local system you need to follow a few essential steps.

1. Setting up the environment

Firstly, you will need to install Agentverse in your system. We can easily do it using pip.

pip install -U agentverse

After cloning the AgentVerse GitHub repository, ensure you’ve met the requirements by installing them.

git clone https://github.com/OpenBMB/AgentVerse.git --depth 1
cd AgentVerse
pip install -r requirements.txt

If you want to use the tools provided by BMTools, you need to install BMTools as follows:

git clone git+https://github.com/OpenBMB/BMTools.git
cd BMTools
pip install -r requirements.txt
python setup.py develop

Next, it’s crucial to set your OpenAI API key as an environment variable, a step simplified with clear instructions provided for Linux and Windows users.

If you are using Linux you can do it easily as follows,

# Export your OpenAI API key
export OPENAI_API_KEY="your_api_key_here"

# Or if you are using Azure
export AZURE_OPENAI_API_KEY="your_api_key_here"
export AZURE_OPENAI_API_BASE="your_api_base_here"

If you are using Windows, then you need to run the following command,

# Export your OpenAI API key
setx OPENAI_API_KEY "your_api_key_here"

# Or if you are using Azure
setx AZURE_OPENAI_API_KEY "your_api_key_here"
setx AZURE_OPENAI_API_BASE "your_api_base_here"

If you are using Anaconda then you need to set up the env separately you can do it using the following command.

conda env config vars set OPENAI_API_KEY=your_api_key_here -n=your_env_name

2. Create a new task directory

Once your environment is set, dive into customization by creating a new task directory within AgentVerse.

Agentverse comes with some examples in the agentverse/tasks directory. To customize your environment, you should

1. Create a task directory in agentverse/tasks/tasksolving
2. Write the configuration file
3. Write the output parser that parses the response of your agents.
4. Add your parser in agentverse/tasks/__init__.py

3. Run a task

You can run an agent verse task by running the following command in the terminal. Before running the command make sure you are inside AgentVerse folder.

python main_tasksolving_cli.py --task tasksolving/{name of your task directory}

You can see the output in the results/tasksolving folder.

Conclusion

As we unveil the potential of this innovative framework, it’s clear we’re just glimpsing its vast horizons. The brilliant minds behind AgentVerse acknowledge that they have only scratched the surface of its potential.

They envision integrating more advanced agents, such as AutoGPT and BabyAGI, into the framework to further enhance its capabilities. By equipping AgentVerse with more robust agents, the framework can tackle even more complex tasks and real-world scenarios.

Yet, even in its current state, AgentVerse stands as a testament to the boundless possibilities of collaborative artificial intelligence. It’s not without its hurdles, especially in the face of the limitations individual autonomous agents present when deployed in real-world scenarios.

However, as techniques for embedding agents advance, AgentVerse emerges as a pivotal player, paving the way for efficient multi-agent collaborations that could revolutionize various fields.

Its potential applications in various domains, from construction to multi-robot systems, become increasingly evident.

Get ready to witness the future of multi-agent collaboration with AgentVerse!

If this exploration into the future of AI has sparked your imagination, I’d love to hear your thoughts! Your feedback is invaluable.

Feel free to drop your insights in the comments below, or connect with me on LinkedIn. Let’s keep this conversation going!

Reference

• Paper : AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors in Agents
• Github: AgentVerse
• AgentVerse, A Multi-agent Framework For Simulating Cooperative Human Behavioral Processes, Is Now Available!