nomosartificial / agent-sim Goto Github PK

Often getting people to understand the power or use of a system comes from having good demonstration examples that immediately convey to people why they would want to use the system and how they should use it.

We should probably expand our example library with more prompts that people can run to immediately get good results and see the use of the system. This could be a Jupyter notebook with a reasonable number of diverse worked examples.

Often, you are going to want to determine agent message style and agent message strategy separately.

For instance, you may want to simulate an email exchange, a text exchange or a Slack or Discord conversation between agents and want the agent conversation to reflect the way humans normally write in these contexts. This could reflect where the agents will be deployed or for another reason, such as to make the text more readable for a human reviewer.

One possible solution for this would be to add a style module:

import json

from langchain.schema import SystemMessage, HumanMessage

from agent_sim.util import extract_json
from agent_sim.prompts_library import (
    STYLIST_USER_PROMPT,
    STYLIST_SYSTEM_PROMPT,
)

class Stylist:
    def __init__(self, model, style):
        self.model = model
        self.style = style

    def stylize(self, current_message):
        llm_messages = [
            SystemMessage(content=STYLIST_SYSTEM_PROMPT.format(style=self.style)),
            HumanMessage(content=STYLIST_USER_PROMPT.format(message=current_message))
        ]
        stylized_message = self.model.predict_messages(llm_messages).content
        return stylized_message

Right now, the simulation ends after a predefined number of turns. This can lead to waste where the desired stopping point occurs before the predefined number of turns.

To solve this, we could add a monitor class that would check for a condition at the end of each agent's "turn" in the simulation and break the simulation loop in the event that the condition has been satisfied.

Example:

import json

from langchain.schema import SystemMessage, HumanMessage

from agent_sim.util import extract_json
from agent_sim.prompts_library import (
    MONITOR_USER_PROMPT,
    MONITOR_SYSTEM_PROMPT,
)


class Monitor:
    def __init__(self, model, condition):
        self.model = model
        self.condition = condition

    def check_condition(self, message_history):
        llm_messages = [
            SystemMessage(content=MONITOR_SYSTEM_PROMPT),
            HumanMessage(content=MONITOR_USER_PROMPT.format(messages=message_history, condition=self.condition))
        ]
        response = self.model.predict_messages(llm_messages).content
        print(response)
        json_response = extract_json(self.model, response)
        return json_response['condition']

Agent deployments may do more than just have conversations. They may also access and manipulate resources. Some of these resources may be shared between agents. For instance, we can imagine agents accessing a shared database or toggling controls on a system.

In order to support these kinds of simulations, we could add an environment class that can be handed to agents, either when they are instantiated or, by the simulation, with certain method calls.