Skip to content

Python: [Bug]: ChatMiddleware isn't called when submitting tool results back to the model #4767

@Ashton-Sidhu

Description

@Ashton-Sidhu

Description

What Happened

When using the ChatMiddleware, after a tool gets ran, the resulting messages array gets sent back to the model but doesn't pass through the ChatMiddleware.

  1. First message correctly goes through the middleware.
  2. Model responds with tool call
  3. Tool call results get sent to Model without going through the Chat Middleware

Code Sample

@tool(approval_mode="never_require")
def get_weather(
    location: Annotated[str, "The location to get the weather for."],
) -> str:
    """Get the weather for a given location."""
    conditions = ["sunny", "cloudy", "rainy", "stormy"]
    return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C.

class ExampleMiddleware(ChatMiddleware):
    def __init__(self):
        super().__init__()

    async def process(
        self,
        context: ChatContext,
        call_next: Callable[[], Awaitable[None]],
    ):
        for message in context.messages:
            print(message.to_json())
        
        await call_next()

agent = OpenAIChatClient(model_id="gpt-4o-mini").as_agent(
        name="WeatherAgent",
        instructions="You are a helpful weather agent.",
        middleware=[
            ExampleMiddleware(),
        ],
        tools=get_weather,
    )

query = "What's the weather like in Seattle and then write a haiku about it?"
result = await agent.run(query)

Error Messages / Stack Traces

Package Versions

agent-framework-rc1.4

Python Version

No response

Additional Context

Expected Results

Before messages get sent to a model, they should pass through the ChatMiddleware and not get bypassed.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpythonv1.0Features being tracked for the version 1.0 GA

Type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions