Action-aware history summarization

[toubatbrian]

Description

Port livekit/agents PR #5099 into agents-js by making history summarization action-aware. The summarizer now consumes tool call and tool result items, and TaskGroup preserves child task tool execution history so those results can be distilled into summaries. This also adds a reusable llm.formatChatHistory(...) helper for readable chat-context logging.

Changes Made

• rewrote ChatContext._summarize() to split the history over the full chat item stream, render messages/tool calls/tool outputs as XML, and emit XML summary messages
• updated TaskGroup summarization to keep the full chat history needed for action-aware summaries by re-merging completed child task contexts after the standard AgentTask handoff
• added llm.formatChatHistory(chatCtx, options?) to render messages, tool calls, tool outputs, and handoffs into a readable multiline debug/log string
• added unit and integration tests covering function-call-aware summarization, TaskGroup summarization with tool outputs, and the new chat history formatter

Pre-Review Checklist

• Build passes: All builds (lint, typecheck, tests) pass locally
• AI-generated code reviewed: Removed unnecessary comments and ensured code quality
• Changes explained: All changes are properly documented and justified above
• Scope appropriate: All changes relate to the PR title, or explanations provided for why they're included
• Video demo: A small video demo showing changes works as expected and did not break any existing functionality using Agent Playground (if applicable)

Testing

• Automated tests added/updated (if applicable)

• All tests pass

• Make sure both restaurant_agent.ts and realtime_agent.ts work properly (for major changes)

• pnpm build:agents

• pnpm exec vitest run agents/src/llm/chat_context.test.ts examples/src/testing/task_group.test.ts

• pnpm exec vitest run agents/src/llm/utils.test.ts

Additional Notes

This restores the TaskGroup-specific re-merge so action-aware summarization also sees child task tool calls and tool outputs end to end, and adds a small formatter utility for logging/debugging chat histories.

---

Note to reviewers: Please ensure the pre-review checklist is completed before starting your review.

  

[changeset-bot]

🦋 Changeset detected

Latest commit: e075841

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 22 packages

Name	Type
@livekit/agents	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-baseten	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-google	Patch
@livekit/agents-plugin-hedra	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-lemonslice	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-phonic	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-sarvam	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugins-test	Patch
@livekit/agents-plugin-trugen	Patch
@livekit/agents-plugin-xai	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 69c25151bd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Escape XML entities before serializing summary input

_summarize() now tells the model the history is XML, but toXml() interpolates raw message/tool text directly into tags, so any content containing <, &, or </...> will produce malformed XML and can change how the summarizer interprets prior turns. This can silently corrupt summaries for common inputs like HTML/XML tool outputs or user text with angle brackets; escaping XML entities in tag bodies (and attributes) is needed to keep the structure faithful.

Useful? React with 👍 / 👎.

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 6 additional findings.