Skip to content

feat(base): add message_tool_names field for per-message tool attribution#74

Merged
hallerite merged 1 commit into
mainfrom
sebastian/tool-names-2026-05-28
May 28, 2026
Merged

feat(base): add message_tool_names field for per-message tool attribution#74
hallerite merged 1 commit into
mainfrom
sebastian/tool-names-2026-05-28

Conversation

@snimu
Copy link
Copy Markdown
Contributor

@snimu snimu commented May 28, 2026

Adds RenderedTokens.message_tool_names: list[str | None] — a sidecar parallel to message_roles that carries the tool function name for each tool-role message in the rendered slice. For each tool message the name is taken from msg["name"] (caller- provided) or recovered by joining msg["tool_call_id"] against any prior assistant's tool_calls[i].function.name in the same list. Tool messages whose issuing assistant lives outside the slice (e.g. on a bridge_to_next_turn call where new_messages covers only the new turn) resolve to None.

Pure metadata: extract_message_tool_names runs independently of the render path, never mutates the caller's messages, and has no effect on the rendered token stream — HF chat-template byte parity is preserved on every renderer. Callers that want the function name to appear in the rendered scaffold (e.g. GPT-OSS Harmony's functions.{name} prefix) continue to attach name themselves before calling render — that responsibility stays with the caller (verifiers does this in _attach_tool_call_names).

Trainers (prime-rl) join this list with message_indices to recover per-token tool attribution — the canonical use case is SFT on tool response bodies of a specific tool while RL acts on assistant tokens.

Wired into every concrete renderer's RenderedTokens(...) construction site (render + bridge_to_next_turn). extract_message_tool_names is exported at package level.

Tests: 5 unit tests covering the case matrix (empty, caller- provided wins, resolves from prior assistant, orphan tool message, non-mutation invariant) + 1 integration test that runs across every renderer in the conftest matrix to catch missed wire-up at any of the ~25 RenderedTokens(...) sites.


Note

Low Risk
Additive metadata and read-only helper; token streams unchanged. Risk is mainly consumers mis-joining with message_indices on partial slices (e.g. bridge-only new_messages).

Overview
Adds RenderedTokens.message_tool_names — a per-message sidecar parallel to message_roles — so trainers can attribute tool-response tokens to a specific function without changing rendered bytes.

extract_message_tool_names in renderers/base.py builds the list: tool messages use msg["name"] when present, otherwise join tool_call_id to the prior assistant’s tool_calls[].function.name in the same message slice; other roles are None. It does not mutate input messages and does not affect tokenization (HF parity for tool messages without name is unchanged).

Every concrete renderer’s render() and bridge_to_next_turn() now sets message_tool_names on RenderedTokens; the helper is exported from renderers. tests/test_message_tool_names.py covers the join cases and a matrix integration test across renderers.

Reviewed by Cursor Bugbot for commit d7c160b. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add message_tool_names field to RenderedTokens for per-message tool attribution

  • Adds extract_message_tool_names(messages) in renderers/base.py that returns a list parallel to the input messages, yielding the tool function name for each tool message and None for non-tool messages. Names are resolved from msg['name'] first, then by matching tool_call_id against prior assistant tool_calls.
  • Adds a message_tool_names: list[str | None] field to the RenderedTokens dataclass, defaulting to an empty list.
  • Populates message_tool_names in all renderer render and render_ids methods across 13 renderer modules.
  • Exports extract_message_tool_names as a top-level symbol from the renderers package.

Macroscope summarized d7c160b.

…bution

Adds ``RenderedTokens.message_tool_names: list[str | None]`` — a
sidecar parallel to ``message_roles`` that carries the tool function
name for each tool-role message in the rendered slice. For each
tool message the name is taken from ``msg["name"]`` (caller-
provided) or recovered by joining ``msg["tool_call_id"]`` against
any prior assistant's ``tool_calls[i].function.name`` in the same
list. Tool messages whose issuing assistant lives outside the slice
(e.g. on a ``bridge_to_next_turn`` call where ``new_messages``
covers only the new turn) resolve to ``None``.

Pure metadata: ``extract_message_tool_names`` runs independently of
the render path, never mutates the caller's messages, and has no
effect on the rendered token stream — HF chat-template byte parity
is preserved on every renderer. Callers that want the function name
to appear in the rendered scaffold (e.g. GPT-OSS Harmony's
``functions.{name}`` prefix) continue to attach ``name`` themselves
before calling ``render`` — that responsibility stays with the
caller (verifiers does this in ``_attach_tool_call_names``).

Trainers (prime-rl) join this list with ``message_indices`` to
recover per-token tool attribution — the canonical use case is SFT
on tool response bodies of a specific tool while RL acts on
assistant tokens.

Wired into every concrete renderer's ``RenderedTokens(...)``
construction site (render + ``bridge_to_next_turn``).
``extract_message_tool_names`` is exported at package level.

Tests: 5 unit tests covering the case matrix (empty, caller-
provided wins, resolves from prior assistant, orphan tool message,
non-mutation invariant) + 1 integration test that runs across
every renderer in the conftest matrix to catch missed wire-up at
any of the ~25 ``RenderedTokens(...)`` sites.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@macroscopeapp
Copy link
Copy Markdown

macroscopeapp Bot commented May 28, 2026

Approvability

Verdict: Approved

Additive metadata field with default value and pure extraction function. Follows established pattern for per-message analytics, doesn't affect render output, and includes comprehensive tests. Author has prior experience with similar features in this codebase.

You can customize Macroscope's approvability policy. Learn more.

@snimu snimu requested a review from hallerite May 28, 2026 16:34
@hallerite hallerite merged commit a11f8d8 into main May 28, 2026
11 checks passed
@hallerite hallerite deleted the sebastian/tool-names-2026-05-28 branch May 28, 2026 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants