Skip to content

Split/tool call args json for qwen3coder tool calls (Qwen3.5) #4433

Open
lapy wants to merge 3 commits intoInternLM:mainfrom
lapy:split/tool-call-args-json
Open

Split/tool call args json for qwen3coder tool calls (Qwen3.5) #4433
lapy wants to merge 3 commits intoInternLM:mainfrom
lapy:split/tool-call-args-json

Conversation

@lapy
Copy link
Contributor

@lapy lapy commented Mar 19, 2026

Preserve OpenAI tool-call argument strings while unblocking Qwen/HF templates

Description

Summary
Qwen3.5 supports qwen3coder tool calls, but the internal chat template fails to parse some tool call arguments.

Using Qwen3.5 with OpenClaw (for example) will result in many tool call failures because of it.

This PR fixes tool-call argument handling for chat-template preprocessing, especially for Qwen-style Hugging Face templates that expect dict-like tool arguments.

Previously, assistant tool_calls[*].function.arguments could be eagerly converted from a JSON string into a dict during shared preprocessing. That helped some templates render, but it also changed the OpenAI-style message shape for downstream consumers.

This change keeps the OpenAI-compatible payload intact while still letting templates render correctly.

What Changed

  • Added shared preprocessing that parses assistant tool-call argument strings only when they decode to JSON objects.
  • Preserved the original function.arguments string and attached a companion parsed_arguments field instead of mutating arguments in place.
  • Restricted this behavior to assistant tool_calls.
  • Added render-only adaptation for Hugging Face chat templates so Qwen-style templates can still iterate over dict-like arguments without changing the canonical message payload.
  • Threaded the same behavior through multimodal preprocessing paths.
  • Expanded tests to cover:
  • OpenAI-style round-trip behavior where arguments remains a string
  • Qwen template rendering with tool calls
  • non-dict JSON payloads
  • malformed JSON payloads
  • no mutation of the original message object

Why
This gives us both sides of the behavior we want:

  • Qwen/HF templates that use arguments|items continue to work.
  • Downstream OpenAI-compatible consumers still see function.arguments as the original JSON string.

Testing

pytest -q tests/test_lmdeploy/test_content_merge.py
pytest -q tests/test_lmdeploy/test_model.py -k "qwen_template_renders_tool_call_arguments_from_string_payload or content_merge"

- Introduced `_parse_tool_call_arguments_dict` and `_prepare_messages_for_hf_template` functions to manage tool call arguments more effectively.
- Updated `MultimodalProcessor` to use `_attach_parsed_tool_call_arguments`, ensuring that original JSON string arguments are preserved while providing parsed companion data.
- Modified tests to validate the new behavior of tool call arguments, ensuring compatibility with both OpenAI-style and Hugging Face templates.
Copilot AI review requested due to automatic review settings March 19, 2026 09:00
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts tool-call argument preprocessing to preserve OpenAI-compatible function.arguments JSON strings for downstream consumers, while still enabling Hugging Face (Qwen-style) chat templates to render dict-like tool arguments during prompt rendering.

Changes:

  • Attach a companion parsed_arguments field for assistant tool calls when function.arguments is a JSON object string (without mutating arguments).
  • Adapt HF template rendering by creating a render-only message copy where function.arguments is converted to a dict when possible.
  • Expand test coverage for round-trip behavior, invalid/non-dict JSON, multi-tool-calls, and multimodal paths.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
lmdeploy/serve/processors/multimodal.py Adds shared preprocessing to attach parsed_arguments for assistant tool calls across text and multimodal normalization.
lmdeploy/model.py Adds HF render-only adaptation that converts tool-call arguments to dicts for templates that require mapping semantics.
tests/test_lmdeploy/test_content_merge.py Adds/updates tests verifying parsed_arguments attachment behavior and non-mutation guarantees.
tests/test_lmdeploy/test_model.py Adds an integration-style test asserting Qwen template rendering works with string tool-call arguments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…ore accessing .get()

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@lvhan028 lvhan028 self-requested a review March 19, 2026 09:13
@lvhan028
Copy link
Collaborator

Cann't qwen3.5's chat template handle the "tools" messages?

@lapy
Copy link
Contributor Author

lapy commented Mar 20, 2026

Cann't qwen3.5's chat template handle the "tools" messages?

It can but it expects a certain structured format for tool arguments in order for the jinja template to render correctly. Using 3.5 with openclaw results in many failed tool calls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants