feat: add XML tool calling support as provider setting#11973
feat: add XML tool calling support as provider setting#11973jthweny wants to merge 8 commits intoRooCodeInc:mainfrom
Conversation
Add a useXmlToolCalling boolean toggle to provider settings that enables text-based XML tool calling instead of native function calling. Phase 1 - System Prompt: - Add useXmlToolCalling to baseProviderSettingsSchema in provider-settings.ts - Modify getSharedToolUseSection() to return XML formatting instructions when useXmlToolCalling is true - Make getToolUseGuidelinesSection() XML-aware with conditional steps - Thread useXmlToolCalling through SYSTEM_PROMPT(), generateSystemPrompt(), and Task.getSystemPrompt() - Add UI toggle checkbox in ApiOptions.tsx settings panel - Add i18n string for the toggle label Phase 2 - Transport Layer: - Add useXmlToolCalling to ApiHandlerCreateMessageMetadata interface - Conditionally omit native tools/tool_choice from Anthropic API requests when useXmlToolCalling is enabled - Same conditional omission for Anthropic Vertex provider - Thread useXmlToolCalling from provider settings into API request metadata in Task.attemptApiRequest() The existing TagMatcher-based text parsing in presentAssistantMessage() automatically handles XML tool calls when the model outputs them as raw text (which occurs when native tools are omitted from the request). Tests: 9 new tool-use.spec.ts tests + 3 new anthropic.spec.ts tests, all passing.
jthweny
requested review from
cte,
daniel-lxs,
hannesrudolph,
jr and
mrubens
as code owners
dosubot
bot
added
size:L
There was a problem hiding this comment.
Pull request overview
Adds a provider setting (useXmlToolCalling) intended to switch Anthropic/Vertex from native tool calling to XML-in-text tool calling, by updating the system prompt and omitting native tool parameters from certain provider API requests.
Changes:
- Add
useXmlToolCallingto provider settings schema and expose it in the webview “Advanced settings” UI. - Thread
useXmlToolCallinginto system prompt generation to emit XML tool-calling instructions. - Update Anthropic + Anthropic Vertex request building to omit
tools/tool_choicewhenuseXmlToolCallingis enabled, with new tests asserting omission.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| webview-ui/src/i18n/locales/en/settings.json | Adds UI strings for the new advanced setting. |
| webview-ui/src/components/settings/ApiOptions.tsx | Adds the “Use XML tool calling” checkbox in Advanced settings. |
| src/core/webview/generateSystemPrompt.ts | Threads the new toggle into system prompt preview generation. |
| src/core/task/Task.ts | Threads the toggle into runtime prompt generation and API handler metadata. |
| src/core/prompts/system.ts | Passes the toggle into tool-use prompt sections. |
| src/core/prompts/sections/tool-use.ts | Adds XML-mode tool-use instructions section. |
| src/core/prompts/sections/tool-use-guidelines.ts | Adds XML-mode reinforcement to tool-use guidelines. |
| src/core/prompts/sections/tests/tool-use.spec.ts | Adds/extends unit tests for XML vs native prompt sections. |
| src/api/providers/anthropic.ts | Omits native tool params when XML mode is enabled. |
| src/api/providers/anthropic-vertex.ts | Omits native tool params when XML mode is enabled. |
| src/api/providers/tests/anthropic.spec.ts | Adds tests asserting omission/presence of tool params based on the toggle. |
| src/api/index.ts | Adds useXmlToolCalling to handler metadata interface and documents intended behavior. |
| packages/types/src/provider-settings.ts | Adds useXmlToolCalling to provider settings schema. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| : "" | ||
|
|
||
| return `# Tool Use Guidelines | ||
|
|
||
| 1. Assess what information you already have and what information you need to proceed with the task. | ||
| 2. Choose the most appropriate tool based on the task and the tool descriptions provided. Assess if you need additional information to proceed, and which of the available tools would be most effective for gathering this information. For example using the list_files tool is more effective than running a command like \`ls\` in the terminal. It's critical that you think about each available tool and use the one that best fits the current step in the task. | ||
| 3. If multiple actions are needed, you may use multiple tools in a single message when appropriate, or use tools iteratively across messages. Each tool use should be informed by the results of previous tool uses. Do not assume the outcome of any tool use. Each step must be informed by the previous step's result. |
There was a problem hiding this comment.
In XML mode, step 3 still says multiple tools may be used in a single message, but other XML instructions reinforce one-at-a-time tool usage. Please reconcile the XML-mode guidance so it consistently reflects the intended/implemented behavior (single-tool vs multi-tool) to avoid conflicting instructions in the system prompt.
| : "" | |
| return `# Tool Use Guidelines | |
| 1. Assess what information you already have and what information you need to proceed with the task. | |
| 2. Choose the most appropriate tool based on the task and the tool descriptions provided. Assess if you need additional information to proceed, and which of the available tools would be most effective for gathering this information. For example using the list_files tool is more effective than running a command like \`ls\` in the terminal. It's critical that you think about each available tool and use the one that best fits the current step in the task. | |
| 3. If multiple actions are needed, you may use multiple tools in a single message when appropriate, or use tools iteratively across messages. Each tool use should be informed by the results of previous tool uses. Do not assume the outcome of any tool use. Each step must be informed by the previous step's result. | |
| : ""; | |
| const step3Guideline = useXmlToolCalling | |
| ? `3. If multiple actions are needed, use tools iteratively across messages, making at most one XML tool call per assistant message. Each tool use should be informed by the results of previous tool uses. Do not assume the outcome of any tool use. Each step must be informed by the previous step's result.` | |
| : `3. If multiple actions are needed, you may use multiple tools in a single message when appropriate, or use tools iteratively across messages. Each tool use should be informed by the results of previous tool uses. Do not assume the outcome of any tool use. Each step must be informed by the previous step's result.`; | |
| return `# Tool Use Guidelines | |
| 1. Assess what information you already have and what information you need to proceed with the task. | |
| 2. Choose the most appropriate tool based on the task and the tool descriptions provided. Assess if you need additional information to proceed, and which of the available tools would be most effective for gathering this information. For example using the list_files tool is more effective than running a command like \`ls\` in the terminal. It's critical that you think about each available tool and use the one that best fits the current step in the task. | |
| ${step3Guideline} |
| <div> | ||
| <Checkbox | ||
| checked={apiConfiguration?.useXmlToolCalling ?? false} | ||
| onChange={handleInputChange("useXmlToolCalling", noTransform)}> | ||
| {t("settings:advancedSettings.useXmlToolCalling")} | ||
| </Checkbox> | ||
| <div className="text-sm text-vscode-descriptionForeground ml-6"> | ||
| {t("settings:advancedSettings.useXmlToolCallingDescription")} | ||
| </div> | ||
| </div> |
There was a problem hiding this comment.
The UI exposes useXmlToolCalling for all providers, but the backend behavior (omitting native tools/tool_choice) is only implemented for Anthropic/Anthropic-Vertex. Enabling this for other providers will still modify the system prompt to demand XML, while requests continue using native tool calling—creating a prompt/API mismatch and likely breaking tool use. Consider gating this checkbox (and/or the prompt toggle) to only providers that actually honor useXmlToolCalling, or implement the request-side behavior for all providers.
| <div> | |
| <Checkbox | |
| checked={apiConfiguration?.useXmlToolCalling ?? false} | |
| onChange={handleInputChange("useXmlToolCalling", noTransform)}> | |
| {t("settings:advancedSettings.useXmlToolCalling")} | |
| </Checkbox> | |
| <div className="text-sm text-vscode-descriptionForeground ml-6"> | |
| {t("settings:advancedSettings.useXmlToolCallingDescription")} | |
| </div> | |
| </div> | |
| {(selectedProvider === "anthropic" || selectedProvider === "vertex") && ( | |
| <div> | |
| <Checkbox | |
| checked={apiConfiguration?.useXmlToolCalling ?? false} | |
| onChange={handleInputChange("useXmlToolCalling", noTransform)}> | |
| {t("settings:advancedSettings.useXmlToolCalling")} | |
| </Checkbox> | |
| <div className="text-sm text-vscode-descriptionForeground ml-6"> | |
| {t("settings:advancedSettings.useXmlToolCallingDescription")} | |
| </div> | |
| </div> | |
| )} |
| ${getSharedToolUseSection(useXmlToolCalling)}${toolsCatalog} | ||
|
|
||
| ${getToolUseGuidelinesSection()} | ||
| ${getToolUseGuidelinesSection(useXmlToolCalling)} |
There was a problem hiding this comment.
useXmlToolCalling is threaded into the system prompt unconditionally. Since only some providers currently change their API request behavior based on this flag, the prompt can instruct XML tool calls while the selected provider still expects native tool calling. Scope the XML prompt sections to providers that actually support this mode (or ensure all providers handle useXmlToolCalling consistently).
| // When useXmlToolCalling is enabled, omit native tool definitions from the API request. | ||
| // The model will rely on XML tool documentation in the system prompt instead, | ||
| // and output tool calls as raw XML text parsed by TagMatcher. | ||
| const nativeToolParams = metadata?.useXmlToolCalling | ||
| ? {} | ||
| : { | ||
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | ||
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | ||
| } |
There was a problem hiding this comment.
When metadata.useXmlToolCalling is true, this omits tools/tool_choice from the Anthropic request, but the codebase currently executes tools via native tool_use blocks with ids/nativeArgs (XML/legacy tool calls are explicitly rejected in presentAssistantMessage/BaseTool). Without an XML-to-ToolUse parser (and tool schema documentation) this will prevent any tool execution. Either implement the XML parsing + tool catalog path end-to-end, or keep sending native tools for Anthropic.
| // When useXmlToolCalling is enabled, omit native tool definitions from the API request. | |
| // The model will rely on XML tool documentation in the system prompt instead, | |
| // and output tool calls as raw XML text parsed by TagMatcher. | |
| const nativeToolParams = metadata?.useXmlToolCalling | |
| ? {} | |
| : { | |
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | |
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | |
| } | |
| // Always send native tool definitions for Anthropic so that tool_use blocks are produced. | |
| // The useXmlToolCalling flag is currently ignored here because the rest of the codebase | |
| // expects native tool_use events and does not support XML-based tool calling. | |
| const nativeToolParams = { | |
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | |
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | |
| } |
| // When useXmlToolCalling is enabled, omit native tool definitions from the API request. | ||
| // The model will rely on XML tool documentation in the system prompt instead, | ||
| // and output tool calls as raw XML text parsed by TagMatcher. | ||
| const nativeToolParams = metadata?.useXmlToolCalling | ||
| ? {} | ||
| : { | ||
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | ||
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | ||
| } |
There was a problem hiding this comment.
Same issue as Anthropic: omitting native tools/tool_choice when useXmlToolCalling is true will leave the system without a working tool-call execution path unless XML tool calls are parsed into ToolUse blocks with ids/nativeArgs. As-is, this will likely break tool use for Anthropic Vertex. Either implement the XML parsing + tool documentation path, or continue sending native tool params.
| // When useXmlToolCalling is enabled, omit native tool definitions from the API request. | |
| // The model will rely on XML tool documentation in the system prompt instead, | |
| // and output tool calls as raw XML text parsed by TagMatcher. | |
| const nativeToolParams = metadata?.useXmlToolCalling | |
| ? {} | |
| : { | |
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | |
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | |
| } | |
| // Always send native tool definitions to the API request so that tool calling | |
| // continues to work even when XML-based tool documentation is used elsewhere. | |
| const nativeToolParams = { | |
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | |
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | |
| } |
| * The model relies solely on XML tool documentation in the system prompt | ||
| * and outputs tool calls as raw XML text, which the existing TagMatcher | ||
| * in presentAssistantMessage() parses into ToolUse objects. |
There was a problem hiding this comment.
The doc comment claims XML tool calls are parsed by TagMatcher in presentAssistantMessage(), but presentAssistantMessage currently treats missing tool_use.id as an invalid legacy/XML tool call and rejects it, and tools generally require nativeArgs. Please update this comment to reflect the actual execution/parsing flow, or add the missing XML parsing implementation and adjust this description accordingly.
| * The model relies solely on XML tool documentation in the system prompt | |
| * and outputs tool calls as raw XML text, which the existing TagMatcher | |
| * in presentAssistantMessage() parses into ToolUse objects. | |
| * The model is expected to rely solely on XML tool documentation in the system prompt | |
| * and may output tool calls as raw XML (or XML-like) text. | |
| * | |
| * This flag only affects how the request is constructed; any parsing of XML tool | |
| * calls into ToolUse objects must be handled by higher-level consumer code. |
|
|
||
| TOOL USE | ||
|
|
||
| You have access to a set of tools that are executed upon the user's approval. You can use one tool per message, and will receive the result of that tool use in the user's response. You use tools step-by-step to accomplish a given task, with each tool use informed by the result of the previous tool use. |
There was a problem hiding this comment.
In XML mode this section says "You can use one tool per message", but the general tool-use guidelines (and native mode) explicitly allow multiple tools per message. This internal inconsistency can confuse the model and cause unpredictable tool behavior. Align the XML instructions with the actual supported behavior (either document single-tool restriction everywhere for XML mode, or remove the single-tool claim here).
| You have access to a set of tools that are executed upon the user's approval. You can use one tool per message, and will receive the result of that tool use in the user's response. You use tools step-by-step to accomplish a given task, with each tool use informed by the result of the previous tool use. | |
| You have access to a set of tools that are executed upon the user's approval. You can use one or more tools per message, and will receive the result of those tool uses in the user's response. You use tools step-by-step to accomplish a given task, with each tool use informed by the result of the previous tool use. |
When useXmlToolCalling is enabled, omit native tool definitions (tools, tool_choice, parallel_tool_calls) from API requests across all 22 providers. The model relies on XML tool documentation in the system prompt instead, fixing 400 errors with servers like vLLM that don't support tool_choice: auto. Providers updated: - OpenAI-style: openai, deepseek, base-openai-compatible-provider, openai-compatible, lm-studio, lite-llm, xai, qwen-code, openrouter, requesty, unbound, vercel-ai-gateway, roo, zai - Responses API: openai-native, openai-codex - Custom formats: bedrock, gemini, minimax, mistral Tests: 5 new tests in openai.spec.ts, 800 total passed
dosubot
bot
added
size:XL
or open system.ts and change the native to xml .. easy |
- Add XmlToolCallParser with streaming XML detection and partial tag handling - Add hand-crafted tool descriptions for attempt_completion and ask_followup_question - Support multiple follow_up formats: JSON arrays, <suggest> tags, comma-less objects - Strip <thinking> tags before XML parsing to prevent hallucination loops - Normalize Meta/Llama tool_call format to standard XML - Prevent XML tags from leaking into chat UI during streaming - Add XML-aware retry messages and missing parameter errors - Graceful degradation: text-only responses shown as followup questions - Compact XML tool descriptions to save context window space - Match Kilo Code/Cline system prompt conventions for better model compliance Made-with: Cursor
dosubot
bot
added
size:XXL
Update tool-use.spec.ts and xml-tool-catalog.spec.ts to match the new compact XML prompt format. Update system prompt snapshots. Made-with: Cursor
Made-with: Cursor
Update presentAssistantMessage tests to match the current error message "missing tool_use.id" instead of the old "XML tool calls are no longer supported" text. Made-with: Cursor
Adds a useXmlToolCalling provider toggle. When enabled, the system prompt includes XML formatting instructions and native tool parameters (tools/tool_choice) are omitted from Anthropic/Vertex API requests, forcing the model to use text-based XML tool calling parsed by the existing TagMatcher. 12 new tests, all passing.
Interactively review PR in Roo Code Cloud