Skip to content

fix: enable native OpenAI-style tool calling for local model providers#964

Open
octo-patch wants to merge 1 commit intovoideditor:mainfrom
octo-patch:fix/issue-857-enable-openai-style-tools-for-local-providers
Open

fix: enable native OpenAI-style tool calling for local model providers#964
octo-patch wants to merge 1 commit intovoideditor:mainfrom
octo-patch:fix/issue-857-enable-openai-style-tools-for-local-providers

Conversation

@octo-patch
Copy link
Copy Markdown

Fixes #857

Problem

Local model providers (Ollama, vLLM, LM Studio, liteLLM, openAICompatible) all expose an OpenAI-compatible API that supports native function/tool calling. However, models served via these providers were missing specialToolFormat: 'openai-style' in their capability definitions.

Without this flag, Void falls back to XML-based tool prompting (embedding tool definitions in the system prompt and parsing XML tags from the model's text output). This approach is unreliable for modern local models that expect to receive tools via the standard tools parameter and return tool_calls in structured JSON.

As a result, users see errors like "No Search/Replace blocks were received!" because:

  1. The tools parameter is never sent in API requests to local providers
  2. The model has no idea it should produce tool calls
  3. The XML parser finds nothing to extract

This is consistently reported across multiple models: qwen3-coder, qwen2.5-coder, llama3.x, devstral, gemma, etc. via Ollama, vLLM, LM Studio, and liteLLM.

Solution

Add specialToolFormat: 'openai-style' to all chat/agent-capable models in:

  • openSourceModelOptions_assumingOAICompat (used as fallback for unrecognized model names)
  • ollamaModelOptions (Ollama-specific recognized models)

This enables:

  1. The tools parameter to be sent in API requests to local providers
  2. Native tool_calls responses to be parsed correctly (already implemented in the streaming handler)
  3. The XML tool-prompting fallback to be skipped (it's not needed when native tool calling works)

Users who need to disable native tool calling for a specific model can override via the specialToolFormat field in their model's advanced settings JSON.

Testing

The fix is targeted at model capability definitions only and affects no runtime logic. The existing streaming handler already processes tool_calls from OpenAI-compatible responses correctly — the missing piece was sending the tools parameter.

fixes voideditor#857)

Local providers (Ollama, vLLM, LM Studio, liteLLM, openAICompatible) all use
the OpenAI-compatible API which supports native function/tool calling. However,
models served via these providers were missing `specialToolFormat: 'openai-style'`
in their capability definitions, causing Void to fall back to XML-based tool
prompting instead of sending the `tools` parameter in API requests.

This meant models like qwen3-coder, llama3.1, devstral, etc. never received tool
definitions and could not perform file edits or other agentic actions, resulting
in 'No Search/Replace blocks were received!' errors.

Fix: add `specialToolFormat: 'openai-style'` to all chat/agent models in
`openSourceModelOptions_assumingOAICompat` and `ollamaModelOptions`. Users who
need to disable this for a specific model can override via `specialToolFormat`
in their model's advanced settings JSON.

Co-Authored-By: Octopus <liyuan851277048@icloud.com>
h2balloon referenced this pull request in h2balloon/voidberg Apr 30, 2026
Cherry-picked from voideditor/void:
- #964: add specialToolFormat: 'openai-style' to all OSS/Ollama model
  entries so local providers send native tool calls instead of falling
  back to broken XML prompting (fixes apply/edit on qwen-coder, llama3.1+)
- #961: ensure MCP tool name prefix starts with a letter to satisfy
  Gemini FunctionDeclaration.name validation (fixes 400 INVALID_ARGUMENT);
  also includes position:relative on .interactive-session for DnD overlay

Co-Authored-By: octo-patch <noreply@github.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
h2balloon referenced this pull request in h2balloon/voidberg Apr 30, 2026
26 tests across 4 suites:
- specialToolFormat: verifies OSS/Ollama models get openai-style tool
  calling via both exact Ollama entries and openAICompatible fallback
- Gemini routing: verifies 2.5/3.x models route to correct buckets and
  that removed gemini-3-pro-preview is correctly unrecognized
- Ordering edge cases: flash-lite disambiguation, deepseek-r1 vs V3,
  and a documented known bug in the llama3.1 fallback rule ordering
- Overrides: specialToolFormat and contextWindow can be overridden

Also discovered Gemini provider has no substring fallback (returns null
for unknown models) unlike openAICompatible — captured in tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] qwen3-coder can't edit file

1 participant