feat: Add ToolSearchToolset for dynamic tool discovery from large tool catalogs #10426
+1,192
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…alogs Implements ToolSearchToolset - a Toolset subclass that enables dynamic tool discovery from large catalogs. Tools are discovered via `search_tools` bm25 based special search tool and become available to the LLM. Key features: - Single discovery mode: "bm25", postpone "embedding" for future - Passthrough mode for small catalogs (< search_threshold) - Self-contained BM25L search engine implementation - Full serialization support (to_dict/from_dict) - Auto warm-up when iterating to ensure bootstrap tool availability
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
vblagoje
added
the
ignore-for-release-notes
Collaborator
Pull Request Test Coverage Report for Build 21243903558Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
The description claimed to return "a JSON array of tool definitions" but actually returns a plain text confirmation message with tool names.
Tools discovered via search_tools were added to _discovered_tools without calling warm_up(), causing tools that require initialization (connections, model loading) to fail when invoked.
The inherited __getitem__ accessed self.tools which is always empty in ToolSearchToolset. This caused IndexError for valid indexes even when tools were available through __iter__.
vblagoje
removed
the
ignore-for-release-notes
Member
Author
|
@sjrl @julian-risch - give me a 1-2 days to test it thoroughly with large mcp toolsets and if all good I'll open this PR. This is the general direction @mpangrazzi and I talked about. LMK if you agree. |
vblagoje
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why
Large tool catalogs overwhelm LLM context windows. Agents need a way to discover tools on-demand rather than receiving all tool definitions upfront.
What
ToolSearchToolset: Toolset subclass with BM25-based tool discovery_BM25SearchEngine(BM25L variant) - no external dependenciessearch_tools(query, k)bootstrap tool for LLM-driven discoverysearch_threshold(default: 8)clear()method for resetting discovered tools between agent runsto_dict/from_dict)How can it be used
How did you test it
Notes for the reviewer