feat: add ask_user tool with host-side LLM interception#1
Open
yash-scaleai wants to merge 1 commit intoscaleapi:scale-customizationsfrom
Open
feat: add ask_user tool with host-side LLM interception#1yash-scaleai wants to merge 1 commit intoscaleapi:scale-customizationsfrom
yash-scaleai wants to merge 1 commit intoscaleapi:scale-customizationsfrom
Conversation
Adds an ask_user tool that allows agents to request clarification on underspecified tasks. The tool simulates user responses using an LLM based on the complete task definition. Key changes: - Add ask_user tool in tools/ask_user/ - Add TaskDefinitionInjectionHook to inject task definitions into containers - Auto-detect task_definitions.json in run_batch.py - **Critical fix**: Intercept ask_user commands on HOST side in agents.py The host-side interception is necessary because Modal containers cannot reach internal API endpoints (e.g., litellm.ml-serving-internal.scale.com). By handling the LLM call on the host (where the agent's own LLM calls are made), we avoid timeout issues while maintaining the same functionality. Based on Bryan's PR #17144 with architectural modification for internal API compatibility. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an
ask_usertool that allows agents to request clarification on underspecified tasks. The tool simulates user responses using an LLM based on the complete task definition.Key changes:
ask_usertool intools/ask_user/TaskDefinitionInjectionHookto inject task definitions into containerstask_definitions.jsoninrun_batch.pyask_usercommands on HOST side inagents.pyWhy host-side interception?
The original implementation (Bryan's PR #17144) runs the LLM call inside the Modal container. However, Modal containers cannot reach internal API endpoints like
litellm.ml-serving-internal.scale.com.This PR intercepts
ask_usercommands inagents.pybefore they are sent to the container, makes the LLM call on the host (where the agent's own LLM calls are made), and returns the response directly.Flow:
Based on Bryan's PR #17144 with architectural modification for internal API compatibility.
Test plan
scripts/test_ask_user_single.py🤖 Generated with Claude Code