fix: support custom LLM clients in agent mode#1678
Conversation
CustomOpenAIClient now implements getLanguageModel() using @ai-sdk/openai, allowing it to work with agent mode which requires a LanguageModelV2 instance. Also improves the error message when getLanguageModel() is missing. Closes #1669 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Matches the OpenAI API spec. Currently dead code in Stagehand's act/extract/observe paths (they don't pass tools), but should be correct regardless. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
No issues found across 3 files
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
Architecture diagram
sequenceDiagram
participant User as User Code
participant Agent as V3AgentHandler
participant Client as CustomOpenAIClient
participant AISDK as @ai-sdk/openai
participant Errors as MissingLLMConfigurationError
User->>Agent: stagehand.agent(options)
Note over Agent,Client: Validation of LLM Client Capabilities
alt NEW: llmClient.getLanguageModel is missing
Agent->>Errors: CHANGED: throw with specific "agent mode" instructions
Errors-->>User: Error: implement getLanguageModel()
else llmClient.getLanguageModel is defined
Agent->>Client: NEW: getLanguageModel()
activate Client
Client->>AISDK: createOpenAI({ apiKey, baseURL })
AISDK-->>Client: provider instance
Client->>Client: provider(modelName)
Client-->>Agent: Return LanguageModelV2 instance
deactivate Client
Agent->>Agent: Use model for agent reasoning loop
Agent-->>User: Result
end
Greptile OverviewGreptile SummaryThese changes make custom OpenAI-compatible clients usable with Confidence Score: 4/5
Important Files Changed
Sequence DiagramsequenceDiagram
autonumber
participant User
participant V3AgentHandler
participant LLMClient
participant CustomOpenAIClient
participant AISDK as ai-sdk/openai
User->>V3AgentHandler: execute()/stream(instruction|options)
V3AgentHandler->>V3AgentHandler: prepareAgent()
V3AgentHandler->>LLMClient: getLanguageModel()
alt getLanguageModel missing
V3AgentHandler-->>User: throw MissingLLMConfigurationError(custom message)
else getLanguageModel present
Note over CustomOpenAIClient: example implementation
CustomOpenAIClient->>AISDK: createOpenAI({ apiKey, baseURL })
AISDK-->>CustomOpenAIClient: provider(modelName)
CustomOpenAIClient-->>V3AgentHandler: LanguageModelV2
V3AgentHandler->>LLMClient: generateText()/streamText(model,...)
end
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
No longer needed now that CustomOpenAIClient implements getLanguageModel(). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
1 issue found across 2 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="packages/core/lib/v3/handlers/v3AgentHandler.ts">
<violation number="1">
P2: Reverting to `MissingLLMConfigurationError()` with no argument produces the generic message *"No LLM API key or LLM Client configured … required to use act, extract, or observe"*, which is misleading when called from agent mode — the user may have a client configured that simply lacks `getLanguageModel()`. The PR description says it improves this error message, but the class constructor takes no arguments and the hardcoded default doesn't mention agent mode or `getLanguageModel()`.
Consider updating `MissingLLMConfigurationError` to accept an optional message, then pass the agent-specific guidance here.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
1 issue found across 1 file (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="packages/core/examples/external_clients/customOpenAI.ts">
<violation number="1">
P0: Bug: `inputSchema` is not a valid field for OpenAI chat completion tool definitions. The OpenAI API expects `parameters`. This change silently breaks tool/function calling via `createChatCompletion()` — the model will see tools with no parameter schema.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| @@ -11,6 +11,8 @@ import { | |||
| LLMClient, | |||
There was a problem hiding this comment.
P0: Bug: inputSchema is not a valid field for OpenAI chat completion tool definitions. The OpenAI API expects parameters. This change silently breaks tool/function calling via createChatCompletion() — the model will see tools with no parameter schema.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/core/examples/external_clients/customOpenAI.ts, line 199:
<comment>Bug: `inputSchema` is not a valid field for OpenAI chat completion tool definitions. The OpenAI API expects `parameters`. This change silently breaks tool/function calling via `createChatCompletion()` — the model will see tools with no parameter schema.</comment>
<file context>
@@ -196,7 +196,7 @@ export class CustomOpenAIClient extends LLMClient {
name: tool.name,
description: tool.description,
- parameters: tool.parameters,
+ inputSchema: tool.parameters,
},
type: "function",
</file context>
| LLMClient, | |
| parameters: tool.parameters, |
| public getLanguageModel(): LanguageModelV2 { | ||
| const provider = createOpenAI({ | ||
| apiKey: this.client.apiKey, | ||
| baseURL: this.client.baseURL, | ||
| }); | ||
| return provider(this.modelName); | ||
| } | ||
|
|
There was a problem hiding this comment.
it seems this just uses openai client meaning the custom one is never actually used?
2 participants
Summary
Adds
getLanguageModel()toCustomOpenAIClientusing@ai-sdk/openai'screateOpenAI, so custom OpenAI-compatible clients (ZhipuAI, Ollama, etc.) work with agent mode.Closes #1669
Test plan
getLanguageModel()returns a validLanguageModelV2(correctmodelId,doGenerate,doStream,specificationVersion: "v2")getLanguageModel()preserves custombaseURLfor OpenAI-compatible providers (ZhipuAI, Ollama)generateText()(the AI SDK path agent mode uses) succeeds with the model fromgetLanguageModel()createChatCompletion()(act/extract/observe path) still works — no regressionMissingLLMConfigurationErrorno-arg constructor still works🤖 Generated with Claude Code