Summary
OpenRouter natively supports combining response_format: { type: 'json_schema', ... } with stream: true, but our adapter currently performs structured output as a separate non-streaming call. This means a logical structured-output run produces two distinct HTTP requests (one streaming free-text, one non-streaming JSON), and callers can't consume the JSON incrementally.
OpenRouter docs: https://openrouter.ai/docs/guides/features/structured-outputs#streaming-with-structured-outputs
Prior discussion: #370 (also related: #195)
Current behavior
packages/typescript/ai-openrouter/src/adapters/text.ts
chatStream() (line 113) — stream: true, no responseFormat. Free-text only.
structuredOutput() (line 266) — stream: false, sends responseFormat: { type: 'json_schema', jsonSchema: { name, schema, strict: true } }. Returns a Promise, no streaming.
The TextAdapter base interface (packages/typescript/ai/src/activities/chat/adapter.ts:97) hardcodes structuredOutput as Promise<StructuredOutputResult<unknown>>, so today there is no surface for incremental structured output even where the provider supports it.
Proposal
Add a streaming structured-output path. Two shapes worth considering:
- New adapter method
structuredOutputStream(options) returning AsyncIterable<StreamChunk>, with chunks carrying partial JSON deltas (we already depend on partial-json for the same purpose elsewhere). Adapters that don't natively support it can fall back to the existing non-streaming structuredOutput.
- Extend
chatStream to accept an outputSchema option and have OpenRouter emit responseFormat alongside stream: true. Simpler wire change, but couples structured-output semantics into the general chat path and makes the typed result harder to expose.
(1) is probably cleaner and parallels the chatStream / structuredOutput split we already have.
Acceptance criteria
@tanstack/ai-openrouter exposes streaming structured output that emits a single HTTP request with stream: true + response_format: json_schema (strict).
- Partial JSON deltas are surfaced as they arrive (parsed via
partial-json).
- Final result validates against the supplied schema (re-using
convertSchemaToJsonSchema(..., { forStructuredOutput: true }) for the strict transform).
- E2E coverage in
testing/e2e (mandatory per CLAUDE.md) — fixture + spec under the structured-output scenarios, with OpenRouter listed in feature-support.ts / test-matrix.ts.
- Other adapters: at minimum, default to the existing non-streaming path so the new method is callable everywhere without breaking changes. Native streaming JSON for OpenAI/Anthropic/Gemini can land separately.
Notes / risks
- Tool calls + structured output in the same request: OpenRouter's docs allow it but behavior varies by upstream provider — worth a smoke test, not a blocker.
- Strict schema transform already lives in
ai; reuse it, don't fork.
- The base
TextAdapter interface change is a breaking surface for downstream adapters — mitigated if we add structuredOutputStream as optional with a default that wraps structuredOutput.
Summary
OpenRouter natively supports combining
response_format: { type: 'json_schema', ... }withstream: true, but our adapter currently performs structured output as a separate non-streaming call. This means a logical structured-output run produces two distinct HTTP requests (one streaming free-text, one non-streaming JSON), and callers can't consume the JSON incrementally.OpenRouter docs: https://openrouter.ai/docs/guides/features/structured-outputs#streaming-with-structured-outputs
Prior discussion: #370 (also related: #195)
Current behavior
packages/typescript/ai-openrouter/src/adapters/text.tschatStream()(line 113) —stream: true, noresponseFormat. Free-text only.structuredOutput()(line 266) —stream: false, sendsresponseFormat: { type: 'json_schema', jsonSchema: { name, schema, strict: true } }. Returns aPromise, no streaming.The
TextAdapterbase interface (packages/typescript/ai/src/activities/chat/adapter.ts:97) hardcodesstructuredOutputasPromise<StructuredOutputResult<unknown>>, so today there is no surface for incremental structured output even where the provider supports it.Proposal
Add a streaming structured-output path. Two shapes worth considering:
structuredOutputStream(options)returningAsyncIterable<StreamChunk>, with chunks carrying partial JSON deltas (we already depend onpartial-jsonfor the same purpose elsewhere). Adapters that don't natively support it can fall back to the existing non-streamingstructuredOutput.chatStreamto accept anoutputSchemaoption and have OpenRouter emitresponseFormatalongsidestream: true. Simpler wire change, but couples structured-output semantics into the general chat path and makes the typed result harder to expose.(1) is probably cleaner and parallels the
chatStream/structuredOutputsplit we already have.Acceptance criteria
@tanstack/ai-openrouterexposes streaming structured output that emits a single HTTP request withstream: true+response_format: json_schema(strict).partial-json).convertSchemaToJsonSchema(..., { forStructuredOutput: true })for the strict transform).testing/e2e(mandatory perCLAUDE.md) — fixture + spec under the structured-output scenarios, with OpenRouter listed infeature-support.ts/test-matrix.ts.Notes / risks
ai; reuse it, don't fork.TextAdapterinterface change is a breaking surface for downstream adapters — mitigated if we addstructuredOutputStreamas optional with a default that wrapsstructuredOutput.