-
-
Notifications
You must be signed in to change notification settings - Fork 144
feat: generation hooks and streaming across all frameworks #327
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
AlemTuzlak
wants to merge
12
commits into
main
Choose a base branch
from
feat/generation-hooks-and-streaming
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
9198188
feat: add generation hooks and streaming for image, speech, video, tr…
AlemTuzlak 1fc91d7
ci: apply automated fixes
autofix-ci[bot] ea42f6e
feat: add unified `stream` flag to all generation activities
AlemTuzlak 7490a44
feat(examples): add streaming/direct tab toggle to generation showcases
AlemTuzlak 31cdfb9
ci: apply automated fixes
autofix-ci[bot] 8288bf0
Merge branch 'main' into feat/generation-hooks-and-streaming
AlemTuzlak 860cfbd
feat: update request handling in generation endpoints to use body.data
AlemTuzlak 127353e
ci: apply automated fixes
autofix-ci[bot] e4288b2
feat(ai-client): pass abort signal to fetcher in generation clients (…
AlemTuzlak a64db3f
chore: add changeset for fetcher abort signal and GenerationFetcher type
AlemTuzlak 1175c07
chore: add @tanstack/ai to changeset
AlemTuzlak aab2d43
Merge branch 'main' into feat/generation-hooks-and-streaming
AlemTuzlak File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| --- | ||
| '@tanstack/ai': patch | ||
| '@tanstack/ai-client': patch | ||
| '@tanstack/ai-react': patch | ||
| '@tanstack/ai-solid': patch | ||
| '@tanstack/ai-vue': patch | ||
| '@tanstack/ai-svelte': patch | ||
| --- | ||
|
|
||
| feat: pass abort signal to generation fetchers and extract GenerationFetcher utility type | ||
|
|
||
| - Generation clients now forward an `AbortSignal` to fetcher functions via an optional `options` parameter, enabling cancellation support when `stop()` is called | ||
| - Introduced `GenerationFetcher<TInput, TResult>` utility type in `@tanstack/ai-client` to centralize the fetcher function signature across all framework integrations | ||
| - All framework hooks/composables (React, Solid, Vue, Svelte) now use the shared `GenerationFetcher` type instead of inline definitions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| { | ||
| "permissions": { | ||
| "allow": [ | ||
| "Bash(pnpm test:lib)", | ||
| "Bash(pnpm test:eslint)", | ||
| "Bash(xargs grep -l \"devtools-event-client\")", | ||
| "Bash(xargs grep -l \"onConfig\\\\|onIterationStart\\\\|onIteration\")", | ||
| "Bash(npx @tanstack/router-cli generate)", | ||
| "Bash(xargs grep -l \"generate\\\\|image\\\\|video\\\\|audio\")", | ||
| "Bash(pnpm --filter @tanstack/ai build)", | ||
| "Bash(pnpm --filter @tanstack/ai-client build)", | ||
| "Bash(npx vitest run tests/generation-client.test.ts)", | ||
| "Bash(npx vitest run tests/stream-generation.test.ts)", | ||
| "Bash(pnpm format)", | ||
| "Bash(pnpm --filter @tanstack/ai-react test:types)", | ||
| "Bash(pnpm --filter @tanstack/ai-solid test:types)", | ||
| "Bash(pnpm --filter @tanstack/ai-vue test:types)", | ||
| "Bash(pnpm --filter @tanstack/ai test:eslint)", | ||
| "Bash(pnpm --filter @tanstack/ai-client test:eslint)", | ||
| "Bash(pnpm --filter @tanstack/ai-react test:eslint)", | ||
| "Bash(pnpm --filter @tanstack/ai-solid test:eslint)", | ||
| "Bash(pnpm --filter @tanstack/ai-vue test:eslint)", | ||
| "Bash(pnpm --filter @tanstack/ai-svelte test:eslint)", | ||
| "Bash(pnpm --filter @tanstack/ai test:types)", | ||
| "Bash(pnpm --filter @tanstack/ai-client test:types)", | ||
| "Bash(pnpm --filter @tanstack/ai test:lib)", | ||
| "Bash(pnpm --filter @tanstack/ai-client test:lib)", | ||
| "Bash(pnpm --filter @tanstack/ai-react build)", | ||
| "Bash(pnpm --filter @tanstack/ai-solid build)", | ||
| "Bash(pnpm --filter @tanstack/ai-vue build)", | ||
| "Bash(pnpm --filter @tanstack/ai-svelte build)", | ||
| "Bash(pnpm --filter @tanstack/ai-react test:lib)", | ||
| "Bash(pnpm --filter @tanstack/ai-solid test:lib)", | ||
| "Bash(pnpm --filter @tanstack/ai-vue test:lib)", | ||
| "Bash(pnpm --filter @tanstack/ai-svelte test:lib)", | ||
| "Bash(pnpm test:docs)", | ||
| "Bash(npx vitest run packages/typescript/ai-vue/tests/use-generation.test.ts)", | ||
| "Bash(pnpm test:types)", | ||
| "Bash(npx tsr generate)", | ||
| "Bash(npx nx run ts-react-media:build)", | ||
| "Bash(npx nx run @tanstack/ai:test:types)", | ||
| "Bash(npx nx run @tanstack/ai:test:types --verbose)", | ||
| "Bash(pnpm install)" | ||
| ] | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,189 @@ | ||
| --- | ||
| title: Generations | ||
| id: generations | ||
| order: 13 | ||
| --- | ||
|
|
||
| # Generations | ||
|
|
||
| TanStack AI provides a unified pattern for non-chat AI activities: **image generation**, **text-to-speech**, **transcription**, **summarization**, and **video generation**. These are collectively called "generations" — single request/response operations (as opposed to multi-turn chat). | ||
|
|
||
| All generations follow the same architecture, making it easy to learn one and apply it to the rest. | ||
|
|
||
| ## Architecture | ||
|
|
||
| ```mermaid | ||
| flowchart TB | ||
| subgraph Server ["Server"] | ||
| direction TB | ||
| activities["generateImage({ ..., stream: true }) | ||
| generateSpeech({ ..., stream: true }) | ||
| generateTranscription({ ..., stream: true }) | ||
| summarize({ ..., stream: true }) | ||
| generateVideo({ ..., stream: true })"] | ||
| transport["toServerSentEventsResponse()"] | ||
| activities --> transport | ||
| end | ||
|
|
||
| transport -- "StreamChunks via SSE | ||
| RUN_STARTED → generation:result → RUN_FINISHED" --> adapter | ||
|
|
||
| subgraph Client ["Client"] | ||
| direction TB | ||
| adapter["fetchServerSentEvents('/api/...')"] | ||
| gc["GenerationClient | ||
| (state machine)"] | ||
| hooks["Framework Hooks | ||
| useGenerateImage() · useGenerateSpeech() | ||
| useGenerateVideo() · useSummarize() | ||
| useTranscription()"] | ||
| adapter --> gc | ||
| gc -- "result, isLoading, error, status" --> hooks | ||
| end | ||
| ``` | ||
|
|
||
| The key insight: **every generation activity on the server is just an async function that returns a result**. By passing `stream: true`, the function returns a `StreamChunk` iterable instead of a plain result, which the client already knows how to consume. | ||
|
|
||
| ## Two Transport Modes | ||
|
|
||
| ### Streaming Mode (Connection Adapter) | ||
|
|
||
| The server passes `stream: true` to the generation function and sends the result as SSE. The client uses `fetchServerSentEvents()` to consume the stream. | ||
|
|
||
| **Server:** | ||
|
|
||
| ```typescript | ||
| import { generateImage, toServerSentEventsResponse } from '@tanstack/ai' | ||
| import { openaiImage } from '@tanstack/ai-openai' | ||
|
|
||
| // In your API route handler | ||
| const stream = generateImage({ | ||
| adapter: openaiImage('dall-e-3'), | ||
| prompt: 'A sunset over mountains', | ||
| stream: true, | ||
| }) | ||
|
|
||
| return toServerSentEventsResponse(stream) | ||
| ``` | ||
|
|
||
| **Client:** | ||
|
|
||
| ```tsx | ||
| import { useGenerateImage } from '@tanstack/ai-react' | ||
| import { fetchServerSentEvents } from '@tanstack/ai-client' | ||
|
|
||
| const { generate, result, isLoading } = useGenerateImage({ | ||
| connection: fetchServerSentEvents('/api/generate/image'), | ||
| }) | ||
| ``` | ||
|
|
||
| ### Direct Mode (Fetcher) | ||
|
|
||
| The client calls a server function directly and receives the result as JSON. No streaming protocol needed. | ||
|
|
||
| **Server:** | ||
|
|
||
| ```typescript | ||
| import { createServerFn } from '@tanstack/react-start' | ||
| import { generateImage } from '@tanstack/ai' | ||
| import { openaiImage } from '@tanstack/ai-openai' | ||
|
|
||
| export const generateImageFn = createServerFn({ method: 'POST' }) | ||
| .inputValidator((data: { prompt: string }) => data) | ||
| .handler(async ({ data }) => { | ||
| return generateImage({ | ||
| adapter: openaiImage('dall-e-3'), | ||
| prompt: data.prompt, | ||
| }) | ||
| }) | ||
| ``` | ||
|
|
||
| **Client:** | ||
|
|
||
| ```tsx | ||
| import { useGenerateImage } from '@tanstack/ai-react' | ||
| import { generateImageFn } from '../lib/server-functions' | ||
|
|
||
| const { generate, result, isLoading } = useGenerateImage({ | ||
| fetcher: (input) => generateImageFn({ data: input }), | ||
| }) | ||
| ``` | ||
|
|
||
| ## How Streaming Works | ||
|
|
||
| When you pass `stream: true` to any generation function, it returns an async iterable of `StreamChunk` events instead of a plain result: | ||
|
|
||
| ``` | ||
| 1. RUN_STARTED → Client sets status to 'generating' | ||
| 2. CUSTOM → Client receives the result | ||
| name: 'generation:result' | ||
| value: <your result> | ||
| 3. RUN_FINISHED → Client sets status to 'success' | ||
| ``` | ||
|
|
||
| If the function throws, a `RUN_ERROR` event is emitted instead: | ||
|
|
||
| ``` | ||
| 1. RUN_STARTED → Client sets status to 'generating' | ||
| 2. RUN_ERROR → Client sets error + status to 'error' | ||
| error: { message: '...' } | ||
| ``` | ||
|
|
||
| This is the same event protocol used by chat streaming, so the same transport layer (`toServerSentEventsResponse`, `fetchServerSentEvents`) works for both. | ||
|
|
||
| ## Common Hook API | ||
|
|
||
| All generation hooks share the same interface: | ||
|
|
||
| | Option | Type | Description | | ||
| |--------|------|-------------| | ||
| | `connection` | `ConnectionAdapter` | Streaming transport (SSE, HTTP stream, custom) | | ||
| | `fetcher` | `(input) => Promise<Result>` | Direct async function (no streaming) | | ||
| | `id` | `string` | Unique identifier for this instance | | ||
| | `body` | `Record<string, any>` | Additional body parameters (connection mode) | | ||
| | `onResult` | `(result) => T \| null \| void` | Transform or react to the result | | ||
| | `onError` | `(error) => void` | Error callback | | ||
| | `onProgress` | `(progress, message?) => void` | Progress updates (0-100) | | ||
|
|
||
| | Return | Type | Description | | ||
| |--------|------|-------------| | ||
| | `generate` | `(input) => Promise<void>` | Trigger generation | | ||
| | `result` | `T \| null` | The result (optionally transformed), or null | | ||
| | `isLoading` | `boolean` | Whether generation is in progress | | ||
| | `error` | `Error \| undefined` | Current error, if any | | ||
| | `status` | `GenerationClientState` | `'idle'` \| `'generating'` \| `'success'` \| `'error'` | | ||
| | `stop` | `() => void` | Abort the current generation | | ||
| | `reset` | `() => void` | Clear all state, return to idle | | ||
|
|
||
| ### Result Transform | ||
|
|
||
| The `onResult` callback can optionally transform the stored result: | ||
|
|
||
| - Return a **non-null value** — replaces the stored result with the transformed value | ||
| - Return **`null`** — keeps the previous result unchanged (useful for filtering) | ||
| - Return **nothing** (`void`) — stores the raw result as-is | ||
|
|
||
| TypeScript automatically infers the result type from your `onResult` return value — no explicit generic parameter needed. | ||
|
|
||
| ```tsx | ||
| const { result } = useGenerateSpeech({ | ||
| connection: fetchServerSentEvents('/api/generate/speech'), | ||
| onResult: (raw) => ({ | ||
| audioUrl: `data:${raw.contentType};base64,${raw.audio}`, | ||
| duration: raw.duration, | ||
| }), | ||
| }) | ||
| // result is typed as { audioUrl: string; duration?: number } | null | ||
| ``` | ||
|
|
||
| ## Available Generations | ||
|
|
||
| | Activity | Server Function | Client Hook (React) | Guide | | ||
| |----------|----------------|---------------------|-------| | ||
| | Image generation | `generateImage()` | `useGenerateImage()` | [Image Generation](./image-generation) | | ||
| | Text-to-speech | `generateSpeech()` | `useGenerateSpeech()` | [Text-to-Speech](./text-to-speech) | | ||
| | Transcription | `generateTranscription()` | `useTranscription()` | [Transcription](./transcription) | | ||
| | Summarization | `summarize()` | `useSummarize()` | - | | ||
| | Video generation | `generateVideo()` | `useGenerateVideo()` | [Video Generation](./video-generation) | | ||
|
|
||
| > **Note:** Video generation uses a jobs/polling architecture. The `useGenerateVideo` hook additionally exposes `jobId`, `videoStatus`, `onJobCreated`, and `onStatusUpdate` for tracking the polling lifecycle. See the [Video Generation](./video-generation) guide for details. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.