feat: add realtime chat support with OpenAI and ElevenLabs adapters

.changeset/realtime-chat.md

@@ -0,0 +1,18 @@
+---
+'@tanstack/ai': minor
+'@tanstack/ai-client': minor
+'@tanstack/ai-openai': minor
+'@tanstack/ai-elevenlabs': minor
+'@tanstack/ai-react': minor
+---
+
+feat: add realtime voice chat with OpenAI and ElevenLabs adapters
+
+Adds realtime voice/text chat capabilities:
+
+- **@tanstack/ai**: `realtimeToken()` function and shared realtime types (`RealtimeToken`, `RealtimeMessage`, `RealtimeSessionConfig`, `RealtimeStatus`, `RealtimeMode`, `AudioVisualization`, events, and error types)
+- **@tanstack/ai-client**: Framework-agnostic `RealtimeClient` class with connection lifecycle, audio I/O, message state management, tool execution, and `RealtimeAdapter`/`RealtimeConnection` interfaces
+- **@tanstack/ai-openai**: `openaiRealtime()` client adapter (WebRTC) and `openaiRealtimeToken()` server token adapter with support for semantic VAD, multiple voices, and all realtime models
+- **@tanstack/ai-elevenlabs**: `elevenlabsRealtime()` client adapter (WebSocket) and `elevenlabsRealtimeToken()` server token adapter for ElevenLabs conversational AI agents
+- **@tanstack/ai-react**: `useRealtimeChat()` hook with reactive state for status, mode, messages, pending transcripts, audio visualization levels, VAD control, text/image input, and interruptions
+- **Docs**: Realtime Voice Chat guide and full API reference for all realtime classes, interfaces, functions, and type aliases

docs/config.json

@@ -78,6 +78,10 @@
           "label": "Runtime Adapter Switching",
           "to": "guides/runtime-adapter-switching"
         },
+        {
+          "label": "Realtime Voice Chat",
+          "to": "guides/realtime-chat"
+        },
         {
           "label": "Text-to-Speech",
           "to": "guides/text-to-speech"
@@ -224,6 +228,10 @@
           "label": "ToolCallManager",
           "to": "reference/classes/ToolCallManager"
         },
+        {
+          "label": "RealtimeClient",
+          "to": "reference/classes/RealtimeClient"
+        },
         {
           "label": "WordBoundaryStrategy",
           "to": "reference/classes/WordBoundaryStrategy"
@@ -311,6 +319,10 @@
           "label": "uiMessageToModelMessages",
           "to": "reference/functions/uiMessageToModelMessages"
         },
+        {
+          "label": "realtimeToken",
+          "to": "reference/functions/realtimeToken"
+        },
         {
           "label": "untilFinishReason",
           "to": "reference/functions/untilFinishReason"
@@ -342,6 +354,10 @@
           "label": "AudioPart",
           "to": "reference/interfaces/AudioPart"
         },
+        {
+          "label": "AudioVisualization",
+          "to": "reference/interfaces/AudioVisualization"
+        },
         {
           "label": "BaseStreamChunk",
           "to": "reference/interfaces/BaseStreamChunk"
@@ -422,6 +438,38 @@
           "label": "ProcessorState",
           "to": "reference/interfaces/ProcessorState"
         },
+        {
+          "label": "RealtimeAdapter",
+          "to": "reference/interfaces/RealtimeAdapter"
+        },
+        {
+          "label": "RealtimeClientOptions",
+          "to": "reference/interfaces/RealtimeClientOptions"
+        },
+        {
+          "label": "RealtimeConnection",
+          "to": "reference/interfaces/RealtimeConnection"
+        },
+        {
+          "label": "RealtimeMessage",
+          "to": "reference/interfaces/RealtimeMessage"
+        },
+        {
+          "label": "RealtimeSessionConfig",
+          "to": "reference/interfaces/RealtimeSessionConfig"
+        },
+        {
+          "label": "RealtimeToken",
+          "to": "reference/interfaces/RealtimeToken"
+        },
+        {
+          "label": "RealtimeTokenAdapter",
+          "to": "reference/interfaces/RealtimeTokenAdapter"
+        },
+        {
+          "label": "RealtimeTokenOptions",
+          "to": "reference/interfaces/RealtimeTokenOptions"
+        },
         {
           "label": "ResponseFormat",
           "to": "reference/interfaces/ResponseFormat"
@@ -577,6 +625,22 @@
           "label": "MessagePart",
           "to": "reference/type-aliases/MessagePart"
         },
+        {
+          "label": "RealtimeEvent",
+          "to": "reference/type-aliases/RealtimeEvent"
+        },
+        {
+          "label": "RealtimeMessagePart",
+          "to": "reference/type-aliases/RealtimeMessagePart"
+        },
+        {
+          "label": "RealtimeMode",
+          "to": "reference/type-aliases/RealtimeMode"
+        },
+        {
+          "label": "RealtimeStatus",
+          "to": "reference/type-aliases/RealtimeStatus"
+        },
         {
           "label": "ModalitiesArrayToUnion",
           "to": "reference/type-aliases/ModalitiesArrayToUnion"