Skip to content

Conversation

@KrishnaShuk
Copy link
Contributor

Description

#868
This PR implements the FallbackAdapter for LLMs, bringing feature parity with the Python SDK. This component allows agents to automatically switch to backup LLM providers if the primary provider fails, times out, or becomes unavailable, ensuring high availability for production agents.

Python(agents) SDK code for fallback adapter: https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/llm/fallback_adapter.py

Changes Made

  • Created packages/agents/src/llm/FallbackAdapter.ts implementing the FallbackAdapter and FallbackLLMStream classes.
  • Implemented Priority Failover: Accepts a list of LLMs and attempts them in order.
  • Implemented Active Health Check: Background recovery logic that uses the actual ChatContext to verify when a failed provider has recovered (instead of a simple timer).
  • Implemented State Tracking: Emits llm_availability_changed events when providers go down or recover.
  • Implemented Glitch Protection: Respects retryOnChunkSent to prevent the agent from repeating partial sentences if a stream fails mid-generation.
  • Exported the new adapter in packages/agents/src/llm/index.ts.

Pre-Review Checklist

  • Build passes: All builds (lint, typecheck, tests) pass locally
  • AI-generated code reviewed: Removed unnecessary comments and ensured code quality
  • Changes explained: All changes are properly documented and justified above
  • Scope appropriate: All changes relate to the PR title, or explanations provided for why they're included
  • Video demo: N/A (Backend logic change, verified via Unit Tests)

Testing

  • Automated tests added/updated
  • All tests pass (8 tests covering initialization, failover, total failure, events, and stream delegation)
  • Make sure both restaurant_agent.ts and realtime_agent.ts work properly (This change is additive/opt-in and does not affect existing agents unless configured to use the adapter)

@changeset-bot
Copy link

changeset-bot bot commented Dec 16, 2025

⚠️ No Changeset found

Latest commit: 7595a17

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@KrishnaShuk KrishnaShuk marked this pull request as draft December 17, 2025 05:10
@KrishnaShuk KrishnaShuk marked this pull request as ready for review December 17, 2025 05:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant