fix(core): don't mark FallbackAdapter primary unavailable on empty LLM turns#1455
Open
ubinatus wants to merge 1 commit into
Open
fix(core): don't mark FallbackAdapter primary unavailable on empty LLM turns#1455ubinatus wants to merge 1 commit into
ubinatus wants to merge 1 commit into
Conversation
…M turns
When the LLM emits a turn with zero spoken text (e.g. a tool-only turn),
`FallbackSynthesizeStream` would throw `APIConnectionError("TTS stream
completed but no audio was received")`, mark the first provider
unavailable, and cascade through the fallback chain — ultimately raising
`all TTS instances failed`. The empty audio is the correct response
because nothing was sent to synthesize.
Track non-sentinel tokens received from the input LLM stream. In the
`!sawRawAudio` branch, await the input stream so the count is settled
and, if zero real tokens were received, treat the empty result as a
clean no-op exit (emit END_OF_STREAM downstream and return) instead of
throwing. Silent failures with real text still trigger fallback.
Adds a unit test that closes the input without enqueuing any text and
asserts no provider is marked unavailable.
🦋 Changeset detectedLatest commit: 2239c17 The changes in this PR will be included in the next version bump. This PR includes changesets to release 31 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1454.
Problem
FallbackSynthesizeStreamthrowsAPIConnectionError("TTS stream completed but no audio was received")and triggersmarkUnAvailable+ cascade to the next provider on turns where the LLM emits zero spoken text — e.g., a turn whose content is a tool call with notext.In
agents/src/tts/fallback_adapter.ts,forwardBufferToTTSmay consume the LLM stream and forward only sentinel tokens without ever callingstream.pushText(...). The provider correctly returns no audio because nothing was sent to synthesize. The adapter then hitsif (!sawRawAudio)and throws, which marks the first provider unavailable, retries through the rest of the chain, and ultimately raisesall TTS instances failed. In production this manifests as recurring spurious failover storms and skewed availability metrics on agents that use tools.Fix
Track non-sentinel tokens received by
readInputLLMStream(realTokensReceived). In the!sawRawAudiobranch, await the input LLM stream so the count is settled, then:realTokensReceived === 0, emitSynthesizeStream.END_OF_STREAMdownstream and return cleanly,The signal is taken from the input side rather than the forward side because the forward task can exit early via
streamOutputCompleted/abortedwhen the inner stream fails, masking text that was actually sitting in the buffer.Test
Added
should not mark the primary unavailable when the LLM emits zero text tokens: closes the adapter's input stream without enqueuing any text and asserts both providers remain available and no frames were produced.Full
agents/suite is green (762 passed, 2 skipped).Affected version
@livekit/agents@1.4.0(same code path in 1.3.x).