Skip to content

fix(personaplex): fix connection and audio streaming#5106

Open
ShayneP wants to merge 33 commits intomainfrom
ShayneP/personaplex-plugin
Open

fix(personaplex): fix connection and audio streaming#5106
ShayneP wants to merge 33 commits intomainfrom
ShayneP/personaplex-plugin

Conversation

@ShayneP
Copy link
Contributor

@ShayneP ShayneP commented Mar 13, 2026

Summary

Fixes the PersonaPlex realtime plugin so it can successfully connect and stream audio with the PersonaPlex server. Based on @milanperovic's work in #1399.

  • Pin sphn>=0.1.4,<0.2 — The PersonaPlex server requires sphn <0.2. The 0.1.x and 0.2.x versions use incompatible wire formats and different APIs (append_pcm()/read_bytes() two-step in 0.1.x vs single return value in 0.2.x). Using the wrong version caused the server's opus_reader.read_pcm() to return None, crashing the server-side opus decode loop.
  • Fix audio frame size (1920 samples, was 2400) — sphn 0.1.x requires valid Opus frame sizes (120, 240, 480, 960, 1920, 2880). The previous value of SAMPLE_RATE // 10 = 2400 is not a valid Opus frame size and raises a ValueError.
  • Wait for handshake before sending audio — The PersonaPlex server runs an is_alive() check during system prompt processing (~2-9s) that consumes incoming WS messages without feeding them to the Opus decoder. Audio sent during this phase is silently dropped. The official JS client only starts streaming mic audio after the handshake arrives. This change matches that behavior — audio frames buffer in the send channel during prompt processing and flush immediately when the handshake arrives.
  • Handle close code 1000 gracefully — When the server closes the connection normally (code 1000), finalize any active generation and reconnect cleanly instead of treating it as a retriable error with exponential backoff.
  • Flush stale audio on reconnect — Close and replace _msg_ch between connections so Opus-encoded frames from a previous session don't leak into a new connection with fresh codec state.
  • Use sphn 0.1.x API — Switch _encode_and_send and _handle_audio_data to the two-step sphn 0.1.x API.

milanperovic and others added 30 commits January 30, 2026 11:54
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Guard audio/text channel sends with ChanClosed suppression
- Wrap recv_task message handlers in try/except to prevent single
  malformed frame from breaking the receive loop
- Replace deprecated asyncio.get_event_loop() with get_running_loop()
- Add response_id to GenerationCreatedEvent emissions
- Add exponential backoff (1s -> 30s max) on reconnect instead of
  fixed 1s delay
- Add 32 unit tests covering init, URL building, audio conversion,
  retry backoff, options, and data structures

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove unused imports in tests (ruff F401)
- Fix import sorting (ruff I001)
- Apply ruff format to realtime_model.py
- Add __pdoc__ dict to realtime/__init__.py for docs generation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Preserve wss:// / https:// scheme for TLS deployments
- Use logger.warning for unsupported dynamic instructions
- Remove redundant length check on opus_bytes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Rename TestBuildWsUrl to TestSessionOptions with proper typed helper
- Rename TestHandleTextToken to TestSpecialTokens to reflect actual scope
- Wrap ResponseGeneration assertions in try/finally for channel cleanup

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Added by upstream in AGT-2474 (#4622). PersonaPlex is full-duplex
and does not support explicit user turn commits.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use ws:// scheme in all example URLs (README, docstrings)
- Sync uv.lock after upstream merge

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@chenghao-mou chenghao-mou requested a review from a team March 13, 2026 20:09
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 6 additional findings.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants