get webrtc adm into rust

[xianshijing-lk]

Summary

This PR implements Platform Audio support for the LiveKit Rust SDK, enabling WebRTC's built-in audio device handling with microphone capture and speaker playout. The implementation introduces a handle-based PlatformAudio API that coexists with the existing NativeAudioSource for manual audio pushing.

Key Features

• Two Audio Source Types:
  • RtcAudioSource::Native (default): Manual audio push via NativeAudioSource for TTS, file streaming, agents
  • RtcAudioSource::Device: WebRTC handles mic capture & speaker playout with echo cancellation (AEC)
• Handle-based API: Create PlatformAudio instances that enable ADM recording; drop to release
• Reference Counting: Multiple PlatformAudio instances share the same underlying ADM
• Device Enumeration & Selection: List and select recording/playout devices
• Hot-swap Device Switching: switch_recording_device() / switch_playout_device() for changing devices during active sessions
• Audio Processing Configuration: AEC, AGC, NS with hardware/software preference
• WebRTC Patching: external_audio_source.patch prevents audio mixing conflicts between device and manual sources

Design Document

See docs/ADM_PROXY_DESIGN.md for full architecture details including:

• Recording gate pattern
• WebRTC patching explanation
• FFI API documentation

API Overview

use livekit::prelude::*;

// Create PlatformAudio instance (enables ADM recording)
let audio = PlatformAudio::new()?;

// Enumerate and select devices
for i in 0..audio.recording_devices() as u16 {
println!("Mic [{}]: {}", i, audio.recording_device_name(i));
}
audio.set_recording_device(0)?;

// Connect and publish
let (room, _) = Room::connect(&url, &token, RoomOptions::default()).await?;
let track = LocalAudioTrack::create_audio_track("mic", audio.rtc_source());
room.local_participant().publish_track(LocalTrack::Audio(track), opts).await?;

// Cleanup - just drop the handle
room.close().await?;
drop(audio); // ADM recording disabled when all handles released

Testing

Run Standalone Tests (no LiveKit server required)

Set custom WebRTC build path

export LK_CUSTOM_WEBRTC="/path/to/webrtc-sys/libwebrtc/mac-arm64-debug"

Run standalone PlatformAudio tests

cargo test -p livekit --test platform_audio_test test_platform_audio_standalone -- --nocapture

Run FFI request handler tests

cargo test -p livekit-ffi requests::tests -- --nocapture

Run E2E Integration Tests (requires LiveKit server)

Start a local LiveKit server first, then:

LIVEKIT_URL=ws://localhost:7880
LIVEKIT_API_KEY=devkey
LIVEKIT_API_SECRET=secret
cargo test -p livekit --test platform_audio_test --features __lk-e2e-test -- --nocapture

Test Coverage
Category │ Tests │ Description │

Standalone - Creation │ 1 │ PlatformAudio creation, device enumeration
Standalone - Ref Counting │ 1 │ Clone, sharing, drop behavior
Standalone - Device Selection │ 1 │ Set devices, invalid index handling
Standalone - Processing │ 1 │ AEC/AGC/NS configuration, hardware availability
Standalone - Reset │ 1 │ reset_platform_audio() function
Standalone - Lifecycle │ 1 │ Full create→configure→use→release cycle
FFI - Handlers │ 6 │ NewPlatformAudio, GetDevices, SetDevice, handle lifecycle
E2E - Room Connection │ 4+ │ Platform audio with room, two participants, device switching

All tests handle missing audio devices gracefully (CI-friendly).
Run the Example

List Audio Devices

cargo run -p basic_room -- --list-devices

Connect with Platform Audio (microphone capture)

LIVEKIT_URL=wss://your-server.livekit.cloud
LIVEKIT_API_KEY=your-key
LIVEKIT_API_SECRET=your-secret
cargo run -p basic_room -- --platform-audio

Connect with File Audio

cargo run -p basic_room -- --file path/to/audio.raw

Connect with Both Platform Audio and File

cargo run -p basic_room -- --platform-audio-and-file path/to/audio.raw

WebRTC Build Requirements

The external_audio_source.patch must be applied to WebRTC. The patch is automatically applied by all platform build scripts:

• build_macos.sh
• build_ios.sh
• build_linux.sh
• build_android.sh
• build_windows.cmd

For local development, set LK_CUSTOM_WEBRTC to point to your patched WebRTC build.

Known Limitations

   Limitation      │                                  Description 

Process-global │ Audio configuration affects all rooms in the process
Device indices │ May change on hot-plug; match by name for persistence
Single device track │ One device audio track per ADM (use NativeAudioSource for additional streams)

[ladvoc]

nitpick: This should probably be defined as a constant somewhere.

[ladvoc]

comment: Generally we should avoid FFI-only methods in the public API—although we do have them in some places. If that is unavoidable in this case, I would recommend annotating with #[doc(hidden)].

[ladvoc]

suggestion: A potentially cleaner way to handle this is to have every room hold a Arc<()> and leverage strong_count to learn the number of active rooms—no need to manually decrement.

[ladvoc]

comment (non-blocking): Currently in CI, all tests are run serially. If we switch over to Nextest (outdated PR, #816), we can configure which tests are run in serial through that config and run everything else in parallel.

[github-actions]

No changeset found

This PR modifies the following packages but doesn't include a changeset:

Directly changed:

• libwebrtc
• livekit
• livekit-ffi
• webrtc-sys

Click here to create a changeset

The link pre-populates a changeset file with patch bumps for all affected packages.
Edit the description and bump types as needed before committing.

If this change doesn't require a version bump, add the internal label to this PR.

[MaxHeimbrock]

This is the original audio input right? Existing Unity clients who want to keep the "Unity" style microphone management would also use this.

[MaxHeimbrock]

Is this also possible from a Unity client? As we discussed, for the lip sync animation Unity clients might want read access to the audio data, but still want output through the platform audio.

[MaxHeimbrock]

How does it handle switching the device at runtime?

[MaxHeimbrock]

Or screen share audio right?