Add UniFFI interface for livekit-wakeword#939
Conversation
Expose WakeWordDetector as a UniFFI Object wrapping WakeWordModel with Mutex for interior mutability. Includes a flat wrapper error type that converts from WakeWordError since its inner types (ort::Error, ndarray::ShapeError, etc.) are not UniFFI-compatible.
Drop the thiserror wrapper error and From impl in favor of #[uniffi::remote(Error)] with #[uniffi(flat_error)], matching the pattern used by AccessTokenError. Removes the thiserror dependency from livekit-uniffi.
Fixes Windows CI linking error by enabling ort to download prebuilt ONNX Runtime binaries on all platforms.
The prebuilt ONNX Runtime binaries from ort's download-binaries feature are built with dynamic CRT (/MD), which conflicts with +crt-static (/MT) causing unresolved symbols for CRT functions.
| /// | ||
| /// Pass ~2 seconds of i16 PCM audio at the sample rate configured in | ||
| /// [`new`](Self::new). Returns a map of classifier name to confidence score. | ||
| pub fn predict(&self, audio_chunk: Vec<i16>) -> Result<HashMap<String, f32>, WakeWordError> { |
There was a problem hiding this comment.
suggestion: This can accept a non-mutable slice &[i16] to avoid cloning. This is also supported by UniFFI.
There was a problem hiding this comment.
I see, I didn't know that, will make a commit
| [dependencies] | ||
| ndarray = "0.17.2" | ||
| ort = { version = "2.0.0-rc.11", default-features = false, features = ["ndarray", "std"] } | ||
| ort = { version = "2.0.0-rc.11", default-features = false, features = ["ndarray", "std", "download-binaries", "tls-native"] } |
There was a problem hiding this comment.
question: Which binaries does ORT need to download? If this is necessary, we shouldn't hardcode tls-native and instead allow the consumer of the crate to choose the TLS feature they want to use based on their requirements (much like how ort does so itself).
There was a problem hiding this comment.
It isn't necessary. It was a suggestion from copilot to solve linking problems on Windows. Would u mind looking into the problem on windows for me @ladvoc ? It has some weird problems with the way our rust config is handled on windows.
ladvoc
left a comment
ladvoc
left a comment
There was a problem hiding this comment.
Looking great! Some initial comments
|
I placed wake word support in |
|
@pblazej, if we were to integrate this into the Swift SDK, what questions/concerns would you have (e.g., dylib size)? As I noted above, wakework support is behind a feature so we can compile without support for it for now. |
|
Surprisingly, we currently use a static lib as you cannot ship a bare We may revisit that in the future, no complaints yet about the bundle size per se. |
would it be fine to just disable building with wakeword on windows? |
3 participants
Summary
WakeWordDetectoras a UniFFI Object wrappingWakeWordModelwithMutexfor interior mutabilitynew,load_model, andpredictmethods across FFI#[uniffi::remote(Error)]with#[uniffi(flat_error)]forWakeWordError, matching the existingAccessTokenErrorpatternTest plan
cargo check -p livekit-uniffipasses