internal(voice): wire DebugMessage over remote-session wire#5855
Open
toubatbrian wants to merge 9 commits into
Open
internal(voice): wire DebugMessage over remote-session wire#5855toubatbrian wants to merge 9 commits into
toubatbrian wants to merge 9 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Picks up the new agent-session messages required for the custom-event work that lands in the next commit: - AgentSessionEvent.custom_event + top-level CustomEvent - AgentSessionEvent.FunctionToolsStarted - AgentSessionEvent.EotPrediction uv lock pinned to 1.1.9. Co-authored-by: Cursor <cursoragent@cursor.com>
…ents into brian/custom-ev-proto
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Member
|
Should we just directly define messages inside livekit/protocol? |
The earlier `chore(deps): bump livekit-protocol to 1.1.9` commit rewrote every entry in uv.lock (~2,584 lines of `+ upload-time = "..."` additions and `revision = 1 -> 2`) because the local uv was newer than the version that wrote main's lockfile. None of that churn was semantic. Restore uv.lock to main so the PR diff stays focused on the pyproject.toml pin bump. CI uses `uv sync --all-extras --dev` (no `--frozen`), so it will auto-relock against the new floor without failing. A future PR can do a clean lockfile-format upgrade in isolation. Co-authored-by: Cursor <cursoragent@cursor.com>
Theo's review: "Should we just directly define messages inside
livekit/protocol?" The Pydantic `CustomEvent` wrapper here was pure
parallel structure -- the proto already has the exact shape we need
(`type: str` + `payload: google.protobuf.Struct`). Wrapping it added a
dict<->Struct round-trip on every emit and an extra type to keep in
sync, with no semantic gain.
Drop the wrapper and the helpers:
- Remove `CustomEvent` BaseModel from `voice/events.py` and from the
`AgentEvent` discriminated union.
- Remove `_dict_to_struct` / `_struct_to_dict` from `voice/remote_session.py`.
- `_on_custom_event` now takes `agent_pb.CustomEvent` and forwards it
through `AgentSessionEvent(custom_event=...)` as-is.
User-facing change: callers emit the proto directly:
session.emit("custom_event", agent_pb.CustomEvent(type=..., payload=struct))
`"custom_event"` stays in `EventTypes` so the listener registration
contract is unchanged; only the payload type narrowed from a wrapper
BaseModel to the proto.
Co-authored-by: Cursor <cursoragent@cursor.com>
toubatbrian
changed the title
toubatbrian
added a commit
to livekit/agents-js
that referenced
this pull request
May 26, 2026
Mirrors the Python-side decision (livekit/agents#5855): with CustomEvent already defined in livekit/protocol, the parallel TS wrapper + dict<->Struct conversion was pure ceremony with no semantic gain. Drop the wrapper and forward the proto unchanged. - events.ts: remove `CustomEvent` type + `createCustomEvent` helper + membership in the `AgentEvent` union. Keep the `AgentSessionEventTypes.CustomEvent` enum entry so the listener topic name stays stable. - agent_session.ts: callback signature is now `(ev: pb.CustomEvent) => void` (import-as-type). - remote_session.ts: - drop `dictToStruct` / `structToDict` and the `JsonValue` import - `RemoteSessionCallbacks.custom_event` typed as `pb.CustomEvent` - `onCustomEvent` forwards the proto as-is via `emitEvent({ case: 'customEvent', value: event })` - dispatch side re-emits `ev.value` directly (no Struct->dict round-trip) User-facing emit: session.emit( 'custom_event', new pb.CustomEvent({ type: 'anomaly_detected', payload: Struct.fromJson({ score: 0.92 }), }), ); `tsc --noEmit` and lint clean (no new warnings beyond the pre-existing test-file `any` noise). Co-authored-by: Cursor <cursoragent@cursor.com>
3 tasks
Sugar for the common case: build agent_pb.CustomEvent from a Python
dict payload and emit it, so callers don't have to import
google.protobuf.Struct + ParseDict at the call site.
session.emit_custom_event("anomaly_detected", {"score": 0.92})
is equivalent to:
from google.protobuf.struct_pb2 import Struct
from google.protobuf.json_format import ParseDict
st = Struct()
ParseDict({"score": 0.92}, st, ignore_unknown_fields=True)
session.emit("custom_event", agent_pb.CustomEvent(type="anomaly_detected", payload=st))
The framework layer stays proto-native (RemoteSession forwards
`agent_pb.CustomEvent` as-is); this is purely a call-site
convenience.
Uses `super().emit(...)` to bypass `AgentSession.emit`'s typed
override, which narrows `arg: AgentEvent` for the Pydantic
event-trace report. Custom events ride the proto and aren't part of
that typed taxonomy.
Co-authored-by: Cursor <cursoragent@cursor.com>
…r counts
- ruff format collapsed the multi-line `emit_custom_event` signature
- the new `session.on('custom_event', ...)` listener wired up in
SessionHost.register_session bumps the on/off counts from 8 to 9 in
test_session_host.py
Co-authored-by: Cursor <cursoragent@cursor.com>
3 tasks
toubatbrian
added a commit
to livekit/protocol
that referenced
this pull request
May 27, 2026
…1593) * agent_session: rename CustomEvent -> DebugMessage; drop type field Renames the agent-session event added in #1588 before any consumer ships it. Repositioned as an internal debug/trace channel surfaced only to the debugger/recorder, not to user code -- the `type` discriminator was unnecessary since callers just emit a free-form JSON payload. Wire: AgentSessionEvent.custom_event (field 21) -> AgentSessionEvent.debug_message (same field number, same type slot). No schema-compat concerns -- nothing has been built downstream against this yet (livekit/agents#5855 and livekit/agents-js#1611 are both still open). Regenerated Go (.pb.go) with protoc 25.1; JS (@livekit/protocol) regenerated via `pnpm --filter @livekit/protocol generate:proto`. Co-authored-by: Cursor <cursoragent@cursor.com> * generated protobuf * changeset: patch bump, not minor Co-authored-by: Cursor <cursoragent@cursor.com> * drop 'free-form' from DebugMessage docs Co-authored-by: Cursor <cursoragent@cursor.com> * generated protobuf --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
…_emit_debug_message Picks up livekit/protocol#1593 which renamed the agent-session event before any consumer shipped it. The message is repositioned as an internal debug/trace channel surfaced only to the debugger/recorder (e.g. agents-cli), not to user code. Proto change (mechanical rename): - AgentSessionEvent.custom_event -> AgentSessionEvent.debug_message (field 21) - agent_pb.CustomEvent (str type, Struct payload) -> agent_pb.DebugMessage (Struct payload) Surface change: - AgentSession.emit_custom_event(event_type, payload) -> AgentSession._emit_debug_message(payload) - underscore prefix + `:meta private:` docstring marker signal: not for user code - type discriminator dropped; callers just emit a JSON payload - EventTypes literal "custom_event" -> "debug_message" - SessionHost._on_custom_event -> _on_debug_message Pin bump: `livekit-protocol>=1.1.9` -> `>=1.1.10` to pick up DebugMessage. (CI type-check is blocked until livekit/python-sdks#689 merges and a new livekit-protocol release is cut to PyPI.) Drive-by deslop on the helper body: - `from google.protobuf...` imports lifted to module scope (avoids the no-inline-imports convention warning). - Dropped the defensive `if payload:` + `dict(payload)` cast and the `ignore_unknown_fields=True` kwarg (Struct fields aren't user fields). - Trimmed the 12-line docstring + 3-line inline comment down to the one non-obvious line that documents *why* we use `super().emit`. ruff check + ruff format --check clean. Co-authored-by: Cursor <cursoragent@cursor.com>
toubatbrian
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wires the new
AgentSessionEvent.debug_message(livekit/protocol#1593) throughSessionHost, and adds an internal-onlyAgentSession._emit_debug_message(payload)helper for the debugger/recorder (e.g.agents-cli).Not a user-facing API — the leading underscore +
:meta private:docstring marker signal that this is not intended to be called from user code. The event rides the existingAgentSessionEventwrapper on the wire; nothing else in the session lifecycle changes.Changes
pyproject.toml— bumpslivekit-protocolfloor>=1.1.9→>=1.1.10(picks upDebugMessage, the renamedAgentSessionEvent.debug_messagefield,FunctionToolsStarted,EotPrediction).voice/events.py—EventTypesliteral"custom_event"→"debug_message". No wrapper BaseModel.voice/agent_session.py:_emit_debug_message(payload: Mapping[str, Any]) -> None— wrapspayloadinto agoogle.protobuf.Structand emitsagent_pb.DebugMessage. Bypasses the typedemitoverride (which narrows toAgentEventfor the Pydantic event-trace report); debug messages ride the proto and aren't part of that typed taxonomy.from google.protobuf.json_format import ParseDict+from google.protobuf.struct_pb2 import Structlifted to module scope (per the no-inline-imports convention).voice/remote_session.py—SessionHostlistens on"debug_message"; the handler takesagent_pb.DebugMessageand forwards it throughAgentSessionEvent(debug_message=...).Usage (internal only)
```python
debugger / recorder code only — not for user code
session._emit_debug_message({"kind": "anomaly_detected", "score": 0.92})
```
External listeners can subscribe to the event topic via the EventEmitter, but the canonical emit path is the underscore method. There is no stable public emit API for debug events.
Test plan
Publishing pipeline
CI type-check is blocked until:
v1.46.4, version to1.1.10)After that the pin in
pyproject.toml(>=1.1.10) will resolve andmypywill seeagent_pb.DebugMessage. Same publishing-pipeline blocker as the previous1.1.9cut — no code change needed once the package is released.Notes
_emitDebugMessageshape on the JS side; published already against@livekit/protocol@1.46.4).