add livekit_agent_simulation service, agent remote session proto & cli dev proto#1404
add livekit_agent_simulation service, agent remote session proto & cli dev proto#1404theomonnom wants to merge 20 commits intomainfrom
Conversation
|
| STATUS_FAILED = 3; | ||
| } | ||
|
|
||
| message Job { |
There was a problem hiding this comment.
do we want to nest this message? seems like it gets complicated to use.. vs SimulationJob
| string error = 9; | ||
| string group_id = 10; | ||
| // Set as participant attributes on the simulation participant's access token. | ||
| map<string, string> metadata = 11; |
There was a problem hiding this comment.
let's be explicit so it's not confused with participant metadata
| map<string, string> metadata = 11; | |
| map<string, string> participant_attributes = 11; |
| string agent_name = 2; | ||
| string agent_description = 3; | ||
| int32 num_simulations = 4; | ||
| string group_id = 5; |
There was a problem hiding this comment.
how do I just run a single simulation scenario?
| int64 created_at = 7; | ||
| Status status = 8; | ||
| string error = 9; | ||
| string group_id = 10; |
There was a problem hiding this comment.
can you create a scenario without a group? and then assign it to a group later? or should it have a default group?
| string instructions = 3; | ||
| string agent_expectations = 4; | ||
| string group_id = 5; | ||
| map<string, string> metadata = 6; |
There was a problem hiding this comment.
should source_room_id be part of this too? same comment about metadata
| } | ||
| message DeleteScenarioResponse {} | ||
|
|
||
| message UpdateScenarioRequest { |
There was a problem hiding this comment.
would it be simpler to just include the Scenario message here and also in Create?
|
Hey @theomonnom, nice work! I have a suggestion for a logically identical structure, just with more nesting. Do what you like with it, but I find it much easier to reason about and the generated struct names are easier to read. &backend.AccessRecord_Subject_LivekitSupport{}
// vs:
&backend.AccessRecordSubjectLivekitSupport{}Branch based of yours: https://github.com/livekit/protocol/blob/js/agent-sim/protobufs/livekit_agent_simulation.proto Like I said, do what you want with it, I just wanted to share this as I found it really valuable. |
- Move response types from SessionRequest into SessionResponse - Rename SendMessage/SendMessageResponse to RunInput/RunInputResponse - Add OverlappingSpeech and SessionUsageUpdated events, remove MetricsCollected - Add model usage types (LLM, TTS, STT, Interruption) - Add GetSessionState, GetRTCStats, GetSessionUsage request/response - Restructure simulation proto with nested CRUD pattern - Migrate simulation created_at from int64 to google.protobuf.Timestamp - Add pagination and status filter to ListSimulationRuns
Moved from livekit.agent package to livekit to break the agent -> livekit import cycle.
Done, using TokenPagination |
theomonnom
changed the title
| string url = 5; | ||
| string token = 6; | ||
| string worker_id = 7; | ||
| bool fake_job = 8; |
| uint32 samples_per_channel = 4; | ||
| } | ||
|
|
||
| enum SessionAgentState { |
There was a problem hiding this comment.
these should be re-usable right? should they just be AgentState & UserState?
also recommend the enums to be abbreviated, i.e.:
AS_INITIALIZING
| repeated ModelUsage model_usage = 1; | ||
| } | ||
|
|
||
| message SessionEvent { |
There was a problem hiding this comment.
prefix with AgentSession to disambiguate?
| message SessionEvent { | |
| message AgentSessionEvent { |
| message SessionAudioPlaybackClear {} | ||
| message SessionAudioPlaybackFinished {} | ||
|
|
||
| message AgentSessionMessage { |
There was a problem hiding this comment.
so these will be text or data streams, while the SessionRequest/Responses are using RPCs?
3 participants
No description provided.