[VoiceLive] Update for agent V2, remove foundry tools, rename filler response #47979
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…emove FoundryAgentTool, add AgentSessionConfig sample
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request updates the Azure VoiceLive SDK to support Agent V2, which introduces a new pattern for connecting directly to Azure AI Foundry agents. The changes replace the previous FoundryAgentTool pattern with a new AgentSessionConfig-based approach where agents are configured at connection time rather than as tools within a session.
Changes:
- Adds
AgentSessionConfigclass for configuring direct agent connections with required agent/project names and optional parameters - Introduces new
startSession(AgentSessionConfig)API methods inVoiceLiveAsyncClientfor agent-based sessions - Removes deprecated FoundryAgentTool pattern and related classes, server events, and enum values
- Adds avatar scene configuration (
Sceneclass) and warning event support (ServerEventWarning) - Updates authentication scope from Cognitive Services to Azure AI (
https://ai.azure.com/.default)
Reviewed changes
Copilot reviewed 31 out of 31 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tsp-location.yaml | Updates TypeSpec commit reference to 9f102a526a6ede52ba997cc751d612c7fe4ff363 |
| AgentSessionConfig.java | New handwritten class for agent session configuration with validation and query parameter conversion |
| AgentSessionConfigTest.java | Comprehensive unit tests for AgentSessionConfig including validation and parameter mapping |
| VoiceLiveAsyncClient.java | Adds new startSession overloads for AgentSessionConfig and updates authentication scope |
| VoiceLiveSessionAsyncClient.java | Changes token scope from Cognitive Services to Azure AI |
| AgentV2Sample.java | New comprehensive sample demonstrating agent V2 pattern with real-time audio |
| VoiceLiveAsyncClientTest.java | Adds tests for new AgentSessionConfig-based startSession methods |
| Scene.java | New generated model for avatar scene configuration (zoom, position, rotation, amplitude) |
| ServerEventWarning.java | New generated model for non-interrupting warning events |
| ServerEventWarningDetails.java | New generated model for warning event details |
| AvatarConfiguration.java | Adds scene and outputAuditAudio properties |
| SessionUpdate.java | Updates discriminator to support warning events, removes foundry agent call events |
| SessionResponseItem.java | Removes foundry_agent_call discriminator |
| VoiceLiveToolDefinition.java | Removes foundry_agent tool type from discriminator logic |
| ItemType.java | Removes FOUNDRY_AGENT_CALL enum value |
| ToolType.java | Removes FOUNDRY_AGENT enum value |
| ServerEventType.java | Removes foundry agent call event types and MCP approval types, adds WARNING |
| FoundryAgentTool.java | Deleted - replaced by AgentSessionConfig pattern |
| FoundryAgentContextType.java | Deleted - no longer needed with new pattern |
| ResponseFoundryAgentCallItem.java | Deleted - foundry agent calls removed |
| ServerEventResponseFoundryAgentCall*.java | Deleted - 5 foundry agent call lifecycle event classes removed |
| CHANGELOG.md | Documents all features added, breaking changes, properly categorized |
| Metadata files | Updated to reflect removed and added types |
| Test files | Deleted tests for removed classes (3 files) |
Contributor
API Change CheckAPIView identified API level changes in this PR and created the following API reviews |
- Regenerate TypeSpec files from commit 347ca2ab38cc4ac37b4733c519f67920ac6c272d - Rename FillerResponseConfigBase to InterimResponseConfigBase - Rename BasicFillerResponseConfig to StaticInterimResponseConfig - Rename LlmFillerResponseConfig to LlmInterimResponseConfig - Rename FillerResponseConfigType to InterimResponseConfigType - Rename FillerTrigger to InterimResponseTrigger - Update VoiceLiveSessionOptions/Response: fillerResponse -> interimResponse - Replace FillerResponseConfigTest with InterimResponseConfigTest - Update VoiceLiveSessionOptionsNewFeaturesTest for interim response API - Update CHANGELOG.md with breaking changes documentation
xitzhang
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…emove FoundryAgentTool, add AgentSessionConfig sample
Description
Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:
General Guidelines and Best Practices
Testing Guidelines