Open
Conversation
Codecov Report❌ Patch coverage is
❌ Your patch check has failed because the patch coverage (0.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage. @@ Coverage Diff @@
## main #421 +/- ##
==========================================
- Coverage 71.04% 70.94% -0.10%
==========================================
Files 624 625 +1
Lines 61569 61659 +90
==========================================
+ Hits 43743 43747 +4
- Misses 14656 14742 +86
Partials 3170 3170
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 2 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
HearyShen
changed the title
Implement async run and debug functionality for agent evaluator type, including: - Add new async methods in evaluator service interface - Implement async handlers in evaluator service - Add request/response types for async operations - Add agent evaluator version conversion logic
Implement methods to fetch async debug evaluator results across service layers. Added new request/response types and implemented the functionality in evaluator source services while maintaining backward compatibility for unsupported evaluator types.
Add evaluator record creation in AsyncRunEvaluator and AsyncDebugEvaluator methods to track async evaluation runs. This provides better observability and audit trail for async operations.
…outes Remove unused async debug result endpoint and its related request/response structs Add missing api routes for evaluator record endpoints
remove unused async debug result related code including handler, service interface, entity and mock implementations to clean up the codebase
The method was not implemented and always returned an error, indicating it was not actually needed for the evaluator services. This cleanup removes dead code from the interface and implementations.
Add new error code for agent evaluator run failures to handle configuration issues
…ed to current agent and haven't matched with AG-UI message protocol
…ed to current agent and haven't matched with AG-UI message protocol
…into feat/agent_evaluator Change-Id: Ibc6898290abb64cf78c0c999b42e6b03843446e5
- Add new EvaluatorExtraOutputContent type to store additional output data - Extend async run methods to return additional extension data - Merge existing ext data when reporting async results
Store async evaluation context in repo for both run and debug operations
Implement LTrim functionality to allow trimming lists in Redis, which is needed for maintaining list sizes efficiently.
Add version field to SkillConfig struct across multiple layers including thrift definition, domain entity, and conversion logic to support skill versioning
…rsions Update SkillConfig struct to use *int64 for SkillID to maintain consistency with other fields. Modify conversion functions to handle the pointer type directly instead of using helper functions.
add file client parameter to experiment application initialization and implement URI to URL conversion for evaluator extra output
Replace full record update with targeted field updates to improve efficiency and reduce unnecessary conversions. Add new method to update only status, score and output data.
Remove redundant evaluator version check and update authorization parameters to use evaluator record data directly. The change aligns authorization with space-level permissions rather than evaluator-level permissions.
Add transformExtraOutputURIToURL method to convert extra output URIs to accessible URLs. This ensures clients can access the output files directly when needed.
Add conversion logic for agent evaluator version in both DO2PO and PO2DO directions. This includes handling meta info, input schemas, and output schemas similar to other evaluator types.
The transaction wrapper was unnecessary since we're performing a single update operation. Removing it simplifies the code while maintaining the same functionality.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Check the PR title
(Optional) Translate the PR title into Chinese
(Optional) More detailed description for this PR(en: English/zh: Chinese)
en:
zh(optional):
(Optional) Which issue(s) this PR fixes