Skip to content

Standardize Output Schema for Evalautors#46436

Draft
m7md7sien wants to merge 36 commits into
mainfrom
mohessie/standardize_output_schema
Draft

Standardize Output Schema for Evalautors#46436
m7md7sien wants to merge 36 commits into
mainfrom
mohessie/standardize_output_schema

Conversation

@m7md7sien
Copy link
Copy Markdown
Contributor

@m7md7sien m7md7sien commented Apr 21, 2026 •
edited
Loading

Description

Standradize Output Scheme for Evalautors

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

m7md7sien and others added 13 commits April 15, 2026 00:35
@m7md7sien
Update Tool Call Accuracy to output unified format
3eb40a8
@m7md7sien
Update tests
d3c4092
@m7md7sien
Merge branch 'main' into mohessie/unify_output/tool_call_accuracy
d076d5c
@m7md7sien
reformatting
5032e26
@m7md7sien
Refactor not applicable result method calls
a525806
@m7md7sien
Fix test assertions for new unified output format and apply black for…
f454ee3 
…matting (#46336)

Agent-Logs-Url: https://github.com/Azure/azure-sdk-for-python/sessions/23f40ca5-7114-46ec-89be-a369e38ac971

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: m7md7sien <16615690+m7md7sien@users.noreply.github.com>
@m7md7sien
Rename tool_call_accuracy reasoning output to reason and update skipp…
aa848fe 
…ed properties handling (#46355)

Agent-Logs-Url: https://github.com/Azure/azure-sdk-for-python/sessions/89b3b528-f2ac-4284-88fb-c484d4c0cce1

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: m7md7sien <16615690+m7md7sien@users.noreply.github.com>
@m7md7sien
Merge branch 'main' into mohessie/unify_output/tool_call_accuracy
83576b4
@m7md7sien
Fix tool call accuracy test for skipped output schema (#46356)
99deb99 
Agent-Logs-Url: https://github.com/Azure/azure-sdk-for-python/sessions/8ab1c161-c24f-4272-95ff-c8e595089e22

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: m7md7sien <16615690+m7md7sien@users.noreply.github.com>
@m7md7sien
Merge branch 'main' into mohessie/unify_output/tool_call_accuracy
1893bc8
@m7md7sien
Standradize Output Scheme
d821299
@m7md7sien
Merge branch 'mohessie/unify_output/tool_call_accuracy' into mohessie…
c489326 
…/standardize_output_schema
@m7md7sien
Add explicit _KEY_PREFIX/_RESULT_KEY
c94d4e7
@github-actions github-actions Bot added the Evaluation Issues related to the client library for Azure AI Evaluation label Apr 21, 2026
@m7md7sien m7md7sien mentioned this pull request Apr 21, 2026
Merged
6 tasks
m7md7sien added 12 commits April 21, 2026 21:18
@m7md7sien
Update recordings tag to solve e2e tests
55b365f
@m7md7sien
Run formatting
8726b93
@m7md7sien
Align evaluator unit tests with unified output schema and refresh rec…
73becf1 
…ordings
@m7md7sien
Merge branch 'main' into mohessie/standardize_output_schema
b4740de
@m7md7sien
Restore legacy _result and bare evaluator-name keys for backward co…
c357054 
…mpat
@m7md7sien
resolve conflict
d7f459b
@m7md7sien
Refresh azure-ai-evaluation test recordings for standardized evaluato…
5cb47ec 
…r output schema
@m7md7sien
Merge branch 'main' into mohessie/standardize_output_schema
d91cbf6
@m7md7sien
Merge branches 'mohessie/standardize_output_schema' and 'mohessie/sta…
bcef3b2 
…ndardize_output_schema' of https://github.com/Azure/azure-sdk-for-python into mohessie/standardize_output_schema
@m7md7sien
Merge branch 'main' into mohessie/standardize_output_schema
b14f476
@m7md7sien
Update multimodal test assertion for new schema and refresh recording…
76cdaf5 
…s tag
@m7md7sien
Remove unused label assignment in navigation efficiency
519c97c 
Remove assignment of match_result to additional_properties_metrics['label']
Copilot AI mentioned this pull request Apr 22, 2026
Merged
This was referenced May 11, 2026
Draft
Merged
@m7md7sien
Revert _evaluate.py changes from #46436 on `mohessie/standardize_ou…
36e0cbb 
…tput_schema` (#46835)

* Initial plan

* Revert _evaluate.py changes from PR 46436 by restoring file from main

Agent-Logs-Url: https://github.com/Azure/azure-sdk-for-python/sessions/8462065c-c6cf-473a-9421-84eaf0a44b5b

Co-authored-by: m7md7sien <16615690+m7md7sien@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: m7md7sien <16615690+m7md7sien@users.noreply.github.com>
@ela-ine ela-ine changed the title Standradize Output Scheme for Evalautors Standardize Output Schema for Evalautors May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

Evaluation Issues related to the client library for Azure AI Evaluation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@m7md7sien