Commit a0edd5f
Foundry Evals integration for Python
Add evaluation framework with local and Foundry-hosted evaluator support:
- EvalItem/EvalResult core types with conversation splitting strategies
- @evaluator decorator for defining custom evaluation functions
- LocalEvaluator for running evaluations locally
- FoundryEvals provider for Azure AI Foundry hosted evaluations
- evaluate_agent() orchestration with expected values support
- evaluate_workflow() for multi-agent workflow evaluation
- Comprehensive test suite and evaluation samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 7c85f98 commit a0edd5f
19 files changed
Lines changed: 6526 additions & 75 deletions
File tree
- python
- packages
- azure-ai
- agent_framework_azure_ai
- tests
- core
- agent_framework
- _workflows
- tests
- core
- workflow
- samples/05-end-to-end/evaluation/foundry_evals
Lines changed: 28 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
5 | 16 | | |
6 | 17 | | |
7 | 18 | | |
| |||
11 | 22 | | |
12 | 23 | | |
13 | 24 | | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
14 | 31 | | |
15 | 32 | | |
16 | 33 | | |
| |||
31 | 48 | | |
32 | 49 | | |
33 | 50 | | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
34 | 57 | | |
35 | 58 | | |
36 | 59 | | |
37 | 60 | | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
38 | 66 | | |
0 commit comments