Python: Foundry Evals integration for Python#4750

Draft

alliscode wants to merge 20 commits intomicrosoft:mainfrom

alliscode:af-foundry-evals-python

Commits on Mar 23, 2026

Foundry Evals integration for Python

alliscode
and
Copilot
committed
fix: resolve mypy redundant-cast errors while keeping pyright happy

alliscode
and
Copilot
committed
fix: CI failures — pyupgrade, evaluator overloads, sample API, reset attr

alliscode
and
Copilot
committed
fix: skip gRPC-dependent observability test

alliscode
and
Copilot
committed
fix: add nosec B101 for bandit assert check

alliscode
and
Copilot
committed
style: align eval samples with repo conventions

alliscode
and
Copilot
committed
Address PR review feedback: async fixes, sample bugs, deprecation warnings

alliscode
and
Copilot
committed
Add test coverage for review feedback items

alliscode
and
Copilot
committed
Fix ruff ISC004 lint error and apply formatter

alliscode
and
Copilot
committed
Remove core type changes (extracted to fix/workflow-stale-session branch)

alliscode
and
Copilot
committed
Address PR review round 2: bugs, tests, and architecture

alliscode
and
Copilot
committed

Commits on Mar 24, 2026

Fix CI: revert test_full_conversation, fix pyright errors

alliscode
and
Copilot
committed
Address PR review round 3: reliability, test gaps, cleanup

alliscode
and
Copilot
committed
Fix CI: ruff S101 assert, pyright and mypy arg-type errors

alliscode
and
Copilot
committed
Address PR review round 4: bugs, reliability, test fixes

alliscode
and
Copilot
committed
Address PR review round 5: reliability, docs, test coverage

alliscode
and
Copilot
committed
Fix error message: evaluate_responses() → evaluate_traces(response_ids=...)

alliscode
and
Copilot
committed
Remove dead to_eval_data() method, fix docstring claims

alliscode
and
Copilot
committed
Reduce default eval timeout from 600s to 180s (3 minutes)

alliscode
and
Copilot
committed

Commits on Mar 25, 2026

Remove dead _evaluate_via_responses method from FoundryEvals

alliscode
and
Copilot
committed