Skip to content

Fix adversarial chat target for Tense, Crescendo, and MultiTurn attack strategies#45776

Merged
slister1001 merged 12 commits intoAzure:mainfrom
slister1001:fix/redteam-encoding
Mar 19, 2026
Merged

Fix adversarial chat target for Tense, Crescendo, and MultiTurn attack strategies#45776
slister1001 merged 12 commits intoAzure:mainfrom
slister1001:fix/redteam-encoding

Conversation

@slister1001
Copy link
Member

Description

Please add an informative description that covers that changes made by the pull request and link all relevant issues.

If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

slister1001 and others added 10 commits March 3, 2026 20:08
Add explicit encoding='utf-8' to all file open() calls in the PyRIT result
processing path. Without this, Windows defaults to the system locale encoding
(charmap/cp1252), causing UnicodeDecodeError when reading JSONL files containing
non-ASCII characters from UnicodeConfusable strategy or CJK languages.

Fixes: Tests 1.7 (UnicodeConfusable), 1.16 (Japanese/Chinese)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Test CJK characters, Unicode confusables, and mixed scripts to prevent
future regressions of the charmap encoding bug on Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…GELOG

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Foundry execution path was incorrectly passing the user's callback
target as adversarial_chat_target to PyRIT's FoundryScenario. This caused
PyRIT's TenseConverter to use the callback as its LLM for prompt
rephrasing, resulting in the callback's fixed response leaking into
converted_value and appearing as the user message in results.

Changes:
- Create AzureRAIServiceTarget with strategy-appropriate template key
  instead of reusing the user's callback target
- Add _get_adversarial_template_key() to select the correct RAI service
  template per attack strategy (crescendo, multi-turn, or tense converter)
- Show original_value for user messages in _build_messages_from_pieces()
  as defense-in-depth against converter output leaking into display
- Add 9 regression tests covering template key selection, wiring
  verification, original_value display, and the exact reported bug
- Fix existing test mocks to set original_value on user-role pieces

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@slister1001 slister1001 requested a review from a team as a code owner March 18, 2026 19:27
Copilot AI review requested due to automatic review settings March 18, 2026 19:27
@github-actions github-actions bot added the Evaluation Issues related to the client library for Azure AI Evaluation label Mar 18, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes Foundry-based red team execution so PyRIT’s adversarial_chat_target uses an AzureRAIServiceTarget (instead of the user’s callback target), preventing callback responses from leaking into converted prompts and ultimately into scan result “user message” content.

Changes:

  • Updated _execute_attacks_with_foundry to construct and pass an AzureRAIServiceTarget as adversarial_chat_target, selecting a template key based on attack strategies.
  • Updated Foundry result message construction to show original_value for user messages (falling back to converted_value when needed).
  • Added regression unit tests and a changelog entry covering the callback-response leakage scenario.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/red_team/_red_team.py Ensures Foundry adversarial chat uses AzureRAIServiceTarget and adds strategy→template selection helper.
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/red_team/_foundry/_foundry_result_processor.py Displays the original adversarial prompt for user messages to avoid showing converter/callback output.
sdk/evaluation/azure-ai-evaluation/tests/unittests/test_redteam/test_foundry.py Updates existing mocks for original_value and adds new regression tests around adversarial chat wiring and message building.
sdk/evaluation/azure-ai-evaluation/CHANGELOG.md Documents the bug fix in the unreleased section.

You can also share your feedback on Copilot code review. Take the survey.

- Convert _get_adversarial_template_key to @staticmethod
- Pass crescendo_format=True when crescendo template is selected
- Remove anti-pattern test and CentralMemory singleton leak
- Update staticmethod test calls to not pass None as self

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Member

@nagkumar91 nagkumar91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good bug fix. The root cause — user's callback being passed as adversarial_chat_target and leaking into converter output — is clearly identified and correctly fixed by creating a proper AzureRAIServiceTarget instead.

  • _get_adversarial_template_key logic is sound — _validate_strategies already prevents Crescendo+MultiTurn coexistence, so the precedence concern is moot ✅
  • original_value preference for user messages is a sensible defense-in-depth ✅
  • crescendo_format flag properly gated on template selection ✅
  • Tests are thorough: 9 regression tests covering template selection, wiring, original_value display, and the exact reported bug ✅

LGTM.

@slister1001 slister1001 enabled auto-merge (squash) March 18, 2026 21:39
@slister1001 slister1001 merged commit 8474221 into Azure:main Mar 19, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Evaluation Issues related to the client library for Azure AI Evaluation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants