Skip to content

Include subdirectory jailbreak templates in listings#1498

Open
biefan wants to merge 1 commit intoAzure:mainfrom
biefan:include-subdirectory-jailbreak-templates
Open

Include subdirectory jailbreak templates in listings#1498
biefan wants to merge 1 commit intoAzure:mainfrom
biefan:include-subdirectory-jailbreak-templates

Conversation

@biefan
Copy link
Contributor

@biefan biefan commented Mar 17, 2026

Summary

  • list jailbreak templates from the cached recursive template scan instead of only the top-level directory
  • add regression coverage for subdirectory template discovery in both TextJailBreak and the AIRT jailbreak scenario

Problem

TextJailBreak already resolves template names from subdirectories such as pliny/amazon/nova.yaml, but get_jailbreak_templates() only scans templates/*.yaml at the top level.

This creates an inconsistent API surface: a template name can be loaded successfully via template_file_name, but the same name is missing from the advertised template list and is rejected by Jailbreak(..., jailbreak_names=[...]) validation.

Testing

  • .venv/bin/pytest tests/unit/datasets/test_jailbreak_text.py tests/unit/scenarios/test_jailbreak.py -q

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant