Skip to content

Reject explicit empty objectives in batch scoring#1503

Open
biefan wants to merge 1 commit intoAzure:mainfrom
biefan:reject-empty-batch-objectives
Open

Reject explicit empty objectives in batch scoring#1503
biefan wants to merge 1 commit intoAzure:mainfrom
biefan:reject-empty-batch-objectives

Conversation

@biefan
Copy link
Contributor

@biefan biefan commented Mar 17, 2026

Summary

  • distinguish omitted objectives from an explicit empty sequence in batch scoring entry points
  • reject objectives=[] when scoring non-empty prompt or image batches
  • add regression coverage for both prompt and image batch scoring paths

Problem

score_prompts_batch_async() treats objectives=[] the same as objectives=None, replacing it with default empty-string objectives and continuing into the scoring pipeline. score_image_batch_async() has the same issue. This skips the documented length validation and can trigger unrelated downstream failures instead of failing fast.

Validation

  • .venv/bin/pytest tests/unit/score -q
  • .venv/bin/ruff check pyrit/score/scorer.py tests/unit/score/test_scorer.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant