Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -453,6 +453,23 @@ Nicolo also dropped this as a reference: http://spec.openapis.org/oas/v3.0.3#ope

</div>

## Score results

Every scorer returns a small `Score` result object. This is the public surface
consumers should read when they need to store, compare, or export evaluation
results:

- `name`: the scorer name
- `score`: a number between 0 and 1, or `None` / `null` when the evaluation is skipped
- `metadata`: optional scorer-specific details, such as rationale text or a
selected choice. Keys are scorer-specific; consumers should not assume
metadata keys are shared across scorer types.
- `error`: deprecated and retained for backward compatibility; some scorers may
still populate it, but callers should primarily handle thrown exceptions

Inputs, expected values, model prompts, and other runtime context are not part
of the `Score` object. Keep those separately if your application needs them.

## Creating custom scorers

You can also create your own scoring functions that do not use LLMs. For example, to test whether the word `'banana'`
Expand Down