Update evals docs: clarify global evaluators opt-in behavior (PR #14682)

build/agents/build-your-agent/evals.mdx

@@ -1,18 +1,18 @@
 ---
 title: 'Evals'
 sidebarTitle: 'Evals'
 description: 'Test and evaluate your AI Agents with scenario-based evaluations and automated Evaluators'
 ---
 
 <Info>
 **Rollout Status**: Evals is currently being rolled out progressively, starting with Enterprise customers. If you're an Enterprise customer and don't see this feature in your account yet, reach out to your account manager to discuss access.
 </Info>
 
 The Evals section is your command center for testing and evaluating AI Agent performance. Located in the **Monitor** tab (next to the Run tab) in the Agent builder, Evals enables you to create Test Suites, define evaluation criteria (Evaluators), run automated evaluations, and monitor ongoing performance—all without manual testing.
 
 ![Evals section showing Test Suites, Evaluators, Runs, and Performance](/images/agent/agent-evals.png)
 
 ## What you can do with Evals
 
 <CardGroup cols={3}>
   <Card title="Conduct Tests" icon="flask-vial">
@@ -28,11 +28,11 @@
 
 ---
 
 ## Evals sections
 
 The Evals section contains five main sections, accessible from the left sidebar:
 
 - **Test Suites** — Create and manage groups of Test scenarios for your Agent. Each Test Suite can contain multiple scenarios with different prompts and evaluation criteria.
 - **Evaluators** — Configure global evaluation criteria that can be applied across any Test Suite or scenario without needing to set them up each time.
 - **Runs** — View your evaluation run history and results. See average scores, number of conversations evaluated, progress status, credit spend, and creation dates for all past runs.
 - **Publish Checks** — Configure which Test Suites must pass before your Agent can be published. Set a pass threshold and optionally block publishing if evaluations fail.
@@ -117,7 +117,7 @@
 6. Click **Create Evaluator**
 
 <Note>
-When you run a Test scenario, scenario-level Evaluators are always included automatically. You can also add or remove global Evaluators (from the Evaluators tab) before each run, allowing you to mix standard criteria with scenario-specific evaluation rules.
+When you run a Test scenario, scenario-level Evaluators are always included automatically. Global Evaluators are not included by default — you must explicitly select them in the evaluation modal (Run Test Set, Run Scenario, or Evaluate Selected Tasks) before each run.
 </Note>
 
 ---
@@ -223,7 +223,7 @@
 You can select specific Test scenarios within a Test Suite to run certain ones at once, or run all scenarios in the Test Suite together. Note that you cannot bulk select and run multiple Test Suites at the same time.
 
 1. Enter a name for the evaluation run (e.g., "Scenario Run - Jan 14, 12:14 PM"). A default name with timestamp is provided.
-2. Select which global Evaluators to include in the run — you can add or remove global Evaluators before starting. Scenario-level Evaluators are always included automatically.
+2. Scenario-level Evaluators are always included automatically. Global Evaluators are not included by default — to include them, tick the ones you want under the **Additional global checks** section.
 3. Click **Run** to begin. The system will simulate conversations with your Agent based on your scenario prompts and evaluate them with your selected Evaluators.
 
 ---
@@ -292,7 +292,7 @@
 
 The Performance tab also includes:
 
 - **Data points** for the overall score over time
 - **Evaluator breakdown** showing individual scoring per Evaluator
 - **Graphs** visualizing Evaluator performance trends
 - **List of evaluation runs** with score, name, and the ability to view the full conversation
@@ -355,6 +355,10 @@
     You can add as many scenarios as needed to a single Test Suite. Each scenario is evaluated independently and can have its own Evaluators.
   </Accordion>
 
+  <Accordion title="How many Evaluators can I add to a scenario?">
+    Each scenario supports up to 10 Evaluators. This applies to scenario-level Evaluators defined within the scenario itself. Global Evaluators added via **Additional global checks** at run time are counted separately.
+  </Accordion>
+
   <Accordion title="How are credits calculated for evaluations?">
     Credits consumed for each scenario are calculated by adding together:
     - The Agent task run (the conversation with your Agent)