Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .changeset/add-si-training-tenant.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
---

feat(training-agent): add `/si` tenant serving SI lifecycle tools (#3940)

Adds a new `si` training-agent tenant at `/si/mcp` that simulates the
brand-agent side of the Sponsored Intelligence session lifecycle. Learners
can now complete the S5 specialist capstone and C3 creative-SI exercises
without hitting `Unknown tool: si_initiate_session`.

**What ships:**
- `v6-si-platform.ts` — minimal `DecisioningPlatform` (no specialism
methods; all four SI tools ride `customTools` pending SDK SI interface).
- `tenants/si.ts` — stub handlers for `si_get_offering`,
`si_initiate_session`, `si_send_message`, `si_terminate_session`
(Nova Brands training fixture, in-memory session state).
- `tenants/registry.ts` — registers the `si` tenant alongside the
existing six.
- `tenants/tool-catalog.ts` — adds `si_*` → `['si']` discovery hints.
- Migration 465 — backfills `tenant_ids` for A3, C3, S5; fixes the C3
`c3_ex2` phantom tool (`connect_to_si_agent` → `si_initiate_session`);
appends stable recertification criterion IDs to S5's `s5_ex1`.

**Not included (human task):**
- Storyboard floors in `.github/workflows/training-agent-storyboards.yml`
for the `si` tenant — add to smoke matrix after this merges.

Refs #3940.
198 changes: 198 additions & 0 deletions server/src/db/migrations/465_si_tenant_module_pins.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
-- Pin A3, C3, and S5 to the new `si` training-agent tenant, which was
-- explicitly left NULL in migration 464 because no `si_*`-serving tenant
-- existed at the time. This migration follows once the tenant ships.
--
-- Additional curriculum fixes bundled here (education-expert review of #3940):
--
-- 1. C3 c3_ex2 phantom tool: `connect_to_si_agent` is an Addie-specific host
-- tool (server/src/addie/mcp/si-host-tools.ts), not a protocol-defined
-- AdCP task. The training agent has never served it. Replace with
-- `si_initiate_session` — the correct entry point for the SI lifecycle.
--
-- 2. S5 s5_ex1 sandbox_actions gap: migration 303 rewrote S5's sandbox_actions
-- to include si_initiate_session and si_send_message but omitted
-- si_get_offering and si_terminate_session. The two new criterion IDs
-- (s5_ex1_sc_session_lifecycle, s5_ex1_sc_offering_integration) require
-- both tools to be prompted actions — adding criterion IDs for behaviors
-- the exercise never asks the learner to perform fails the ASTM E3416-24
-- required-demonstration test. Append both tools to sandbox_actions first.
--
-- 3. S5 criterion IDs: The s5_ex1 success_criteria were defined as plain text
-- strings (migration 270 seed, overwritten verbatim by migrations 303/298).
-- The _append_criterion helper (migration 407) adds stable IDs the
-- recertification engine can target when SI experimental surfaces change.
-- Adds s5_ex1_sc_session_lifecycle and s5_ex1_sc_offering_integration —
-- the two criteria covering the graded competencies at highest assessment
-- weight in S5's protocol_mastery dimension.

-- ── 1. Pin tenant_ids ───────────────────────────────────────────────────────

-- A3: add si to the landscape tour (the tour already lists the other six
-- tenants from migration 464; si is the missing stop).
UPDATE certification_modules
SET tenant_ids = ARRAY['sales', 'signals', 'governance', 'creative', 'brand', 'si']
WHERE id = 'A3' AND tenant_ids IS NULL;

-- C3: creative + sponsored intelligence — si is the primary SI surface;
-- creative and brand remain for sync_creatives / creative_approval exercises.
-- Drops creative-builder (per #3930 review: creative-builder is S2-specific).
UPDATE certification_modules
SET tenant_ids = ARRAY['creative', 'brand', 'si']
WHERE id = 'C3' AND tenant_ids IS NULL;

-- S5: specialist deep-dive — si is the sole primary tenant. brand was
-- considered (creative_approval for commerce handoff) but excluded: the
-- si_terminate_session reason enum already covers handoff_transaction and
-- handoff_complete as first-class termination reasons, and creative_approval
-- does not appear in S5's sandbox_actions. The ACP checkout handoff is
-- post-session and out of scope for the SI lifecycle exercise.
UPDATE certification_modules
SET tenant_ids = ARRAY['si']
WHERE id = 'S5' AND tenant_ids IS NULL;

-- ── 2. Fix C3 c3_ex2 phantom tool ──────────────────────────────────────────
--
-- Replace `connect_to_si_agent` in sandbox_actions with `si_initiate_session`.
-- The exercise intent (connecting to a brand SI agent) is preserved; only the
-- tool name and guidance text change to match the actual protocol tool.
-- Safe to replay: the WHERE guard matches only when the phantom name is present.

UPDATE certification_modules
SET exercise_definitions = (
SELECT jsonb_agg(
CASE
WHEN ex->>'id' = 'c3_ex2'
THEN jsonb_set(
ex,
'{sandbox_actions}',
(
SELECT jsonb_agg(
CASE
WHEN act->>'tool' = 'connect_to_si_agent'
THEN jsonb_build_object(
'tool', 'si_initiate_session',
'guidance', 'Initiate a session with the training SI brand agent. Provide an intent describing what the user is looking for. Examine the session_id, negotiated_capabilities, and the brand''s opening message — note how the brand already incorporates your intent into its response. That bidirectional personalization is what distinguishes SI from impression-based formats.'
)
ELSE act
END
)
FROM jsonb_array_elements(ex->'sandbox_actions') act
)
)
ELSE ex
END
)
FROM jsonb_array_elements(exercise_definitions) ex
)
WHERE id = 'C3'
AND exercise_definitions::text LIKE '%connect_to_si_agent%';

-- ── 3. S5 s5_ex1 sandbox_actions: add missing si_get_offering and si_terminate_session
--
-- Migration 303 wrote si_initiate_session and si_send_message but omitted the
-- bookend tools. Both must be present as prompted actions before the criterion
-- IDs in section 4 can be treated as demonstrated competencies under
-- ASTM E3416-24. Safe to replay: WHERE guard requires the current actions list
-- to lack si_get_offering (idempotent).

UPDATE certification_modules
SET exercise_definitions = (
SELECT jsonb_agg(
CASE
WHEN ex->>'id' = 's5_ex1'
THEN jsonb_set(
ex,
'{sandbox_actions}',
ex->'sandbox_actions'
|| jsonb_build_array(
jsonb_build_object(
'tool', 'si_get_offering',
'guidance', 'Call si_get_offering with the training brand''s offering_id. Examine the returned offering_token — you will pass it to si_initiate_session to demonstrate session-continuity handoff.'
),
jsonb_build_object(
'tool', 'si_terminate_session',
'guidance', 'Terminate the session with an appropriate reason code (user_exit, handoff_transaction, or handoff_complete). Verify turns_completed in the response.'
)
)
)
ELSE ex
END
)
FROM jsonb_array_elements(exercise_definitions) ex
)
WHERE id = 'S5'
AND exercise_definitions::text NOT LIKE '%si_get_offering%';

-- ── 4. S5 stable criterion IDs ──────────────────────────────────────────────
--
-- Re-declare the _append_criterion helper (CREATE OR REPLACE — idempotent)
-- then stamp two semantic IDs onto s5_ex1. These IDs let the recertification
-- engine identify credential holders who need re-assessment when the SI
-- experimental surface changes.

CREATE OR REPLACE FUNCTION _append_criterion(
p_module_id text,
p_exercise_id text,
p_criterion_id text,
p_text text
) RETURNS void AS $$
DECLARE
defs jsonb;
updated jsonb := '[]'::jsonb;
ex jsonb;
criteria jsonb;
already_present boolean;
exercise_matched boolean := false;
BEGIN
SELECT exercise_definitions INTO defs
FROM certification_modules
WHERE id = p_module_id;

IF defs IS NULL OR jsonb_typeof(defs) <> 'array' THEN
RAISE EXCEPTION 'Module % not found or has no exercise_definitions array', p_module_id;
END IF;

FOR ex IN SELECT * FROM jsonb_array_elements(defs)
LOOP
IF ex->>'id' = p_exercise_id THEN
exercise_matched := true;
criteria := COALESCE(ex->'success_criteria', '[]'::jsonb);

SELECT EXISTS (
SELECT 1 FROM jsonb_array_elements(criteria) c
WHERE c->>'id' = p_criterion_id
) INTO already_present;

IF NOT already_present THEN
criteria := criteria || jsonb_build_array(
jsonb_build_object('id', p_criterion_id, 'text', p_text)
);
ex := jsonb_set(ex, '{success_criteria}', criteria);
END IF;
END IF;
updated := updated || jsonb_build_array(ex);
END LOOP;

IF NOT exercise_matched THEN
RAISE EXCEPTION 'Exercise % not found in module %', p_exercise_id, p_module_id;
END IF;

UPDATE certification_modules
SET exercise_definitions = updated
WHERE id = p_module_id;
END;
$$ LANGUAGE plpgsql;

SELECT _append_criterion(
'S5',
's5_ex1',
's5_ex1_sc_session_lifecycle',
'Demonstrates the full SI session lifecycle: calls si_get_offering, si_initiate_session, si_send_message (at least one turn), and si_terminate_session in correct protocol order.'
);

SELECT _append_criterion(
'S5',
's5_ex1',
's5_ex1_sc_offering_integration',
'Uses si_get_offering before session initiation and passes the returned offering_token to si_initiate_session, demonstrating the session-continuity handoff.'
);
9 changes: 6 additions & 3 deletions server/src/training-agent/tenants/registry.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
/**
* Multi-tenant TenantRegistry setup.
*
* Six per-specialism tenants — `/sales`, `/signals`, `/governance`,
* `/creative`, `/creative-builder`, `/brand` — each with its own
* Seven per-specialism tenants — `/sales`, `/signals`, `/governance`,
* `/creative`, `/creative-builder`, `/brand`, `/si` — each with its own
* `DecisioningPlatform` impl, ephemeral signing key, and specialism
* declarations. Path-routed: tenants register with `agentUrl` like
* `${CANONICAL_BASE}/<tenantId>`, the router binds tenantId at route
Expand Down Expand Up @@ -41,6 +41,7 @@ import { buildGovernanceTenantConfig } from './governance.js';
import { buildCreativeTenantConfig } from './creative.js';
import { buildCreativeBuilderTenantConfig } from './creative-builder.js';
import { buildBrandTenantConfig } from './brand.js';
import { buildSiTenantConfig } from './si.js';
import { createLogger } from '../../logger.js';

const logger = createLogger('training-agent-tenants');
Expand Down Expand Up @@ -212,6 +213,7 @@ export function createRegistryHolder(): RegistryHolder {
const creative = buildCreativeTenantConfig(hostBase);
const creativeBuilder = buildCreativeBuilderTenantConfig(hostBase);
const brand = buildBrandTenantConfig(hostBase);
const si = buildSiTenantConfig(hostBase);
// awaitFirstValidation:true blocks until the no-op validator
// promotes the tenant to 'healthy'. Without it the first request
// would race the background validation and see 'pending' (refused
Expand All @@ -223,9 +225,10 @@ export function createRegistryHolder(): RegistryHolder {
reg.register(creative.tenantId, creative.config, { awaitFirstValidation: true }),
reg.register(creativeBuilder.tenantId, creativeBuilder.config, { awaitFirstValidation: true }),
reg.register(brand.tenantId, brand.config, { awaitFirstValidation: true }),
reg.register(si.tenantId, si.config, { awaitFirstValidation: true }),
]);
logger.info(
{ hostBase, tenants: ['signals', 'sales', 'governance', 'creative', 'creative-builder', 'brand'] },
{ hostBase, tenants: ['signals', 'sales', 'governance', 'creative', 'creative-builder', 'brand', 'si'] },
'Tenant registry initialized',
);
registry = reg;
Expand Down
Loading
Loading