Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,9 +128,9 @@ Use narrower verification for small edits.

## Agent Studio (`pkg/cmd/agents/...`, `api/agentstudio/`)

Top-level command group: `algolia agents`. Verbs: `list`, `get`, `create`, `update`, `delete`, `publish`, `unpublish`, `duplicate`, `try`, `run`. Backend source of truth: `github.com/algolia/conversational-ai`.
Top-level command group: `algolia agents`. Verbs: `list`, `get`, `create`, `update`, `delete`, `publish`, `unpublish`, `duplicate`, `try`, `run`. Sub-groups: `cache` (`invalidate`). Backend source of truth: `github.com/algolia/conversational-ai`.

**Naming note**: `try` (not `test`) — see "On `--dry-run`" below for why. All other verbs are single-word lowercase to match the CLI-wide convention; no hyphenated subcommand names exist anywhere in the tree.
**Naming note**: `try` (not `test`) — see "On `--dry-run`" below for why. All flat verbs are single-word lowercase to match the CLI-wide convention; no hyphenated subcommand names exist anywhere in the tree. Sub-groups (`cache`, future `providers` / `conversations` / `keys` / `domains`) read as noun-then-verb (`agents cache invalidate`) — also a single word per token.

### API client (`api/agentstudio/`)

Expand All @@ -139,6 +139,9 @@ Top-level command group: `algolia agents`. Verbs: `list`, `get`, `create`, `upda
- Errors: `*APIError` with `StatusCode`, `Detail`, optional `Sentinel`. The detail extractor prefers structured FastAPI `detail[].msg` arrays over the generic `message` field — backends that return both pair them as `{"message":"Input is invalid, see detail/body:","detail":[{"msg":"..."}]}` and the structured form is the actionable one.
- `CreateAgent` / `UpdateAgent` accept `json.RawMessage` bodies on purpose. The backend's `AgentConfigCreate` schema is large, deeply validated, and evolves often. The CLI is a pass-through; the backend validates; our 422-detail surfacing makes errors actionable.
- `Completions(...)` returns the raw `*http.Response`. Caller checks `Content-Type` (`text/event-stream` → `ParseStream`; else copy verbatim). One method, two output shapes.
- `CompletionOptions.No*` fields (`NoCache`, `NoMemory`, `NoAnalytics`) are **inverted** from the backend's query polarity. Two reasons: the backend defaults all three to true (only the negative is interesting at the CLI), and `memory` in particular has an `anyOf [{const false}, {type null}]` schema — sending `memory=true` would 422. Therefore the wire form omits the param when the No* field is false, and sends `<param>=false` when true. Polarity is enforced end-to-end by `TestCompletions_QueryFlagsAndSecureUserToken` in `api/agentstudio/completions_test.go`.
- `CompletionOptions.SecureUserToken` populates the `X-Algolia-Secure-User-Token` header when non-empty. It carries a signed JWT scoping the conversation/memory/analytics partition to a specific end-user (see `rag/dependencies/secure_user_token.py` in the backend). Empty means no header — `X-Algolia-User-ID` fallback applies.
- `InvalidateAgentCache(id, before)` calls `DELETE /1/agents/{id}/cache?before=YYYY-MM-DD` (query omitted when `before` is empty). Date format validation is **deliberately not done client-side** — the backend's Pydantic parser is the source of truth, and our 422 surfacing turns malformed input into an actionable message verbatim. Mirroring the parser in Go would create silent skew.

### Streaming (`api/agentstudio/sse.go`)

Expand All @@ -151,6 +154,21 @@ The wire format is **not** standard SSE. Two protocols, both served as `text/eve

Streaming output convention: NDJSON to stdout regardless of TTY, one `{"type":"...","data":{...}}` per line. Plays well with `jq -r 'select(.type=="text-delta") | .data.delta'`. Don't fork rendering between TTY/non-TTY for streaming responses.

### Completion runtime knobs (`agents try` / `agents run`)

Both commands expose the same set of completion-time flags, mapping directly to backend query params + headers:

| Flag | Wire | Default | Notes |
|---|---|---|---|
| `--no-stream` | `?stream=false` | stream | Buffered single-JSON response instead of SSE |
| `--compatibility v4\|v5` | `?compatibilityMode=ai-sdk-{4,5}` | v5 | **Required** server-side; CLI promotes empty → v5 |
| `--no-cache` | `?cache=false` | cache on | Bypasses backend completion cache for this call |
| `--no-memory` | `?memory=false` | memory on | Disables agent memory retrieval/write for this call |
| `--no-analytics` | `?analytics=false` | analytics on | Skips Agent Studio analytics for this call |
| `--secure-user-token <jwt>` | `X-Algolia-Secure-User-Token` header | (omitted) | Signed JWT, end-user scoping |

The flag set is intentionally duplicated across `try.go` and `run.go` rather than extracted into a `RegisterCompletionFlags` shared helper — there are exactly two consumers and the duplication is mechanical (8 lines per command). If a third consumer appears, extract following the "second use" rule (same as `PrintDryRun` / `NormalizeCompatibility`).

### On `--dry-run`

Two distinct concepts share the name and they MUST NOT be conflated:
Expand Down
93 changes: 89 additions & 4 deletions api/agentstudio/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -130,19 +130,46 @@ func (c *Client) ListAgents(ctx context.Context, params ListAgentsParams) (*Pagi
return &out, nil
}

// CompletionOptions configures Completions(...) query parameters.
// CompletionOptions configures Completions(...) query parameters and
// per-request headers.
//
// Stream maps to ?stream=true|false; the default zero value (false) gives
// a buffered single-JSON response. Set explicitly via the command layer
// (`agents test` / `agents run` set Stream=true unless --no-stream).
// (`agents try` / `agents run` set Stream=true unless --no-stream).
//
// Compatibility maps to ?compatibilityMode=ai-sdk-4|ai-sdk-5. The backend
// requires this query param (no server-side default), so empty here is
// promoted to CompatV5 — its frames are standard SSE with [DONE], easier
// to parse defensively than v4's `<type>:<json>\n` line format.
//
// NoCache, NoMemory, and NoAnalytics are inverted from the backend's
// query-param polarity for two reasons:
//
// - The backend defaults all three to true; only the negated case is
// interesting from the CLI surface.
// - The flag layer ships them as `--no-cache`/`--no-memory`/`--no-analytics`
// so the option fields keep that polarity end-to-end.
//
// When a No*-field is false (the zero value) the corresponding query
// param is omitted entirely, which matches the backend's "default ON"
// behavior. The `memory` schema in particular is `anyOf [{const: false},
// {type: null}]` — false is the ONLY valid passable value, so always
// emitting `memory=true` would be a server-side validation error.
//
// SecureUserToken populates the X-Algolia-Secure-User-Token header when
// non-empty. It carries a signed JWT that scopes the conversation /
// memory / analytics partition to a specific end-user; required by the
// backend whenever a feature behind SecureUserTokenDep is enabled (see
// rag/dependencies/secure_user_token.py in algolia/conversational-ai).
// Empty here means no header is sent — the existing X-Algolia-User-ID
// fallback applies.
type CompletionOptions struct {
Stream bool
Compatibility CompatibilityMode
Stream bool
Compatibility CompatibilityMode
NoCache bool
NoMemory bool
NoAnalytics bool
SecureUserToken string
}

// Completions calls POST /1/agents/{agentID}/completions and returns the
Expand Down Expand Up @@ -191,6 +218,19 @@ func (c *Client) Completions(
q := url.Values{}
q.Set("stream", boolToWire(opts.Stream))
q.Set("compatibilityMode", string(mode))
// Only emit the negative cases — backend defaults match the omitted
// state, so adding `cache=true`/`analytics=true` would be wire noise,
// and `memory=true` would actually be a 422 (the schema only allows
// `false` or null). See CompletionOptions godoc for the full reasoning.
if opts.NoCache {
q.Set("cache", "false")
}
if opts.NoMemory {
q.Set("memory", "false")
}
if opts.NoAnalytics {
q.Set("analytics", "false")
}

endpoint := c.cfg.BaseURL + "/1/agents/" + url.PathEscape(agentID) + "/completions?" + q.Encode()

Expand All @@ -200,6 +240,9 @@ func (c *Client) Completions(
}
c.setHeaders(req)
req.Header.Set("Content-Type", "application/json")
if opts.SecureUserToken != "" {
req.Header.Set("X-Algolia-Secure-User-Token", opts.SecureUserToken)
}
// Preferred Accept: streaming responses come back as text/event-stream
// (both v4 and v5); buffered ones as application/json. Listing both
// is safe — the server picks based on ?stream and we inspect the
Expand Down Expand Up @@ -260,6 +303,48 @@ func (c *Client) UpdateAgent(ctx context.Context, id string, body json.RawMessag
return c.doAgentMutation(ctx, http.MethodPatch, endpoint, body, "update agent")
}

// InvalidateAgentCache calls DELETE /1/agents/{id}/cache. The backend
// removes cached completion responses for this agent.
//
// `before` is an optional YYYY-MM-DD date string. When non-empty, only
// cache entries created strictly before that date are invalidated
// (exclusive). When empty, all cache entries for the agent are wiped.
//
// The format is intentionally not pre-parsed in Go — the backend
// accepts the literal string and returns a 422 with a structured detail
// on a malformed value, which our extractDetail surfaces unchanged. Any
// client-side date parsing here would diverge from whatever Pydantic
// version the backend ships and create silent skew.
//
// Returns nil on the backend's HTTP 204 No Content. Wraps the standard
// 4xx/5xx APIError otherwise.
func (c *Client) InvalidateAgentCache(ctx context.Context, id, before string) error {
if strings.TrimSpace(id) == "" {
return fmt.Errorf("agent studio: agent id is required")
}

endpoint := c.cfg.BaseURL + "/1/agents/" + url.PathEscape(id) + "/cache"
if before != "" {
q := url.Values{}
q.Set("before", before)
endpoint += "?" + q.Encode()
}

req, err := http.NewRequestWithContext(ctx, http.MethodDelete, endpoint, nil)
if err != nil {
return err
}
c.setHeaders(req)

resp, err := c.httpClient.Do(req)
if err != nil {
return fmt.Errorf("agent studio: invalidate agent cache: %w", err)
}
defer resp.Body.Close()

return checkResponse(resp)
}

// DeleteAgent calls DELETE /1/agents/{id}.
//
// Returns nil on the backend's HTTP 204 No Content. The backend
Expand Down
75 changes: 75 additions & 0 deletions api/agentstudio/completions_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,81 @@ func readAllString(t *testing.T, r io.Reader) string {
return strings.TrimSpace(string(b))
}

func TestCompletions_QueryFlagsAndSecureUserToken(t *testing.T) {
// Phase 5: validates the new --no-cache / --no-memory / --no-analytics
// / --secure-user-token plumbing all the way through the wire.
//
// Polarity matters here: the No*-fields are inverted from the
// backend's query polarity (see CompletionOptions godoc). A `false`
// value MUST omit the param — sending `cache=true` would still
// match server defaults, but sending `memory=true` would 422 (the
// `memory` schema only allows {const false, null}). This is the
// regression net for that.
cases := []struct {
name string
opts CompletionOptions
wantHas map[string]string // params that must equal a value
wantNot []string // params that must be ABSENT
wantHdr string // expected X-Algolia-Secure-User-Token; "" = absent
}{
{
name: "all defaults: only stream + compatibilityMode set",
opts: CompletionOptions{Stream: true},
wantHas: map[string]string{"stream": "true", "compatibilityMode": "ai-sdk-5"},
wantNot: []string{"cache", "memory", "analytics"},
wantHdr: "",
},
{
name: "--no-cache only",
opts: CompletionOptions{Stream: true, NoCache: true},
wantHas: map[string]string{"cache": "false"},
wantNot: []string{"memory", "analytics"},
},
{
name: "--no-memory only (the most semantically constrained)",
opts: CompletionOptions{Stream: true, NoMemory: true},
wantHas: map[string]string{"memory": "false"},
wantNot: []string{"cache", "analytics"},
},
{
name: "--no-analytics only",
opts: CompletionOptions{Stream: true, NoAnalytics: true},
wantHas: map[string]string{"analytics": "false"},
wantNot: []string{"cache", "memory"},
},
{
name: "all three negative + secure user token header",
opts: CompletionOptions{Stream: true, NoCache: true, NoMemory: true, NoAnalytics: true, SecureUserToken: "ey.signed.jwt"},
wantHas: map[string]string{"cache": "false", "memory": "false", "analytics": "false"},
wantHdr: "ey.signed.jwt",
},
}

for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/1/agents/test/completions", func(w http.ResponseWriter, r *http.Request) {
for k, v := range tc.wantHas {
assert.Equal(t, v, r.URL.Query().Get(k), "query param %q", k)
}
for _, k := range tc.wantNot {
assert.False(t, r.URL.Query().Has(k), "query param %q must be absent", k)
}
assert.Equal(t, tc.wantHdr, r.Header.Get("X-Algolia-Secure-User-Token"))

w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{}`))
})
_, c := newTestClient(t, mux)

resp, err := c.Completions(context.Background(), "test",
json.RawMessage(`{"messages":[{"role":"user","content":"x"}]}`), tc.opts)
require.NoError(t, err)
_ = resp.Body.Close()
})
}
}

func TestCompletions_BodyContentRoundTrip(t *testing.T) {
// Confirms we POST exactly the bytes we were handed (no re-encode).
wire := `{"messages":[{"role":"user","content":"x"}],"id":"conv-1"}`
Expand Down
87 changes: 87 additions & 0 deletions api/agentstudio/mutations_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,93 @@ func TestLifecycle_RejectsEmptyID(t *testing.T) {
}
}

func TestInvalidateAgentCache(t *testing.T) {
cases := []struct {
name string
id string
before string
serverFn func(t *testing.T) http.HandlerFunc
wantErr string // substring; "" = expect success
isSentinel error
}{
{
name: "no before -> DELETE without query",
id: "abc-123",
before: "",
serverFn: func(t *testing.T) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, http.MethodDelete, r.Method)
assert.Equal(t, "", r.URL.RawQuery, "no before -> no query string")
w.WriteHeader(http.StatusNoContent)
}
},
},
{
name: "with before -> DELETE with ?before",
id: "abc-123",
before: "2026-01-15",
serverFn: func(t *testing.T) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, http.MethodDelete, r.Method)
assert.Equal(t, "2026-01-15", r.URL.Query().Get("before"))
w.WriteHeader(http.StatusNoContent)
}
},
},
{
name: "404 from backend surfaces as ErrNotFound",
id: "missing",
before: "",
serverFn: func(t *testing.T) http.HandlerFunc {
return func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNotFound)
_, _ = w.Write([]byte(`{"detail":"Agent not found"}`))
}
},
wantErr: "Agent not found",
isSentinel: ErrNotFound,
},
{
name: "422 with structured detail (e.g. malformed before) surfaces backend message verbatim",
id: "abc-123",
before: "not-a-date",
serverFn: func(t *testing.T) http.HandlerFunc {
return func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusUnprocessableEntity)
_, _ = w.Write([]byte(`{"detail":[{"msg":"Input should be a valid date in YYYY-MM-DD format","loc":["query","before"]}]}`))
}
},
wantErr: "valid date",
},
}

for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/1/agents/"+tc.id+"/cache", tc.serverFn(t))
_, c := newTestClient(t, mux)

err := c.InvalidateAgentCache(context.Background(), tc.id, tc.before)
if tc.wantErr == "" {
require.NoError(t, err)
return
}
require.Error(t, err)
assert.Contains(t, err.Error(), tc.wantErr)
if tc.isSentinel != nil {
assert.True(t, errors.Is(err, tc.isSentinel))
}
})
}
}

func TestInvalidateAgentCache_RejectsEmptyID(t *testing.T) {
_, c := newTestClient(t, http.NewServeMux())
err := c.InvalidateAgentCache(context.Background(), " ", "")
require.Error(t, err)
assert.Contains(t, err.Error(), "agent id is required")
}

func TestLifecycle_NotFound(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/1/agents/missing/publish", func(w http.ResponseWriter, _ *http.Request) {
Expand Down
33 changes: 33 additions & 0 deletions e2e/testscripts/agents/cache.txtar
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# `agents cache invalidate` — flag-validation + dry-run contract.
# None of these hit the network: the failing cases trip cobra/our own
# validators before client construction; the passing case uses --dry-run.
#
# Ungated: runs whenever the standard ALGOLIA_APPLICATION_ID +
# ALGOLIA_API_KEY are present.

# `cache` parent without verb prints help (cobra default for groups
# with no Run; exits 0).
exec algolia agents cache
stdout 'invalidate'

# `invalidate` requires <agent-id> positional
! exec algolia agents cache invalidate
stderr 'requires exactly 1 argument'

# Non-TTY without --confirm is refused (matches `agents delete` rule)
! exec algolia agents cache invalidate abc-123
stderr '--confirm required'

# --dry-run skips the confirm requirement and prints the would-be call
exec algolia agents cache invalidate abc-123 --dry-run
! stderr .
stdout 'Dry run: would DELETE /1/agents/abc-123/cache'
stdout 'all cached completions for this agent'
! stdout '\?before='

# --dry-run with --before describes the bounded scope and includes the
# query string in the previewed URL
exec algolia agents cache invalidate abc-123 --before 2026-01-15 --dry-run
! stderr .
stdout 'Dry run: would DELETE /1/agents/abc-123/cache\?before=2026-01-15'
stdout 'before 2026-01-15'
Loading