-
Notifications
You must be signed in to change notification settings - Fork 9
docs: refresh recent behavior notes #262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
f0c0781
d9e72ee
61d49ad
bbaec28
096bb68
a69370d
815a869
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,10 +1,31 @@ | ||||||
| # @poe-code/acp-telemetry | ||||||
|
|
||||||
| Pure ACP -> trace converters plus Braintrust/OTEL emitters. | ||||||
| Pure ACP event-to-trace conversion plus Braintrust and OpenTelemetry emitters. | ||||||
|
|
||||||
| ## Public Exports | ||||||
|
|
||||||
| - `@poe-code/acp-telemetry` | ||||||
| - `acpToTrace(ctx)` converts an `@poe-code/agent-spawn` ACP spawn context into an `AcpTrace`. | ||||||
| - `emitToBraintrust(trace, parent)` writes the trace as nested Braintrust task/tool spans. | ||||||
| - `emitToOtel(trace, tracer)` writes the trace as OpenTelemetry-style spans and attributes. | ||||||
| - `redact(value)` removes sensitive prompt, tool, and metadata fields before emission. | ||||||
| - Types: `AcpTrace`, `AcpTraceSpan`, `BraintrustSpanLike`, `OtelSpanLike`, `OtelTracerLike`. | ||||||
|
|
||||||
| ## Trace shape | ||||||
|
|
||||||
| `acpToTrace` creates one root `agent:<agent>:<model>` span with redacted prompt input, accumulated assistant output, token/cost/duration metrics when present, session/thread metadata, and one child span per ACP tool call. Tool child spans include redacted inputs, assembled tool outputs, tool call metadata, and start/end timestamps when the ACP event metadata includes them. | ||||||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| ```ts | ||||||
| import { acpToTrace, emitToOtel } from "@poe-code/acp-telemetry"; | ||||||
|
|
||||||
| const trace = acpToTrace(spawnContext); | ||||||
| emitToOtel(trace, tracer); | ||||||
| ``` | ||||||
|
|
||||||
| ## Emitters | ||||||
|
|
||||||
| Braintrust emission expects a parent span-like object with `startSpan`, `log`, and `end`. The root is emitted as a `task`; children are emitted as `tool` spans. | ||||||
|
|
||||||
| OpenTelemetry emission expects a tracer-like object with `startSpan`. Agent spans set `gen_ai.system`, request model, agent name, token usage, and Poe Code session/thread attributes. Tool spans set tool name and tool-call id attributes. Non-primitive inputs and outputs are serialized as JSON attributes. | ||||||
|
|
||||||
| ## Configuration | ||||||
|
|
||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -64,6 +64,14 @@ Notes: | |||||
| - Cancellation is only possible in the interactive prompt path. If `select(...)` returns a value that `isCancel(...)` recognizes, the function returns `{ cancelled: true }` and does not throw. Callers are responsible for showing the cancellation message and stopping the command cleanly. | ||||||
| - `pipeline`, `experiment`, `ralph`, and `superintendent` all route their single-agent loop selection through this function so the precedence stays aligned across commands. | ||||||
|
|
||||||
| ## Plan document helpers | ||||||
|
|
||||||
| `openPlanList`, `discoverPlans`, and `archivePlan` expose numbered Markdown plan folders through `@poe-code/task-list`. They resolve the configured plan directory with the same cwd/home rules as workflow docs, open it as a `markdown-dir` single-list named `plans`, and use `frontmatterMode: "passthrough"` so plan-specific metadata survives task updates. | ||||||
|
|
||||||
| - `discoverPlans({ cwd, homeDir, planDirectory, kinds? })` returns plan ids, names, kinds, absolute paths, and display paths. Numeric filename prefixes such as `04-api-shape-providers.md` are stripped from ids. | ||||||
| - `archivePlan({ cwd, homeDir, planDirectory, id })` fires the task-list `archive` event, moves the document under `archive/`, and repacks active plan prefixes. | ||||||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| - `openPlanList(...)` returns the underlying `TaskList` for commands that need direct task operations. | ||||||
|
|
||||||
| ## Configuration | ||||||
|
|
||||||
| This package does not read config files directly, but Poe Code callers commonly pass `configuredDefaultAgent` from merged `core.defaultAgent`. | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,40 @@ | ||
| # @poe-code/agent-harness | ||
|
|
||
| Shared package for agent harness types and runtime orchestration APIs. | ||
| Shared harness loader, template, schema, and runtime orchestration APIs for `.md` + `.ajs` agent-script harness pairs. | ||
|
|
||
| This package is currently a scaffold. Loader, module, template, codegen, and CLI behavior will be added in later tasks. | ||
| ## Public API | ||
|
|
||
| - `runHarnessPair(mdPath, options)` resolves the matching `.ajs` file, validates frontmatter against any exported `schema`, lints the script, and runs it through `@poe-code/agent-script`. | ||
| - `listBuiltinTemplates()` returns bundled template pairs: `ralph-demo`, `coverage-demo`, `experiment-demo`, `pipeline-demo`, and `superintendent-demo`. | ||
| - `extractSchema(source, filename)` reads a harness script's exported schema for frontmatter validation. | ||
| - `resolvePair(mdPath)` resolves the Markdown/script pair for a harness document. | ||
| - `LintError` wraps lint diagnostics raised before execution. | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
|
||
| ## Harness pairs | ||
|
|
||
| A harness is a Markdown document plus a sibling `.ajs` script. The Markdown frontmatter configures the run; the body is passed to the harness import metadata. The `.ajs` file must export a default entry point and may export `schema` to validate frontmatter before execution. | ||
|
|
||
| `runHarnessPair` locks the Markdown file while it runs, injects the `schema` module, wraps host modules for deterministic replay across resumes, writes snapshots, and cleans up completed snapshots after successful runs. | ||
|
|
||
| ## Snapshots and resume | ||
|
|
||
| Pass `snapshotPath` to control where snapshots are read and written. `resume` defaults to `true`; set `resume: false` to remove a completed snapshot and force a fresh run. If a snapshot exists, the underlying agent-script source hash must still match the `.ajs` source. | ||
|
|
||
| The CLI mirrors these options: | ||
|
|
||
| ```sh | ||
| poe-code harness run harness.md --snapshot-path .poe-code/harnesses/demo/snapshot.json --resume | ||
| poe-code harness new coverage-demo coverage.md | ||
| ``` | ||
|
|
||
| ## Built-in templates | ||
|
|
||
| `listBuiltinTemplates()` exposes template metadata with `kind`, `mdPath`, and `ajsPath`. `poe-code harness new <kind> <path>` copies both files into a new harness pair. | ||
|
|
||
| ## Environment Variables | ||
|
|
||
| This package does not read any environment variables. | ||
|
|
||
| ## Configuration | ||
|
|
||
| This package does not read any configuration options. | ||
| This package does not read package-level configuration. Runtime behavior is supplied through `runHarnessPair` options: `modulesFor`, `allowedGlobals`, `resume`, `signal`, and `snapshotPath`. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
acpToTracedoes not currently include any cost metric fromctx.usage(it only maps token counts/cache counts anddurationMsinbuildMetrics), so documenting cost here is misleading.