Bootstrap developer documentation surface

docs/agents-runtime-tools.md

@@ -0,0 +1,171 @@
+# Agents, runtime, and tools
+
+This page documents the core runtime path: registered agent definitions, normalized messages, the multi-turn conversation loop, compaction and budget controls, and generic tool mediation.
+
+## Agent definitions and registry
+
+`WP_Agent` is a declarative value object, not a database row or runtime base class. Constructing or registering one does not create storage, token records, scaffold files, or directories.
+
+Recognized registration fields include:
+
+- `label` and `description` for diagnostics and UI surfaces owned by consumers.
+- `memory_seeds`, a map of scaffold filename to source path.
+- `owner_resolver`, an optional callable used by owner-based effective-agent resolution.
+- `default_config`, caller-owned runtime config such as provider/model/tool policy defaults.
+- `supports_conversation_compaction` and `conversation_compaction_policy`.
+- `meta`, including reserved provenance keys `source_plugin`, `source_type`, `source_package`, and `source_version`.
+- `subagents`, a sanitized list of agent slugs coordinated by this agent.
+
+Register agents inside `wp_agents_api_init`:
+
+```php
+add_action(
+    'wp_agents_api_init',
+    static function () {
+        wp_register_agent(
+            'example-agent',
+            array(
+                'label'          => 'Example Agent',
+                'default_config' => array(
+                    'tool_policy' => array(
+                        'mode'       => 'allow',
+                        'categories' => array( 'read' ),
+                    ),
+                ),
+                'meta'           => array(
+                    'source_plugin' => 'example-plugin/example-plugin.php',
+                ),
+            )
+        );
+    }
+);
+```
+
+`WP_Agents_Registry` stores definitions in memory for the request. Public helpers are `wp_register_agent()`, `wp_get_agent()`, `wp_get_agents()`, `wp_has_agent()`, and `wp_unregister_agent()`.
+
+## Message envelopes
+
+`AgentsAPI\AI\WP_Agent_Message` normalizes both plain `role`/`content` messages and typed envelopes into a JSON-friendly shape:
+
+```php
+array(
+    'schema'   => 'agents-api.message',
+    'version'  => 1,
+    'type'     => 'text',
+    'role'     => 'assistant',
+    'content'  => 'Hello',
+    'payload'  => array(),
+    'metadata' => array(),
+)
+```
+
+Supported message types include `text`, `tool_call`, `tool_result`, `input_required`, `approval_required`, `final_result`, `error`, `delta`, and `multimodal_part`. Helpers such as `text()`, `toolCall()`, `toolResult()`, and `approvalRequired()` build canonical envelopes. `to_provider_message()` projects an envelope back to a provider-facing `role`/`content` shape while preserving metadata needed by adapters.
+
+## Conversation loop boundary
+
+`AgentsAPI\AI\WP_Agent_Conversation_Loop::run()` sequences a conversation around caller-owned adapters. The loop owns:
+
+- message normalization;
+- optional compaction before each turn;
+- invoking a caller-supplied turn runner;
+- validating/normalizing turn results;
+- optional tool-call mediation;
+- typed completion policy checks;
+- transcript persistence through `WP_Agent_Transcript_Persister`;
+- optional transcript locking through `WP_Agent_Conversation_Lock`;
+- iteration budget enforcement;
+- lifecycle event emission through `on_event` and `agents_api_loop_event`.
+
+The loop does **not** assemble prompts, select providers/models, implement concrete tools, choose storage, expose UI, or define product-specific workflow semantics.
+
+Minimal caller-managed use:
+
+```php
+$result = AgentsAPI\AI\WP_Agent_Conversation_Loop::run(
+    $messages,
+    static function ( array $messages, array $context ): array {
+        return $provider_adapter->run_turn( $messages, $context );
+    },
+    array(
+        'max_turns' => 4,
+        'context'   => array( 'agent_id' => 'example-agent' ),
+    )
+);
+```
+
+When `tool_executor` and `tool_declarations` are provided, the turn runner can return `tool_calls`; the loop validates, executes, and appends tool-call/tool-result messages. In mediation mode, `should_continue` defaults to continue so natural completion, budgets, completion policy, and `max_turns` become the stop signals.
+
+## Loop options and failure modes
+
+Important options:
+
+- `max_turns`: defaults to `1`; synthesized into a `turns` budget unless an explicit budget is provided.
+- `budgets`: `WP_Agent_Iteration_Budget` instances keyed by budget name.
+- `completion_policy`: `WP_Agent_Conversation_Completion_Policy` implementation.
+- `transcript_persister`: called on success and failure; persister exceptions are swallowed so persistence cannot change the runtime result.
+- `transcript_lock` / `transcript_lock_store`: advisory session lock; contention returns status `transcript_lock_contention`.
+- `on_event`: caller-owned observer. Exceptions are swallowed.
+
+The loop emits `turn_started`, `tool_call`, `tool_result`, `budget_exceeded`, `completed`, `failed`, and lock-contention events. The same event snapshots are also published through `do_action( 'agents_api_loop_event', $event, $payload )`.
+
+## Compaction and conservation
+
+`WP_Agent_Conversation_Compaction` is provider-neutral. Consumers pass a policy and summarizer callable. The policy controls whether compaction is enabled, message thresholds, recent message retention, summary role/prefix/provider/model metadata, tool-boundary preservation, conservation checks, and deterministic overflow archiving.
+
+If summarization fails or conservation fails with fail-closed enabled, the original transcript is returned with a failure event rather than silently dropping history. Overflow archiving can deterministically split old messages into archive items and insert an archive stub without a model call.
+
+Related generic primitives include `WP_Agent_Compaction_Item`, `WP_Agent_Compaction_Conservation`, and `WP_Agent_Markdown_Section_Compaction_Adapter`.
+
+## Iteration budgets
+
+`WP_Agent_Iteration_Budget` is a stateful per-run counter with a name and ceiling. The loop natively understands:
+
+- `turns`;
+- `tool_calls`;
+- `tool_calls_<tool_name>`.
+
+When an explicit budget is exceeded, the loop returns `status: budget_exceeded` and `budget: <name>` and emits a `budget_exceeded` event.
+
+## Tool declarations and execution
+
+`AgentsAPI\AI\Tools\WP_Agent_Tool_Declaration` describes a tool in a normalized runtime shape. `WP_Agent_Tool_Call`, `WP_Agent_Tool_Parameters`, and `WP_Agent_Tool_Result` normalize invocation and result data.
+
+`WP_Agent_Tool_Execution_Core` performs product-neutral mediation:
+
+1. Find the requested tool declaration.
+2. Validate required parameters.
+3. Build normalized parameters with runtime context.
+4. Call a consumer-supplied `WP_Agent_Tool_Executor`.
+5. Catch executor exceptions and normalize success/error results.
+
+Concrete execution remains a consumer adapter.
+
+## Tool visibility policy
+
+`WP_Agent_Tool_Policy` resolves the visible tool map. Inputs include:
+
+- runtime mode (`chat`, `pipeline`, or `system`);
+- tool-declared `mode` / `modes`;
+- `tool_access_checker` callable;
+- registered agent or runtime `tool_policy`;
+- `WP_Agent_Tool_Access_Policy` providers from constructor, context, or `agents_api_tool_policy_providers`;
+- runtime `categories`, `allow_only`, and `deny` lists.
+
+The final map can be filtered with `agents_api_resolved_tools`.
+
+## Action policy
+
+`WP_Agent_Action_Policy_Resolver` resolves whether a visible tool executes as `direct`, `preview`, or `forbidden`.
+
+Resolution order:
+
+1. Explicit runtime `deny` list.
+2. Agent/runtime per-tool action policy.
+3. Agent/runtime per-category action policy.
+4. Host `WP_Agent_Action_Policy_Provider` providers from constructor, context, or `agents_api_action_policy_providers`.
+5. Tool-declared `action_policy`.
+6. Tool-declared mode-specific `action_policy_<mode>`.
+7. Default `direct`.
+8. Final `agents_api_tool_action_policy` filter.
+
+The resolver returns only canonical values from `AgentsAPI\AI\Tools\WP_Agent_Action_Policy`.

docs/architecture.md

@@ -0,0 +1,69 @@
+# Architecture and bootstrap lifecycle
+
+Agents API is a standalone WordPress plugin that loads a provider-neutral agent runtime substrate. The implementation is intentionally organized as contracts, value objects, registries, and narrow orchestration helpers rather than a product application.
+
+## Requirements and package shape
+
+- Plugin entry point: `agents-api.php`.
+- Composer package: `automattic/agents-api` with PHP `>=8.1`.
+- Plugin metadata declares version `0.1.0`, GPL-2.0-or-later licensing, and WordPress `Requires at least: 7.0` in the inspected ref.
+- `woocommerce/action-scheduler` is suggested, not required. Workflow and routine scheduling bridges detect Action Scheduler at runtime and no-op when it is unavailable.
+- Tests run through `composer test`, which executes the smoke-test suite listed in `composer.json`.
+
+## Layer boundary
+
+```text
+wp-ai-client -> provider/model prompt execution and provider capabilities
+Agents API   -> identity, runtime contracts, orchestration contracts, tool mediation contracts, memory/transcripts/sessions
+Consumers    -> product UX, concrete tools, workflows, prompt policy, storage/materialization policy
+```
+
+Agents API does not own provider request code, admin screens, product REST controllers, workflow editors, concrete tool execution adapters, concrete persistence schemas, prompt assembly policy, support routing, or product-specific runtime semantics.
+
+## Bootstrap flow
+
+`agents-api.php` guards against double-loading with `AGENTS_API_LOADED`, defines `AGENTS_API_PATH` and `AGENTS_API_PLUGIN_FILE`, then requires the source modules explicitly. The load order matters because the repository uses WordPress-shaped class files rather than Composer autoloading.
+
+At WordPress `init`:
+
+- `WP_Guidelines_Substrate::register()` runs at priority `9`, registering the guideline CPT/taxonomy polyfill when enabled and when no upstream provider already registered those objects.
+- `WP_Agents_Registry::init()` runs at priority `10`, opens the `wp_agents_api_init` registration window, and fires `do_action( 'wp_agents_api_init', $registry )`.
+
+Agent registrations are expected inside `wp_agents_api_init`; reads such as `wp_get_agent()` are safe after `init` has fired.
+
+## Source module map
+
+| Module | Responsibility |
+| --- | --- |
+| `src/Registry` | Declarative `WP_Agent` value object, in-memory `WP_Agents_Registry`, and public agent helper functions. |
+| `src/Runtime` | Message envelopes, execution principals, effective-agent resolution, conversation request/result/runner contracts, loop sequencing, compaction, iteration budgets, transcript persister interfaces. |
+| `src/Tools` | Tool declarations, calls, parameters, execution core, executor contracts, tool visibility policy, action policy, and policy provider contracts. |
+| `src/Auth` | Access grants, token metadata/store/authenticator, authorization policies, capability ceilings, and cross-site caller context. |
+| `src/Approvals` | Pending action, approval decision, status, store, resolver, and handler contracts. |
+| `src/Consent` | Generic consent operations, decisions, policy interface, and conservative default policy. |
+| `src/Context` | Memory/context source registry, context section registry, composable context, retrieved-context authority tiers, conflict kinds, conflict resolver contracts. |
+| `src/Memory` | Store-neutral memory scope, metadata, query, validation, read/write/list results, store capabilities, validators, and store interface. |
+| `src/Guidelines` | `wp_guideline` / `wp_guideline_type` substrate polyfill and explicit guideline capability mapping. |
+| `src/Channels` | Direct channel base class, canonical chat ability, external message normalization, session map/store, webhook signature/idempotency, remote bridge services. |
+| `src/Workflows` | Workflow spec, validator, binding expansion, runner, registry, store/recorder contracts, canonical workflow abilities, Action Scheduler bridge/listener. |
+| `src/Routines` | Persistent scheduled routine value object, registry, public helpers, Action Scheduler bridge/listener. |
+| `src/Transcripts` | Conversation store and lock contracts plus null lock implementation. |
+| `src/Identity` | Agent identity scope, materialized identity, and identity store contract. |
+| `src/Workspace` | Generic workspace identity value object. |
+| `src/Packages` | Agent package, package artifact, artifact type, artifact registry, adoption diff/result/adopter contracts. |
+
+## Design principles
+
+1. **Provider neutrality.** Consumers provide the model/provider dispatch adapter; Agents API only normalizes runtime contracts around it.
+2. **WordPress-shaped integration.** Public registration helpers, `init` lifecycle, `do_action`/`apply_filters`, capability checks, and optional CPT/taxonomy substrate follow WordPress conventions.
+3. **Contracts before implementations.** Memory, transcript, token, access, workflow store, run recorder, pending-action store, and default stores are interfaces or proposals; concrete storage is consumer-owned or belongs in a companion package.
+4. **Durability boundaries are explicit.** Locks, compare-and-swap hashes, queue acknowledgements, approval resolution fields, and metadata capability declarations make failure modes visible to consumers.
+5. **Fail closed where trust is involved.** Token authentication rejects expired tokens and malformed caller context before touching tokens; caller headers are claims only; guideline private memory uses explicit owner/workspace metadata and capabilities rather than ordinary private-post semantics.
+6. **No product imports.** `tests/no-product-imports-smoke.php` guards the boundary that Agents API must not import Data Machine or other consumer product classes.
+
+## Operational workflows
+
+- Run all smoke tests with `composer test`.
+- Validate individual PHP files with `php -l <file>` for focused changes.
+- Use `homeboy`/Playground CI workflows only as consumer infrastructure; the substrate itself remains ordinary PHP/WordPress plugin code.
+- Documentation maintenance should keep `docs/coverage-map.md` aligned when source modules, public hooks, workflow/routine abilities, or contract interfaces change.

docs/index.md

@@ -0,0 +1,33 @@
+# Agents API developer documentation
+
+Agents API is a WordPress-shaped, provider-neutral substrate for agent runtime contracts. It provides value objects, registries, orchestration seams, and policy contracts that product plugins can compose without copying the same agent primitives. Product plugins still own concrete provider dispatch, product UI, durable storage implementations, prompt policy, runtime adapters, and product-specific workflows.
+
+This documentation surface was bootstrapped from `README.md`, `docs/**`, `agents-api.php`, the `src/**` modules, `composer.json`, smoke tests, open and closed issues, and recent pull requests. The source-derived coverage map is maintained in [Coverage map](coverage-map.md).
+
+## Start here
+
+| Topic | Scope |
+| --- | --- |
+| [Architecture and bootstrap lifecycle](architecture.md) | Plugin loading, module boundaries, initialization hooks, requirements, testing, and design principles. |
+| [Agents, runtime, and tools](agents-runtime-tools.md) | Agent registration, conversation loop, messages, compaction, iteration budgets, tool declarations, execution mediation, tool visibility, and action policy. |
+| [Authorization, identity, and permissions](auth-identity-permissions.md) | Execution principals, bearer tokens, access grants, caller context, capability ceilings, effective-agent resolution, identity, and workspace scope. |
+| [Memory, context, and guidelines](memory-context-guidelines.md) | Memory store contracts, provenance metadata, context registries, retrieved-context authority, composable sections, and `wp_guideline` substrate. |
+| [Channels and bridges](channels-bridges.md) | `agents/chat`, direct channels, external message normalization, session mapping, webhook safety, and remote bridge queue/pending/ack semantics. |
+| [Workflows and routines](workflows-routines.md) | Workflow specs, validation, bindings, runner, canonical workflow abilities, optional Action Scheduler bridge, routines, and subagent declarations. |
+| [Persistence, approvals, consent, and packages](persistence-approvals-consent-packages.md) | Transcript stores and locks, pending action approval contracts, consent decisions, package artifact contracts, and default-store boundary. |
+| [Coverage map](coverage-map.md) | Source/test/docs/issues/PR evidence, coverage status by source area, known out-of-scope areas, and maintenance checklist. |
+
+## Public extension points at a glance
+
+- **Registration hooks:** `wp_agents_api_init`, `wp_abilities_api_categories_init`, `wp_abilities_api_init`.
+- **Agent registry helpers:** `wp_register_agent()`, `wp_get_agent()`, `wp_get_agents()`, `wp_has_agent()`, `wp_unregister_agent()`.
+- **Workflow/routine helpers:** `wp_register_workflow()`, `wp_get_workflow()`, `wp_register_routine()`, `wp_get_routine()` and the workflow handler helper in `AgentsAPI\AI\Workflows\register_workflow_handler()`.
+- **Canonical abilities:** `agents/chat`, `agents/run-workflow`, `agents/validate-workflow`, and `agents/describe-workflow`.
+- **Runtime hooks and filters:** `agents_api_loop_event`, `agents_api_execution_principal`, `agents_api_resolved_tools`, `agents_api_tool_action_policy`, `agents_api_tool_policy_providers`, `agents_api_action_policy_providers`, `wp_agent_chat_handler`, `wp_agent_workflow_handler`, `wp_agent_workflow_step_handlers`, `wp_agent_workflow_known_step_types`, `wp_agent_workflow_known_trigger_types`, `agents_run_workflow_permission`, `agents_validate_workflow_permission`, `wp_guidelines_substrate_enabled`, `agents_api_memory_sources`, and `agents_api_context_sections`.
+- **Scheduling hooks:** `wp_agent_workflow_schedule_requested`, `wp_agent_workflow_run_scheduled`, `wp_agent_routine_schedule_requested`, `wp_agent_routine_run_scheduled`, and routine registry lifecycle hooks.
+
+## Ownership boundary
+
+Agents API owns reusable contracts and provider-neutral runtime mechanics: agent definitions, message envelopes, conversation loop sequencing, tool mediation, authorization shapes, context/memory contracts, channel/bridge substrate, workflow/routine plumbing, transcript and approval interfaces, consent value objects, package artifact contracts, and guideline substrate vocabulary.
+
+Consumers own concrete provider calls, product UI, REST controllers, durable storage, store schemas, workflow editors, concrete tools, prompt assembly, support/escalation routing, approval UX, connector onboarding screens, and product-specific policy decisions.