Skip to content

[ddev/ai/phases]: Introduce AgenticPhase and make Phase an abstract lifecycle base#23663

Open
luisorofino wants to merge 11 commits into
loa/ddev-validatefrom
loa/agent-phase-refactor
Open

[ddev/ai/phases]: Introduce AgenticPhase and make Phase an abstract lifecycle base#23663
luisorofino wants to merge 11 commits into
loa/ddev-validatefrom
loa/agent-phase-refactor

Conversation

@luisorofino
Copy link
Copy Markdown
Contributor

@luisorofino luisorofino commented May 11, 2026

What does this PR do?

Refactors the phases layer of the AI framework to separate the lifecycle contract from the agent-driven implementation.

  • Splits Phase into two classes:
    • Phase (abstract): owns the immutable lifecycle skeleton — should_process_message, process_message, on_success, on_error, checkpoint writing, memory persistence. Subclasses must implement async execute(context) -> PhaseOutcome.
    • AgenticPhase(Phase): owns the LLM pipeline — builds the agent and ReActProcess, runs tasks (run_tasks), runs the memory step (_run_memory_step), and exposes before_react / after_react hooks.
  • Introduces a PhaseOutcome dataclass (memory_text, total_input_tokens, total_output_tokens, extra_checkpoint) as the single return value of execute(). The base Phase.process_message consumes it to assemble the checkpoint payload, so subclasses no longer reach into checkpoint internals.
  • Adds Phase.validate_config classmethod (no-op by default) and invokes it from the orchestrator for every phase scheduled in flow:. Orphan phases (defined but not in flow:) are skipped. AgenticPhase.validate_config enforces agent is set, refers to a known agent, and that tasks is non-empty.
  • Makes PhaseConfig.agent and PhaseConfig.tasks optional (with type default \"AgenticPhase\"), so non-agentic Phase subclasses can be configured without dummy agent/tasks entries.
  • Moves the agent_config / anthropic_client constructor parameters off of Phase and onto AgenticPhase, where they belong. The orchestrator only passes them when instantiating an AgenticPhase subclass.
  • Reorganizes tests: agent-driven behaviour now lives in test_agentic_phase.py; test_base.py covers the lifecycle contract via a _StubPhase. Shared mock helpers and fixtures (MockAgent, make_agent_phase, flow_dir, message_queue) moved into phases/conftest.py.

Motivation

Until now, Phase was both the lifecycle contract and the agent-driven implementation, which made it impossible to add a non-LLM Phase subclass (e.g. a deterministic validation phase, a phase that just reads files, etc.) without dragging along the entire agent machinery. Separating the two layers keeps the framework generic and reusable, and allows us to define non-LLM-driven phases.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the `qa/skip-qa` label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the `backport/` label to the PR and it will automatically open a backport PR once this one is merged

- Add PhaseOutcome dataclass (memory_text, token counts, extra_checkpoint)
- Add validate_config() classmethod to Phase (no-op default)
- Add execute() method that implements the agent pipeline (later to be overridden by AgentPhase)
- Rewrite process_message() to call execute() and assemble the checkpoint from PhaseOutcome
- Create agent_phase.py with AgentPhase(Phase) that owns the LLM pipeline:
  before_react/after_react hooks, run_tasks, execute()
- Move render_task_prompt and render_memory_prompt to agent_phase.py
- AgentPhase.validate_config enforces agent, known-agent, and non-empty tasks
- Phase.execute() now raises NotImplementedError — subclasses must implement it
- Strip base.py of all agent-specific code and imports
- Split test_base.py into lifecycle-only tests (using _StubPhase) and
  test_agent_phase.py for the agent-driven behaviour tests
- type default: "Phase" → "AgentPhase"
- agent: str (required) → str | None = None
- tasks: list[TaskConfig] (required) → list[TaskConfig] = []
- Remove at_least_one_task field validator (now enforced by AgentPhase.validate_config)
- FlowConfig.cross_references: skip unknown-agent check when agent is None
- orchestrator: guard agent_config lookup against None, import AgentConfig
- test_config.py: update type assertion, remove empty_tasks test, add
  test_flow_config_phase_without_agent_validates
- test_base.py / test_agent_phase.py: drop model_construct workarounds
- Call phase_cls.validate_config(phase_id, config, agents) immediately after
  resolving the phase class in on_initialize — only for phases scheduled in flow:
- Orphan phases (defined but absent from flow:) are skipped before the call
- test_orchestrator.py: drop explicit type: Phase lines from fixtures (use AgentPhase default),
  assert AgentPhase is registered by discovery, add tests for validate_config
  invocation and orphan-skip behaviour
@dd-octo-sts dd-octo-sts Bot added the ddev label May 11, 2026
@datadog-official
Copy link
Copy Markdown
Contributor

datadog-official Bot commented May 11, 2026

Tests

Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 7 Tests failed

❄️ Known flaky: test_config[Metric collection disabled] from test_config.py   View in Datadog   (Fix with Cursor)
RetryError[<Future at 0x7ff9e4f1b890 state=finished raised RetryError>]
❄️ Known flaky: test_config[Resource stats collection disabled; message_flows enabled] from test_config.py   View in Datadog   (Fix with Cursor)
RetryError[<Future at 0x7ff9e4f1b890 state=finished raised RetryError>]
❄️ Known flaky: test_config[Resource stats collection enabled; message_flows disabled] from test_config.py   View in Datadog   (Fix with Cursor)
RetryError[<Future at 0x7ff9e4f1b890 state=finished raised RetryError>]
View all

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 98.09%
Overall Coverage: 87.59% (-0.00%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 36d0a4b | Docs | Datadog PR Page | Give us feedback!

@codecov
Copy link
Copy Markdown

codecov Bot commented May 11, 2026

Codecov Report

❌ Patch coverage is 98.08917% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.99%. Comparing base (b288a9b) to head (36d0a4b).

Additional details and impacted files
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@luisorofino luisorofino added the qa/skip-qa Automatically skip this PR for the next QA label May 12, 2026
@luisorofino luisorofino changed the title Loa/agent phase refactor refactor(ai/phases): introduce AgenticPhase and make Phase an abstract lifecycle base May 12, 2026
@luisorofino luisorofino changed the title refactor(ai/phases): introduce AgenticPhase and make Phase an abstract lifecycle base [ddev/ai/phases]: Introduce AgenticPhase and make Phase an abstract lifecycle base May 12, 2026
@luisorofino luisorofino marked this pull request as ready for review May 12, 2026 12:39
@luisorofino luisorofino requested a review from a team as a code owner May 12, 2026 12:39
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f725277fea

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread ddev/src/ddev/ai/phases/base.py
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 12, 2026

Validation Report

All 20 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and Codecov settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ddev qa/skip-qa Automatically skip this PR for the next QA team/agent-integrations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant