Skip to content

feat: validate with signatures — PipeSignature contract-only pipes + --allow-signatures#953

Open
lchoquel wants to merge 25 commits into
devfrom
feature/Validate-with-signatures-3
Open

feat: validate with signatures — PipeSignature contract-only pipes + --allow-signatures#953
lchoquel wants to merge 25 commits into
devfrom
feature/Validate-with-signatures-3

Conversation

@lchoquel
Copy link
Copy Markdown
Member

@lchoquel lchoquel commented May 31, 2026

Summary

Adds PipeSignature — contract-only pipes for top-down pipeline design — and strict signature validation, and merges the latest dev into the branch.

  • PipeSignature pipe type (type = "PipeSignature"): declares a pipe's inputs, output, and description without an implementation, so an author or agent can sketch a complete pipeline before committing to the operator that will do the work. Slots into PipeBlueprintUnion/PipeSpecUnion, produced by PipeSignatureFactory, with its own runtime in pipelex/pipe_signature/. Dry-run mints a mock Stuff matching the declared output; live-run raises PipeSignatureNotExecutableError.
  • --allow-signatures flag on pipelex validate pipe/bundle and every pipelex-agent validate subcommand (default off). Without it, strict validation refuses any pipeline whose dependency graph reaches a signature, raising SignaturesNotAllowedError and reporting every reachable signature plus the controller chain (signature_refs, dep_paths).
  • Graph walk (collect_signature_refs() / collect_signature_paths() in pipelex/pipe_signature/signature_walk.py) powers the strict-mode pre-check in dry_run_pipe/dry_run_pipes and validate_bundle.
  • dry_run_pipe, dry_run_pipes, validate_bundle, validate_bundles_from_directory now accept allow_signatures: bool = False; ValidateBundleError gained signature_check_error: SignaturesNotAllowedError | None so the CLI renders the dep chain.

Merge of dev

This branch had diverged structurally from dev's validation/error-handling refactor (#943/#948/#949). The merge was resolved semantically:

  • Adopted dev's _translate_to_validate_bundle_error context manager + library-lifecycle teardown, and re-applied the signature feature on top (re-enabled dry-run, threaded allow_signatures, single SignaturesNotAllowedError arm shared by all entry points).
  • Honored dev's class-location convention: ValidateBundleError lives in pipelex/pipeline/exceptions.py, extended there with the signature fields.
  • Combined both [Unreleased] CHANGELOG sections.

Verification

  • make agent-check — pyright 0 errors / 0 warnings, mypy clean.
  • make tb (boot/config) green.
  • Signature/validate targeted tests green; make agent-test (full offline suite) passing.

🤖 Generated with Claude Code


Summary by cubic

Adds PipeSignature for contract-only pipes and makes validation strict by default across pipelex and pipelex-agent; pass --allow-signatures to dry-run with mocked outputs. Bumps pipelex to 0.31.0.

  • New Features

    • PipeSignature: declares inputs/output; dry-run returns a mock (multiplicity supported); live-run raises PipeSignatureNotExecutableError.
    • --allow-signatures on pipelex validate and all pipelex-agent validate commands; summaries note signature count in lenient mode.
    • Strict pre-check walks dependencies to find reachable signatures (signature_walk); validation APIs accept allow_signatures; ValidateBundleError carries signature_check_error; CLI renders friendly messages.
  • Migration

    • Validation is strict by default; pipelines that reach a PipeSignature fail unless you pass --allow-signatures. Live execution of a signature always fails.
    • validate bundle --pipe now dry-runs only the requested slice, so unrelated signatures elsewhere don’t block it.
    • This branch uses a branch-local .mthds schema that includes PipeSignature; the Makefile generates the schema before running plxt lint and merge checks.

Written for commit 58ca53b. Summary will update on new commits.

Review in cubic

lchoquel and others added 20 commits May 14, 2026 13:27
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Added PipeSignature and related classes to support contract-only pipe definitions.
- Updated PipeSpecUnion to include PipeSignature.
- Enhanced WorkingMemoryFactory to create mock Stuff for PipeSignatures.
- Modified PipeBlueprint and PipeAbstract to recognize PipeSignature category.
- Implemented PipeSignatureBlueprint and PipeSignatureRuntime for runtime behavior.
- Added tests for PipeSignature integration and validation.
- Updated schema generation to include PipeSignature in definitions.
- Add SignaturesNotAllowedError to handle cases where PipeSignature placeholders are reachable in strict mode.
- Update validate_bundle to accept an allow_signatures flag, allowing lenient validation.
- Modify dry_run_pipes to respect the allow_signatures flag during execution.
- Enhance error messages to provide detailed information about unreachable signatures and their dependency paths.
- Introduce tests for validating bundles with signatures in both strict and lenient modes, ensuring correct error handling and success paths.
- Update existing tests to accommodate changes in signature validation logic.
Resolves four confirmed issues from cubic, greptile, and codex review on PR #899:

- Makefile: make plxt-lint and merge-check-plxt-lint generate the MTHDS schema
  first, so CI no longer fails on clean checkouts where derived/ is unpopulated.
- PipeSignatureBlueprint: reject signature_for=PipeSignature at the language
  layer (was only guarded at the spec layer).
- SignaturesNotAllowedError: carry the set of offending pipe_refs instead of
  a single pipes[0].pipe_ref, so the aggregated message in dry_run_pipes names
  the actual offender(s) rather than the first iterated pipe. Multi-offender
  case uses a plural header.
- _validate_pipe_or_bundle (single-pipe path): wrap SignaturesNotAllowedError
  in a Rich-formatted handler + typer.Exit(1), matching the bundle path's
  treatment; users no longer see a raw traceback.

Includes regression tests for each fix (TDD red-then-green) under
tests/unit/pipelex/pipe_signature/, tests/integration/pipelex/pipe_signature/,
and tests/integration/pipelex/cli/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sNotAllowedError

The property had zero callers — added defensively when the constructor signature
flipped to `offending_pipe_refs`. Per project rule "no backward compatibility",
remove it now rather than carry dead code. `offending_pipe_refs` is the only
remaining identifier for offenders.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…oving pipe_dependencies and enhancing error handling for signatures
- Renamed `PipeSignature` to `PipeSignatureSpec` to clarify its role as a contract-only pipe.
- Introduced a new `PipeSignature` class for runtime execution, which raises errors for unimplemented signatures.
- Updated various modules to reflect the new naming and structural changes, including `pipe_spec_map.py`, `pipe_spec_union.py`, and `registry_models.py`.
- Enhanced the `PipeSignatureSpec` with detailed documentation and validation logic.
- Modified tests to accommodate the new structure and ensure proper functionality of the signature system.
- Updated related documentation to reflect changes in the signature handling and validation processes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
# Conflicts:
#	CHANGELOG.md
#	TODOS.md
…alk module

The collect_signature_refs / collect_signature_paths walk previously lived
on PipeAbstract and took a pipe_lookup callable parameter to dodge the
pipe_abstract -> hub -> library -> pipe_library -> pipe_abstract import
cycle. That parameter was a code smell.

Move the walk into a new module of free functions,
pipelex/pipe_signature/signature_walk.py, which sits downstream of
pipelex.hub and so imports get_optional_pipe directly with no cycle.
PipeAbstract keeps only pipe_dependencies() and is_signature.

Recursion, visited-set cycle protection, sorted iteration, and the
longest/first-path semantics are preserved exactly. All call sites and
tests updated; CHANGELOG corrected.

Also archive the signature-based-validation TDD plan to wip/archive/ and
add a PR-story HTML page briefing the work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a dated post-archive update note recording that the graph walk moved
off PipeAbstract into pipelex/pipe_signature/signature_walk.py, and
correct the now-stale collect_signature_refs(pipe_lookup=...) references
in the always-current sections (Current state, architecture sketch, file
reference). The per-phase deviation notes are left verbatim as the
historical implementation record.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolved conflicts across CHANGELOG.md, the agent-CLI validate commands,
working_memory_factory.py and the other conflicting files — keeping both
the PipeSignature feature and dev's PipeStructure / error-handling /
Temporal work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Semantic resolution of the validation/error-handling divergence:

- Adopted dev's refactored validate_bundle.py (shared _translate_to_validate_bundle_error
  context manager + library-lifecycle teardown guard) and re-applied the signature
  feature on top: allow_signatures threaded into both entry points, dry-run re-enabled
  (required for the strict-mode signature check to fire), and a single
  SignaturesNotAllowedError arm added to the context manager so every entry point wraps
  it into ValidateBundleError(signature_check_error=...).
- Honored dev's class-location convention: ValidateBundleError stays in
  pipelex/pipeline/exceptions.py, extended there with signature_check_error,
  pipe_concept_instantiation_errors, and the pipe_validation_error_data property.
- error_handlers.py: kept both the SignaturesNotAllowedError import and dev's relocated
  ValidateBundleError import path.
- CHANGELOG.md: combined both [Unreleased] sections (signatures + error infra).

make agent-check (pyright/mypy) clean; make agent-test green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 31, 2026

Greptile Summary

This PR adds contract-only signature pipes and validation controls for them. The main changes are:

  • New PipeSignature spec, blueprint, factory, and runtime.
  • Strict signature detection through dependency graph walking.
  • --allow-signatures flags for validation commands.
  • Dry-run support that mocks signature outputs in lenient mode.
  • Docs and tests covering signature validation behavior.

Confidence Score: 3/5

This should be fixed before merging.

  • Strict bundle validation can reject valid bundles that contain unrelated signature stubs.
  • Agent validate bundle --pipe can fail before checking the selected pipe.
  • Main validate method has no way to use the new lenient signature mode.

pipelex/pipe_run/dry_run.py, pipelex/cli/agent_cli/commands/validate/_validate_core.py, and pipelex/cli/commands/validate/method_cmd.py need follow-up.

Important Files Changed

Filename Overview
pipelex/pipe_run/dry_run.py Adds strict signature pre-checks, but batch validation treats standalone signatures as blockers.
pipelex/cli/agent_cli/commands/validate/_validate_core.py Threads allow_signatures through agent validation, but selected-pipe bundle validation still runs whole-bundle strict checks first.
pipelex/cli/commands/validate/method_cmd.py Main method validation was not updated with the new --allow-signatures option.
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
pipelex/pipe_run/dry_run.py:149-154
**Unreached signatures fail bundles**

`dry_run_pipes` raises whenever any pipe in the batch is a `PipeSignature`, even when no non-signature pipe depends on it. `validate_bundle` passes every loaded pipe from the bundle into this function, so a bundle with a complete `main_pipe` plus an unrelated draft signature fails strict validation. That contradicts the advertised contract that strict mode rejects pipelines whose dependency graph reaches a signature, while unreached signatures are allowed.

### Issue 2 of 2
pipelex/cli/agent_cli/commands/validate/_validate_core.py:164-168
**Selected pipe is blocked**

`validate_pipe_in_bundle_core` first calls `validate_bundle` in the same strict mode before it dry-runs the requested pipe. For `pipelex-agent validate bundle draft.mthds --pipe implemented_pipe`, an unrelated `PipeSignature` elsewhere in the bundle makes the preliminary bundle validation fail before the selected pipe is checked. This makes `--pipe` behave like whole-bundle validation for signatures instead of validating only the requested pipe.

Reviews (1): Last reviewed commit: "Merge branch 'dev' into feature/Validate..." | Re-trigger Greptile

Comment thread pipelex/pipe_run/dry_run.py
Comment thread pipelex/cli/agent_cli/commands/validate/_validate_core.py
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2d754e167d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pipelex/cli/agent_cli/commands/validate/_validate_core.py
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 issues found across 64 files

You’re at about 90% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread .pipelex/plxt.toml
Comment thread pipelex/cli/error_handlers.py
Comment thread pipelex/cli/commands/validate/_validate_core.py
Comment thread pipelex/builder/pipe/pipe_signature_spec.py Outdated
Comment thread pipelex/pipe_run/dry_run.py
lchoquel and others added 5 commits May 31, 2026 16:49
- validate bundle --pipe now dry-runs only the requested pipe, so an
  unrelated PipeSignature (or otherwise-broken pipe) elsewhere in the
  bundle no longer blocks validating an implemented slice. Adds a
  dry_run_pipe_codes param to validate_bundle. [greptile/codex]
- validate --all renders SignaturesNotAllowedError as a friendly CLI
  error instead of bubbling an unhandled traceback. [cubic]
- handle_signatures_not_allowed_error honors --traceback like every
  other handle_* function. [cubic]
- escape dynamic values in PipeSignatureSpec.rendered_pretty so concept
  multiplicity (Doc[], Img[3]) and bracketed descriptions render
  literally instead of being parsed as Rich markup. [cubic]
- clarify strict-validation docstrings: whole-bundle strict rejects any
  bundle containing a signature; --pipe validates one implemented slice,
  --allow-signatures enables lenient mocking. [greptile]

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reconcile the signature-validation tracking so a reviewer can verify
execution state against current reality:

- Rewrite wip/signature-based-validation.md from a "no code yet" design
  proposal into a current-state tracker with a file->test verification
  map, the strict-by-default behavior for both CLIs, the post-merge
  review fixes, and Phase 7.4 as the single open item.
- Repoint the .pipelex/plxt.toml "tracked in TODOS.md" reference to the
  live tracker: the docs-tidy merge had overwritten root TODOS.md with an
  unrelated recap and moved the signature plan into wip/archive/.
- Mark the archived TDD plan as superseded and correct its stale "agent
  CLI defaults to lenient" claim (the shipped default is strict).
- List the signature work in wip/README.md (it was absent from the index).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant