feat: validate with signatures — PipeSignature contract-only pipes + --allow-signatures#953
feat: validate with signatures — PipeSignature contract-only pipes + --allow-signatures#953lchoquel wants to merge 25 commits into
Conversation
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Added PipeSignature and related classes to support contract-only pipe definitions. - Updated PipeSpecUnion to include PipeSignature. - Enhanced WorkingMemoryFactory to create mock Stuff for PipeSignatures. - Modified PipeBlueprint and PipeAbstract to recognize PipeSignature category. - Implemented PipeSignatureBlueprint and PipeSignatureRuntime for runtime behavior. - Added tests for PipeSignature integration and validation. - Updated schema generation to include PipeSignature in definitions.
- Add SignaturesNotAllowedError to handle cases where PipeSignature placeholders are reachable in strict mode. - Update validate_bundle to accept an allow_signatures flag, allowing lenient validation. - Modify dry_run_pipes to respect the allow_signatures flag during execution. - Enhance error messages to provide detailed information about unreachable signatures and their dependency paths. - Introduce tests for validating bundles with signatures in both strict and lenient modes, ensuring correct error handling and success paths. - Update existing tests to accommodate changes in signature validation logic.
Resolves four confirmed issues from cubic, greptile, and codex review on PR #899: - Makefile: make plxt-lint and merge-check-plxt-lint generate the MTHDS schema first, so CI no longer fails on clean checkouts where derived/ is unpopulated. - PipeSignatureBlueprint: reject signature_for=PipeSignature at the language layer (was only guarded at the spec layer). - SignaturesNotAllowedError: carry the set of offending pipe_refs instead of a single pipes[0].pipe_ref, so the aggregated message in dry_run_pipes names the actual offender(s) rather than the first iterated pipe. Multi-offender case uses a plural header. - _validate_pipe_or_bundle (single-pipe path): wrap SignaturesNotAllowedError in a Rich-formatted handler + typer.Exit(1), matching the bundle path's treatment; users no longer see a raw traceback. Includes regression tests for each fix (TDD red-then-green) under tests/unit/pipelex/pipe_signature/, tests/integration/pipelex/pipe_signature/, and tests/integration/pipelex/cli/. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sNotAllowedError The property had zero callers — added defensively when the constructor signature flipped to `offending_pipe_refs`. Per project rule "no backward compatibility", remove it now rather than carry dead code. `offending_pipe_refs` is the only remaining identifier for offenders. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…oving pipe_dependencies and enhancing error handling for signatures
- Renamed `PipeSignature` to `PipeSignatureSpec` to clarify its role as a contract-only pipe. - Introduced a new `PipeSignature` class for runtime execution, which raises errors for unimplemented signatures. - Updated various modules to reflect the new naming and structural changes, including `pipe_spec_map.py`, `pipe_spec_union.py`, and `registry_models.py`. - Enhanced the `PipeSignatureSpec` with detailed documentation and validation logic. - Modified tests to accommodate the new structure and ensure proper functionality of the signature system. - Updated related documentation to reflect changes in the signature handling and validation processes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ands to support PipeSignature placeholders
# Conflicts: # CHANGELOG.md # TODOS.md
…alk module The collect_signature_refs / collect_signature_paths walk previously lived on PipeAbstract and took a pipe_lookup callable parameter to dodge the pipe_abstract -> hub -> library -> pipe_library -> pipe_abstract import cycle. That parameter was a code smell. Move the walk into a new module of free functions, pipelex/pipe_signature/signature_walk.py, which sits downstream of pipelex.hub and so imports get_optional_pipe directly with no cycle. PipeAbstract keeps only pipe_dependencies() and is_signature. Recursion, visited-set cycle protection, sorted iteration, and the longest/first-path semantics are preserved exactly. All call sites and tests updated; CHANGELOG corrected. Also archive the signature-based-validation TDD plan to wip/archive/ and add a PR-story HTML page briefing the work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a dated post-archive update note recording that the graph walk moved off PipeAbstract into pipelex/pipe_signature/signature_walk.py, and correct the now-stale collect_signature_refs(pipe_lookup=...) references in the always-current sections (Current state, architecture sketch, file reference). The per-phase deviation notes are left verbatim as the historical implementation record. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolved conflicts across CHANGELOG.md, the agent-CLI validate commands, working_memory_factory.py and the other conflicting files — keeping both the PipeSignature feature and dev's PipeStructure / error-handling / Temporal work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Semantic resolution of the validation/error-handling divergence: - Adopted dev's refactored validate_bundle.py (shared _translate_to_validate_bundle_error context manager + library-lifecycle teardown guard) and re-applied the signature feature on top: allow_signatures threaded into both entry points, dry-run re-enabled (required for the strict-mode signature check to fire), and a single SignaturesNotAllowedError arm added to the context manager so every entry point wraps it into ValidateBundleError(signature_check_error=...). - Honored dev's class-location convention: ValidateBundleError stays in pipelex/pipeline/exceptions.py, extended there with signature_check_error, pipe_concept_instantiation_errors, and the pipe_validation_error_data property. - error_handlers.py: kept both the SignaturesNotAllowedError import and dev's relocated ValidateBundleError import path. - CHANGELOG.md: combined both [Unreleased] sections (signatures + error infra). make agent-check (pyright/mypy) clean; make agent-test green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Greptile SummaryThis PR adds contract-only signature pipes and validation controls for them. The main changes are:
Confidence Score: 3/5This should be fixed before merging.
Important Files Changed
Prompt To Fix All With AIFix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
pipelex/pipe_run/dry_run.py:149-154
**Unreached signatures fail bundles**
`dry_run_pipes` raises whenever any pipe in the batch is a `PipeSignature`, even when no non-signature pipe depends on it. `validate_bundle` passes every loaded pipe from the bundle into this function, so a bundle with a complete `main_pipe` plus an unrelated draft signature fails strict validation. That contradicts the advertised contract that strict mode rejects pipelines whose dependency graph reaches a signature, while unreached signatures are allowed.
### Issue 2 of 2
pipelex/cli/agent_cli/commands/validate/_validate_core.py:164-168
**Selected pipe is blocked**
`validate_pipe_in_bundle_core` first calls `validate_bundle` in the same strict mode before it dry-runs the requested pipe. For `pipelex-agent validate bundle draft.mthds --pipe implemented_pipe`, an unrelated `PipeSignature` elsewhere in the bundle makes the preliminary bundle validation fail before the selected pipe is checked. This makes `--pipe` behave like whole-bundle validation for signatures instead of validating only the requested pipe.
Reviews (1): Last reviewed commit: "Merge branch 'dev' into feature/Validate..." | Re-trigger Greptile |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2d754e167d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
5 issues found across 64 files
You’re at about 90% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
- validate bundle --pipe now dry-runs only the requested pipe, so an unrelated PipeSignature (or otherwise-broken pipe) elsewhere in the bundle no longer blocks validating an implemented slice. Adds a dry_run_pipe_codes param to validate_bundle. [greptile/codex] - validate --all renders SignaturesNotAllowedError as a friendly CLI error instead of bubbling an unhandled traceback. [cubic] - handle_signatures_not_allowed_error honors --traceback like every other handle_* function. [cubic] - escape dynamic values in PipeSignatureSpec.rendered_pretty so concept multiplicity (Doc[], Img[3]) and bracketed descriptions render literally instead of being parsed as Rich markup. [cubic] - clarify strict-validation docstrings: whole-bundle strict rejects any bundle containing a signature; --pipe validates one implemented slice, --allow-signatures enables lenient mocking. [greptile] Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reconcile the signature-validation tracking so a reviewer can verify execution state against current reality: - Rewrite wip/signature-based-validation.md from a "no code yet" design proposal into a current-state tracker with a file->test verification map, the strict-by-default behavior for both CLIs, the post-merge review fixes, and Phase 7.4 as the single open item. - Repoint the .pipelex/plxt.toml "tracked in TODOS.md" reference to the live tracker: the docs-tidy merge had overwritten root TODOS.md with an unrelated recap and moved the signature plan into wip/archive/. - Mark the archived TDD plan as superseded and correct its stale "agent CLI defaults to lenient" claim (the shipped default is strict). - List the signature work in wip/README.md (it was absent from the index). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Summary
Adds
PipeSignature— contract-only pipes for top-down pipeline design — and strict signature validation, and merges the latestdevinto the branch.PipeSignaturepipe type (type = "PipeSignature"): declares a pipe'sinputs,output, anddescriptionwithout an implementation, so an author or agent can sketch a complete pipeline before committing to the operator that will do the work. Slots intoPipeBlueprintUnion/PipeSpecUnion, produced byPipeSignatureFactory, with its own runtime inpipelex/pipe_signature/. Dry-run mints a mockStuffmatching the declared output; live-run raisesPipeSignatureNotExecutableError.--allow-signaturesflag onpipelex validate pipe/bundleand everypipelex-agent validatesubcommand (default off). Without it, strict validation refuses any pipeline whose dependency graph reaches a signature, raisingSignaturesNotAllowedErrorand reporting every reachable signature plus the controller chain (signature_refs,dep_paths).collect_signature_refs()/collect_signature_paths()inpipelex/pipe_signature/signature_walk.py) powers the strict-mode pre-check indry_run_pipe/dry_run_pipesandvalidate_bundle.dry_run_pipe,dry_run_pipes,validate_bundle,validate_bundles_from_directorynow acceptallow_signatures: bool = False;ValidateBundleErrorgainedsignature_check_error: SignaturesNotAllowedError | Noneso the CLI renders the dep chain.Merge of
devThis branch had diverged structurally from
dev's validation/error-handling refactor (#943/#948/#949). The merge was resolved semantically:_translate_to_validate_bundle_errorcontext manager + library-lifecycle teardown, and re-applied the signature feature on top (re-enabled dry-run, threadedallow_signatures, singleSignaturesNotAllowedErrorarm shared by all entry points).ValidateBundleErrorlives inpipelex/pipeline/exceptions.py, extended there with the signature fields.[Unreleased]CHANGELOG sections.Verification
make agent-check— pyright 0 errors / 0 warnings, mypy clean.make tb(boot/config) green.make agent-test(full offline suite) passing.🤖 Generated with Claude Code
Summary by cubic
Adds
PipeSignaturefor contract-only pipes and makes validation strict by default acrosspipelexandpipelex-agent; pass--allow-signaturesto dry-run with mocked outputs. Bumpspipelexto 0.31.0.New Features
PipeSignature: declares inputs/output; dry-run returns a mock (multiplicity supported); live-run raisesPipeSignatureNotExecutableError.--allow-signaturesonpipelex validateand allpipelex-agent validatecommands; summaries note signature count in lenient mode.signature_walk); validation APIs acceptallow_signatures;ValidateBundleErrorcarriessignature_check_error; CLI renders friendly messages.Migration
PipeSignaturefail unless you pass--allow-signatures. Live execution of a signature always fails.validate bundle --pipenow dry-runs only the requested slice, so unrelated signatures elsewhere don’t block it..mthdsschema that includesPipeSignature; the Makefile generates the schema before runningplxt lintand merge checks.Written for commit 58ca53b. Summary will update on new commits.