refactor(forks/lstar): split spec.py into per-concern mixins#1
Closed
leolara wants to merge 118 commits into
Closed
refactor(forks/lstar): split spec.py into per-concern mixins#1leolara wants to merge 118 commits into
leolara wants to merge 118 commits into
Conversation
) The single 1422-line event_source.py grew to mix three concerns: the abstract event-source contract, the inbound gossip wire-format parser, and the live network orchestrator. This PR splits it into a focused package without rewriting any documentation or changing any behavior. New layout: - event_source/protocol.py: EventSource Protocol, GossipMessageError, SUPPORTED_PROTOCOLS allow-list (the dependency-light shared surface). - event_source/gossip.py: GossipHandler and read_gossip_message (the inbound gossipsub wire-format parser). - event_source/live.py: LiveNetworkEventSource (the orchestrator that owns connections, streams, and the gossipsub behavior). - event_source/__init__.py: re-exports the full public surface so every existing import path keeps working unchanged. Dependency direction: protocol -> gossip -> live, strict and one-way. Inside live.py, two structural cleanups: - _accept_streams (243 lines, three nested concerns) decomposes into _negotiate_inbound_stream, _handle_gossipsub_inbound_stream, _setup_outbound_gossipsub_after_delay (was a nested closure), and _handle_reqresp_inbound_stream. The orchestrator becomes ~30 lines: accept loop plus protocol-id dispatch. - _handle_inbound_quic_connection collapses into _handle_inbound_connection (the two were byte-identical except for one log line). _listen_quic now passes the unified handler. Every existing docstring and inline comment is preserved verbatim from main; only the structural pieces above changed. No external import path moves; no test file is touched. Verified: - uvx tox -e all-checks (ruff, format, ty, codespell, mdformat) passes. - tests/lean_spec/subspecs/networking/client and gossipsub tests (159 cases) pass. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eum#682) The time check used to admit any vote within one full slot of the local clock. That window let an adversary pre-publish next-slot aggregates ~800 ms before any honest validator could produce them, with the next proposer happily including them. The bound is now expressed in interval units against `Store.time` and gated by a new `GOSSIP_DISPARITY_INTERVALS` constant (one interval, the lean analogue of mainnet's `MAXIMUM_GOSSIP_CLOCK_DISPARITY`). Tests reference the constant rather than hardcoding numbers, so future changes propagate automatically. Several existing spec tests gossiped attestations carrying a future data.slot to compress timelines; they have been restructured to follow the natural Lean flow (gossip during the producer's own slot, then tick to migrate). Three new boundary regressions are added to both gossip validation files plus a unit-level `TestValidateAttestationTimeCheck` class. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tion (leanEthereum#690) Add 3 new fork choice test vectors covering attestation target selection, block production, and equivocation handling. All tests use the ForkChoiceTestFiller pattern, generating JSON fixtures for client implementations.
- Equivocation test now asserts per-validator vote tracking via AttestationCheck so the first-vote-wins rule is exercised at the validator level, not just inferred from the head. - Drop the redundant n==9 or n==11 head-slot skip in the block production test so every extension slot is checked. - Trim repeated post-finality assertions on slots 10/11 in the target selection test; slot 9 keeps the full check. - Document timing arithmetic and gossip ordering inline. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eanEthereum#638) * feat: add multi-fork architecture with ForkProtocol and SpecRunner Introduce a protocol-and-fork-class architecture that enables multiple devnet forks to coexist in the codebase using Python class inheritance. Key changes: - Add ForkProtocol ABC defining the consensus interface each fork implements - Add SpecRunner for era-aware fork dispatch - Create Devnet4Spec wrapping current State/Store as the base fork - Create Devnet5Spec skeleton inheriting from Devnet4Spec - Move State and Store from subspecs/ into forks/devnet4/ (fork-specific) - Wire ForkProtocol into Node, __main__, genesis, and storage - Rename testing fork from Devnet to Devnet4, add Devnet5 - Update all consumer imports to use lean_spec.forks re-exports - Update all test markers from valid_until("Devnet") to valid_until("Devnet4") The architecture follows the coworker's proposal: subspecs/ contains only fork-agnostic shared libraries (ssz, xmss, networking, chain). Fork-specific processing logic (State, Store) lives in forks/devnetN/. Consumer modules import State/Store from lean_spec.forks (the package re-export). Construction goes through ForkProtocol (fork.generate_genesis(), fork.create_store()). Adding a new fork requires: 1. forks/devnet5/state.py — Devnet5State(State) overriding changed methods 2. forks/devnet5/spec.py — Devnet5Spec(Devnet4Spec) with generate_genesis/upgrade_state 3. Python virtual dispatch handles the rest — no consumer code changes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: update --fork=Devnet to --fork=Devnet4 in workflows Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(forks): move consensus containers into forks/devnet4/ and genericize ForkProtocol Relocate every consensus-dependent container (Attestation, Block, Checkpoint, Config, Slot, State, Validator) from subspecs/containers/ into forks/devnet4/containers/ so per-fork divergence of container shapes (e.g. a future Devnet5 Attestation) becomes physically possible without mutating devnet4 symbols. Pydantic inheritance across forks is documented as unsafe for SSZ hash_tree_root stability — the supported pattern is copy-then-diverge inside the new fork's package. Strip ForkProtocol to its irreducible surface: five ClassVars (NAME, VERSION, state_class, block_class, store_class) and three methods (generate_genesis, create_store, upgrade_state). The protocol module imports nothing from any devnet package and exposes SpecStateType / SpecStoreType structural Protocols for type-hinting the classmethod contract. Runner rejects duplicate NAMEs and non-strictly-increasing VERSIONs; adds DEFAULT_RUNNER convenience. Reinstate Devnet5Spec as a genuine registered placeholder: own NAME, own VERSION, inherits method logic from Devnet4Spec, binds container types via explicit aliases that are one-line swappable when divergence lands. The multi-fork pipeline (runner, fixture filler, test framework) now exercises two forks end to end. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(forks): use AST walk instead of text search for fork-agnostic check The previous test asserted the literal string 'devnet4' did not appear in forks/protocol.py, which tripped on the docstring example for the NAME ClassVar. Replace with an AST-based check that inspects Import and ImportFrom nodes only — docstrings and comments are free to reference fork names as examples. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(forks): trim consensus-testing fork module docstring Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(forks): stage 1 trivial cleanups for ForkProtocol surface Bundles four small, behavior-preserving improvements to the multi-fork skeleton from PR leanEthereum#638. Tests + all-checks green. * Expose `GOSSIP_DIGEST: ClassVar[str]` on `ForkProtocol`. The gossipsub fork digest is fork metadata, not a top-level constant. Devnet4Spec binds it to "devnet0" (preserving the existing network agreement); Devnet5Spec inherits. Removes the hardcoded `GOSSIP_FORK_DIGEST` constant and the now-unused `Final` import from `__main__.py`, and routes every consumer through `fork.GOSSIP_DIGEST`. * Use `DEFAULT_RUNNER` directly in `__main__.py` instead of building a second `SpecRunner(FORK_SEQUENCE)`. The forks package already exports the singleton; the redundant construction is dropped along with the unused `FORK_SEQUENCE` and `SpecRunner` imports. * Add `previous: ClassVar[type[ForkProtocol] | None]` linking each fork to its predecessor. `Devnet4Spec.previous = None`; `Devnet5Spec.previous = Devnet4Spec`. Sets up the foundation for a future registry that derives ordering from topology and for chained state migrations via `upgrade_state`. * Tidy `devnet5/spec.py`: drop the module-level `_Devnet5State` and `_Devnet5Store` identity aliases; bind the container classes directly on `Devnet5Spec`. When devnet5 grows its own `forks/devnet5/containers/` package, the swap is a file-level diff in the imports rather than an alias rebinding. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(forks): rename SpecRunner to ForkRegistry The class is a registry — it stores a sequence of forks and looks them up by name — not a dispatcher routing spec calls per fork. The old name implied the latter and set wrong expectations for follow-ups. * Renames the file `runner.py` -> `registry.py`. * Renames the class `SpecRunner` -> `ForkRegistry`. * Renames the singleton `DEFAULT_RUNNER` -> `DEFAULT_REGISTRY`. * Updates the docstring on `FORK_SEQUENCE` and the error message in the empty-list guard. * Updates the test class name and assertions. No behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(forks): make upgrade_state abstract on ForkProtocol Replaces the silent identity default with an abstract method, forcing every concrete fork to declare its migration explicitly. This prevents the silent-no-op footgun: a future fork that adds a State field but forgets to override upgrade_state would otherwise migrate by simply not migrating. * `ForkProtocol.upgrade_state` becomes `@abstractmethod` with a docstring-only body. * `Devnet4Spec.upgrade_state` is identity, documented as "root fork, no predecessor, no migration". * `Devnet5Spec.upgrade_state` is identity, documented as "currently mirrors devnet4; replace when devnet5's State diverges". * Adds a test that asserts `ForkProtocol()` raises TypeError, locking in the abstract-class contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(forks): drop devnet5 placeholder and rename devnet4 to lstar Delete forks/devnet5/ entirely. We are running one devnet roughly per month, so per-devnet placeholders rot fast and lock the spec into a sequential upgrade story we are not committing to. Rename forks/devnet4/ to forks/lstar/ (via git mv, history preserved): - Devnet4Spec -> LstarSpec - Devnet4 (test BaseFork class) -> Lstar - "devnet4" / "Devnet4" string identifiers -> "lstar" / "Lstar" - lean_spec.forks.devnet4.* import paths -> lean_spec.forks.lstar.* - --fork=Devnet4 / --fork=devnet4 in CI workflows and docs -> Lstar / lstar Test surface follows: TestDevnet4Spec -> TestLstarSpec, TestDevnet5Spec removed entirely. Multi-fork ForkRegistry tests now use a synthetic in-test successor class instead of a real second fork. Untouched on purpose: - pyproject.toml lean-multisig-py branch="devnet4" (external repo, not ours). - tests/consensus/devnet/ folder name (generic test path, no version number). - GOSSIP_DIGEST="devnet0" — the cross-client gossip network name set by PR leanEthereum#622. Renaming it touches the whole networking layer and is a separate cleanup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(networking): rename fork_digest to network_name The value sitting on the gossipsub topic layer is the cross-client network name (currently "devnet0", set by PR leanEthereum#622), not a 4-byte fork digest hash. The "fork_digest" naming was a holdover from Ethereum mainline where it really is a digest; here it just confuses readers into expecting a hash where there is a string identifier. Renamed in the network-name layer: - ForkProtocol.GOSSIP_DIGEST -> ForkProtocol.NETWORK_NAME - GossipTopic.fork_digest -> GossipTopic.network_name - GossipTopic.{block,committee_aggregation,attestation_subnet} fork_digest parameter -> network_name - GossipTopic.validate_fork(expected_fork_digest=...) -> validate_fork(expected_network_name=...) - GossipTopic.from_string_validated(..., expected_fork_digest=...) -> expected_network_name - GossipHandler.fork_digest -> GossipHandler.network_name - LiveNetworkEventSource._fork_digest -> _network_name - LiveNetworkEventSource.set_fork_digest() -> set_network_name() - NetworkService.fork_digest -> NetworkService.network_name - NodeConfig.fork_digest -> NodeConfig.network_name Plus matching test renames: - test_gossip_digest -> test_network_name - test_gossip_topic_fork_digest_{matches,mismatch,...} -> test_gossip_topic_network_name_* - All test variables, parameters, and prose updated. Untouched on purpose: - Eth2Data.fork_digest: ForkDigest in subspecs/networking/enr/eth2.py is a real 4-byte ENR hash per the Ethereum p2p spec. The ForkDigest type stays. enr/enr.py, peer.py, discovery/routing.py all interact with this real digest and are unchanged. - JSON fixture keys "forkDigest" / "expectedForkDigest" in cross-client test vectors. Other clients consume those keys; renaming would break the wire format. Internal Python uses network_name; the JSON keys are read into network_name on the way in. - ForkMismatchError class name. A fork mismatch is still a fork mismatch semantically. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Leo Lara <leo@leolara.me>
…m#686) (leanEthereum#694) * refactor(forks): tighten ForkProtocol surface (Stage 1) Drop Any from ForkProtocol method signatures. Replace with fork-stable primitives (Uint64) and structural protocols (SpecStateType, SpecBlockType, SpecStoreType). Drop ClassVar from state_class/block_class/store_class so subclasses can narrow them under pyright/ty invariance rules. Rename NETWORK_NAME to GOSSIP_DIGEST: the field already serves the gossipsub fork-digest role. Remove the None-fork fallback in generate_pre_state. Every call now dispatches through fork.generate_genesis. The pre pytest fixture threads fork from the framework's fork fixture. Refs leanEthereum#686 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(forks): type validators as SSZList[Any] object was too loose for the structural validators parameter on generate_genesis. SSZList is the fork-stable concrete base; the element type stays generic because each fork owns its own Validator class. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: trim stale legacy reference from _DEFAULT_FORK docstring Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: add initial classes and constants * feat: implement BlocksByRange networking protocol and sync optimization This commit adds the server-side handler for the BlocksByRange protocol, integrates it into the SyncService, and enhances the backfill synchronization with range-based fetching. It also includes a strict SSZ validation fix for fixed-size containers to ensure malformed oversized payloads are correctly rejected. * fix: resolve linting, build errors, and missing imports in tests This commit addresses multiple linting violations (E501 line length), fixes missing imports in head_sync.py and test helpers, and restores/enhances test coverage for malformed SSZ payloads in the request/response layer. * chore: remove unnecessary scope check * chore: remove dependency from Store in backfill_sync * fix: fix ci * review: address consensus + py-architect feedback on BlocksByRange Apply review fixes spanning protocol semantics, client correctness, and test discipline. The headline change is a sliding history window: replace the absolute MIN_BLOCK_REQUESTS_HISTORY_SLOT floor with a window relative to the responder's current slot, matching upstream consensus-spec behavior and unblocking sync from genesis. Protocol semantics: - Sliding window: handle_blocks_by_range now rejects start_slot < max(0, current_slot - MIN_SLOTS_FOR_BLOCK_REQUESTS). Add CurrentSlotLookup callback on RequestHandler; missing lookup returns SERVER_ERROR rather than silently misreporting history. - Crash guard: head_sync rejects gossip blocks at or below the finalized slot, fixing the SSZValueError underflow path on adversarial input. - Gap-detection floor: head_slot + 1 (not finalized_slot) so slots already in the Store are never redownloaded. Client correctness (reqresp_client.request_blocks_by_range): - Propagate CodecError so peer downscoring fires on protocol violations (was being swallowed by the broad except Exception). - Parent-root continuity check applies across empty slots, not just consecutive slots, matching the canonical-chain MUST. - Range bounds use int() arithmetic to avoid Slot overflow. - Local validation: count <= MAX_REQUEST_BLOCKS and start_slot + count <= UINT64_MAX before opening a stream. - Raise CodecError when peer sends more than count chunks. - Log violations with conn.peer_id, not the raw connection repr. - Bug fix: count == 0 compares against Uint64(0) (was raising TypeError). Architecture: - New StoreView Protocol (has_root / finalized_slot / head_slot) replaces two loose Callable | None callbacks on BackfillSync. _SyncStoreView adapter in service.py reads through a getter so live store mutations are observed. - _max_range_slot watermark advances only after a completed (success or empty) range fetch; failed fetches stay retryable. reset() clears it. - LiveNetworkEventSource exposes set_block_by_slot_lookup and set_current_slot_lookup, symmetric with set_block_lookup. __main__ wires the current-slot side. Block-by-slot wiring requires SignedBlock storage and is a follow-up. Tests and style: - Drop MagicMock store fixture; use a concrete FakeStoreView dataclass. - Drop the get_finalized_slot=None mutation hack; that test now builds its own no-store-view backfill. - Split MockNetworkRequester.request_log into typed root/range logs. - Full-equality assertions across new tests; new coverage for failed- range non-advance, reset-clears-watermark, sliding-window genesis edge case, missing current_slot_lookup SERVER_ERROR. - Drop step-numbered comments, nested test import, dead add_block(root=) parameter, and dead duplicate MAX_CONCURRENT_REQUESTS constant. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: cover BlocksByRange client and head-sync routing paths Add the unit tests that were missing after the previous review-fix commit: - tests/lean_spec/subspecs/networking/client/test_reqresp_client_range.py: 14 tests for the outbound BlocksByRange flow. Zero-count short-circuit, count-above-max rejection, start_slot+count overflow, no-connection, full-range happy path, partial response on early close, RESOURCE_UNAVAILABLE chunk skipping, slot monotonicity violation, out-of-range slot violation, parent-root continuity violation across a skipped slot (the upstream- aligned tightening), parent-root continuity holds across skipped slots, more-than-count chunk enforcement, timeout returns empty, SERVER_ERROR halts reading and returns the partial list. - tests/lean_spec/subspecs/sync/test_head_sync_backfill_routing.py: 5 tests for the post-review _cache_and_backfill routing. Silent rejection at the finalized slot, silent rejection below the finalized slot, single-slot gap above head uses root recursion, multi-slot gap above head uses a single range fetch, alt-fork gossip at-or-below head uses root recursion. Drop the local UINT64_MAX constant in the reqresp client; use the canonical Uint64.max_value() helper from the typed API instead. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…leanEthereum#686) (leanEthereum#695) Promote the data primitives whose shape is stable across the leanSpec fork chain out of the lstar fork package and into lean_spec.types: - Slot (with its 3SF-mini justifiability methods) - ValidatorIndex, SubnetId - Checkpoint Validator and Validators stay in the fork because their XMSS-bound key shape is signature-scheme specific. ValidatorIndices stays for now because to_aggregation_bits couples it to the fork's AggregationBits; that decomposition can land as a follow-up. Config also stays in the fork: the single-field shape is the most likely container to grow per fork (deposits, RANDAO, fee parameters), so promoting it to subspecs/chain/ would just churn it back out later. Tighten the ForkProtocol surface using the freshly promoted types: SpecStoreType.from_anchor and ForkProtocol.create_store now take ValidatorIndex | None instead of Uint64 | None. Also drop the fork-side re-exports per CLAUDE.md (no compat shims) and update all call sites across src/, tests/, and packages/. Refs leanEthereum#686 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…thereum#686) (leanEthereum#698) Subspecs no longer hard-import any concrete fork module. They reach fork-specific types through three channels: fork-stable types in `lean_spec.types`, structural protocols in `lean_spec.forks.protocol`, and re-exports of the active fork from `lean_spec.forks`. Concrete class pointers are injected through `ForkProtocol` (`fork.state_class`, `fork.block_class`, ...) for subspecs that construct or decode SSZ instances. `AggregationBits`, `ValidatorIndices`, and `VALIDATOR_REGISTRY_LIMIT` move into `lean_spec.types`; `subspecs.chain.config` re-exports the limit so existing call sites keep working. `SQLiteDatabase` now takes `state_class`, `block_class`, and `attestation_data_class` as constructor arguments instead of hard-importing `forks.lstar` containers. A new AST-walk test (`test_subspecs_do_not_import_concrete_fork`) guards the property going forward by asserting no file under `src/lean_spec/subspecs/` imports from `lean_spec.forks.lstar.*`. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…#686) (leanEthereum#702) * refactor(forks): add Spec delegator surface (Stage 4A of leanEthereum#686) Adds the LstarSpec method surface that mirrors the existing State / Store / SignedBlock methods one-for-one: - State transition: state_transition, process_slots, process_block, process_block_header, process_attestations, build_block. - Forkchoice: on_block, on_tick, on_gossip_attestation, on_gossip_aggregated_attestation, produce_attestation_data, produce_block_with_signatures, get_proposal_head. - Block signatures: verify_signatures. Each method is a pure delegator to the corresponding container method. No call sites change — the new surface is unused initially. Stage 4B will rewrite call sites to go through the spec; Stage 4C will move the bodies in and replace literal Block/State/Store references with self.*_class(...). Tests use unittest.mock.patch.object to verify each delegator forwards its arguments unchanged and returns the container method's result verbatim. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: tighten delegator surface docstrings Rewrite the per-method docstrings on the fork-class delegators and the delegator test suite to follow the project documentation rules: - Each docstring describes the operation in plain English. - No explicit class or method names that rot when renamed. - No issue/stage references that belong in the PR description. - No banner-style separator comments inside the class body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(forks): add inner-type class pointers and structural protocols Extend ForkProtocol with class pointers for the inner container types (BlockBody, BlockHeader, AggregatedAttestations, AttestationSignatures) and the matching structural protocols. Hook them up on LstarSpec. This is the scaffolding the next stages of leanEthereum#686 need so that moving container method bodies into the spec can replace literal Block(...), BlockBody(...), AggregatedAttestations(...) constructors with self.<name>_class(...) — keeping inheriting forks free to swap any single inner type without re-implementing the parent's logic. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…leanEthereum#686) (leanEthereum#703) Tests that drive state transition, fork choice, block production, and signature verification now go through the fork's spec class instead of calling methods directly on the State / Store / SignedBlock containers. The pattern: state.process_block(block) → spec.process_block(state, block) store.on_block(signed_block) → spec.on_block(store, signed_block) signed_block.verify_signatures(vs) → spec.verify_signatures(signed_block, vs) store.produce_attestation_data(slot) → spec.produce_attestation_data(store, slot) ... Test functions consume `spec` via a new session-scoped pytest fixture defined in `tests/lean_spec/conftest.py`. Helper / library code uses a module-level `_SPEC = LstarSpec()` constant — `LstarSpec` is stateless, so a single shared instance is fine. Internal Store helpers (`aggregate`, `update_head`, `accept_new_attestations`, `update_safe_target`, `tick_interval`, `validate_attestation`, `prune_stale_attestation_data`, `extract_attestations_from_aggregated_payloads`, `compute_block_weights`) are not yet on the spec surface and are left as direct method calls — they migrate alongside the body move in the next stage. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… part 1 of leanEthereum#686) (leanEthereum#704) * refactor(forks): move verify_signatures body into the spec class Moves the full XMSS signature verification logic from SignedBlock.verify_signatures into LstarSpec.verify_signatures. SignedBlock becomes a pure SSZ data container. Internal callers that still hold a Store (notably Store.on_block, which itself moves to the spec class in a follow-up) reach the verification path via a deferred import of LstarSpec to sidestep the spec ↔ store module-load cycle. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(forks): move State and Store bodies into the spec class Migrates every State and Store method body into LstarSpec. Containers (State, Store, SignedBlock) become thin Pydantic data classes whose methods are one-line forwarders to the active fork spec, reached via a deferred import that breaks the spec ↔ container module-load cycle. Inside the moved bodies, every literal Block(...), BlockBody(...), BlockHeader(...), Config(...), AggregatedAttestations(...), and other container constructor is now self.<name>_class(...). An inheriting fork that swaps a single container type therefore receives the parent fork's logic for free. Observability hooks (observe_state_transition, observe_on_block, observe_on_attestation) ride along with the bodies, preserving the metrics surface. The obsolete delegator-forwarding test file is removed; behavioural coverage now lives in the existing state-transition, fork-choice, and block-production test suites. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(forks): delete the trivial container forwarders (Stage 4D of leanEthereum#686) State, Store, and SignedBlock become pure Pydantic data containers. All forwarder methods that delegated through the lazy spec singleton are removed; the lazy singleton helpers are removed alongside them. Every remaining call site that used to go through a container method now goes through the active fork spec: - Tests use the session-scoped `spec` fixture from `tests/lean_spec/conftest.py`. - Subspec services (`sync`, `validator`, `chain`, plus the fork-choice API endpoint) carry a module-level `_SPEC = LstarSpec()` constant. - Library helpers (`tests/lean_spec/helpers/builders.py`, `packages/testing/...`) follow the same `_SPEC` pattern. `ForkProtocol.generate_genesis` and `ForkProtocol.create_store` become abstract; the previous default implementations referenced container methods that no longer exist. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(forks): resolve Block forward reference in BlockLookup The Store.blocks field is annotated as `BlockLookup`, which was defined as `dict[Bytes32, "Block"]` — a string forward reference. After the container methods moved off Store, Pydantic's model rebuild could no longer resolve `Block` because it was never imported into store.py alongside the alias. Drop the forward-reference quoting: `BlockLookup` lives in the same module as `Block`, so the type can refer to the class directly. Pydantic then resolves `Store.blocks` correctly through the alias re-export. Without this, every consensus filler that constructed a Store via `spec.create_store(...)` raised `PydanticUserError: 'Store' is not fully defined`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(forks): route mock-store tests through autouse spec patches The forwarders that used to live on `Store` (and were patched in `tests/lean_spec/subspecs/{sync,chain,networking,validator}/`) are gone after Stage 4D. The mocks (`MockStore`, `MockForkchoiceStore`) still implement the same method surface, but the service code now calls the real spec, which expects a Pydantic Store. Add an autouse fixture per affected subspec that patches the active spec's methods to delegate back to `store.method(...)`. The mocks intercept calls in-place, preserving every test's recording semantics without touching service code. The validator service tests that previously patched `Store.produce_block_with_signatures` and `Store.on_gossip_attestation` now patch the matching methods on the validator service's `_SPEC` — same intent, current attribute path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(forks): drop cast(Store, ...) at call sites Casts were a workaround for Liskov violations on LstarSpec.create_store when it returned the SpecStoreType protocol while concretely producing Store. cast had no runtime effect and pushed type-checker noise into fixtures and tests. Move the imprecision into the fork itself: create_store now declares its concrete Store return and suppresses the override warning at the single definition site. Callers receive Store directly and need no cast. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(forks): replace 17-site _SPEC duplication with DI Each module declared its own _SPEC = LstarSpec(), creating value-equal but identity-distinct instances. The pattern also baked LstarSpec into 17 import sites; a future fork would have to grep-and-replace them all. Production services (chain, sync, validator, api) now take spec as a dataclass field with default_factory=LstarSpec — explicit at the composition root in node.py, optional in tests. node.py narrows config.fork (ForkProtocol) to LstarSpec once with isinstance, which also lets the cast(State, ...) and cast(Store, ...) at the genesis construction sites drop. Test conftests that intercept spec calls now monkey-patch LstarSpec at the class level (not the deleted module-level _SPEC instance). Test types and fixtures instantiate LstarSpec at call time — no module-level cache, no shared mutable state to alias. ForkProtocol still declares only the three abstract construction methods (generate_genesis, create_store, upgrade_state). Services and tests that drive consensus methods (process_slots, build_block, tick_interval, ...) keep the concrete LstarSpec type until the protocol surface is widened in a follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tests): replace conftest monkey-patches with injected spec Three autouse fixtures in chain/sync/networking conftests patched LstarSpec class methods so MockStore / MockForkchoiceStore could intercept consensus calls in place. Class-level patching mutates shared state and runs against every test in the directory whether needed or not. Now that services accept a spec field, tests inject a small StoreInterceptingSpec subclass that forwards each spec call back to the store argument. make_store() (used by sync/networking tests) hands the intercepting spec to the real SyncService transparently. Chain test_service.py threads it through ChainService directly. Two conftests delete entirely; the sync conftest keeps only its sample_checkpoint / sample_status fixtures. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(tests): patch validator spec on the instance, not the dropped module Two validator tests still resolved `lean_spec.subspecs.validator.service._SPEC`, which was removed when the spec moved onto the service as a field. Patching now targets `service.spec` directly via `patch.object`, which also exercises the single instance the test actually calls into. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eanEthereum#686) (leanEthereum#705) Per the multi-fork roadmap, Store now declares its State and Block types as type parameters so future forks can specialize them with a typedef instead of copy-pasting the whole class. Design (per the py-architect agent): - `Store(StrictBaseModel, Generic[StateT, BlockT])` in forks/lstar/store.py. StateT and BlockT are bound to Container so Pydantic can build a real schema at parameterization time; structural protocols would not. - `LstarStore = Store[State, Block]` in forks/lstar/spec.py is the concrete binding owned by the lstar fork. `LstarSpec.store_class` and `create_store` are typed against it. - `BlockLookup` is dropped — it was a single-line alias used in two helper signatures that read just as clearly as `dict[Bytes32, Block]`. - Public `from lean_spec.forks import Store` resolves to `LstarStore`, so every existing call site keeps working without change. `LstarStore` is also exported under its canonical name for callers that prefer to be explicit. - Mutable Store defaults move from bare `= {}` to `Field(default_factory= dict)` to silence the Pydantic generic-default warning ty raised intermittently and to keep the schema honest. The third Stage 5 item (a `Devnet5Store` typedef demonstrating the pattern) is dropped because devnet5 was unified back into lstar. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tage 6 of leanEthereum#686) (leanEthereum#706) Per the multi-fork roadmap, test files that target fork-specific containers or fork choice belong next to the fork they exercise. Until the roadmap is finished, only lstar exists, so the tree starts as forks/lstar/{state, forkchoice}/. Each conftest moves with its only consumers: - containers/conftest.py provided container_key_manager, used solely by test_state_aggregation.py — both move to forks/lstar/state/. - forkchoice/conftest.py provided pruning_store and sample_store, used by tests now under forks/lstar/forkchoice/. The empty subspecs/forkchoice/ directory is removed. Files in subspecs/containers/ that target fork-stable types (test_checkpoint.py) or attestation containers stay where they are; Stage 6 was scoped to state and forkchoice tests only. The Stage 6 item that mirrors the structure under devnet5 is dropped because devnet5 was unified back into lstar. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nEthereum#699) Adds a state-transition fixture that fills the registry to capacity (4096 placeholder validators), then justifies block 1 with an attestation from the 2731/4096 supermajority threshold. The placeholder validators carry zero pubkeys to skip XMSS key generation; signatures are appended via forced_attestations to bypass the builder's signing path. The state transition still tallies the votes and applies the same supermajority rule as the production path. Closes leanEthereum#580 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(chore): migrating from tox to just for better devex * chore(tools): remove tox, use just; add list descriptions; cleanup --------- Co-authored-by: shriraj pawar <shripawar0411@gmail.com>
…thereum#710) * chore: rm old references to ``devnet`` fork; rm exec fork refs * fix(test): remaining fork updates ``Devnet`` -> ``Lstar`` for pytest marks * chore: rename devnet -> lstar
…hor block (leanEthereum#713) * feat(api,sync): add /lean/v0/blocks/finalized for checkpoint-sync anchor Store.create_store(state, anchor_block, ...) requires both the finalized state and the anchor block, asserts state_root pairing, and seeds store.blocks[anchor_root] = anchor_block. The only checkpoint-sync API endpoint shipped state alone, so a syncing node had no spec-conformant way to obtain the matching block. Implementations worked around this by fabricating a SignedBlock from state.latest_block_header with an empty body and zero signature. Such synthetic blocks have hash_tree_root != anchor_root whenever the real anchor body contained attestations, breaking BlocksByRoot reqresp verification on receiving peers. Changes: - New endpoint /lean/v0/blocks/finalized returning SignedBlock SSZ - ApiServer.signed_block_getter callable wired into the request app - Checkpoint-sync client gains fetch_finalized_block() and fetch_finalized_anchor() returning the (state, signed_block) pair with pairing verification (state_root must match hash_tree_root(state)) Tests cover endpoint contract, error paths (404, network, corrupt SSZ, mismatched pair) and client-server integration. Closes leanEthereum#712. * test(api): serialize sequential_posts_converge to fix Py3.14 macOS flake asyncio.gather of 3 admin POSTs (T,F,T) raced server-side arrival order; the 'converge to True' assertion only held when the third request landed last. Switch to sequential await chain, matching the test's name and intent. Drop now-unused asyncio import.
* feat: add validator sync-lag duty gate * chore: inline logging * refactor(validator): harden sync-lag gate per consensus + py review Address review findings on the sync-lag duty gate. Decision logic - Replace peer-reported head-slot signal with local-store evidence: the freshest slot across blocks already validated into the store. Drops the unauthenticated peer.status.head.slot path entirely. - Add NETWORK_STALL_THRESHOLD (= 2 * SYNC_LAG_THRESHOLD) distinct from the local threshold so jitter at the local boundary cannot also trip the network-wide branch. - Add HYSTERESIS_BAND so a closed gate reopens only when lag drops to threshold - band, preventing slot-over-slot flap. - Persist gate state on the service; log only on state transitions instead of every query. - Saturate the future-head case at zero lag rather than trusting the chain unconditionally. API and types - duty parameter typed as Literal["block", "attestation"]. - Split _duties_skipped_lag into _blocks_skipped_lag and _attestations_skipped_lag, owned by the run loop. Attribution flattened so wrong-interval slots never tick the gate counter. - Drop redundant int(slot) casts where Slot arithmetic suffices. - Remove PeerManager.get_network_head_slot and its tests now that no caller remains. Tests - Replace patch.object(PeerManager, ...) mocks with real PeerManager and store manipulation, matching repo policy. - New helpers preserve the block-map key-equals-root invariant. - Add hysteresis flap test, split-counter test, transition-only log assertion. Use substring checks instead of exact log strings. Documentation - Constants, gate method, properties, fields, and inline comments rewritten per /doc rules: structured Why/Effect/Decision matrix labels, one idea per line, no function or variable names in prose, concrete numbers throughout. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(validator): use first-person voice in threshold rationale Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…leanEthereum#686) (leanEthereum#715) * feat(forks): add SigScheme capability and @requires marker (Stage 7 of leanEthereum#686) Introduces the first fork-level capability and a pytest marker to gate tests on capability presence. Capability ---------- - New `SigScheme` runtime-checkable Protocol in `forks/capabilities.py` asserts a `sig_scheme: ClassVar[GeneralizedXmssScheme]` attribute. - `LstarSpec` binds `sig_scheme = TARGET_SIGNATURE_SCHEME` so `isinstance(LstarSpec(), SigScheme)` returns True. - The three spec methods that previously took `scheme=TARGET_SIGNATURE_SCHEME` (`verify_signatures`, `on_gossip_attestation`, `on_block`) drop the parameter and read `self.sig_scheme` directly. The capability becomes the runtime source of truth. Marker ------ - `requires(*capabilities)` pytest marker, registered in `pytest_plugins/filler.py`. Composes (AND) with the existing `valid_from` / `valid_until` / `valid_at` fork-range markers. - `_check_markers_valid_for_fork` instantiates the active spec once and runs `isinstance(spec, capability)` per required capability. - A `requires(...)` helper in `framework.markers` works around pytest's auto-detect-class shortcut (which trips on Protocol args to `@pytest.mark.requires(...)`). Tests ----- - 11 unit tests in `tests/lean_spec/forks/test_capabilities.py` cover the Protocol and the dispatch helper (composition with the fork-range markers, multiple `@requires` markers, error path for non-runtime_checkable Protocol). - One smoke filler test in `tests/consensus/lstar/test_capability_gating.py` exercises the marker through pytest's live collection: one test marked with SigScheme runs, one marked with a synthetic absent capability is deselected. Filler scheme override ---------------------- The three filler call sites that previously passed `scheme=LEAN_ENV_TO_SCHEMES[self.lean_env]` to spec methods (in `test_fixtures/fork_choice.py` and `test_types/block_spec.py`) drop the kwarg. The PR description has the trade-off note and revert path if that override is in fact needed somewhere we missed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(forks): tighten capability marker and apply review feedback - Rename the marker helper to requires_capability and validate runtime-checkable Protocols at call time (fail at import, not at collection) - Cache the fork spec instance in marker dispatch instead of constructing it once per test - Re-export the capabilities namespace from lean_spec.forks so future capabilities don't need new import-site edits - Register valid_from / valid_at / requires markers in pyproject so unit tests can build real pytest Marks under strict-markers - Drop the hand-rolled Mark stand-in in tests; build real Marks via the MarkDecorator path; drop is True / is False on bool predicates - Tighten docstrings per project style (no paragraph blocks, no backtick references, no internal-name references) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(forks): tighten dispatcher-guard test docstring Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>
* fix: generating leanSpec test assets * Update prod-vectors.yml --------- Co-authored-by: Unnawut Leepaisalsuwanna <921194+unnawut@users.noreply.github.com>
…#716) * fix(forks/lstar): allow older-but-justified sources in build_block The fixed-point loop required `att.source` to equal `current_justified` exactly. That rejected legitimate gap-closing attestations whose source is at or before the head chain's latest justified slot (e.g. a genesis source when the head chain has already justified slot 1). Block production aborted with "Fixed-point attestation loop did not converge" when the store's justified checkpoint advanced via a sibling fork. Replace the Checkpoint-equality with a slot bound: skip atts whose source slot is past the current justified slot. This minimum change unblocks the gap-closing path; the right shape of the broader filter is left as an open question for review. * test(forks/lstar): assert build_block closes the justification gap Build a fork tree where the canonical head (block_5) lags behind the store's justified checkpoint (block_2, advanced by sibling block_6). Assert that produce_block_with_signatures succeeds, the produced block's post-state catches up to block_2, and the body includes block_6's gap-closing attestation. * docs: fix example diagram * fix(forks/lstar): tighten build_block filter with chain + target checks Layer additional filters onto the source-slot bound so build_block matches process_attestations more closely without re-introducing the strict source-equality rule: - Source and target roots must match the chain at their slots (historical_block_hashes for [0, parent.slot - 1], parent_root at parent.slot, ZERO_HASH for empty slots between parent and the candidate). Mirrors the STF's own source/target consistency check. - Target slot must not already be justified on the chain, with one exception: the degenerate source.slot == target.slot == 0 genesis self-vote stays selectable so existing fixtures keep working. The STF still drops these, but they're tolerated in the block body. Track the justified-slot bitfield and finalized slot through the fixed-point loop so the target check stays accurate after each iteration advances justification or finalization. With these additions, the previously failing test_build_block_skips_non_matching_source, test_block_builder_fixed_point_advances_justification, and test_block_builder_recovers_finality_after_non_zero_boundary_stall all pass without modification. * refactor(forks/lstar): drop redundant processed_att_data guard Selecting an attestation already adds its data to processed_att_data, and the cap check at the top of the loop short-circuits before doing anything with already-processed entries. The explicit duplicate-check before the add was redundant. * docs(forks/lstar): tighten build_block filter comments Trim the explanatory comments around the chain-match and already-justified checks now that the surrounding code is settled. * fix: rename test_build_block_skips_non_matching_source to better match actual test * refactor(forks/lstar): extract attestation chain-match helper Both process_attestations and build_block need to verify that an attestation's source and target checkpoints reference the chain at their respective slots. Lift that logic into _attestation_data_matches_chain so the two call sites share a single implementation. build_block constructs the chain view that process_block_header would produce on the candidate block (parent_state.historical_block_hashes plus parent_root, plus ZERO_HASH for empty slots) and passes it to the helper, instead of branching inline on source.slot and target.slot. * fix(forks/lstar): use HistoricalBlockHashes in chain-match helper signature ty doesn't recognize the SSZ HistoricalBlockHashes container as a Sequence[Bytes32]. Use the concrete type to satisfy the typechecker without introducing structural-typing concessions. * fix(forks/lstar): reject zero-hash source or target in chain match Empty slots between blocks carry ZERO_HASH in historical_block_hashes. An attestation whose source or target root is ZERO_HASH could otherwise satisfy the chain-match helper by colliding with one of those entries, even though it doesn't reference a real block. Inline the explicit zero-hash rejection into the helper so both process_attestations and build_block benefit. * fix(forks/lstar): require source slot is actually justified The previous filter only bounded source.slot by the latest justified slot. That admits attestations whose source lies on an unjustified slot before the latest justified one (gaps inside the bitfield). Match the STF: skip the attestation unless its source slot is set in the justified-slots bitfield.
…cates (leanEthereum#718) * fix(forks/lstar): harden build_block filter against crashes and duplicates Reorder filters so chain-match runs first. A source slot beyond the candidate slot no longer crashes the producer with IndexError on the justified-slot bitfield. Restore the processed_att_data dedup guard. Without it, attestations whose target fails to justify get re-selected on every fixed-point iteration and appended as duplicates to the candidate block body. Add state-transition parity filters with a genesis-anchor bypass. Target must be strictly after source. Target must fall on a slot the state transition accepts. Without these, gossip-pool entries the STF will silently drop can starve real votes out of MAX_ATTESTATIONS_DATA. Make the chain-match helper a staticmethod. Swap its parameter order to (data, chain). Widen its chain parameter to Sequence[Bytes32] so callers can pass plain lists without reconstructing SSZ list instances. Use full-Checkpoint equality in the new test per testing-style.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(forks/lstar): drop STF-parity filters that strip fork-choice signal Remove two filters added in the previous commit: - target.slot > source.slot - target.slot.is_justifiable_after(current_finalized_slot) Both are enforced by the state transition, but the state transition is not the only consumer of body attestations. Block processing also merges every body attestation into the known aggregated payload pool, which is what fork choice reads for head votes. Filtering these entries at production time hides head votes that the producer's chain would otherwise have seen. The MAX_ATTESTATIONS_DATA cap test and the deep-fork-split / reorg tests rely on this behavior: without the filters they pass, with them they shrink the body or move the head away from the expected subtree. The chain-match-first reorder, the processed_att_data dedup guard, the target-already-justified check with its genesis self-vote bypass, and the helper signature changes are unaffected and remain. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eum#719) * refactor(types): rename aggregation.py to participation.py The module holds both AggregationBits and ValidatorIndices, which represent validator participation in two equivalent forms (bitlist and index list). "participation" reflects the unifying concept better than "aggregation", which is narrower and only fits one of the two types. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(types): drop redundant bool() in to_validator_indices Boolean inherits from int, so truthiness is already correct without an explicit bool() conversion. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(types): add dedicated test_participation.py with full coverage Moves the existing AggregationBits/ValidatorIndices tests out of test_attestation_aggregation.py into a dedicated file mirroring the source location at src/lean_spec/types/participation.py. Adds previously-uncovered cases for ValidatorIndices.to_aggregation_bits: empty input, index at and above VALIDATOR_REGISTRY_LIMIT, deduplication of repeated indices, unsorted input ordering, and bitfield length. Coverage of participation.py is now 100%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ereum#720) Establishes a coherent documentation system for the project: - New /doc skill at .claude/skills/doc/SKILL.md with scope handling, standard section vocabulary, and a Python gold-standard exemplar. - Expanded .claude/rules/documentation.md from 3 thin rules to a full checklist including one-sentence-per-line, bullet decomposition, structured labels, and a consolidated anti-pattern list. - Refactored .claude/agents/doc-writer.md to defer atomic rules to the above files and keep only the consensus-domain patterns and persona. Then applies the new system to bitfields.py as a live demonstration: - Module + class docstrings use bullets and worked examples. - encode/decode bodies use phase labels and bullet-decomposed bit math with concrete worked examples (byte-boundary crossings, delimiter spill, integer-interpretation mapping). - Replaces the (N + 7) // 8 ceiling trick with math.ceil(N / 8). - Replaces the byte-by-byte delimiter scan with a single int.from_bytes + bit_length call. - Drops redundant tuple() wrap; passes list[Boolean] directly. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…est coverage (leanEthereum#721) * refactor(types): tighten Boolean strictness, simplify dunders, improve test coverage Boolean is now strict-by-default, matching the established BaseUint pattern across SSZ primitive types: - __eq__ and __ne__ now raise TypeError on any non-Boolean operand. The previous loose behavior (Boolean(1) == 1 returning True) silently bypassed the type contract; the asymmetry with bitwise ops (which already raised) was a real footgun. - __new__ uses int(value) before the (0, 1) membership test so the new strict __eq__ does not fire when wrapping an existing Boolean. - The four no-op arithmetic dunders (__add__/__radd__/__sub__/__rsub__) collapse into a single _no_arithmetic method with class-level aliasing. One source of truth, ~13 lines deleted. - deserialize drops a redundant length check; decode_bytes already validates and produces the right error. Tests: - Coverage of boolean.py is now 100% (every line, every branch). - Removed a duplicate test_strict_equality_with_same_type that was fully subsumed by the parametrized equality/inequality tests. - Added gap-filling tests for: Boolean(Boolean(x)) construction, reverse bitwise dunders (int & Boolean etc), reflected __ne__, the Pydantic is_instance branch, and Pydantic serialization back to a plain bool. - Every pytest.raises match= now asserts the full error message (via re.escape for dynamic strings, anchored r"^...$" for static). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(tests): suppress ty arg-type warning on Pydantic strict bool The Pydantic model declares value: Boolean, so ty rejects passing a raw True at the call site. Matches the existing suppression on the strict-validation test above. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(boolean): align with /doc rules and bitfields.py style Rewrites docstrings and inline comments to match the project's documentation conventions used in bitfields.py: - No backticks anywhere in docstrings or comments. - One-sentence-per-line; bullet decompositions over prose paragraphs. - Plain English in prose (lowercase "boolean") instead of class-name references in descriptive text. - Class docstring becomes a 4-bullet behavior summary plus a wire-format example showing how true/false encode to bytes. - __get_pydantic_core_schema__ body uses bullet decomposition of the two-step validator chain and the two-branch union schema. - encode_bytes uses a raw string with single backslashes so help() renders b"\x01" correctly (was b"\\x01" before). - decode_bytes and deserialize gain Args/Returns/Raises sections. No behavior change; code is byte-identical. 72 tests still pass at 100% coverage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(boolean): use bullet lists for multi-case Raises clauses Converts the "If X, or Y" prose in decode_bytes and deserialize Raises sections into two bullet points each, matching the bullet-decomposition style used elsewhere in the file. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eanEthereum#722) Brings the bitfields.py test suite to: - 100% line and branch coverage (was 90%). - Full-message matches on every pytest.raises via re.escape — both static strings and dynamic f-strings carrying class names, sizes, and element counts. Coverage gaps filled: - BaseBitvector validator: generator input materialized to a tuple. - BaseBitlist validator: generator input materialized to a list. - BaseBitlist validator: non-iterable input (int/None/float) and the str/bytes rejection branch. - BaseBitlist.__getitem__ both int and slice branches. - BaseBitlist.__add__ returning NotImplemented for unsupported types. - BaseBitlist.decode_bytes "no delimiter bit found" for non-empty all-zero input. - BaseBitlist.decode_bytes "exceeds limit of N" for an encoding whose recovered bit count is above LIMIT. Duplicates removed: - TestBitfieldSerialization.test_bitvector_serialization_deserialization and ..._bitlist_... were strict subsets of TestBitfieldSSZ's encode_decode tests. The TestBitfieldSerialization class is gone; its remaining decode-error tests moved into TestBitfieldSSZ. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…m#789) * refactor: drop model_copy in favour of in-place mutation StrictBaseModel was previously frozen, which forced every state update to go through model_copy(update={...}) and produced a verbose functional style throughout the codebase. The frozen constraint was introduced as groundwork for formal verification, which we are not yet using; in the meantime the indirection hurts readability. Changes: - src/lean_spec/base.py: drop frozen=True from StrictBaseModel and update the docstring (no longer claims immutability). - .claude/rules/ssz-patterns.md: drop "immutability" from the SSZModel principle bullet so the rule matches the new behavior. - All 115 model_copy(update=...) call sites across src/, tests/, and packages/testing/ converted to direct field assignment. Intermediate dict-building variables inlined where the result was used only once. Exceptions: - The three JustifiedSlots collection helpers (with_justified, extend_to_slot, shift_window) still return new instances, but via type(self)(data=...) instead of model_copy. This preserves the return-new contract that callers rely on for SSZ collection methods. - The api_endpoint test fixture's "return self.model_copy(update=handler( store, self))" pattern becomes a setattr loop over handler's output dict, then return self. - MockStore in tests/lean_spec/node/chain/test_service.py loses its hand-rolled model_copy method (no longer needed; it's a plain dataclass and is mutable by default). just check passes (ruff lint + format, ty type check, codespell, mdformat). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: restore hashability and pure-function semantics where required Dropping frozen globally lost two side effects the codebase relied on: hashability of value types used as dict keys / set elements, and the "returns a new state" contract that callers of process_slots and state_transition were depending on. Both are restored surgically. Hashability: - Re-freeze the value types used as dict keys / set elements: Checkpoint, AttestationData, Attestation, Signature, TypeOneMultiSignature, TypeTwoMultiSignature, KeyPair, ValidatorKeyPair, Eth2Data. Per-class model_config override leaves the stateful types (Store, State, Block, BlockHeader, ENR, etc.) unfrozen and mutable. - Add explicit __hash__ on Signature, TypeOneMultiSignature, and TypeTwoMultiSignature: their nested list-bearing fields break the auto-derived hash, so the SSZ-encoded bytes drive the hash instead. Pure-function semantics: - process_slots now deepcopies its input at entry, matching the existing docstring ("returns a new state with slot == target_slot"). This makes state_transition and build_block trivially pure. - Slot assertion relaxed from < to <= so process_slots is idempotent under repeated calls with the same target (build_block needs this). - Two Checkpoint mutations in process_block_header replaced with construction now that Checkpoint is frozen again. Test fixture in BlockSpec.build_signed_block_with_store deepcopies the store at entry: the simulation pipeline (on_tick, on_gossip_attestation, aggregate, accept_new_attestations) mutates store directly, so the caller's store needs an explicit barrier. Test fixes: - _replace_head_at_slot and _add_block_at_slot in test_service.py construct new Block instances instead of mutating the original (the AST conversion had broken them). - Corrupted-proof tests construct new TypeOneMultiSignature rather than mutating .proof on a frozen instance. - test_combined_path_rejects_{depth_mismatch,odd_depth} construct new HashSubTree to avoid polluting the prf_trees fixture across tests. - Dropped test_frozen_rejects_assignment on StrictBaseModel (the base itself is no longer frozen by design). - Dropped the "store is not store_before" identity check that was only meaningful when on_gossip_attestation returned a new store. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(spec): restore strict process_slots assertion Loosening the assertion to state.slot <= target_slot was a workaround for the mutation cascade through build_block. Now that process_slots deepcopies its input at entry, the caller's state.slot is never advanced across repeated calls, so the original strict inequality holds again. The spec filler test test_process_slots_target_equal_to_state_slot_rejected was relying on the strict form. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(forks/lstar): restore Checkpoint frozen override after merge The merge from main clobbered the per-class `frozen=True` model_config override on Checkpoint. That override was added in 24a0e7b alongside the same override on eight other value types used as dict keys / set members; Checkpoint was the only one that moved files in the merge (the deleted lean_spec/types/checkpoint.py from leanEthereum#790 collided with the new home spec/forks/lstar/containers.py from leanEthereum#785) and so it was the only one whose override was lost. Without the override AttestationData (which embeds Checkpoint) is no longer hashable, and on_gossip_attestation crashes when it inserts into store.attestation_signatures. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…eanEthereum#792) Five sites in spec.py rebuilt store dicts (and their inner sets) before mutating them — defensive copies left over from the model_copy era. Since fb5ff3c the store is mutated in place, so these rebuilds protect nothing and actively mislead readers into thinking the store is immutable. The aggregate() filter is replaced by an in-place pop loop with identical semantics. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… per-bit loop (leanEthereum#793) Both BaseBitvector.encode_bytes and BaseBitlist.encode_bytes used a Python for-loop to set each bit of a bytearray. For a 4096-validator AggregationBits that is 4096 Python iterations per encoding. Use the same trick already in merkleization.py: build the packed bits as a single Python int, then call int.to_bytes once. The bit-to-byte split runs in C. For BaseBitlist, the delimiter folds into the integer construction as bit num_bits, and the byte-spill special case disappears — the byte width ceil((num_bits + 1) / 8) already covers it. The on-the-wire format is unchanged; existing decode paths and SSZ round-trip tests cover the new encoders unchanged. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eanEthereum#794) The previous behavior silently popped mode and by_alias from kwargs so the serializer could pin them to "json" and True. A caller passing either almost certainly expected the override to apply, so silently honoring the pin produced output that did not match their request — a footgun. Reject the kwargs with a clear TypeError instead. Other kwargs still forward unchanged. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…chemas (leanEthereum#795) Four field validators across the codebase each rebuilt the same "if str, decode hex" logic on top of a Pydantic field. Promote the parsing to the schema layer so it happens once per type. - BaseBytes.__get_pydantic_core_schema__: add a str branch that routes any string through the constructor. - Container: add a model_validator(mode="wrap") that decodes a hex string input via from_hex, with other shapes passing through field-by-field validation. Removes: - validator/registry.py parse_pubkey — covered by the Bytes52 schema. - xmss/containers.py _decode_public_key, _decode_secret_key — covered by the Container wrap validator. Simplifies: - genesis/config.py keeps only the YAML quirk where an unquoted 0x-prefixed value is parsed as an int. Adds TestHexStringValidator in test_container.py covering every branch of the new wrap validator: hex success paths with prefix and case variants, empty-hex edge case, dict and instance pass-through, the class-name-tagged error on wrong length, and the nested-container field case. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…leanEthereum#796) * refactor(forks/lstar): move multi-signature types out of crypto layer The crypto subspec imported AggregationBits, Slot, ValidatorIndex, and ValidatorIndices from the consensus layer to define TypeOneMultiSignature and TypeTwoMultiSignature. That layering inversion forced three mid-file deferred imports with # noqa: E402 in lstar/containers.py and made the crypto layer reach into consensus for types it should never know about. Move the multi-signature classes (plus AggregationError) into the consensus layer, where they belong as domain-typed wrappers around the crypto byte-level primitives. The crypto layer keeps only the Rust prover bindings. Slot moves into its own small module so the crypto layer can name a slot without pulling the full consensus container module. The crypto API still uses Slot, not Uint64 — the layering fix preserves semantics. Net result: - crypto/xmss/aggregation.py shrinks from 348 lines to 38; no consensus imports remain. - crypto/xmss/{containers,interface}.py import Slot from lstar/slot.py. - lstar/containers.py promotes all three previously deferred imports (HISTORICAL_ROOTS_LIMIT, multi-sig types, PublicKey/Signature) to the top of the file. The # noqa: E402 block is gone. - 17 importers updated to fetch TypeOneMultiSignature etc. from the consensus layer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(crypto/xmss): delete the aggregation shim After the multi-signature containers moved to the consensus layer, the aggregation module shrank to a re-export shell with one import-time side effect. With exactly one consumer (lstar/containers.py), the indirection added cost without abstraction. Distribute the three concerns: - Rust binding imports go directly into lstar/containers.py from lean_multisig_py. - LOG_INV_RATE moves next to its sole caller in lstar/containers.py. - setup_prover(mode=LEAN_ENV) moves into crypto/xmss/__init__.py, alongside the other LEAN_ENV-driven xmss bootstrap. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eanEthereum#797) The sync and validator services defaulted their publish callbacks to no-op async functions used purely as sentinels. Replace them with optional callables defaulting to None and guard each call site. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…es (leanEthereum#798) The networking and snappy modules each carried a self-contained LEB128 implementation. The two algorithms were identical, differing only in byte cap (10 vs 5). One implementation was a slow drift away from the other waiting to happen. Keep the networking varint as the single source of truth and give both encode and decode a max_bytes parameter (default 10 = uint64 cap). Snappy now imports the canonical codec and passes 5 via a new SNAPPY_VARINT_MAX_BYTES constant. Encode now also enforces the cap, so per-cap semantics are symmetric. The snappy length-prefix tests run against the canonical codec, plus an integration test that asserts an oversize prefix surfaces as a SnappyDecompressionError. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ure types (leanEthereum#799) * refactor(forks/lstar): type SignedBlock.proof as TypeTwoMultiSignature The field was typed as raw ByteList512KiB and producers manually called encode_bytes before storing the result; verify_signatures and the post-block split path then decode_bytes-ed it back. Promote the field to its real type so SSZ handles the (de)serialization once at the container boundary. - Removes the encode-then-wrap dance in the validator service and test builders. - Removes the try/decode_bytes/except blocks in verify_signatures and the sync service post-block handler. - Updates SSZ round-trip tests, fork choice tests, and reqresp client helpers to construct the typed envelope directly. - Deletes four tautological decode-smoke assertions in test_service.py now that the Pydantic-validated field guarantees the shape. The on-wire SSZ encoding of SignedBlock is unchanged; hash_tree_root will change because Container merkleization differs from List[byte] merkleization, so consensus fixtures that hardcode block roots need regeneration. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(forks/lstar): inline single-use type_two alias Both call sites used the local alias exactly once. Drop it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(forks/lstar): rename multi-signature types to SingleMessageAggregate / MultiMessageAggregate TypeOneMultiSignature and TypeTwoMultiSignature were named after the underlying Rust prover convention, not what they semantically are. Rename them to express the actual data: an aggregate over a single message versus an aggregate over many distinct messages. - Classes: TypeOneMultiSignature -> SingleMessageAggregate, TypeTwoMultiSignature -> MultiMessageAggregate - Variables and locals follow the snake_case form: type_1 -> single_message_aggregate, type_2 -> multi_message_aggregate (plus the compound forms type1_wire, type2_wire, block_t1, proposer_type_1, etc.) - Test function names match: test_type_one_* -> test_single_message_*, test_type_two_* -> test_multi_message_* - Prose mentions of "Type-1" / "Type-2" in docstrings and comments become "single-message aggregate" / "multi-message aggregate" External Rust binding names (aggregate_type_1, verify_type_1, merge_many_type_1, split_type_2_by_msg, verify_type_2_with_messages) come from the leanMultisig-py package and are left untouched. The "Type 1" / "Type 2" mentions in node/snappy/encoding.py refer to Snappy's Copy Type 1 / Copy Type 2 wire encodings and are unrelated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eum#800) * refactor: expand all abbreviated identifiers to full words A reference specification should read as explicitly as possible, so every abbreviated identifier is spelled out in full across src, tests, and the packages/ testing framework (~280 distinct renames). Highlights: - validator_id -> validator_index (the domain-correct term) - att -> attestation, msg -> message, sig -> signature, sk -> secret_key, pk/pubkey -> public_key, idx -> index, prev -> previous, agg -> aggregate, prop -> proposal, conn -> connection, privkey -> private_key, len -> length - function/method/class names too (split_by_msg -> split_by_message, Pubkey -> PublicKey, get_attestation_pubkey -> get_attestation_public_key) - MSG_LEN_FE -> MESSAGE_LENGTH_FIELD_ELEMENTS Kept verbatim: canonical protocol IDs (peer_id, node_id, protocol_id, subnet_id, stream_id), ENR fields (attnets, seq), num_* prefixes, the reqresp protocol name, library APIs (argparse dest, model_config, tmp_path), and external symbols (e.g. lean_multisig_py functions like split_type_2_by_msg). Internal config/fixture string keys were updated to match the renamed fields (genesis YAML keys, validator registry manifest keys, one metrics coverage-section label). Test assertions/fixtures referencing identifiers via strings (parametrize names) were aligned. Adds a NO ABBREVIATIONS IN IDENTIFIERS rule to CLAUDE.md documenting the convention and its canonical exceptions. just check passes (ruff, format, ty, codespell, mdformat). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(tests): correct XmssKeyManager __slots__ entry after rename The attribute self._keys_dir was renamed to self._keys_directory, but the matching __slots__ string entry must be renamed too -- otherwise setting the attribute raises AttributeError (slotted classes have no __dict__). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…um#801) Move AttestationSignatureEntry and Store next to the other lstar container types, then delete the standalone store module so the fork exposes a single container namespace. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…heck proof verification (leanEthereum#786) * test(consensus): add VerifyProofsTest fixture and Type-1 valid vectors Introduces a new consensus fixture format that emits self-contained multi-signature verification vectors so cross-client implementations can run their own Type-1 verifier and compare outcomes. Three positive vectors land alongside the fixture: a single-validator baseline, a four-validator all-participating case, and a four-validator non-contiguous bitfield case ([1, 0, 1, 1]). The fixture also surfaces the spec-layer binding between attestation data and proof: clients recompute hash_tree_root(attestation_data) and must match the emitted message field before running the verifier. * test(consensus): add Type-1 verify_proofs rejection vectors Adds four negative vectors exercising spec-layer bindings between inputs and the multi-signature proof. Each vector uses a tamper operation on the fixture to produce a structurally valid bundle that must be rejected by a conformant verifier: - wrong_message: proof bound to an alternate head root inside the attestation data - wrong_slot: emitted slot field shifted while the proof binding stays on the original slot - wrong_public_keys: one emitted pubkey replaced with another validator's - aggregation_bits_length_mismatch: emitted bits truncated while the pubkey count stays unchanged Vectors covering malformed or truncated proof bytes are intentionally out of scope: leanSpec consumes the multi-signature primitive as a black box and primitive integrity belongs to its own conformance suite. Pubkey ordering is also not a binding to test: the aggregator sorts participants internally, so the verifier is order-insensitive. * test(consensus): align VerifyProofsTest with sibling fixture conventions Brings the new fixture in line with the patterns the other consensus test fixtures follow: - Drop ``from __future__ import annotations`` (PR leanEthereum#759 removed it from Pydantic-defining files); quote the one self-reference instead. - Replace the bespoke ``expect_valid: bool`` field with the inherited ``expect_exception`` field already used by SSZTest, NetworkingCodec, and VerifySignaturesTest. Rejection vectors now pin ``AggregationError`` and the framework serializes the class name to JSON. - Switch the tamper dispatch in ``_apply_tamper`` from ``if/elif`` to ``match/case`` to follow the pattern in slot_clock and networking_codec. - Expand the module-level docstring from one line to a short paragraph describing what the fixture covers. - Normalize the ``public_keys`` default from ``[]`` to ``| None = None`` to match every other output field on the model. * test(consensus): drop aggregation_bits_length_mismatch rejection vector The check that fires here is the early-reject in the spec wrapper's verify method (len(public_keys) != participants.count(True)), not a consensus-critical binding. In real consensus the inconsistency cannot arise because clients resolve public keys from the bitfield plus the validator registry as one operation. A client that did pass a wrong pubkey count would also be rejected by the underlying recursive verifier on its internal pubkey-set commitment, so the wrapper check is at best an early exit with a nicer error message. The remaining three rejection vectors still exercise the meaningful spec-layer bindings: message hash, slot, and pubkey set. * refactor(consensus): tighten VerifyProofsTest and absorb post-rebase renames Rebase onto main and apply the renames/cleanups landed since the branch was opened: - PR leanEthereum#799: TypeOneMultiSignature → SingleMessageAggregate, proof_type literal "type_1" → "single_message", file renames test_type_1_{valid,invalid}.py → test_single_message_{valid,invalid}.py. - PR leanEthereum#800: validator_ids → validator_indices, with_validator_id → with_validator_index, vid/pubkey expansions. - post-leanEthereum#788/leanEthereum#790/leanEthereum#796 imports: lean_spec.spec.forks / .spec.ssz / .spec.crypto.*. Replace the stringly-typed tamper dict with a Pydantic discriminated union (RebindToAlternateHeadRoot, IncrementEmittedSlot, SwapParticipantPublicKey). The match dispatch on the typed variants drops the two type: ignore[index] casts and lets the test sites read tamper=SwapParticipantPublicKey(index=0, with_validator_index=1) instead of a magic-string dict. Inline the four single-call helpers (_apply_tamper plus three _tamper_*) and the verification helper into make_fixture so the four phases — generate / tamper / verify / publish — are visible in one method. Drop both model_copy(update=...) calls per leanEthereum#789: direct field construction for the frozen AttestationData rebuild, direct field assignment for the mutable fixture self-update. Replace the internal dict[str, Any] bundle with named locals; the bundle never escapes make_fixture, so the dict adds nothing. Guard SwapParticipantPublicKey against silent no-op swaps where the replacement key happens to equal the original. Broaden the verifier exception catch to surface unexpected exception types as "expected X got Y" instead of crashing the filler. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…eum#802) * fix stale finalized source finalization * fix: rebase onto main and correct finalization test Rebase the stale-source finalization fix onto current main: the spec file moved under the spec/ package and attestation specs renamed validator_ids to validator_indices. Also fix the new test expectation. Building justified_slots via model_copy(update=...) bypassed Pydantic validation, leaving data as a list instead of a tuple, so the post-state comparison failed even with the fix applied. Use the canonical constructor instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: pin source-at-finalized-boundary finalization case Add the boundary case raised in review: a source whose slot equals the finalized slot. Such a source is already final, so it may justify a newer target but must never re-finalize. This locks the > (not >=) intent of the finalization-advance guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…hereum#803) Expands the `exc` abbreviation used in `except X as exc:` clauses to the full word `exception`, matching the no-abbreviations rule in CLAUDE.md. Adds the rule explicitly so future code keeps the full spelling. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…eanEthereum#804) The store protocol declared from_anchor, on_block, on_gossip_attestation, and on_gossip_aggregated_attestation, but the concrete Store is a plain data model that implements none of them. Those operations live on the fork spec and are invoked as spec.on_block(store, ...), so the declarations were dead and contradicted the comment claiming the store conforms structurally. Remove the four declarations along with the three SignedBlock / SignedAttestation / SignedAggregatedAttestation structural protocols that only existed to type those signatures and were referenced nowhere else. SpecStoreType now exposes only the read surface the store actually provides, making the structural-conformance comment true. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eum#805) The diagnostic block-weight computation and the LMD-GHOST head walk both climbed from each validator's head vote up through its ancestors, accumulating one unit of weight per visited block above a start slot. The two loops were byte-identical apart from the start slot, so the API's reported weights could silently diverge from the actual head decision if one copy changed. Extract the walk into a single helper parameterized on the start slot and call it from both. No behavior change. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nEthereum#806) The state transition function took a valid_signatures flag whose only non-default value raised immediately, so any path that proceeded had it set to True. The sole caller passed the result of signature verification, which already returns True or raises. The flag therefore implied the state transition could optionally skip signature checks, when in fact it never inspects signatures at all. Remove the parameter and its dead branch. Signature verification stays where it belongs, before the call, and aborts the import on failure. Update the docstring and the test and fixture call sites accordingly. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ereum#808) Anchoring the genesis block as justified and finalized swapped in the parent root but kept the pre-state checkpoint slot. That was correct only because both checkpoint slots are still zero when the first post-genesis block is processed, a hidden dependency a reader had to reconstruct. Write the slot as zero directly at both the header-processing anchor and its mirror in the block builder, so the genesis anchor reads as a fixed slot-zero checkpoint. No behavior change. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…le_message_proofs (leanEthereum#811) The verify_proofs/ directory now houses both single-message (Type-1) and multi-message (Type-2) primitive verification vectors. Rename the Type-1 fixture format, class, and test signatures so they sit symmetrically next to the incoming Type-2 fixture.
* refactor(forks/lstar): small Pythonic cleanups Five no-behavior-change simplifications in the lstar spec: - Drop a redundant ternary guarding the justifications dict comprehension; enumerating an empty root list already yields an empty dict. - Replace a manual root-to-slot loop with the equivalent dict comprehension. - Convert the lookback count to int before range() instead of relying on the implicit integer coercion, matching the int() usage elsewhere. - Add a constructor that builds aggregation bits straight from a set of validator indices, route the existing index-list conversion through it, and use it in the prover path to avoid building a throwaway index list. - Trim the per-slot housekeeping comment to its load-bearing invariant: the state root is cached at most once per block, on the first empty slot. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(forks/lstar): drop the to_aggregation_bits wrapper With the index-set logic now living in the aggregation-bits constructor, the conversion method on the index list was a pure delegating wrapper. Most of its call sites also built a throwaway index list purely to call it. Remove the wrapper, widen the constructor to accept any iterable of indices, and call it directly everywhere, dropping the intermediate index lists. The conversion test suite now exercises the constructor directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…um#810) * test(testing): add VerifyMultiMessageProofsTest fixture Mirror the single-message fixture for the Type-2 multi-message aggregate primitive: per-component validator lists, per-component attestation data, emitted parallel lists of messages, slots, public keys, and aggregation bits, plus the merged proof bytes. Three tampers target one component at a time: - RebindComponentToAlternateHeadRoot regenerates one component against an alternate head root and re-merges, so the emitted layout stays honest but the merged proof bytes carry the off-target binding. - IncrementComponentSlot bumps one component's emitted slot past its bound slot. - SwapComponentParticipantPublicKey swaps one participant's key for another validator's, breaking the layout the merged proof verifies against. * test(consensus): add multi-message verify vectors for lstar Three positive vectors covering two- and three-component bundles with single-, four-, and mixed-sized participant lists; three rejection vectors covering wrong message, wrong slot, and wrong public key applied to one component at a time. * test(testing): cover multi-message-specific rejection paths and dedup the verify ladder Add the failure modes that only a multi-message (Type-2) proof can suffer, which the initial vector set did not reach: - SwapComponentMessageBindings transposes two components' emitted message-slot bindings after an honest merge, with distinct head roots per component, so each component's proof faces the other's binding. This is the canonical Type-2 attack the positional binding exists to reject. - DropComponentMessageBinding removes one component's binding while keeping its keys, exercising the verifier's binding-count guard. Add two valid vectors for parity with the single-message suite: a single component bundle (the n=1 boundary) and a non-contiguous committee whose aggregation bits resolve to [1, 0, 1, 1]. Factor the identical expectation-comparison ladder into BaseConsensusFixture.assert_expected_outcome and reuse it from both verify fixtures. Hoist the repeated component-index range check into a helper, and guard the slot-increment tamper against landing on a neighbour's slot. Inline the per-test attestation data to match the single-message sibling files, rename the ambiguous swap field to participant_index, and correct the union docstring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add attestation proof order fixture * test: remove banner-separator comments The code style bans banner-style separator comments, since they add visual clutter and blank lines already mark logical sections. Drop the 63 pure dash and equals divider lines across the QUIC, service, and fork-choice test files, keeping the section heading text as plain single-line comments. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Adam Mohammed A Latif <latifkasuli@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
leanEthereum#813) The multi-message proof fixture still built a throwaway index list and called the conversion method that was removed when the index-to-bits logic moved onto the aggregation-bits constructor. The two changes landed in a merge order that left a call to the deleted method, so type checking failed on the main branch. Build the participation bitfield directly from the indices and drop the now unused import. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapse a multi-line AggregationBits.from_indices call onto a single line to satisfy ruff format. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…um#815) Move PoseidonParams field descriptions into attribute docstrings, matching the convention used elsewhere in the codebase. Replace the rounds_f even-count field validator with the native multiple_of=2 constraint, preserving the rationale in the field docstring and dropping the now-unused field_validator import. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Break the single large LstarSpec class into one file per concern (state transition, fork choice, block production, signatures, aggregation, timeline, validator duties), assembled back into LstarSpec via mixins over a shared typing contract (LstarSpecContract). Pure structural move: every method body is byte-for-byte identical, with no change to any external call site or type annotation. The only edits are relocating the chain-match static helper to a module-level function and moving the LstarStore alias into the contract module (still re-exported from spec.py).
c69249b to
c93eed2
Compare
Owner
Author
|
Superseded by leanEthereum#817, which targets upstream main with a clean 9-file diff. This fork-internal PR showed a large diff only because the fork's main was stale. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
refactor(forks/lstar): split spec.py into per-concern mixins
Summary
src/lean_spec/spec/forks/lstar/spec.pyhad grown to a single 1891-line class,LstarSpec, owning every concern of the fork: state transition, fork choice,block production, signature verification, aggregation, interval ticking, and
validator duties. This PR splits that class into one file per concern, assembled
back into the same
LstarSpecvia mixins.The split is a pure structural move: every method body is byte-for-byte
identical to before, and there are no changes to any external call site or type
annotation.
LstarSpecis still a single concrete class exposing the same 28methods with the same signatures.
Motivation
and the state-transition function are large, independent subsystems that were
interleaved in one class body.
while preserving the exact public surface the node services and tests depend on.
changing the agnostic
ForkProtocolor the way callers use the spec.What changed
LstarSpecis now assembled from seven mixins plus a small typing contract:_contract.pyLstarSpecContract(ForkProtocol)— the only new construct. Declares the concrete container-factory types and the cross-mixin method surface. Also hosts theLstarStorealias.state_transition.pyStateTransitionMixin+ the module functionattestation_data_matches_chain.signatures.pySignatureMixin.block_production.pyBlockProductionMixin.fork_choice.pyForkChoiceMixin.aggregation.pyAggregationMixin.timeline.pyTimelineMixin.validator_duties.pyValidatorDutiesMixin.spec.pyLstarSpec(...the seven mixins, LstarSpecContract)— identity fields, the*_classvalues, and theLstarStorere-export.Two deliberate edits, everything else verbatim:
_attestation_data_matches_chainstatic method became a module-levelfunction
attestation_data_matches_chain(it never usedself). Its two callsites now call it as a free function;
block_production.pyimports it fromstate_transition.py.LstarStore = Store[State, Block]alias moved into_contract.pyand isre-exported from
spec.py, sofrom .spec import LstarStore(used by both__init__.pyfiles) keeps working.The typing contract
The mixins call each other through
self(e.g. fork choice's block import callsself.state_transition(...)). For the type checker to accept those cross-filecalls, each mixin needs to know the signatures of the siblings it calls and the
concrete types of the container factories it uses.
LstarSpecContractprovidesexactly that:
*_classattributes, annotated with the concrete lstar types(
type[State], nottype[SpecStateType]), soself.state_class(...)callstype-check. No
ClassVar(it would block subclass narrowing).@abstractmethodsignatures. (Methods only called within their own file are notin the contract.)
LstarSpecContractlives inside thelstarpackage, never inprotocol.py,because it references concrete lstar containers and
ForkProtocolmust stayfork-agnostic (enforced by an existing AST import-guard test).
How callers are unaffected
ForkProtocoldeclares only three abstract methods (generate_genesis,create_store,upgrade_state); every other method is called through theconcrete
LstarSpectype — by the node services (spec: LstarSpec) and thetests. Because the split keeps
LstarSpecas one class exposing all those methodsunchanged, no caller, annotation, or import outside this package changes.
Extending a fork: descendants override methods directly, not mixins
This is the part most relevant to the multi-fork roadmap. The mixins are an
internal organization detail of lstar. A descendant fork does not need to
know they exist, and does not add or re-compose its own mixins.
A new fork subclasses the assembled class and overrides what it needs, directly on
the fork class:
Why overriding on the fork class is enough — late binding. Every sibling call
goes through
self, so it is resolved against the runtime type on every call.Lstar's inherited
tick_interval(in the timeline mixin) callsself.update_safe_target(...); when invoked on aDevnet5Specinstance,selfis
Devnet5Spec, so it dispatches to Devnet5's override — even thoughtick_intervalwas written long before Devnet5 existed and lives in a differentfile. Change one method on the fork class, and every inherited caller picks it up.
This is identical to overriding a method on any ordinary subclass; the mixin
decomposition changes nothing about it.
The only condition: a method must call its sibling as
self.B(...)(which thesplit preserves everywhere). To extend rather than replace, use
super():And if a fork changes only a container (not logic), it overrides no method at all:
the inherited bodies already build via
self.block_class(...), which resolves tothe fork's container at runtime.
So: subclass
LstarSpec, override the attributes/methods that differ, done. Nonew mixins, no new contract base.
When a mixin is not the right tool (and you want components instead)
Mixins are excellent for method-level reuse and override, which is what the
roadmap's "swap a container, inherit the logic, occasionally tweak a method" needs.
They are a poor fit for subsystem-level swapping, and that is the signal to
reach for a different structure (composed components — "Option 3" in the design
notes).
Reach for a component, not a mixin override, when any of these hold:
different fork-choice rule. Overriding a dozen interrelated methods scattered
across the MRO is fragile; a replacement still has to silently honor the implicit
contract the other mixins rely on, with no enforced interface.
Mixins bind one implementation per class via the MRO; you cannot hold two. A
component is an object you can choose, inject, or swap.
selfnamespace and cannot cleanly own its own fields or collaborators; acomponent can.
replacement is complete). The mixin contract is an internal convenience, not a
public API; a component exposes an explicit protocol.
assemble a fork. That re-composition is the smell: the concern wants to be a
swappable object, not a class fragment glued by inheritance order.
When that day comes, the migration is incremental: each mixin already encapsulates
one concern and can be promoted to a held collaborator (e.g.
self.fork_choice)exposing a typed interface, without disturbing the other concerns.
Rule of thumb: tweak a method → override on the fork subclass (mixins are
invisible). Replace a subsystem or need two of something → make it a component.
Verification
confirms byte-identical bodies, modulo the two intended edits above.
exactly; an independent recomputation of the cross-file call graph confirms the
contract covers exactly those 10 methods (none missing, none unused).
LstarSpec()instantiates; the MRO is the expected linear chain;__abstractmethods__is empty.just checkpasses (ruff lint + format,tytype check, codespell, mdformat,lock).
just testpasses (3084 tests, ~93% coverage); the consensus fixture path runsthrough
fillagainst the refactored spec.API compatibility, docs/style, plan adherence) found no blocker or major
issues.
Compatibility notes
LstarStoreisrelocated, not aliased twice; the only re-export is the pre-existing public path.
ForkProtocol/protocol.pyis untouched and stays fork-agnostic.*_classnarrowing as an invariantoverride; this is pre-existing — the original
LstarSpecnarrowed the sameattributes — and accepted by
ty, which is the project's type checker.Follow-ups (out of scope)
aggregation logic. It violates the comment-style rule but predates this change;
leaving it keeps this PR a pure move. Worth a separate doc-cleanup pass.