Skip to content

refactor(forks/lstar): split spec.py into per-concern mixins#1

Closed
leolara wants to merge 118 commits into
mainfrom
refactor/lstar-spec-mixins
Closed

refactor(forks/lstar): split spec.py into per-concern mixins#1
leolara wants to merge 118 commits into
mainfrom
refactor/lstar-spec-mixins

Conversation

@leolara
Copy link
Copy Markdown
Owner

@leolara leolara commented Jun 1, 2026

refactor(forks/lstar): split spec.py into per-concern mixins

Summary

src/lean_spec/spec/forks/lstar/spec.py had grown to a single 1891-line class,
LstarSpec, owning every concern of the fork: state transition, fork choice,
block production, signature verification, aggregation, interval ticking, and
validator duties. This PR splits that class into one file per concern, assembled
back into the same LstarSpec via mixins.

The split is a pure structural move: every method body is byte-for-byte
identical to before, and there are no changes to any external call site or type
annotation
. LstarSpec is still a single concrete class exposing the same 28
methods with the same signatures.

Motivation

  • One 1891-line file is hard to navigate, review, and reason about. Fork choice
    and the state-transition function are large, independent subsystems that were
    interleaved in one class body.
  • The split makes each subsystem a self-contained, independently reviewable unit
    while preserving the exact public surface the node services and tests depend on.
  • It sets up the multi-fork direction (see "Extending a fork" below) without
    changing the agnostic ForkProtocol or the way callers use the spec.

What changed

LstarSpec is now assembled from seven mixins plus a small typing contract:

File Contents
_contract.py LstarSpecContract(ForkProtocol) — the only new construct. Declares the concrete container-factory types and the cross-mixin method surface. Also hosts the LstarStore alias.
state_transition.py StateTransitionMixin + the module function attestation_data_matches_chain.
signatures.py SignatureMixin.
block_production.py BlockProductionMixin.
fork_choice.py ForkChoiceMixin.
aggregation.py AggregationMixin.
timeline.py TimelineMixin.
validator_duties.py ValidatorDutiesMixin.
spec.py LstarSpec(...the seven mixins, LstarSpecContract) — identity fields, the *_class values, and the LstarStore re-export.

Two deliberate edits, everything else verbatim:

  1. The _attestation_data_matches_chain static method became a module-level
    function attestation_data_matches_chain (it never used self). Its two call
    sites now call it as a free function; block_production.py imports it from
    state_transition.py.
  2. The LstarStore = Store[State, Block] alias moved into _contract.py and is
    re-exported from spec.py, so from .spec import LstarStore (used by both
    __init__.py files) keeps working.

The typing contract

The mixins call each other through self (e.g. fork choice's block import calls
self.state_transition(...)). For the type checker to accept those cross-file
calls, each mixin needs to know the signatures of the siblings it calls and the
concrete types of the container factories it uses. LstarSpecContract provides
exactly that:

  • The nine *_class attributes, annotated with the concrete lstar types
    (type[State], not type[SpecStateType]), so self.state_class(...) calls
    type-check. No ClassVar (it would block subclass narrowing).
  • The ten methods that are actually called across file boundaries, as
    @abstractmethod signatures. (Methods only called within their own file are not
    in the contract.)

LstarSpecContract lives inside the lstar package, never in protocol.py,
because it references concrete lstar containers and ForkProtocol must stay
fork-agnostic (enforced by an existing AST import-guard test).

How callers are unaffected

ForkProtocol declares only three abstract methods (generate_genesis,
create_store, upgrade_state); every other method is called through the
concrete LstarSpec type — by the node services (spec: LstarSpec) and the
tests. Because the split keeps LstarSpec as one class exposing all those methods
unchanged, no caller, annotation, or import outside this package changes.

Extending a fork: descendants override methods directly, not mixins

This is the part most relevant to the multi-fork roadmap. The mixins are an
internal organization detail of lstar. A descendant fork does not need to
know they exist, and does not add or re-compose its own mixins.

A new fork subclasses the assembled class and overrides what it needs, directly on
the fork class:

class Devnet5Spec(LstarSpec):                       # inherits all seven mixins transitively
    block_class: type[Devnet5Block] = Devnet5Block  # swap one container type

    def upgrade_state(self, state: State) -> Devnet5State:
        ...                                          # the migration every fork overrides

    def process_attestations(self, state, attestations):  # OPTIONAL: override one method
        ...                                               # lives in StateTransitionMixin on lstar

Why overriding on the fork class is enough — late binding. Every sibling call
goes through self, so it is resolved against the runtime type on every call.
Lstar's inherited tick_interval (in the timeline mixin) calls
self.update_safe_target(...); when invoked on a Devnet5Spec instance, self
is Devnet5Spec, so it dispatches to Devnet5's override — even though
tick_interval was written long before Devnet5 existed and lives in a different
file. Change one method on the fork class, and every inherited caller picks it up.
This is identical to overriding a method on any ordinary subclass; the mixin
decomposition changes nothing about it.

# Minimal illustration: A in one mixin, B in another.
class StateTransitionMixin(LstarSpecContract):
    def A(self): return self.B() + 1     # cross-mixin call via self

class ForkChoiceMixin(LstarSpecContract):
    def B(self): return 10

class LstarSpec(StateTransitionMixin, ForkChoiceMixin, LstarSpecContract): ...

class Devnet5Spec(LstarSpec):
    def B(self): return 100              # override B only; leave A inherited

Devnet5Spec().A()   # -> 101  (inherited A used the overridden B)
LstarSpec().A()     # -> 11

The only condition: a method must call its sibling as self.B(...) (which the
split preserves everywhere). To extend rather than replace, use super():

class Devnet5Spec(LstarSpec):
    def update_safe_target(self, store):
        store = super().update_safe_target(store)   # run lstar's version
        ...                                          # then add behavior

And if a fork changes only a container (not logic), it overrides no method at all:
the inherited bodies already build via self.block_class(...), which resolves to
the fork's container at runtime.

So: subclass LstarSpec, override the attributes/methods that differ, done. No
new mixins, no new contract base.

When a mixin is not the right tool (and you want components instead)

Mixins are excellent for method-level reuse and override, which is what the
roadmap's "swap a container, inherit the logic, occasionally tweak a method" needs.
They are a poor fit for subsystem-level swapping, and that is the signal to
reach for a different structure (composed components — "Option 3" in the design
notes).

Reach for a component, not a mixin override, when any of these hold:

  • A fork replaces a whole subsystem, not a method or two — e.g. an entirely
    different fork-choice rule. Overriding a dozen interrelated methods scattered
    across the MRO is fragile; a replacement still has to silently honor the implicit
    contract the other mixins rely on, with no enforced interface.
  • Two implementations must coexist or be selected at runtime/config time.
    Mixins bind one implementation per class via the MRO; you cannot hold two. A
    component is an object you can choose, inject, or swap.
  • The subsystem needs its own state or dependencies. A mixin shares the single
    self namespace and cannot cleanly own its own fields or collaborators; a
    component can.
  • You want a real, enforced interface boundary (so the type checker verifies a
    replacement is complete). The mixin contract is an internal convenience, not a
    public API; a component exposes an explicit protocol.
  • You find yourself writing per-fork mixins and re-ordering base classes to
    assemble a fork. That re-composition is the smell: the concern wants to be a
    swappable object, not a class fragment glued by inheritance order.

When that day comes, the migration is incremental: each mixin already encapsulates
one concern and can be promoted to a held collaborator (e.g. self.fork_choice)
exposing a typed interface, without disturbing the other concerns.

Rule of thumb: tweak a method → override on the fork subclass (mixins are
invisible). Replace a subsystem or need two of something → make it a component.

Verification

  • Verbatim: an AST diff of all 29 original methods against their new homes
    confirms byte-identical bodies, modulo the two intended edits above.
  • Contract parity: the 10 abstract signatures match their implementations
    exactly; an independent recomputation of the cross-file call graph confirms the
    contract covers exactly those 10 methods (none missing, none unused).
  • Runtime: LstarSpec() instantiates; the MRO is the expected linear chain;
    __abstractmethods__ is empty.
  • just check passes (ruff lint + format, ty type check, codespell, mdformat,
    lock).
  • just test passes (3084 tests, ~93% coverage); the consensus fixture path runs
    through fill against the refactored spec.
  • A multi-dimension adversarial review (verbatim, imports/cycles, contract/typing,
    API compatibility, docs/style, plan adherence) found no blocker or major
    issues
    .

Compatibility notes

  • No backward-compatibility shims were added (per repo policy). LstarStore is
    relocated, not aliased twice; the only re-export is the pre-existing public path.
  • ForkProtocol / protocol.py is untouched and stays fork-agnostic.
  • Pyright (not the repo's checker) reports the *_class narrowing as an invariant
    override; this is pre-existing — the original LstarSpec narrowed the same
    attributes — and accepted by ty, which is the project's type checker.

Follow-ups (out of scope)

  • One pre-existing banner-style separator comment carried over verbatim inside the
    aggregation logic. It violates the comment-style rule but predates this change;
    leaving it keeps this PR a pure move. Worth a separate doc-cleanup pass.

tcoratger and others added 30 commits April 27, 2026 12:38
)

The single 1422-line event_source.py grew to mix three concerns: the
abstract event-source contract, the inbound gossip wire-format parser,
and the live network orchestrator. This PR splits it into a focused
package without rewriting any documentation or changing any behavior.

New layout:

- event_source/protocol.py: EventSource Protocol, GossipMessageError,
  SUPPORTED_PROTOCOLS allow-list (the dependency-light shared surface).
- event_source/gossip.py: GossipHandler and read_gossip_message (the
  inbound gossipsub wire-format parser).
- event_source/live.py: LiveNetworkEventSource (the orchestrator that
  owns connections, streams, and the gossipsub behavior).
- event_source/__init__.py: re-exports the full public surface so every
  existing import path keeps working unchanged.

Dependency direction: protocol -> gossip -> live, strict and one-way.

Inside live.py, two structural cleanups:

- _accept_streams (243 lines, three nested concerns) decomposes into
  _negotiate_inbound_stream, _handle_gossipsub_inbound_stream,
  _setup_outbound_gossipsub_after_delay (was a nested closure), and
  _handle_reqresp_inbound_stream. The orchestrator becomes ~30 lines:
  accept loop plus protocol-id dispatch.
- _handle_inbound_quic_connection collapses into _handle_inbound_connection
  (the two were byte-identical except for one log line). _listen_quic
  now passes the unified handler.

Every existing docstring and inline comment is preserved verbatim from
main; only the structural pieces above changed. No external import path
moves; no test file is touched.

Verified:

- uvx tox -e all-checks (ruff, format, ty, codespell, mdformat) passes.
- tests/lean_spec/subspecs/networking/client and gossipsub tests
  (159 cases) pass.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eum#682)

The time check used to admit any vote within one full slot of the local
clock. That window let an adversary pre-publish next-slot aggregates
~800 ms before any honest validator could produce them, with the next
proposer happily including them.

The bound is now expressed in interval units against `Store.time` and
gated by a new `GOSSIP_DISPARITY_INTERVALS` constant (one interval, the
lean analogue of mainnet's `MAXIMUM_GOSSIP_CLOCK_DISPARITY`). Tests
reference the constant rather than hardcoding numbers, so future
changes propagate automatically.

Several existing spec tests gossiped attestations carrying a future
data.slot to compress timelines; they have been restructured to follow
the natural Lean flow (gossip during the producer's own slot, then
tick to migrate). Three new boundary regressions are added to both
gossip validation files plus a unit-level `TestValidateAttestationTimeCheck`
class.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tion (leanEthereum#690)

Add 3 new fork choice test vectors covering attestation target selection,
block production, and equivocation handling. All tests use the
ForkChoiceTestFiller pattern, generating JSON fixtures for client
implementations.
- Equivocation test now asserts per-validator vote tracking via
  AttestationCheck so the first-vote-wins rule is exercised at the
  validator level, not just inferred from the head.
- Drop the redundant n==9 or n==11 head-slot skip in the block
  production test so every extension slot is checked.
- Trim repeated post-finality assertions on slots 10/11 in the target
  selection test; slot 9 keeps the full check.
- Document timing arithmetic and gossip ordering inline.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eanEthereum#638)

* feat: add multi-fork architecture with ForkProtocol and SpecRunner

Introduce a protocol-and-fork-class architecture that enables multiple
devnet forks to coexist in the codebase using Python class inheritance.

Key changes:

- Add ForkProtocol ABC defining the consensus interface each fork implements
- Add SpecRunner for era-aware fork dispatch
- Create Devnet4Spec wrapping current State/Store as the base fork
- Create Devnet5Spec skeleton inheriting from Devnet4Spec
- Move State and Store from subspecs/ into forks/devnet4/ (fork-specific)
- Wire ForkProtocol into Node, __main__, genesis, and storage
- Rename testing fork from Devnet to Devnet4, add Devnet5
- Update all consumer imports to use lean_spec.forks re-exports
- Update all test markers from valid_until("Devnet") to valid_until("Devnet4")

The architecture follows the coworker's proposal: subspecs/ contains only
fork-agnostic shared libraries (ssz, xmss, networking, chain). Fork-specific
processing logic (State, Store) lives in forks/devnetN/. Consumer modules
import State/Store from lean_spec.forks (the package re-export). Construction
goes through ForkProtocol (fork.generate_genesis(), fork.create_store()).

Adding a new fork requires:
1. forks/devnet5/state.py — Devnet5State(State) overriding changed methods
2. forks/devnet5/spec.py — Devnet5Spec(Devnet4Spec) with generate_genesis/upgrade_state
3. Python virtual dispatch handles the rest — no consumer code changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ci: update --fork=Devnet to --fork=Devnet4 in workflows

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(forks): move consensus containers into forks/devnet4/ and genericize ForkProtocol

Relocate every consensus-dependent container (Attestation, Block, Checkpoint,
Config, Slot, State, Validator) from subspecs/containers/ into
forks/devnet4/containers/ so per-fork divergence of container shapes (e.g. a
future Devnet5 Attestation) becomes physically possible without mutating
devnet4 symbols. Pydantic inheritance across forks is documented as unsafe
for SSZ hash_tree_root stability — the supported pattern is copy-then-diverge
inside the new fork's package.

Strip ForkProtocol to its irreducible surface: five ClassVars (NAME, VERSION,
state_class, block_class, store_class) and three methods (generate_genesis,
create_store, upgrade_state). The protocol module imports nothing from any
devnet package and exposes SpecStateType / SpecStoreType structural Protocols
for type-hinting the classmethod contract. Runner rejects duplicate NAMEs and
non-strictly-increasing VERSIONs; adds DEFAULT_RUNNER convenience.

Reinstate Devnet5Spec as a genuine registered placeholder: own NAME, own
VERSION, inherits method logic from Devnet4Spec, binds container types via
explicit aliases that are one-line swappable when divergence lands. The
multi-fork pipeline (runner, fixture filler, test framework) now exercises
two forks end to end.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(forks): use AST walk instead of text search for fork-agnostic check

The previous test asserted the literal string 'devnet4' did not appear in
forks/protocol.py, which tripped on the docstring example for the NAME
ClassVar. Replace with an AST-based check that inspects Import and
ImportFrom nodes only — docstrings and comments are free to reference fork
names as examples.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(forks): trim consensus-testing fork module docstring

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): stage 1 trivial cleanups for ForkProtocol surface

Bundles four small, behavior-preserving improvements to the multi-fork
skeleton from PR leanEthereum#638. Tests + all-checks green.

* Expose `GOSSIP_DIGEST: ClassVar[str]` on `ForkProtocol`. The gossipsub
  fork digest is fork metadata, not a top-level constant. Devnet4Spec
  binds it to "devnet0" (preserving the existing network agreement);
  Devnet5Spec inherits. Removes the hardcoded `GOSSIP_FORK_DIGEST`
  constant and the now-unused `Final` import from `__main__.py`, and
  routes every consumer through `fork.GOSSIP_DIGEST`.

* Use `DEFAULT_RUNNER` directly in `__main__.py` instead of building a
  second `SpecRunner(FORK_SEQUENCE)`. The forks package already exports
  the singleton; the redundant construction is dropped along with the
  unused `FORK_SEQUENCE` and `SpecRunner` imports.

* Add `previous: ClassVar[type[ForkProtocol] | None]` linking each fork
  to its predecessor. `Devnet4Spec.previous = None`;
  `Devnet5Spec.previous = Devnet4Spec`. Sets up the foundation for a
  future registry that derives ordering from topology and for chained
  state migrations via `upgrade_state`.

* Tidy `devnet5/spec.py`: drop the module-level `_Devnet5State` and
  `_Devnet5Store` identity aliases; bind the container classes
  directly on `Devnet5Spec`. When devnet5 grows its own
  `forks/devnet5/containers/` package, the swap is a file-level diff
  in the imports rather than an alias rebinding.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): rename SpecRunner to ForkRegistry

The class is a registry — it stores a sequence of forks and looks them
up by name — not a dispatcher routing spec calls per fork. The old name
implied the latter and set wrong expectations for follow-ups.

* Renames the file `runner.py` -> `registry.py`.
* Renames the class `SpecRunner` -> `ForkRegistry`.
* Renames the singleton `DEFAULT_RUNNER` -> `DEFAULT_REGISTRY`.
* Updates the docstring on `FORK_SEQUENCE` and the error message in
  the empty-list guard.
* Updates the test class name and assertions.

No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(forks): make upgrade_state abstract on ForkProtocol

Replaces the silent identity default with an abstract method, forcing
every concrete fork to declare its migration explicitly. This prevents
the silent-no-op footgun: a future fork that adds a State field but
forgets to override upgrade_state would otherwise migrate by simply
not migrating.

* `ForkProtocol.upgrade_state` becomes `@abstractmethod` with a
  docstring-only body.
* `Devnet4Spec.upgrade_state` is identity, documented as "root fork,
  no predecessor, no migration".
* `Devnet5Spec.upgrade_state` is identity, documented as "currently
  mirrors devnet4; replace when devnet5's State diverges".
* Adds a test that asserts `ForkProtocol()` raises TypeError, locking
  in the abstract-class contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): drop devnet5 placeholder and rename devnet4 to lstar

Delete forks/devnet5/ entirely. We are running one devnet roughly per month,
so per-devnet placeholders rot fast and lock the spec into a sequential
upgrade story we are not committing to.

Rename forks/devnet4/ to forks/lstar/ (via git mv, history preserved):

- Devnet4Spec    -> LstarSpec
- Devnet4 (test BaseFork class) -> Lstar
- "devnet4" / "Devnet4" string identifiers -> "lstar" / "Lstar"
- lean_spec.forks.devnet4.* import paths -> lean_spec.forks.lstar.*
- --fork=Devnet4 / --fork=devnet4 in CI workflows and docs -> Lstar / lstar

Test surface follows: TestDevnet4Spec -> TestLstarSpec, TestDevnet5Spec
removed entirely. Multi-fork ForkRegistry tests now use a synthetic
in-test successor class instead of a real second fork.

Untouched on purpose:

- pyproject.toml lean-multisig-py branch="devnet4" (external repo, not ours).
- tests/consensus/devnet/ folder name (generic test path, no version number).
- GOSSIP_DIGEST="devnet0" — the cross-client gossip network name set by
  PR leanEthereum#622. Renaming it touches the whole networking layer and is a
  separate cleanup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(networking): rename fork_digest to network_name

The value sitting on the gossipsub topic layer is the cross-client
network name (currently "devnet0", set by PR leanEthereum#622), not a 4-byte fork
digest hash. The "fork_digest" naming was a holdover from Ethereum
mainline where it really is a digest; here it just confuses readers
into expecting a hash where there is a string identifier.

Renamed in the network-name layer:

- ForkProtocol.GOSSIP_DIGEST -> ForkProtocol.NETWORK_NAME
- GossipTopic.fork_digest -> GossipTopic.network_name
- GossipTopic.{block,committee_aggregation,attestation_subnet}
  fork_digest parameter -> network_name
- GossipTopic.validate_fork(expected_fork_digest=...)
  -> validate_fork(expected_network_name=...)
- GossipTopic.from_string_validated(..., expected_fork_digest=...)
  -> expected_network_name
- GossipHandler.fork_digest -> GossipHandler.network_name
- LiveNetworkEventSource._fork_digest -> _network_name
- LiveNetworkEventSource.set_fork_digest() -> set_network_name()
- NetworkService.fork_digest -> NetworkService.network_name
- NodeConfig.fork_digest -> NodeConfig.network_name

Plus matching test renames:

- test_gossip_digest -> test_network_name
- test_gossip_topic_fork_digest_{matches,mismatch,...}
  -> test_gossip_topic_network_name_*
- All test variables, parameters, and prose updated.

Untouched on purpose:

- Eth2Data.fork_digest: ForkDigest in subspecs/networking/enr/eth2.py
  is a real 4-byte ENR hash per the Ethereum p2p spec. The ForkDigest
  type stays. enr/enr.py, peer.py, discovery/routing.py all interact
  with this real digest and are unchanged.
- JSON fixture keys "forkDigest" / "expectedForkDigest" in cross-client
  test vectors. Other clients consume those keys; renaming would break
  the wire format. Internal Python uses network_name; the JSON keys are
  read into network_name on the way in.
- ForkMismatchError class name. A fork mismatch is still a fork
  mismatch semantically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Leo Lara <leo@leolara.me>
…m#686) (leanEthereum#694)

* refactor(forks): tighten ForkProtocol surface (Stage 1)

Drop Any from ForkProtocol method signatures. Replace with fork-stable
primitives (Uint64) and structural protocols (SpecStateType, SpecBlockType,
SpecStoreType). Drop ClassVar from state_class/block_class/store_class so
subclasses can narrow them under pyright/ty invariance rules.

Rename NETWORK_NAME to GOSSIP_DIGEST: the field already serves the
gossipsub fork-digest role.

Remove the None-fork fallback in generate_pre_state. Every call now
dispatches through fork.generate_genesis. The pre pytest fixture threads
fork from the framework's fork fixture.

Refs leanEthereum#686

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): type validators as SSZList[Any]

object was too loose for the structural validators parameter on
generate_genesis. SSZList is the fork-stable concrete base; the element
type stays generic because each fork owns its own Validator class.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: trim stale legacy reference from _DEFAULT_FORK docstring

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: add initial classes and constants

* feat: implement BlocksByRange networking protocol and sync optimization

This commit adds the server-side handler for the BlocksByRange protocol, integrates it into the SyncService, and enhances the backfill synchronization with range-based fetching. It also includes a strict SSZ validation fix for fixed-size containers to ensure malformed oversized payloads are correctly rejected.

* fix: resolve linting, build errors, and missing imports in tests

This commit addresses multiple linting violations (E501 line length), fixes missing imports in head_sync.py and test helpers, and restores/enhances test coverage for malformed SSZ payloads in the request/response layer.

* chore: remove unnecessary scope check

* chore: remove dependency from Store in backfill_sync

* fix: fix ci

* review: address consensus + py-architect feedback on BlocksByRange

Apply review fixes spanning protocol semantics, client correctness, and
test discipline. The headline change is a sliding history window: replace
the absolute MIN_BLOCK_REQUESTS_HISTORY_SLOT floor with a window relative
to the responder's current slot, matching upstream consensus-spec behavior
and unblocking sync from genesis.

Protocol semantics:

- Sliding window: handle_blocks_by_range now rejects start_slot < max(0,
  current_slot - MIN_SLOTS_FOR_BLOCK_REQUESTS). Add CurrentSlotLookup
  callback on RequestHandler; missing lookup returns SERVER_ERROR rather
  than silently misreporting history.
- Crash guard: head_sync rejects gossip blocks at or below the finalized
  slot, fixing the SSZValueError underflow path on adversarial input.
- Gap-detection floor: head_slot + 1 (not finalized_slot) so slots already
  in the Store are never redownloaded.

Client correctness (reqresp_client.request_blocks_by_range):

- Propagate CodecError so peer downscoring fires on protocol violations
  (was being swallowed by the broad except Exception).
- Parent-root continuity check applies across empty slots, not just
  consecutive slots, matching the canonical-chain MUST.
- Range bounds use int() arithmetic to avoid Slot overflow.
- Local validation: count <= MAX_REQUEST_BLOCKS and start_slot + count
  <= UINT64_MAX before opening a stream.
- Raise CodecError when peer sends more than count chunks.
- Log violations with conn.peer_id, not the raw connection repr.
- Bug fix: count == 0 compares against Uint64(0) (was raising TypeError).

Architecture:

- New StoreView Protocol (has_root / finalized_slot / head_slot) replaces
  two loose Callable | None callbacks on BackfillSync. _SyncStoreView
  adapter in service.py reads through a getter so live store mutations
  are observed.
- _max_range_slot watermark advances only after a completed (success or
  empty) range fetch; failed fetches stay retryable. reset() clears it.
- LiveNetworkEventSource exposes set_block_by_slot_lookup and
  set_current_slot_lookup, symmetric with set_block_lookup. __main__
  wires the current-slot side. Block-by-slot wiring requires SignedBlock
  storage and is a follow-up.

Tests and style:

- Drop MagicMock store fixture; use a concrete FakeStoreView dataclass.
- Drop the get_finalized_slot=None mutation hack; that test now builds
  its own no-store-view backfill.
- Split MockNetworkRequester.request_log into typed root/range logs.
- Full-equality assertions across new tests; new coverage for failed-
  range non-advance, reset-clears-watermark, sliding-window genesis edge
  case, missing current_slot_lookup SERVER_ERROR.
- Drop step-numbered comments, nested test import, dead add_block(root=)
  parameter, and dead duplicate MAX_CONCURRENT_REQUESTS constant.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: cover BlocksByRange client and head-sync routing paths

Add the unit tests that were missing after the previous review-fix commit:

- tests/lean_spec/subspecs/networking/client/test_reqresp_client_range.py:
  14 tests for the outbound BlocksByRange flow. Zero-count short-circuit,
  count-above-max rejection, start_slot+count overflow, no-connection,
  full-range happy path, partial response on early close, RESOURCE_UNAVAILABLE
  chunk skipping, slot monotonicity violation, out-of-range slot violation,
  parent-root continuity violation across a skipped slot (the upstream-
  aligned tightening), parent-root continuity holds across skipped slots,
  more-than-count chunk enforcement, timeout returns empty, SERVER_ERROR
  halts reading and returns the partial list.

- tests/lean_spec/subspecs/sync/test_head_sync_backfill_routing.py:
  5 tests for the post-review _cache_and_backfill routing. Silent rejection
  at the finalized slot, silent rejection below the finalized slot,
  single-slot gap above head uses root recursion, multi-slot gap above
  head uses a single range fetch, alt-fork gossip at-or-below head uses
  root recursion.

Drop the local UINT64_MAX constant in the reqresp client; use the canonical
Uint64.max_value() helper from the typed API instead.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…leanEthereum#686) (leanEthereum#695)

Promote the data primitives whose shape is stable across the leanSpec fork
chain out of the lstar fork package and into lean_spec.types:

- Slot (with its 3SF-mini justifiability methods)
- ValidatorIndex, SubnetId
- Checkpoint

Validator and Validators stay in the fork because their XMSS-bound key shape
is signature-scheme specific. ValidatorIndices stays for now because
to_aggregation_bits couples it to the fork's AggregationBits; that
decomposition can land as a follow-up.

Config also stays in the fork: the single-field shape is the most likely
container to grow per fork (deposits, RANDAO, fee parameters), so promoting
it to subspecs/chain/ would just churn it back out later.

Tighten the ForkProtocol surface using the freshly promoted types:
SpecStoreType.from_anchor and ForkProtocol.create_store now take
ValidatorIndex | None instead of Uint64 | None.

Also drop the fork-side re-exports per CLAUDE.md (no compat shims) and
update all call sites across src/, tests/, and packages/.

Refs leanEthereum#686

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…thereum#686) (leanEthereum#698)

Subspecs no longer hard-import any concrete fork module. They reach
fork-specific types through three channels: fork-stable types in
`lean_spec.types`, structural protocols in `lean_spec.forks.protocol`,
and re-exports of the active fork from `lean_spec.forks`. Concrete
class pointers are injected through `ForkProtocol` (`fork.state_class`,
`fork.block_class`, ...) for subspecs that construct or decode SSZ
instances.

`AggregationBits`, `ValidatorIndices`, and `VALIDATOR_REGISTRY_LIMIT`
move into `lean_spec.types`; `subspecs.chain.config` re-exports the
limit so existing call sites keep working.

`SQLiteDatabase` now takes `state_class`, `block_class`, and
`attestation_data_class` as constructor arguments instead of
hard-importing `forks.lstar` containers.

A new AST-walk test (`test_subspecs_do_not_import_concrete_fork`)
guards the property going forward by asserting no file under
`src/lean_spec/subspecs/` imports from `lean_spec.forks.lstar.*`.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…#686) (leanEthereum#702)

* refactor(forks): add Spec delegator surface (Stage 4A of leanEthereum#686)

Adds the LstarSpec method surface that mirrors the existing
State / Store / SignedBlock methods one-for-one:

- State transition: state_transition, process_slots, process_block,
  process_block_header, process_attestations, build_block.
- Forkchoice: on_block, on_tick, on_gossip_attestation,
  on_gossip_aggregated_attestation, produce_attestation_data,
  produce_block_with_signatures, get_proposal_head.
- Block signatures: verify_signatures.

Each method is a pure delegator to the corresponding container method.
No call sites change — the new surface is unused initially. Stage 4B
will rewrite call sites to go through the spec; Stage 4C will move
the bodies in and replace literal Block/State/Store references with
self.*_class(...).

Tests use unittest.mock.patch.object to verify each delegator
forwards its arguments unchanged and returns the container method's
result verbatim.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: tighten delegator surface docstrings

Rewrite the per-method docstrings on the fork-class delegators and
the delegator test suite to follow the project documentation rules:

- Each docstring describes the operation in plain English.
- No explicit class or method names that rot when renamed.
- No issue/stage references that belong in the PR description.
- No banner-style separator comments inside the class body.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(forks): add inner-type class pointers and structural protocols

Extend ForkProtocol with class pointers for the inner container types
(BlockBody, BlockHeader, AggregatedAttestations, AttestationSignatures)
and the matching structural protocols. Hook them up on LstarSpec.

This is the scaffolding the next stages of leanEthereum#686 need so that moving
container method bodies into the spec can replace literal Block(...),
BlockBody(...), AggregatedAttestations(...) constructors with
self.<name>_class(...) — keeping inheriting forks free to swap any
single inner type without re-implementing the parent's logic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…leanEthereum#686) (leanEthereum#703)

Tests that drive state transition, fork choice, block production, and
signature verification now go through the fork's spec class instead of
calling methods directly on the State / Store / SignedBlock containers.

The pattern:

    state.process_block(block)             →  spec.process_block(state, block)
    store.on_block(signed_block)           →  spec.on_block(store, signed_block)
    signed_block.verify_signatures(vs)     →  spec.verify_signatures(signed_block, vs)
    store.produce_attestation_data(slot)   →  spec.produce_attestation_data(store, slot)
    ...

Test functions consume `spec` via a new session-scoped pytest fixture
defined in `tests/lean_spec/conftest.py`. Helper / library code uses a
module-level `_SPEC = LstarSpec()` constant — `LstarSpec` is stateless,
so a single shared instance is fine.

Internal Store helpers (`aggregate`, `update_head`, `accept_new_attestations`,
`update_safe_target`, `tick_interval`, `validate_attestation`,
`prune_stale_attestation_data`, `extract_attestations_from_aggregated_payloads`,
`compute_block_weights`) are not yet on the spec surface and are left as
direct method calls — they migrate alongside the body move in the next stage.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… part 1 of leanEthereum#686) (leanEthereum#704)

* refactor(forks): move verify_signatures body into the spec class

Moves the full XMSS signature verification logic from
SignedBlock.verify_signatures into LstarSpec.verify_signatures.
SignedBlock becomes a pure SSZ data container.

Internal callers that still hold a Store (notably Store.on_block,
which itself moves to the spec class in a follow-up) reach the
verification path via a deferred import of LstarSpec to sidestep
the spec ↔ store module-load cycle.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): move State and Store bodies into the spec class

Migrates every State and Store method body into LstarSpec. Containers
(State, Store, SignedBlock) become thin Pydantic data classes whose
methods are one-line forwarders to the active fork spec, reached via a
deferred import that breaks the spec ↔ container module-load cycle.

Inside the moved bodies, every literal Block(...), BlockBody(...),
BlockHeader(...), Config(...), AggregatedAttestations(...), and other
container constructor is now self.<name>_class(...). An inheriting fork
that swaps a single container type therefore receives the parent fork's
logic for free.

Observability hooks (observe_state_transition, observe_on_block,
observe_on_attestation) ride along with the bodies, preserving the
metrics surface.

The obsolete delegator-forwarding test file is removed; behavioural
coverage now lives in the existing state-transition, fork-choice, and
block-production test suites.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): delete the trivial container forwarders (Stage 4D of leanEthereum#686)

State, Store, and SignedBlock become pure Pydantic data containers.
All forwarder methods that delegated through the lazy spec singleton
are removed; the lazy singleton helpers are removed alongside them.

Every remaining call site that used to go through a container method
now goes through the active fork spec:

- Tests use the session-scoped `spec` fixture from `tests/lean_spec/conftest.py`.
- Subspec services (`sync`, `validator`, `chain`, plus the fork-choice
  API endpoint) carry a module-level `_SPEC = LstarSpec()` constant.
- Library helpers (`tests/lean_spec/helpers/builders.py`,
  `packages/testing/...`) follow the same `_SPEC` pattern.

`ForkProtocol.generate_genesis` and `ForkProtocol.create_store` become
abstract; the previous default implementations referenced container
methods that no longer exist.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(forks): resolve Block forward reference in BlockLookup

The Store.blocks field is annotated as `BlockLookup`, which was defined
as `dict[Bytes32, "Block"]` — a string forward reference. After the
container methods moved off Store, Pydantic's model rebuild could no
longer resolve `Block` because it was never imported into store.py
alongside the alias.

Drop the forward-reference quoting: `BlockLookup` lives in the same
module as `Block`, so the type can refer to the class directly. Pydantic
then resolves `Store.blocks` correctly through the alias re-export.

Without this, every consensus filler that constructed a Store via
`spec.create_store(...)` raised
`PydanticUserError: 'Store' is not fully defined`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(forks): route mock-store tests through autouse spec patches

The forwarders that used to live on `Store` (and were patched in
`tests/lean_spec/subspecs/{sync,chain,networking,validator}/`) are gone
after Stage 4D. The mocks (`MockStore`, `MockForkchoiceStore`) still
implement the same method surface, but the service code now calls the
real spec, which expects a Pydantic Store.

Add an autouse fixture per affected subspec that patches the active
spec's methods to delegate back to `store.method(...)`. The mocks
intercept calls in-place, preserving every test's recording semantics
without touching service code.

The validator service tests that previously patched
`Store.produce_block_with_signatures` and `Store.on_gossip_attestation`
now patch the matching methods on the validator service's `_SPEC` —
same intent, current attribute path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): drop cast(Store, ...) at call sites

Casts were a workaround for Liskov violations on LstarSpec.create_store
when it returned the SpecStoreType protocol while concretely producing
Store. cast had no runtime effect and pushed type-checker noise into
fixtures and tests.

Move the imprecision into the fork itself: create_store now declares
its concrete Store return and suppresses the override warning at the
single definition site. Callers receive Store directly and need no cast.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): replace 17-site _SPEC duplication with DI

Each module declared its own _SPEC = LstarSpec(), creating value-equal
but identity-distinct instances. The pattern also baked LstarSpec into
17 import sites; a future fork would have to grep-and-replace them all.

Production services (chain, sync, validator, api) now take spec as a
dataclass field with default_factory=LstarSpec — explicit at the
composition root in node.py, optional in tests. node.py narrows
config.fork (ForkProtocol) to LstarSpec once with isinstance, which
also lets the cast(State, ...) and cast(Store, ...) at the genesis
construction sites drop.

Test conftests that intercept spec calls now monkey-patch LstarSpec
at the class level (not the deleted module-level _SPEC instance).
Test types and fixtures instantiate LstarSpec at call time — no
module-level cache, no shared mutable state to alias.

ForkProtocol still declares only the three abstract construction
methods (generate_genesis, create_store, upgrade_state). Services and
tests that drive consensus methods (process_slots, build_block,
tick_interval, ...) keep the concrete LstarSpec type until the
protocol surface is widened in a follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(tests): replace conftest monkey-patches with injected spec

Three autouse fixtures in chain/sync/networking conftests patched
LstarSpec class methods so MockStore / MockForkchoiceStore could
intercept consensus calls in place. Class-level patching mutates
shared state and runs against every test in the directory whether
needed or not.

Now that services accept a spec field, tests inject a small
StoreInterceptingSpec subclass that forwards each spec call back to
the store argument. make_store() (used by sync/networking tests)
hands the intercepting spec to the real SyncService transparently.
Chain test_service.py threads it through ChainService directly.

Two conftests delete entirely; the sync conftest keeps only its
sample_checkpoint / sample_status fixtures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(tests): patch validator spec on the instance, not the dropped module

Two validator tests still resolved `lean_spec.subspecs.validator.service._SPEC`,
which was removed when the spec moved onto the service as a field. Patching now
targets `service.spec` directly via `patch.object`, which also exercises the
single instance the test actually calls into.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eanEthereum#686) (leanEthereum#705)

Per the multi-fork roadmap, Store now declares its State and Block types as
type parameters so future forks can specialize them with a typedef instead
of copy-pasting the whole class.

Design (per the py-architect agent):

- `Store(StrictBaseModel, Generic[StateT, BlockT])` in forks/lstar/store.py.
  StateT and BlockT are bound to Container so Pydantic can build a real
  schema at parameterization time; structural protocols would not.
- `LstarStore = Store[State, Block]` in forks/lstar/spec.py is the concrete
  binding owned by the lstar fork. `LstarSpec.store_class` and
  `create_store` are typed against it.
- `BlockLookup` is dropped — it was a single-line alias used in two helper
  signatures that read just as clearly as `dict[Bytes32, Block]`.
- Public `from lean_spec.forks import Store` resolves to `LstarStore`, so
  every existing call site keeps working without change. `LstarStore` is
  also exported under its canonical name for callers that prefer to be
  explicit.
- Mutable Store defaults move from bare `= {}` to `Field(default_factory=
  dict)` to silence the Pydantic generic-default warning ty raised
  intermittently and to keep the schema honest.

The third Stage 5 item (a `Devnet5Store` typedef demonstrating the pattern)
is dropped because devnet5 was unified back into lstar.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tage 6 of leanEthereum#686) (leanEthereum#706)

Per the multi-fork roadmap, test files that target fork-specific containers
or fork choice belong next to the fork they exercise. Until the roadmap is
finished, only lstar exists, so the tree starts as forks/lstar/{state,
forkchoice}/.

Each conftest moves with its only consumers:

- containers/conftest.py provided container_key_manager, used solely by
  test_state_aggregation.py — both move to forks/lstar/state/.
- forkchoice/conftest.py provided pruning_store and sample_store, used by
  tests now under forks/lstar/forkchoice/.

The empty subspecs/forkchoice/ directory is removed. Files in
subspecs/containers/ that target fork-stable types (test_checkpoint.py)
or attestation containers stay where they are; Stage 6 was scoped to
state and forkchoice tests only.

The Stage 6 item that mirrors the structure under devnet5 is dropped
because devnet5 was unified back into lstar.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nEthereum#699)

Adds a state-transition fixture that fills the registry to capacity
(4096 placeholder validators), then justifies block 1 with an attestation
from the 2731/4096 supermajority threshold. The placeholder validators
carry zero pubkeys to skip XMSS key generation; signatures are appended
via forced_attestations to bypass the builder's signing path. The state
transition still tallies the votes and applies the same supermajority
rule as the production path.

Closes leanEthereum#580

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(chore): migrating from tox to just for better devex

* chore(tools): remove tox, use just; add list descriptions; cleanup

---------

Co-authored-by: shriraj pawar <shripawar0411@gmail.com>
…thereum#710)

* chore: rm old references to ``devnet`` fork; rm exec fork refs

* fix(test): remaining fork updates ``Devnet`` -> ``Lstar`` for pytest marks

* chore: rename devnet -> lstar
…hor block (leanEthereum#713)

* feat(api,sync): add /lean/v0/blocks/finalized for checkpoint-sync anchor

Store.create_store(state, anchor_block, ...) requires both the finalized
state and the anchor block, asserts state_root pairing, and seeds
store.blocks[anchor_root] = anchor_block. The only checkpoint-sync API
endpoint shipped state alone, so a syncing node had no spec-conformant
way to obtain the matching block.

Implementations worked around this by fabricating a SignedBlock from
state.latest_block_header with an empty body and zero signature. Such
synthetic blocks have hash_tree_root != anchor_root whenever the real
anchor body contained attestations, breaking BlocksByRoot reqresp
verification on receiving peers.

Changes:
- New endpoint /lean/v0/blocks/finalized returning SignedBlock SSZ
- ApiServer.signed_block_getter callable wired into the request app
- Checkpoint-sync client gains fetch_finalized_block() and
  fetch_finalized_anchor() returning the (state, signed_block) pair
  with pairing verification (state_root must match hash_tree_root(state))

Tests cover endpoint contract, error paths (404, network, corrupt SSZ,
mismatched pair) and client-server integration.

Closes leanEthereum#712.

* test(api): serialize sequential_posts_converge to fix Py3.14 macOS flake

asyncio.gather of 3 admin POSTs (T,F,T) raced server-side arrival order;
the 'converge to True' assertion only held when the third request landed
last. Switch to sequential await chain, matching the test's name and
intent. Drop now-unused asyncio import.
* feat: add validator sync-lag duty gate

* chore: inline logging

* refactor(validator): harden sync-lag gate per consensus + py review

Address review findings on the sync-lag duty gate.

Decision logic
- Replace peer-reported head-slot signal with local-store evidence:
  the freshest slot across blocks already validated into the store.
  Drops the unauthenticated peer.status.head.slot path entirely.
- Add NETWORK_STALL_THRESHOLD (= 2 * SYNC_LAG_THRESHOLD) distinct
  from the local threshold so jitter at the local boundary cannot
  also trip the network-wide branch.
- Add HYSTERESIS_BAND so a closed gate reopens only when lag drops
  to threshold - band, preventing slot-over-slot flap.
- Persist gate state on the service; log only on state transitions
  instead of every query.
- Saturate the future-head case at zero lag rather than trusting
  the chain unconditionally.

API and types
- duty parameter typed as Literal["block", "attestation"].
- Split _duties_skipped_lag into _blocks_skipped_lag and
  _attestations_skipped_lag, owned by the run loop. Attribution
  flattened so wrong-interval slots never tick the gate counter.
- Drop redundant int(slot) casts where Slot arithmetic suffices.
- Remove PeerManager.get_network_head_slot and its tests now that
  no caller remains.

Tests
- Replace patch.object(PeerManager, ...) mocks with real
  PeerManager and store manipulation, matching repo policy.
- New helpers preserve the block-map key-equals-root invariant.
- Add hysteresis flap test, split-counter test, transition-only
  log assertion. Use substring checks instead of exact log strings.

Documentation
- Constants, gate method, properties, fields, and inline comments
  rewritten per /doc rules: structured Why/Effect/Decision matrix
  labels, one idea per line, no function or variable names in
  prose, concrete numbers throughout.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(validator): use first-person voice in threshold rationale

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…leanEthereum#686) (leanEthereum#715)

* feat(forks): add SigScheme capability and @requires marker (Stage 7 of leanEthereum#686)

Introduces the first fork-level capability and a pytest marker to gate
tests on capability presence.

Capability
----------
- New `SigScheme` runtime-checkable Protocol in `forks/capabilities.py`
  asserts a `sig_scheme: ClassVar[GeneralizedXmssScheme]` attribute.
- `LstarSpec` binds `sig_scheme = TARGET_SIGNATURE_SCHEME` so
  `isinstance(LstarSpec(), SigScheme)` returns True.
- The three spec methods that previously took
  `scheme=TARGET_SIGNATURE_SCHEME` (`verify_signatures`,
  `on_gossip_attestation`, `on_block`) drop the parameter and read
  `self.sig_scheme` directly. The capability becomes the runtime source
  of truth.

Marker
------
- `requires(*capabilities)` pytest marker, registered in
  `pytest_plugins/filler.py`. Composes (AND) with the existing
  `valid_from` / `valid_until` / `valid_at` fork-range markers.
- `_check_markers_valid_for_fork` instantiates the active spec once and
  runs `isinstance(spec, capability)` per required capability.
- A `requires(...)` helper in `framework.markers` works around pytest's
  auto-detect-class shortcut (which trips on Protocol args to
  `@pytest.mark.requires(...)`).

Tests
-----
- 11 unit tests in `tests/lean_spec/forks/test_capabilities.py` cover
  the Protocol and the dispatch helper (composition with the
  fork-range markers, multiple `@requires` markers, error path for
  non-runtime_checkable Protocol).
- One smoke filler test in
  `tests/consensus/lstar/test_capability_gating.py` exercises the
  marker through pytest's live collection: one test marked with
  SigScheme runs, one marked with a synthetic absent capability is
  deselected.

Filler scheme override
----------------------
The three filler call sites that previously passed
`scheme=LEAN_ENV_TO_SCHEMES[self.lean_env]` to spec methods (in
`test_fixtures/fork_choice.py` and `test_types/block_spec.py`) drop the
kwarg. The PR description has the trade-off note and revert path if
that override is in fact needed somewhere we missed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks): tighten capability marker and apply review feedback

- Rename the marker helper to requires_capability and validate
  runtime-checkable Protocols at call time (fail at import, not
  at collection)
- Cache the fork spec instance in marker dispatch instead of
  constructing it once per test
- Re-export the capabilities namespace from lean_spec.forks so
  future capabilities don't need new import-site edits
- Register valid_from / valid_at / requires markers in pyproject
  so unit tests can build real pytest Marks under strict-markers
- Drop the hand-rolled Mark stand-in in tests; build real Marks
  via the MarkDecorator path; drop is True / is False on bool
  predicates
- Tighten docstrings per project style (no paragraph blocks, no
  backtick references, no internal-name references)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(forks): tighten dispatcher-guard test docstring

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>
* fix: generating leanSpec test assets

* Update prod-vectors.yml

---------

Co-authored-by: Unnawut Leepaisalsuwanna <921194+unnawut@users.noreply.github.com>
…#716)

* fix(forks/lstar): allow older-but-justified sources in build_block

The fixed-point loop required `att.source` to equal `current_justified`
exactly. That rejected legitimate gap-closing attestations whose source
is at or before the head chain's latest justified slot (e.g. a genesis
source when the head chain has already justified slot 1). Block
production aborted with "Fixed-point attestation loop did not converge"
when the store's justified checkpoint advanced via a sibling fork.

Replace the Checkpoint-equality with a slot bound: skip atts whose
source slot is past the current justified slot. This minimum change
unblocks the gap-closing path; the right shape of the broader filter
is left as an open question for review.

* test(forks/lstar): assert build_block closes the justification gap

Build a fork tree where the canonical head (block_5) lags behind the
store's justified checkpoint (block_2, advanced by sibling block_6).
Assert that produce_block_with_signatures succeeds, the produced
block's post-state catches up to block_2, and the body includes
block_6's gap-closing attestation.

* docs: fix example diagram

* fix(forks/lstar): tighten build_block filter with chain + target checks

Layer additional filters onto the source-slot bound so build_block
matches process_attestations more closely without re-introducing the
strict source-equality rule:

- Source and target roots must match the chain at their slots
  (historical_block_hashes for [0, parent.slot - 1], parent_root at
  parent.slot, ZERO_HASH for empty slots between parent and the
  candidate). Mirrors the STF's own source/target consistency check.

- Target slot must not already be justified on the chain, with one
  exception: the degenerate source.slot == target.slot == 0 genesis
  self-vote stays selectable so existing fixtures keep working. The
  STF still drops these, but they're tolerated in the block body.

Track the justified-slot bitfield and finalized slot through the
fixed-point loop so the target check stays accurate after each
iteration advances justification or finalization.

With these additions, the previously failing
test_build_block_skips_non_matching_source,
test_block_builder_fixed_point_advances_justification, and
test_block_builder_recovers_finality_after_non_zero_boundary_stall
all pass without modification.

* refactor(forks/lstar): drop redundant processed_att_data guard

Selecting an attestation already adds its data to processed_att_data,
and the cap check at the top of the loop short-circuits before doing
anything with already-processed entries. The explicit duplicate-check
before the add was redundant.

* docs(forks/lstar): tighten build_block filter comments

Trim the explanatory comments around the chain-match and
already-justified checks now that the surrounding code is settled.

* fix: rename test_build_block_skips_non_matching_source to better match actual test

* refactor(forks/lstar): extract attestation chain-match helper

Both process_attestations and build_block need to verify that an
attestation's source and target checkpoints reference the chain at
their respective slots. Lift that logic into
_attestation_data_matches_chain so the two call sites share a single
implementation.

build_block constructs the chain view that process_block_header would
produce on the candidate block (parent_state.historical_block_hashes
plus parent_root, plus ZERO_HASH for empty slots) and passes it to
the helper, instead of branching inline on source.slot and
target.slot.

* fix(forks/lstar): use HistoricalBlockHashes in chain-match helper signature

ty doesn't recognize the SSZ HistoricalBlockHashes container as a
Sequence[Bytes32]. Use the concrete type to satisfy the typechecker
without introducing structural-typing concessions.

* fix(forks/lstar): reject zero-hash source or target in chain match

Empty slots between blocks carry ZERO_HASH in historical_block_hashes.
An attestation whose source or target root is ZERO_HASH could
otherwise satisfy the chain-match helper by colliding with one of
those entries, even though it doesn't reference a real block. Inline
the explicit zero-hash rejection into the helper so both
process_attestations and build_block benefit.

* fix(forks/lstar): require source slot is actually justified

The previous filter only bounded source.slot by the latest justified
slot. That admits attestations whose source lies on an unjustified
slot before the latest justified one (gaps inside the bitfield).
Match the STF: skip the attestation unless its source slot is set in
the justified-slots bitfield.
…cates (leanEthereum#718)

* fix(forks/lstar): harden build_block filter against crashes and duplicates

Reorder filters so chain-match runs first.
A source slot beyond the candidate slot no longer crashes
the producer with IndexError on the justified-slot bitfield.

Restore the processed_att_data dedup guard.
Without it, attestations whose target fails to justify get
re-selected on every fixed-point iteration and appended as
duplicates to the candidate block body.

Add state-transition parity filters with a genesis-anchor bypass.
Target must be strictly after source.
Target must fall on a slot the state transition accepts.
Without these, gossip-pool entries the STF will silently drop
can starve real votes out of MAX_ATTESTATIONS_DATA.

Make the chain-match helper a staticmethod.
Swap its parameter order to (data, chain).
Widen its chain parameter to Sequence[Bytes32] so callers can
pass plain lists without reconstructing SSZ list instances.

Use full-Checkpoint equality in the new test per testing-style.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(forks/lstar): drop STF-parity filters that strip fork-choice signal

Remove two filters added in the previous commit:
- target.slot > source.slot
- target.slot.is_justifiable_after(current_finalized_slot)

Both are enforced by the state transition, but the state transition
is not the only consumer of body attestations. Block processing also
merges every body attestation into the known aggregated payload pool,
which is what fork choice reads for head votes.

Filtering these entries at production time hides head votes that the
producer's chain would otherwise have seen. The MAX_ATTESTATIONS_DATA
cap test and the deep-fork-split / reorg tests rely on this behavior:
without the filters they pass, with them they shrink the body or
move the head away from the expected subtree.

The chain-match-first reorder, the processed_att_data dedup guard,
the target-already-justified check with its genesis self-vote bypass,
and the helper signature changes are unaffected and remain.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eum#719)

* refactor(types): rename aggregation.py to participation.py

The module holds both AggregationBits and ValidatorIndices, which represent
validator participation in two equivalent forms (bitlist and index list).
"participation" reflects the unifying concept better than "aggregation",
which is narrower and only fits one of the two types.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(types): drop redundant bool() in to_validator_indices

Boolean inherits from int, so truthiness is already correct without an
explicit bool() conversion.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(types): add dedicated test_participation.py with full coverage

Moves the existing AggregationBits/ValidatorIndices tests out of
test_attestation_aggregation.py into a dedicated file mirroring the
source location at src/lean_spec/types/participation.py.

Adds previously-uncovered cases for ValidatorIndices.to_aggregation_bits:
empty input, index at and above VALIDATOR_REGISTRY_LIMIT, deduplication
of repeated indices, unsorted input ordering, and bitfield length.

Coverage of participation.py is now 100%.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ereum#720)

Establishes a coherent documentation system for the project:
- New /doc skill at .claude/skills/doc/SKILL.md with scope handling,
  standard section vocabulary, and a Python gold-standard exemplar.
- Expanded .claude/rules/documentation.md from 3 thin rules to a full
  checklist including one-sentence-per-line, bullet decomposition,
  structured labels, and a consolidated anti-pattern list.
- Refactored .claude/agents/doc-writer.md to defer atomic rules to the
  above files and keep only the consensus-domain patterns and persona.

Then applies the new system to bitfields.py as a live demonstration:
- Module + class docstrings use bullets and worked examples.
- encode/decode bodies use phase labels and bullet-decomposed bit math
  with concrete worked examples (byte-boundary crossings, delimiter
  spill, integer-interpretation mapping).
- Replaces the (N + 7) // 8 ceiling trick with math.ceil(N / 8).
- Replaces the byte-by-byte delimiter scan with a single
  int.from_bytes + bit_length call.
- Drops redundant tuple() wrap; passes list[Boolean] directly.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…est coverage (leanEthereum#721)

* refactor(types): tighten Boolean strictness, simplify dunders, improve test coverage

Boolean is now strict-by-default, matching the established BaseUint
pattern across SSZ primitive types:

- __eq__ and __ne__ now raise TypeError on any non-Boolean operand.
  The previous loose behavior (Boolean(1) == 1 returning True) silently
  bypassed the type contract; the asymmetry with bitwise ops (which
  already raised) was a real footgun.
- __new__ uses int(value) before the (0, 1) membership test so the
  new strict __eq__ does not fire when wrapping an existing Boolean.
- The four no-op arithmetic dunders (__add__/__radd__/__sub__/__rsub__)
  collapse into a single _no_arithmetic method with class-level
  aliasing. One source of truth, ~13 lines deleted.
- deserialize drops a redundant length check; decode_bytes already
  validates and produces the right error.

Tests:

- Coverage of boolean.py is now 100% (every line, every branch).
- Removed a duplicate test_strict_equality_with_same_type that was
  fully subsumed by the parametrized equality/inequality tests.
- Added gap-filling tests for: Boolean(Boolean(x)) construction,
  reverse bitwise dunders (int & Boolean etc), reflected __ne__,
  the Pydantic is_instance branch, and Pydantic serialization
  back to a plain bool.
- Every pytest.raises match= now asserts the full error message
  (via re.escape for dynamic strings, anchored r"^...$" for static).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(tests): suppress ty arg-type warning on Pydantic strict bool

The Pydantic model declares value: Boolean, so ty rejects passing
a raw True at the call site. Matches the existing suppression on
the strict-validation test above.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(boolean): align with /doc rules and bitfields.py style

Rewrites docstrings and inline comments to match the project's
documentation conventions used in bitfields.py:

- No backticks anywhere in docstrings or comments.
- One-sentence-per-line; bullet decompositions over prose paragraphs.
- Plain English in prose (lowercase "boolean") instead of class-name
  references in descriptive text.
- Class docstring becomes a 4-bullet behavior summary plus a wire-format
  example showing how true/false encode to bytes.
- __get_pydantic_core_schema__ body uses bullet decomposition of the
  two-step validator chain and the two-branch union schema.
- encode_bytes uses a raw string with single backslashes so help()
  renders b"\x01" correctly (was b"\\x01" before).
- decode_bytes and deserialize gain Args/Returns/Raises sections.

No behavior change; code is byte-identical. 72 tests still pass at
100% coverage.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(boolean): use bullet lists for multi-case Raises clauses

Converts the "If X, or Y" prose in decode_bytes and deserialize
Raises sections into two bullet points each, matching the
bullet-decomposition style used elsewhere in the file.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eanEthereum#722)

Brings the bitfields.py test suite to:

- 100% line and branch coverage (was 90%).
- Full-message matches on every pytest.raises via re.escape — both
  static strings and dynamic f-strings carrying class names, sizes,
  and element counts.

Coverage gaps filled:

- BaseBitvector validator: generator input materialized to a tuple.
- BaseBitlist validator: generator input materialized to a list.
- BaseBitlist validator: non-iterable input (int/None/float) and
  the str/bytes rejection branch.
- BaseBitlist.__getitem__ both int and slice branches.
- BaseBitlist.__add__ returning NotImplemented for unsupported types.
- BaseBitlist.decode_bytes "no delimiter bit found" for non-empty
  all-zero input.
- BaseBitlist.decode_bytes "exceeds limit of N" for an encoding
  whose recovered bit count is above LIMIT.

Duplicates removed:

- TestBitfieldSerialization.test_bitvector_serialization_deserialization
  and ..._bitlist_... were strict subsets of TestBitfieldSSZ's
  encode_decode tests. The TestBitfieldSerialization class is gone;
  its remaining decode-error tests moved into TestBitfieldSSZ.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tcoratger and others added 27 commits May 28, 2026 21:42
…m#789)

* refactor: drop model_copy in favour of in-place mutation

StrictBaseModel was previously frozen, which forced every state update to
go through model_copy(update={...}) and produced a verbose functional
style throughout the codebase. The frozen constraint was introduced as
groundwork for formal verification, which we are not yet using; in the
meantime the indirection hurts readability.

Changes:
- src/lean_spec/base.py: drop frozen=True from StrictBaseModel and update
  the docstring (no longer claims immutability).
- .claude/rules/ssz-patterns.md: drop "immutability" from the SSZModel
  principle bullet so the rule matches the new behavior.
- All 115 model_copy(update=...) call sites across src/, tests/, and
  packages/testing/ converted to direct field assignment. Intermediate
  dict-building variables inlined where the result was used only once.

Exceptions:
- The three JustifiedSlots collection helpers (with_justified,
  extend_to_slot, shift_window) still return new instances, but via
  type(self)(data=...) instead of model_copy. This preserves the
  return-new contract that callers rely on for SSZ collection methods.
- The api_endpoint test fixture's "return self.model_copy(update=handler(
  store, self))" pattern becomes a setattr loop over handler's output dict,
  then return self.
- MockStore in tests/lean_spec/node/chain/test_service.py loses its
  hand-rolled model_copy method (no longer needed; it's a plain dataclass
  and is mutable by default).

just check passes (ruff lint + format, ty type check, codespell, mdformat).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix: restore hashability and pure-function semantics where required

Dropping frozen globally lost two side effects the codebase relied on:
hashability of value types used as dict keys / set elements, and the
"returns a new state" contract that callers of process_slots and
state_transition were depending on. Both are restored surgically.

Hashability:
- Re-freeze the value types used as dict keys / set elements:
  Checkpoint, AttestationData, Attestation, Signature,
  TypeOneMultiSignature, TypeTwoMultiSignature, KeyPair,
  ValidatorKeyPair, Eth2Data. Per-class model_config override leaves
  the stateful types (Store, State, Block, BlockHeader, ENR, etc.)
  unfrozen and mutable.
- Add explicit __hash__ on Signature, TypeOneMultiSignature, and
  TypeTwoMultiSignature: their nested list-bearing fields break the
  auto-derived hash, so the SSZ-encoded bytes drive the hash instead.

Pure-function semantics:
- process_slots now deepcopies its input at entry, matching the
  existing docstring ("returns a new state with slot == target_slot").
  This makes state_transition and build_block trivially pure.
- Slot assertion relaxed from < to <= so process_slots is idempotent
  under repeated calls with the same target (build_block needs this).
- Two Checkpoint mutations in process_block_header replaced with
  construction now that Checkpoint is frozen again.

Test fixture in BlockSpec.build_signed_block_with_store deepcopies
the store at entry: the simulation pipeline (on_tick,
on_gossip_attestation, aggregate, accept_new_attestations) mutates
store directly, so the caller's store needs an explicit barrier.

Test fixes:
- _replace_head_at_slot and _add_block_at_slot in test_service.py
  construct new Block instances instead of mutating the original
  (the AST conversion had broken them).
- Corrupted-proof tests construct new TypeOneMultiSignature rather
  than mutating .proof on a frozen instance.
- test_combined_path_rejects_{depth_mismatch,odd_depth} construct
  new HashSubTree to avoid polluting the prf_trees fixture across
  tests.
- Dropped test_frozen_rejects_assignment on StrictBaseModel (the
  base itself is no longer frozen by design).
- Dropped the "store is not store_before" identity check that was
  only meaningful when on_gossip_attestation returned a new store.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(spec): restore strict process_slots assertion

Loosening the assertion to state.slot <= target_slot was a workaround
for the mutation cascade through build_block. Now that process_slots
deepcopies its input at entry, the caller's state.slot is never
advanced across repeated calls, so the original strict inequality
holds again. The spec filler test
test_process_slots_target_equal_to_state_slot_rejected was relying on
the strict form.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(forks/lstar): restore Checkpoint frozen override after merge

The merge from main clobbered the per-class `frozen=True` model_config
override on Checkpoint. That override was added in 24a0e7b alongside
the same override on eight other value types used as dict keys / set
members; Checkpoint was the only one that moved files in the merge
(the deleted lean_spec/types/checkpoint.py from leanEthereum#790 collided with the
new home spec/forks/lstar/containers.py from leanEthereum#785) and so it was the
only one whose override was lost.

Without the override AttestationData (which embeds Checkpoint) is no
longer hashable, and on_gossip_attestation crashes when it inserts
into store.attestation_signatures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…eanEthereum#792)

Five sites in spec.py rebuilt store dicts (and their inner sets) before
mutating them — defensive copies left over from the model_copy era. Since
fb5ff3c the store is mutated in place, so these rebuilds protect nothing
and actively mislead readers into thinking the store is immutable.

The aggregate() filter is replaced by an in-place pop loop with identical
semantics.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… per-bit loop (leanEthereum#793)

Both BaseBitvector.encode_bytes and BaseBitlist.encode_bytes used a
Python for-loop to set each bit of a bytearray. For a 4096-validator
AggregationBits that is 4096 Python iterations per encoding.

Use the same trick already in merkleization.py: build the packed bits
as a single Python int, then call int.to_bytes once. The bit-to-byte
split runs in C.

For BaseBitlist, the delimiter folds into the integer construction as
bit num_bits, and the byte-spill special case disappears — the byte
width ceil((num_bits + 1) / 8) already covers it.

The on-the-wire format is unchanged; existing decode paths and SSZ
round-trip tests cover the new encoders unchanged.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eanEthereum#794)

The previous behavior silently popped mode and by_alias from kwargs so the
serializer could pin them to "json" and True. A caller passing either
almost certainly expected the override to apply, so silently honoring the
pin produced output that did not match their request — a footgun.

Reject the kwargs with a clear TypeError instead. Other kwargs still
forward unchanged.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…chemas (leanEthereum#795)

Four field validators across the codebase each rebuilt the same
"if str, decode hex" logic on top of a Pydantic field. Promote the
parsing to the schema layer so it happens once per type.

- BaseBytes.__get_pydantic_core_schema__: add a str branch that
  routes any string through the constructor.
- Container: add a model_validator(mode="wrap") that decodes a hex
  string input via from_hex, with other shapes passing through
  field-by-field validation.

Removes:
- validator/registry.py parse_pubkey — covered by the Bytes52 schema.
- xmss/containers.py _decode_public_key, _decode_secret_key — covered
  by the Container wrap validator.

Simplifies:
- genesis/config.py keeps only the YAML quirk where an unquoted
  0x-prefixed value is parsed as an int.

Adds TestHexStringValidator in test_container.py covering every
branch of the new wrap validator: hex success paths with prefix and
case variants, empty-hex edge case, dict and instance pass-through,
the class-name-tagged error on wrong length, and the nested-container
field case.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…leanEthereum#796)

* refactor(forks/lstar): move multi-signature types out of crypto layer

The crypto subspec imported AggregationBits, Slot, ValidatorIndex, and
ValidatorIndices from the consensus layer to define TypeOneMultiSignature
and TypeTwoMultiSignature. That layering inversion forced three mid-file
deferred imports with # noqa: E402 in lstar/containers.py and made the
crypto layer reach into consensus for types it should never know about.

Move the multi-signature classes (plus AggregationError) into the
consensus layer, where they belong as domain-typed wrappers around the
crypto byte-level primitives. The crypto layer keeps only the Rust
prover bindings.

Slot moves into its own small module so the crypto layer can name a
slot without pulling the full consensus container module. The crypto
API still uses Slot, not Uint64 — the layering fix preserves semantics.

Net result:
- crypto/xmss/aggregation.py shrinks from 348 lines to 38; no consensus
  imports remain.
- crypto/xmss/{containers,interface}.py import Slot from lstar/slot.py.
- lstar/containers.py promotes all three previously deferred imports
  (HISTORICAL_ROOTS_LIMIT, multi-sig types, PublicKey/Signature) to the
  top of the file. The # noqa: E402 block is gone.
- 17 importers updated to fetch TypeOneMultiSignature etc. from the
  consensus layer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(crypto/xmss): delete the aggregation shim

After the multi-signature containers moved to the consensus layer, the
aggregation module shrank to a re-export shell with one import-time
side effect. With exactly one consumer (lstar/containers.py), the
indirection added cost without abstraction.

Distribute the three concerns:

- Rust binding imports go directly into lstar/containers.py from
  lean_multisig_py.
- LOG_INV_RATE moves next to its sole caller in lstar/containers.py.
- setup_prover(mode=LEAN_ENV) moves into crypto/xmss/__init__.py,
  alongside the other LEAN_ENV-driven xmss bootstrap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eanEthereum#797)

The sync and validator services defaulted their publish callbacks to
no-op async functions used purely as sentinels. Replace them with
optional callables defaulting to None and guard each call site.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…es (leanEthereum#798)

The networking and snappy modules each carried a self-contained LEB128
implementation. The two algorithms were identical, differing only in
byte cap (10 vs 5). One implementation was a slow drift away from the
other waiting to happen.

Keep the networking varint as the single source of truth and give both
encode and decode a max_bytes parameter (default 10 = uint64 cap).
Snappy now imports the canonical codec and passes 5 via a new
SNAPPY_VARINT_MAX_BYTES constant.

Encode now also enforces the cap, so per-cap semantics are symmetric.
The snappy length-prefix tests run against the canonical codec, plus
an integration test that asserts an oversize prefix surfaces as a
SnappyDecompressionError.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ure types (leanEthereum#799)

* refactor(forks/lstar): type SignedBlock.proof as TypeTwoMultiSignature

The field was typed as raw ByteList512KiB and producers manually called
encode_bytes before storing the result; verify_signatures and the
post-block split path then decode_bytes-ed it back. Promote the field
to its real type so SSZ handles the (de)serialization once at the
container boundary.

- Removes the encode-then-wrap dance in the validator service and test
  builders.
- Removes the try/decode_bytes/except blocks in verify_signatures and
  the sync service post-block handler.
- Updates SSZ round-trip tests, fork choice tests, and reqresp client
  helpers to construct the typed envelope directly.
- Deletes four tautological decode-smoke assertions in test_service.py
  now that the Pydantic-validated field guarantees the shape.

The on-wire SSZ encoding of SignedBlock is unchanged; hash_tree_root
will change because Container merkleization differs from List[byte]
merkleization, so consensus fixtures that hardcode block roots need
regeneration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks/lstar): inline single-use type_two alias

Both call sites used the local alias exactly once. Drop it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(forks/lstar): rename multi-signature types to SingleMessageAggregate / MultiMessageAggregate

TypeOneMultiSignature and TypeTwoMultiSignature were named after the
underlying Rust prover convention, not what they semantically are.
Rename them to express the actual data: an aggregate over a single
message versus an aggregate over many distinct messages.

- Classes: TypeOneMultiSignature -> SingleMessageAggregate,
  TypeTwoMultiSignature -> MultiMessageAggregate
- Variables and locals follow the snake_case form:
  type_1 -> single_message_aggregate, type_2 -> multi_message_aggregate
  (plus the compound forms type1_wire, type2_wire, block_t1,
  proposer_type_1, etc.)
- Test function names match: test_type_one_* -> test_single_message_*,
  test_type_two_* -> test_multi_message_*
- Prose mentions of "Type-1" / "Type-2" in docstrings and comments
  become "single-message aggregate" / "multi-message aggregate"

External Rust binding names (aggregate_type_1, verify_type_1,
merge_many_type_1, split_type_2_by_msg, verify_type_2_with_messages)
come from the leanMultisig-py package and are left untouched.

The "Type 1" / "Type 2" mentions in node/snappy/encoding.py refer to
Snappy's Copy Type 1 / Copy Type 2 wire encodings and are unrelated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eum#800)

* refactor: expand all abbreviated identifiers to full words

A reference specification should read as explicitly as possible, so every
abbreviated identifier is spelled out in full across src, tests, and the
packages/ testing framework (~280 distinct renames).

Highlights:
- validator_id -> validator_index (the domain-correct term)
- att -> attestation, msg -> message, sig -> signature, sk -> secret_key,
  pk/pubkey -> public_key, idx -> index, prev -> previous, agg -> aggregate,
  prop -> proposal, conn -> connection, privkey -> private_key, len -> length
- function/method/class names too (split_by_msg -> split_by_message,
  Pubkey -> PublicKey, get_attestation_pubkey -> get_attestation_public_key)
- MSG_LEN_FE -> MESSAGE_LENGTH_FIELD_ELEMENTS

Kept verbatim: canonical protocol IDs (peer_id, node_id, protocol_id,
subnet_id, stream_id), ENR fields (attnets, seq), num_* prefixes, the
reqresp protocol name, library APIs (argparse dest, model_config, tmp_path),
and external symbols (e.g. lean_multisig_py functions like split_type_2_by_msg).

Internal config/fixture string keys were updated to match the renamed
fields (genesis YAML keys, validator registry manifest keys, one metrics
coverage-section label). Test assertions/fixtures referencing identifiers
via strings (parametrize names) were aligned.

Adds a NO ABBREVIATIONS IN IDENTIFIERS rule to CLAUDE.md documenting the
convention and its canonical exceptions.

just check passes (ruff, format, ty, codespell, mdformat).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(tests): correct XmssKeyManager __slots__ entry after rename

The attribute self._keys_dir was renamed to self._keys_directory, but the
matching __slots__ string entry must be renamed too -- otherwise setting the
attribute raises AttributeError (slotted classes have no __dict__).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…um#801)

Move AttestationSignatureEntry and Store next to the other lstar
container types, then delete the standalone store module so the fork
exposes a single container namespace.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…heck proof verification (leanEthereum#786)

* test(consensus): add VerifyProofsTest fixture and Type-1 valid vectors

Introduces a new consensus fixture format that emits self-contained
multi-signature verification vectors so cross-client implementations
can run their own Type-1 verifier and compare outcomes.

Three positive vectors land alongside the fixture: a single-validator
baseline, a four-validator all-participating case, and a
four-validator non-contiguous bitfield case ([1, 0, 1, 1]).

The fixture also surfaces the spec-layer binding between attestation
data and proof: clients recompute hash_tree_root(attestation_data)
and must match the emitted message field before running the verifier.

* test(consensus): add Type-1 verify_proofs rejection vectors

Adds four negative vectors exercising spec-layer bindings between
inputs and the multi-signature proof. Each vector uses a tamper
operation on the fixture to produce a structurally valid bundle that
must be rejected by a conformant verifier:

- wrong_message: proof bound to an alternate head root inside the
  attestation data
- wrong_slot: emitted slot field shifted while the proof binding
  stays on the original slot
- wrong_public_keys: one emitted pubkey replaced with another
  validator's
- aggregation_bits_length_mismatch: emitted bits truncated while
  the pubkey count stays unchanged

Vectors covering malformed or truncated proof bytes are intentionally
out of scope: leanSpec consumes the multi-signature primitive as a
black box and primitive integrity belongs to its own conformance
suite. Pubkey ordering is also not a binding to test: the aggregator
sorts participants internally, so the verifier is order-insensitive.

* test(consensus): align VerifyProofsTest with sibling fixture conventions

Brings the new fixture in line with the patterns the other consensus
test fixtures follow:

- Drop ``from __future__ import annotations`` (PR leanEthereum#759 removed it from
  Pydantic-defining files); quote the one self-reference instead.
- Replace the bespoke ``expect_valid: bool`` field with the inherited
  ``expect_exception`` field already used by SSZTest, NetworkingCodec,
  and VerifySignaturesTest. Rejection vectors now pin
  ``AggregationError`` and the framework serializes the class name to
  JSON.
- Switch the tamper dispatch in ``_apply_tamper`` from ``if/elif`` to
  ``match/case`` to follow the pattern in slot_clock and
  networking_codec.
- Expand the module-level docstring from one line to a short
  paragraph describing what the fixture covers.
- Normalize the ``public_keys`` default from ``[]`` to ``| None =
  None`` to match every other output field on the model.

* test(consensus): drop aggregation_bits_length_mismatch rejection vector

The check that fires here is the early-reject in the spec wrapper's
verify method (len(public_keys) != participants.count(True)), not a
consensus-critical binding. In real consensus the inconsistency cannot
arise because clients resolve public keys from the bitfield plus the
validator registry as one operation. A client that did pass a wrong
pubkey count would also be rejected by the underlying recursive
verifier on its internal pubkey-set commitment, so the wrapper check
is at best an early exit with a nicer error message.

The remaining three rejection vectors still exercise the meaningful
spec-layer bindings: message hash, slot, and pubkey set.

* refactor(consensus): tighten VerifyProofsTest and absorb post-rebase renames

Rebase onto main and apply the renames/cleanups landed since the branch
was opened:

- PR leanEthereum#799: TypeOneMultiSignature → SingleMessageAggregate, proof_type
  literal "type_1" → "single_message", file renames
  test_type_1_{valid,invalid}.py → test_single_message_{valid,invalid}.py.
- PR leanEthereum#800: validator_ids → validator_indices, with_validator_id →
  with_validator_index, vid/pubkey expansions.
- post-leanEthereum#788/leanEthereum#790/leanEthereum#796 imports: lean_spec.spec.forks / .spec.ssz /
  .spec.crypto.*.

Replace the stringly-typed tamper dict with a Pydantic discriminated
union (RebindToAlternateHeadRoot, IncrementEmittedSlot,
SwapParticipantPublicKey). The match dispatch on the typed variants
drops the two type: ignore[index] casts and lets the test sites read
tamper=SwapParticipantPublicKey(index=0, with_validator_index=1)
instead of a magic-string dict.

Inline the four single-call helpers (_apply_tamper plus three
_tamper_*) and the verification helper into make_fixture so the four
phases — generate / tamper / verify / publish — are visible in one
method.

Drop both model_copy(update=...) calls per leanEthereum#789: direct field
construction for the frozen AttestationData rebuild, direct field
assignment for the mutable fixture self-update.

Replace the internal dict[str, Any] bundle with named locals; the
bundle never escapes make_fixture, so the dict adds nothing.

Guard SwapParticipantPublicKey against silent no-op swaps where the
replacement key happens to equal the original. Broaden the verifier
exception catch to surface unexpected exception types as
"expected X got Y" instead of crashing the filler.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…eum#802)

* fix stale finalized source finalization

* fix: rebase onto main and correct finalization test

Rebase the stale-source finalization fix onto current main:
the spec file moved under the spec/ package and attestation specs
renamed validator_ids to validator_indices.

Also fix the new test expectation. Building justified_slots via
model_copy(update=...) bypassed Pydantic validation, leaving data
as a list instead of a tuple, so the post-state comparison failed
even with the fix applied. Use the canonical constructor instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: pin source-at-finalized-boundary finalization case

Add the boundary case raised in review: a source whose slot equals
the finalized slot. Such a source is already final, so it may justify
a newer target but must never re-finalize. This locks the > (not >=)
intent of the finalization-advance guard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…hereum#803)

Expands the `exc` abbreviation used in `except X as exc:` clauses to
the full word `exception`, matching the no-abbreviations rule in
CLAUDE.md. Adds the rule explicitly so future code keeps the full
spelling.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…eanEthereum#804)

The store protocol declared from_anchor, on_block, on_gossip_attestation,
and on_gossip_aggregated_attestation, but the concrete Store is a plain
data model that implements none of them. Those operations live on the
fork spec and are invoked as spec.on_block(store, ...), so the
declarations were dead and contradicted the comment claiming the store
conforms structurally.

Remove the four declarations along with the three SignedBlock /
SignedAttestation / SignedAggregatedAttestation structural protocols that
only existed to type those signatures and were referenced nowhere else.
SpecStoreType now exposes only the read surface the store actually
provides, making the structural-conformance comment true.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eum#805)

The diagnostic block-weight computation and the LMD-GHOST head walk both
climbed from each validator's head vote up through its ancestors,
accumulating one unit of weight per visited block above a start slot. The
two loops were byte-identical apart from the start slot, so the API's
reported weights could silently diverge from the actual head decision if
one copy changed.

Extract the walk into a single helper parameterized on the start slot and
call it from both. No behavior change.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nEthereum#806)

The state transition function took a valid_signatures flag whose only
non-default value raised immediately, so any path that proceeded had it
set to True. The sole caller passed the result of signature verification,
which already returns True or raises. The flag therefore implied the state
transition could optionally skip signature checks, when in fact it never
inspects signatures at all.

Remove the parameter and its dead branch. Signature verification stays
where it belongs, before the call, and aborts the import on failure. Update
the docstring and the test and fixture call sites accordingly.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ereum#808)

Anchoring the genesis block as justified and finalized swapped in the
parent root but kept the pre-state checkpoint slot. That was correct only
because both checkpoint slots are still zero when the first post-genesis
block is processed, a hidden dependency a reader had to reconstruct.

Write the slot as zero directly at both the header-processing anchor and
its mirror in the block builder, so the genesis anchor reads as a fixed
slot-zero checkpoint. No behavior change.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…le_message_proofs (leanEthereum#811)

The verify_proofs/ directory now houses both single-message (Type-1) and
multi-message (Type-2) primitive verification vectors. Rename the Type-1
fixture format, class, and test signatures so they sit symmetrically next
to the incoming Type-2 fixture.
* refactor(forks/lstar): small Pythonic cleanups

Five no-behavior-change simplifications in the lstar spec:

- Drop a redundant ternary guarding the justifications dict comprehension;
  enumerating an empty root list already yields an empty dict.
- Replace a manual root-to-slot loop with the equivalent dict comprehension.
- Convert the lookback count to int before range() instead of relying on
  the implicit integer coercion, matching the int() usage elsewhere.
- Add a constructor that builds aggregation bits straight from a set of
  validator indices, route the existing index-list conversion through it,
  and use it in the prover path to avoid building a throwaway index list.
- Trim the per-slot housekeeping comment to its load-bearing invariant: the
  state root is cached at most once per block, on the first empty slot.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(forks/lstar): drop the to_aggregation_bits wrapper

With the index-set logic now living in the aggregation-bits constructor,
the conversion method on the index list was a pure delegating wrapper.
Most of its call sites also built a throwaway index list purely to call it.

Remove the wrapper, widen the constructor to accept any iterable of
indices, and call it directly everywhere, dropping the intermediate index
lists. The conversion test suite now exercises the constructor directly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…um#810)

* test(testing): add VerifyMultiMessageProofsTest fixture

Mirror the single-message fixture for the Type-2 multi-message aggregate
primitive: per-component validator lists, per-component attestation data,
emitted parallel lists of messages, slots, public keys, and aggregation
bits, plus the merged proof bytes.

Three tampers target one component at a time:
- RebindComponentToAlternateHeadRoot regenerates one component against an
  alternate head root and re-merges, so the emitted layout stays honest
  but the merged proof bytes carry the off-target binding.
- IncrementComponentSlot bumps one component's emitted slot past its
  bound slot.
- SwapComponentParticipantPublicKey swaps one participant's key for
  another validator's, breaking the layout the merged proof verifies
  against.

* test(consensus): add multi-message verify vectors for lstar

Three positive vectors covering two- and three-component bundles with
single-, four-, and mixed-sized participant lists; three rejection
vectors covering wrong message, wrong slot, and wrong public key applied
to one component at a time.

* test(testing): cover multi-message-specific rejection paths and dedup the verify ladder

Add the failure modes that only a multi-message (Type-2) proof can suffer,
which the initial vector set did not reach:

- SwapComponentMessageBindings transposes two components' emitted
  message-slot bindings after an honest merge, with distinct head roots per
  component, so each component's proof faces the other's binding. This is the
  canonical Type-2 attack the positional binding exists to reject.
- DropComponentMessageBinding removes one component's binding while keeping
  its keys, exercising the verifier's binding-count guard.

Add two valid vectors for parity with the single-message suite: a single
component bundle (the n=1 boundary) and a non-contiguous committee whose
aggregation bits resolve to [1, 0, 1, 1].

Factor the identical expectation-comparison ladder into
BaseConsensusFixture.assert_expected_outcome and reuse it from both verify
fixtures. Hoist the repeated component-index range check into a helper, and
guard the slot-increment tamper against landing on a neighbour's slot.

Inline the per-test attestation data to match the single-message sibling
files, rename the ambiguous swap field to participant_index, and correct the
union docstring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Thomas Coratger <60488569+tcoratger@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add attestation proof order fixture

* test: remove banner-separator comments

The code style bans banner-style separator comments, since they add visual
clutter and blank lines already mark logical sections. Drop the 63 pure
dash and equals divider lines across the QUIC, service, and fork-choice
test files, keeping the section heading text as plain single-line comments.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Adam Mohammed A Latif <latifkasuli@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
leanEthereum#813)

The multi-message proof fixture still built a throwaway index list and
called the conversion method that was removed when the index-to-bits logic
moved onto the aggregation-bits constructor. The two changes landed in a
merge order that left a call to the deleted method, so type checking failed
on the main branch.

Build the participation bitfield directly from the indices and drop the now
unused import.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapse a multi-line AggregationBits.from_indices call onto a single
line to satisfy ruff format.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…um#815)

Move PoseidonParams field descriptions into attribute docstrings,
matching the convention used elsewhere in the codebase.

Replace the rounds_f even-count field validator with the native
multiple_of=2 constraint, preserving the rationale in the field
docstring and dropping the now-unused field_validator import.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Break the single large LstarSpec class into one file per concern (state
transition, fork choice, block production, signatures, aggregation, timeline,
validator duties), assembled back into LstarSpec via mixins over a shared
typing contract (LstarSpecContract).

Pure structural move: every method body is byte-for-byte identical, with no
change to any external call site or type annotation. The only edits are
relocating the chain-match static helper to a module-level function and
moving the LstarStore alias into the contract module (still re-exported from
spec.py).
@leolara leolara force-pushed the refactor/lstar-spec-mixins branch from c69249b to c93eed2 Compare June 1, 2026 12:47
@leolara
Copy link
Copy Markdown
Owner Author

leolara commented Jun 1, 2026

Superseded by leanEthereum#817, which targets upstream main with a clean 9-file diff. This fork-internal PR showed a large diff only because the fork's main was stale.

@leolara leolara closed this Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.