refactor(spec-tools): use testing pydantic models in t8n#2924
Open
gurukamath wants to merge 7 commits into
Open
refactor(spec-tools): use testing pydantic models in t8n#2924gurukamath wants to merge 7 commits into
gurukamath wants to merge 7 commits into
Conversation
Add a CONSTRUCTION/LIVE/FROZEN lifecycle to the testing-side Alloc so it directly satisfies ethereum.state.PreState. The first PreState read transitions the alloc to LIVE and rejects further __setitem__/ __delitem__; apply_diff(BlockDiff) is the sole mutation entry point in LIVE, and freeze() locks the allocation for assertion use. Groundwork for the t8n refactor: with Alloc directly usable as a PreState, fork.BlockState(pre_state=alloc) works without an adapter, and t8n can drop its bespoke Alloc/Env/Result/Txs JSON types.
T8N now wires the testing-package types end-to-end:
* __init__ accepts an optional ``t8n_data`` and otherwise parses the
JSON inputs into testing ``Alloc``/``Environment``/``Transaction``
via ``model_validate``. The CLI ingress is structured so the future
in-process lift only needs to swap the caller, not the constructor.
* ``env.py`` drops the bespoke ``Env`` class in favour of
``build_block_environment(fork, env, pre_state, chain_id, ommers,
state_test)`` plus a handful of ``_resolve_*`` helpers. ``Ommer``
stays as a small dataclass for the pre-PoS reward path.
* ``convert_transaction`` routes through ``TransactionLoad`` rather
than ``rlp.decode_to`` so contract-creating typed txs (Blob /
SetCode with ``to=null``) construct successfully and let
``check_transaction`` raise the canonical
``TransactionTypeContractCreationError``.
* The bespoke ``Result`` / ``Txs`` / ``Alloc`` classes are gone;
``build_result`` and ``get_receipts_from_output`` produce a
``cli_types.Result`` directly, and ``T8N.run()`` emits the
``TransitionToolOutput``-shaped JSON.
* The per-tx ``backup_state`` / ``restore_state`` pattern disappears
with the snapshot-based State: a failed ``process_transaction`` no
longer reaches ``incorporate_tx_into_block``, so ``BlockState`` is
untouched without explicit rollback. After execution the block
diff is applied in-place via ``Alloc.apply_diff``.
JSON ingress smooths two boundary mismatches: ``yParity`` on auth
tuples (duplicated by the testing serializer, rejected by the
validator) and ``secretKey`` left on already-signed txs (rejected by
``InvalidSignaturePrivateKeyError``). Unsigned txs with only a
``secretKey`` are signed post-validation; pre-Spurious-Dragon forks
get ``protected=False`` so the v-value stays in {27, 28}.
The testing-side EELS caller no longer marshals the input through a JSON ``StringIO`` and back. ``_evaluate`` now hands the testing ``TransitionToolInput`` directly to ``T8N`` via the existing ``t8n_data`` kwarg and assembles the ``TransitionToolOutput`` from ``T8N``'s in-memory ``alloc``/``result``/``body``. To make the in-process path symmetric with the CLI path, ``T8N``: * pulls ``blob_params`` from ``t8n_data.blob_params`` (camelCase dump matches the existing parse), so BPO-fork blob schedules don't have to be re-serialized through ``--input.blobParams=stdin``;
T8N now takes a testing ``TransitionTool.TransitionToolData`` and nothing else from the JSON/CLI surface. The CLI plumbing (``argparse`` namespace, ``--input.*``/``--output.*`` flags, stdin, file paths, tracer construction from CLI flags) lives in a new ``t8n.cli`` module: * ``build_t8n_from_cli_options(options, in_file, cache) -> T8N`` reads the JSON inputs (stdin / files), validates each piece into testing pydantic types, resolves the fork, bundles everything into a ``TransitionToolData``, builds tracers from the CLI flags, and hands them to ``T8N``. * ``write_t8n_outputs(t8n, output, options, out_file)`` serialises the t8n output + opcode counts per ``--output.*``. * ``run_t8n_cli(options, out_file, in_file, cache) -> int`` chains the two for the CLI entry point. ``T8N`` internally calls ``resolve_fork(t8n_data.fork_name, t8n_data.env)`` to translate the testing-side fork name into a spec ``Hardfork`` + optional ``ByBlockNumber`` criteria (handles both canonical names and CLI exception aliases like ``Paris``, ``ConstantinopleFix``, ``HomesteadToDaoAt5``). ``T8N.run()`` returns the ``TransitionToolOutput`` directly — no more out_file writing. Callers updated: * ``evm_tools.__init__.main`` now calls ``run_t8n_cli``. * ``statetest`` and ``tests/json_loader`` use ``build_t8n_from_cli_options``. * ``tests/evm_tools/test_count_opcodes`` uses ``run_t8n_cli``. * ``ExecutionSpecsTransitionTool._evaluate`` hands its ``transition_tool_data`` straight to ``T8N`` — no argparse dance. The CLI ↔ testing fork-name mapping is title-case + a one-entry override for ``DAOFork`` (testing's irregular capitalisation). ``state_reward=None`` is resolved to the fork's ``BLOCK_REWARD`` (or ``-1`` for PoS forks) in the wrapper before constructing ``TransitionToolData.reward: int``.
``Alloc`` used to maintain its own parallel ``State`` dataclass plus ``set_account``/``set_storage``/``state_root``/``storage_root`` free functions to compute its root. Now that ``Alloc`` implements the ``PreState`` protocol, ``state_root()`` can route through ``_materialize_state()`` and ``ethereum.state.state_root``, so the in-package trie machinery is redundant. * ``Alloc.state_root()`` reduced to a one-liner over ``spec_state.state_root(self._materialize_state())``. The materialize call doesn't transition the alloc out of ``CONSTRUCTION``, so existing callers that compute a genesis root and then keep mutating the alloc are unaffected. * Local ``State`` dataclass + trie helpers (``set_account``, ``set_storage``, ``storage_root``, ``state_root``) removed; they had no consumers outside the deleted ``Alloc.state_root`` body. * ``test_types/trie.py`` deleted along with its now-tautological ``test_eest_trie_keccak256_matches_eels`` keccak-dispatch check (the module just re-exported ``ethereum.crypto.hash.keccak256``).
Contributor
Author
|
Note to reviewers: I have already run |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## forks/amsterdam #2924 +/- ##
===================================================
- Coverage 90.44% 90.33% -0.11%
===================================================
Files 535 535
Lines 32439 32430 -9
Branches 3012 3012
===================================================
- Hits 29338 29296 -42
- Misses 2573 2613 +40
+ Partials 528 521 -7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
gurukamath
commented
May 27, 2026
| """ | ||
| Convert a testing ``Transaction`` into the fork's tx object. | ||
|
|
||
| TODO: Replace with ``self.fork.decode_transaction(tx.rlp())`` |
Contributor
Author
There was a problem hiding this comment.
Transactions are the only part of the t8n which use some sort of a json round trip workaround in this PR. In order to avoid this workaround (and fully make use of the rlp that the testing framework generates) we will need to make some small changes on the specs side, particularly wrt. the decode_transaction function. I aim to tackle this in a subsequent PR.
See #2925
petertdavies
requested changes
May 28, 2026
Contributor
petertdavies
left a comment
petertdavies
left a comment
There was a problem hiding this comment.
This looks great! I have a few minor comments.
petertdavies
approved these changes
May 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🗒️ Description
The PR re-shapes the boundary between the testing package and
ethereum_spec_toolsso that theT8Ntransition tool becomes a smallJSON-free engine that consumes testing pydantic types directly. Six
commits, one architectural thread per commit:
Allocbecomes aPreState(24b3e4d492c). The testing-sideAllocnow implementsethereum.state.PreState(
get_account_optional,get_storage,get_code,account_has_storage,compute_state_root_and_trie_changes),removing the need for a bespoke adapter between testing's
allocation model and the spec's state-transition entry point. A
_Phasemachine (CONSTRUCTION → LIVE → FROZEN) governs when freemutations are allowed; once any
PreStatemethod is called, thealloc transitions to
LIVEand onlyapply_diffmay mutate it.T8Nrewritten to consume testing pydantic types(
11eeda46d97). The bespokeAlloc/Env/Result/TxsJSON-parsing classes (t8n_types.py, 440 lines)are deleted.
T8Nno longer parses argparse output into customcontainers — it accepts a
TransitionTool.TransitionToolDatadirectly.
In-process EELS path stops round-tripping through JSON
(
f71df5daa49).ExecutionSpecsTransitionTool._evaluatehands itsTransitionToolDatastraight toT8Nand assembles theTransitionToolOutputfromT8N's in-memory state.CLI surface extracted into
t8n/cli.py(5e366b57c9a). Argparse setup,--input.*/--output.*flags,stdin/file readers, tracer construction from CLI flags, and the
t8n-output JSON serialization all live in a new module. Core
T8Nknows nothing about argparse, JSON, or stdin/stdout.
Duplicate testing-package state machinery removed
(
84a416cab69). The parallelStatedataclass andset_account/set_storage/state_root/storage_rootfreehelpers in
account_types.pydeleted.Alloc.state_root()nowroutes through
spec_state.state_root(_materialize_state()). Thetesting package no longer maintains a fork of trie/state plumbing.
Final cleanup pass (
3ac3e80300b) - Small miscellanous clean-ups.🔗 Related Issues or PRs
#1883
#1359
✅ Checklist
just statictype(scope):.mkdocs servelocally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.Cute Animal Picture