test: Match blockchain test exceptions#1556
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1556 +/- ##
==========================================
- Coverage 97.09% 97.07% -0.02%
==========================================
Files 162 162
Lines 14440 14480 +40
Branches 3370 3380 +10
==========================================
+ Hits 14021 14057 +36
- Misses 303 305 +2
- Partials 116 118 +2
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
chfast
commented
May 29, 2026
chfast
force-pushed
the
test/block-validation-errors
branch
2 times, most recently
from
094b74f to
ecc26f5
Compare
There was a problem hiding this comment.
Pull request overview
This PR tightens blockchain test validation by requiring invalid blocks to fail for the expected reason (matching fixtures’ expectException), rather than treating “any rejection” as a pass. It introduces block-level error codes and threads the expected exception through loading and execution so the runner can compare actual vs. expected failure reasons.
Changes:
- Replace
TestBlock::validwithTestBlock::expected_exception(empty means valid). - Change
validate_block()to returnstd::error_codewith new block-levelstate::ErrorCodevalues whose messages match EESTBlockException.*constants. - Map legacy ethereum/tests exception names to the EEST constants during fixture loading, and substring-match the resulting expected exception against the actual validation error message in the runner.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
test/utils/blockchaintest.hpp |
Switches block validity signaling from a boolean to an expected_exception string (empty => valid). |
test/utils/blockchaintest_loader.cpp |
Captures expectException and rewrites legacy exception names to the BlockException.* constant evmone reports. |
test/state/errors.hpp |
Appends block-level ErrorCode entries and maps them to BlockException.*-style messages via the error category. |
test/blockchaintest/blockchaintest_runner.cpp |
Uses std::error_code results from validate_block() and checks invalid blocks’ rejection reasons against fixture expectations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
chfast
force-pushed
the
test/block-validation-errors
branch
from
ecc26f5 to
90571ec
Compare
The blockchain test runner treated block validity as a bool: for a block expected to be invalid it accepted ANY rejection and never checked that the reason matched the fixture's `expectException`. So a block rejected for the wrong reason still passed. Make `validate_block` return a `std::error_code` instead of `bool`, with new block-level entries appended (at the back, keeping existing values stable) to `state::ErrorCode`. Each entry's message is the matching execution-spec-tests `BlockException` constant (INVALID_GASLIMIT, INVALID_BASEFEE_PER_GAS, INCORRECT_EXCESS_BLOB_GAS, RLP_BLOCK_LIMIT_EXCEEDED, INVALID_BLOCK_TIMESTAMP_OLDER_THAN_PARENT, INVALID_BLOCK_NUMBER, INCORRECT_BLOCK_FORMAT for the rest), except the missing-parent check which uses evmone's own INVALID_BLOCK_PARENT (a broader condition than EEST's UNKNOWN_PARENT). The loader captures the `expectException` string; the runner substring-matches the actual error's constant against it, which also handles the fixtures' `|`-separated alternatives. Legacy ethereum/tests fixtures use older, sometimes finer-grained exception names (e.g. uncles after Paris, a number gap, a wrong base fee). The loader maps those (many-to-one) onto the `BlockException` constant evmone reports, so they match by reason as well.
chfast
force-pushed
the
test/block-validation-errors
branch
from
90571ec to
dc1ce13
Compare
Comment on lines
+392
to
396
| EXPECT_NE(test_block.expected_exception.find(block_error.message()), | ||
| std::string::npos) | ||
| << "Block invalidity reason mismatch: got " << block_error.message() | ||
| << ", expected " << test_block.expected_exception; | ||
| continue; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The blockchain test runner treated block validity as a bool: for a block expected to be invalid it accepted ANY rejection and never checked that the reason matched the fixture's
expectException. So a block rejected for the wrong reason still passed.Make
validate_blockreturn astd::error_codeinstead ofbool, with new block-level entries appended (at the back, keeping existing values stable) tostate::ErrorCode. Each entry's message is the matching execution-spec-testsBlockExceptionconstant (INVALID_GASLIMIT, INVALID_BASEFEE_PER_GAS, INCORRECT_EXCESS_BLOB_GAS, RLP_BLOCK_LIMIT_EXCEEDED, INVALID_BLOCK_TIMESTAMP_OLDER_THAN_PARENT, INCORRECT_BLOCK_FORMAT for the rest). The loader captures theexpectExceptionstring; the runner substring-matches the actual error's constant against it, which also handles the fixtures'|-separated alternatives.Legacy ethereum/tests fixtures use older, sometimes finer-grained exception names (e.g. uncles after Paris, a number gap, a wrong base fee). The loader maps those (many-to-one) onto the
BlockExceptionconstant evmone reports, so they match by reason as well.