test(e2e): fix expiration_timestamp prune race + flake-group e2e_slashing#23432
Open
AztecBot wants to merge 2 commits into
Open
test(e2e): fix expiration_timestamp prune race + flake-group e2e_slashing#23432AztecBot wants to merge 2 commits into
AztecBot wants to merge 2 commits into
Conversation
The 'invalidates the transaction' tests warp L1 past the proposer-pipelined
block's slot, which makes the archiver prune that block asynchronously. The
next test's beforeEach + body each call aztecNode.getBlockData('latest'); the
second call races the prune and resolves a block number that has been deleted
by the time the row is read, returning undefined.
Cache the header timestamp from beforeEach and reuse it in runInvalidatesTest
instead of refetching mid-cycle. Drop the matching error_regex flake pattern
added in the previous commit on this branch.
Collaborator
Author
Flakey Tests🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Re-enables the merge queue on
merge-train/spartanafter PR #23344 was dequeued by three test flakes in theci/x5-fullheavy-grind shard (run 26169283519). None are caused by #23344's diff.What failed
src/e2e_expiration_timestamp.test.tsBlock header not found in invalidates-the-transaction setup(line 147)src/e2e_p2p/sentinel_status_slash.parallel.test.tscheckpoint-invalid, gotcheckpoint-missedsrc/e2e_slashing/attested_invalid_proposal.test.tsFix — e2e_expiration_timestamp (real fix, not just patterning)
Root-caused to a real pipelining race. The
with no enqueued public calls / invalidates the transactiontest callscheatCodes.eth.warp(provedExpiration + aztecSlotDuration, { resetBlockInterval: true }), jumping L1 ~37 slots ahead. The proposer had pre-built block 9 for slot 10 (pipelining), but L1 is now past slot 10 without that block being checkpointed, so the archiver prunes it (Pruning blocks after block 8 due to slot 10 not being checkpointed). The prune is asynchronous and lands during the next test cycle. That next test'sbeforeEachreadsgetBlockData('latest')cleanly, then 3ms laterrunInvalidatesTestreads it again and races the in-flight prune —resolveBlockTag('proposed')returns block 9's number, butstores.blocks.getBlockData({ number: 9 })returns undefined because the row was just deleted.Fix: cache the header timestamp from
beforeEachand reuse it inrunInvalidatesTestinstead of refetching mid-cycle. The first read tolerates the warp's reorg because it's after a fresh test transition; the second read was redundant.Flake patterns — e2e_slashing
The
attested_invalid_proposaltimeout (5min budget) under heavy grind isn't a missing-hash race; it's slot-counting timing. Addssrc/e2e_slashing/.*\.test\.tsto the existinge2e-p2p-epoch-flakesgroup (owner: palla), alongside the existing p2p/epoch patterns. Thee2e_p2p/sentinel_status_slashfailure is already in that group via thesrc/e2e_p2p/.*regex.Group threshold stays at 5, so up to 4 simultaneous flakes are tolerated.
Details in https://gist.github.com/AztecBot/3600f093693517fa69c8981f6f31dbe7