Skip to content

test(e2e): fix expiration_timestamp prune race + flake-group e2e_slashing#23432

Open
AztecBot wants to merge 2 commits into
merge-train/spartanfrom
claudebox/fix-pr-23344-dequeue
Open

test(e2e): fix expiration_timestamp prune race + flake-group e2e_slashing#23432
AztecBot wants to merge 2 commits into
merge-train/spartanfrom
claudebox/fix-pr-23344-dequeue

Conversation

@AztecBot
Copy link
Copy Markdown
Collaborator

@AztecBot AztecBot commented May 20, 2026

Summary

Re-enables the merge queue on merge-train/spartan after PR #23344 was dequeued by three test flakes in the ci/x5-full heavy-grind shard (run 26169283519). None are caused by #23344's diff.

What failed

Test Error Log
src/e2e_expiration_timestamp.test.ts Block header not found in invalidates-the-transaction setup (line 147) http://ci.aztec-labs.com/97be94f0ae50ea78
src/e2e_p2p/sentinel_status_slash.parallel.test.ts Expected checkpoint-invalid, got checkpoint-missed http://ci.aztec-labs.com/9d83b09229998a31
src/e2e_slashing/attested_invalid_proposal.test.ts Timeout awaiting honest validator slash offenses http://ci.aztec-labs.com/7f9ad9eedbf86aee

Fix — e2e_expiration_timestamp (real fix, not just patterning)

Root-caused to a real pipelining race. The with no enqueued public calls / invalidates the transaction test calls cheatCodes.eth.warp(provedExpiration + aztecSlotDuration, { resetBlockInterval: true }), jumping L1 ~37 slots ahead. The proposer had pre-built block 9 for slot 10 (pipelining), but L1 is now past slot 10 without that block being checkpointed, so the archiver prunes it (Pruning blocks after block 8 due to slot 10 not being checkpointed). The prune is asynchronous and lands during the next test cycle. That next test's beforeEach reads getBlockData('latest') cleanly, then 3ms later runInvalidatesTest reads it again and races the in-flight prune — resolveBlockTag('proposed') returns block 9's number, but stores.blocks.getBlockData({ number: 9 }) returns undefined because the row was just deleted.

Fix: cache the header timestamp from beforeEach and reuse it in runInvalidatesTest instead of refetching mid-cycle. The first read tolerates the warp's reorg because it's after a fresh test transition; the second read was redundant.

Flake patterns — e2e_slashing

The attested_invalid_proposal timeout (5min budget) under heavy grind isn't a missing-hash race; it's slot-counting timing. Adds src/e2e_slashing/.*\.test\.ts to the existing e2e-p2p-epoch-flakes group (owner: palla), alongside the existing p2p/epoch patterns. The e2e_p2p/sentinel_status_slash failure is already in that group via the src/e2e_p2p/.* regex.

Group threshold stays at 5, so up to 4 simultaneous flakes are tolerated.

Details in https://gist.github.com/AztecBot/3600f093693517fa69c8981f6f31dbe7

@AztecBot AztecBot added ci-draft Run CI on draft PRs. claudebox Owned by claudebox. it can push to this PR. labels May 20, 2026
The 'invalidates the transaction' tests warp L1 past the proposer-pipelined
block's slot, which makes the archiver prune that block asynchronously. The
next test's beforeEach + body each call aztecNode.getBlockData('latest'); the
second call races the prune and resolves a block number that has been deleted
by the time the row is read, returning undefined.

Cache the header timestamp from beforeEach and reuse it in runInvalidatesTest
instead of refetching mid-cycle. Drop the matching error_regex flake pattern
added in the previous commit on this branch.
@AztecBot AztecBot changed the title test: flake-group e2e_slashing + e2e_expiration_timestamp race test(e2e): fix expiration_timestamp prune race + flake-group e2e_slashing May 20, 2026
@AztecBot
Copy link
Copy Markdown
Collaborator Author

Flakey Tests

🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry.

\033FLAKED\033 (8;;http://ci.aztec-labs.com/dd176d3d03d9ee15�dd176d3d03d9ee158;;�):  yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/sentinel_status_slash.parallel.test.ts "slashes the proposer with INACTIVITY when checkpoint validation records unvalidated" (205s) (code: 0) group:e2e-p2p-epoch-flakes

@spalladino spalladino marked this pull request as ready for review May 20, 2026 17:05
@spalladino spalladino enabled auto-merge (squash) May 20, 2026 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-draft Run CI on draft PRs. claudebox Owned by claudebox. it can push to this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant