Skip to content

starknet_committer: create storage tries concurrently#13632

Merged
nimrod-starkware merged 1 commit intomainfrom
nimrod/parallel-reads/concurrently
Apr 15, 2026
Merged

starknet_committer: create storage tries concurrently#13632
nimrod-starkware merged 1 commit intomainfrom
nimrod/parallel-reads/concurrently

Conversation

@nimrod-starkware
Copy link
Copy Markdown
Contributor

@nimrod-starkware nimrod-starkware commented Mar 31, 2026

Note

Medium Risk
Introduces a new concurrent code path for building per-contract storage tries and tightens NodeLayout trait bounds (Send/Sync), which can affect implementors and has typical concurrency/correctness risks despite the new path currently being unused (dead_code).

Overview
Adds scaffolding to build per-contract storage trie original skeletons concurrently via ImmutableReadOnlyStorage::gather: introduces TrieReadTask implementing StorageTask and a new create_storage_tries_concurrently helper that fans out one task per contract and collects results.

Updates starknet_patricia::NodeLayout to require Send on NodeData and Send + Sync on SubTree to support cross-task execution.

Reviewed by Cursor Bugbot for commit 767c891. Bugbot is set up for automated code reviews on this repo. Configure here.

@reviewable-StarkWare
Copy link
Copy Markdown

This change is Reviewable

Copy link
Copy Markdown
Contributor

@ArielElp ArielElp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ArielElp reviewed 1 file and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on nimrod-starkware).


crates/starknet_committer/src/db/trie_traversal.rs line 494 at r1 (raw file):

        Send + Sync,
    for<'b> <Layout as NodeLayout<'b, <Layout as NodeLayoutFor<StarknetStorageValue>>::DbLeaf>>::NodeData:
        Send,

Lose these bounds everywhere in this file and add them at the trait level in db_layout.rs, add Send on NodeData and Send+Sync on SubTree. A little less general but worth it IMO, we only use simple types here.

Code quote:

    for<'b> <Layout as NodeLayout<'b, <Layout as NodeLayoutFor<StarknetStorageValue>>::DbLeaf>>::SubTree:
        Send + Sync,
    for<'b> <Layout as NodeLayout<'b, <Layout as NodeLayoutFor<StarknetStorageValue>>::DbLeaf>>::NodeData:
        Send,

@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from ffd8c2b to 108778e Compare April 9, 2026 09:59
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/gatherable-cached-storage branch from 9c65e69 to 0bd0e8d Compare April 9, 2026 09:59
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/gatherable-cached-storage branch from 0bd0e8d to f5347be Compare April 9, 2026 10:19
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from 108778e to 11b6b54 Compare April 9, 2026 10:19
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/gatherable-cached-storage branch from f5347be to 86480d5 Compare April 9, 2026 13:25
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from 11b6b54 to 1aa095e Compare April 9, 2026 13:25
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/gatherable-cached-storage branch from 86480d5 to 701127d Compare April 12, 2026 07:18
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from 1aa095e to 4a27ae2 Compare April 12, 2026 07:18
Copy link
Copy Markdown
Contributor Author

@nimrod-starkware nimrod-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nimrod-starkware made 1 comment.
Reviewable status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on ArielElp and nimrod-starkware).


crates/starknet_committer/src/db/trie_traversal.rs line 494 at r1 (raw file):

Previously, ArielElp wrote…

Lose these bounds everywhere in this file and add them at the trait level in db_layout.rs, add Send on NodeData and Send+Sync on SubTree. A little less general but worth it IMO, we only use simple types here.

Done.

@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/gatherable-cached-storage branch from 701127d to b1c33a3 Compare April 12, 2026 10:35
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from 4a27ae2 to 767c891 Compare April 12, 2026 10:35
Copy link
Copy Markdown
Contributor

@ArielElp ArielElp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@ArielElp reviewed 3 files and all commit messages, made 1 comment, and resolved 1 discussion.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on nimrod-starkware).

@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from 767c891 to b0023e2 Compare April 12, 2026 14:02
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/gatherable-cached-storage branch from b1c33a3 to 6633956 Compare April 12, 2026 14:02
@cursor
Copy link
Copy Markdown

cursor bot commented Apr 12, 2026

PR Summary

Medium Risk
Introduces concurrency-related trait bounds and a new GatherableStorage-driven task path for building storage tries, which may ripple to all NodeLayout implementers and affect compile-time and runtime threading assumptions.

Overview
Adds an (currently unused) concurrent pathway for building per-contract storage trie skeletons by introducing TrieReadTask and create_storage_tries_concurrently, designed to run via GatherableStorage::gather with per-task ReadsCollectorStorage snapshots.

Updates starknet_patricia::NodeLayout to be thread-safe (NodeData: Send, SubTree: Send + Sync) to support running trie traversal work across concurrent tasks.

Reviewed by Cursor Bugbot for commit 8cca25e. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown
Collaborator

@dorimedini-starkware dorimedini-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dorimedini-starkware reviewed 3 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on nimrod-starkware).


crates/starknet_committer/src/db/trie_traversal.rs line 520 at r4 (raw file):

    <Layout as NodeLayoutFor<StarknetStorageValue>>::DbLeaf:
        HasStaticPrefix<KeyContext = ContractAddress>,
{

better to be consistent, at least in a single signature

Suggestion:

async fn create_storage_tries_concurrently<'a, S, Layout>(
    storage: &mut S,
    actual_storage_updates: &'a HashMap<ContractAddress, LeafModifications<StarknetStorageValue>>,
    original_contracts_trie_leaves: &HashMap<NodeIndex, ContractState>,
    warn_on_trivial_modifications: bool,
    storage_tries_sorted_indices: &'a HashMap<ContractAddress, SortedLeafIndices<'a>>,
) -> ForestResult<HashMap<ContractAddress, OriginalSkeletonTreeImpl<'a>>>
where
    S: ImmutableReadOnlyStorage,
    Layout: NodeLayoutFor<StarknetStorageValue> + Send + 'static,
    <Layout as NodeLayoutFor<StarknetStorageValue>>::DbLeaf:
        HasStaticPrefix<KeyContext = ContractAddress>,
{

@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from b0023e2 to f06cd35 Compare April 12, 2026 14:35
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/gatherable-cached-storage branch from 6633956 to 51ca741 Compare April 12, 2026 14:35
Copy link
Copy Markdown
Contributor Author

@nimrod-starkware nimrod-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nimrod-starkware made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on dorimedini-starkware).


crates/starknet_committer/src/db/trie_traversal.rs line 520 at r4 (raw file):

Previously, dorimedini-starkware wrote…

better to be consistent, at least in a single signature

Done.

Copy link
Copy Markdown
Collaborator

@dorimedini-starkware dorimedini-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dorimedini-starkware reviewed 1 file and all commit messages, and resolved 1 discussion.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on nimrod-starkware).

@graphite-app graphite-app bot changed the base branch from nimrod/parallel-reads/gatherable-cached-storage to graphite-base/13632 April 13, 2026 07:40
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from f06cd35 to f636314 Compare April 14, 2026 09:53
@nimrod-starkware nimrod-starkware changed the base branch from graphite-base/13632 to nimrod/parallel-reads/gatherable-cached-storage April 14, 2026 09:53
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from f636314 to f32b92e Compare April 14, 2026 10:02
Copy link
Copy Markdown
Collaborator

@dorimedini-starkware dorimedini-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dorimedini-starkware reviewed 1 file and all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on nimrod-starkware).

@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/gatherable-cached-storage branch from fa2b94b to c9282b4 Compare April 14, 2026 15:21
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from f32b92e to c1ab4af Compare April 14, 2026 15:21
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from c1ab4af to 6253637 Compare April 15, 2026 08:24
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/gatherable-cached-storage branch from c9282b4 to 1b7c56c Compare April 15, 2026 08:24
@nimrod-starkware nimrod-starkware changed the base branch from nimrod/parallel-reads/gatherable-cached-storage to main April 15, 2026 08:54
@nimrod-starkware nimrod-starkware force-pushed the nimrod/parallel-reads/concurrently branch from 6253637 to 8cca25e Compare April 15, 2026 09:17
Copy link
Copy Markdown
Contributor Author

@nimrod-starkware nimrod-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nimrod-starkware reviewed all commit messages.
Reviewable status: 3 of 4 files reviewed, all discussions resolved.

Copy link
Copy Markdown
Collaborator

@dorimedini-starkware dorimedini-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dorimedini-starkware reviewed 1 file and all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on nimrod-starkware).

@nimrod-starkware nimrod-starkware added this pull request to the merge queue Apr 15, 2026
Merged via the queue into main with commit 1739332 Apr 15, 2026
26 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Apr 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants