refactor: more robust tagging index sync as sender #17611
Merged
+1,844
−385
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Oct 9, 2025
Contributor
Author
benesjan
force-pushed
the
10-08-refactor_renaming_indexedtaggingsecret_as_pretag
branch
from
73a8f6d to
778aeaa
Compare
benesjan
force-pushed
the
10-09-feat_more_robust_unconstrained_tagging
branch
from
588cef2 to
39f19c1
Compare
benesjan
force-pushed
the
10-09-feat_more_robust_unconstrained_tagging
branch
from
ceb0929 to
d100cf8
Compare
benesjan
force-pushed
the
10-08-refactor_renaming_indexedtaggingsecret_as_pretag
branch
from
778aeaa to
0a26d3b
Compare
benesjan
force-pushed
the
10-08-refactor_renaming_indexedtaggingsecret_as_pretag
branch
from
0a26d3b to
16795d9
Compare
benesjan
force-pushed
the
10-09-feat_more_robust_unconstrained_tagging
branch
from
977b053 to
dfec582
Compare
Base automatically changed from
10-08-refactor_renaming_indexedtaggingsecret_as_pretag
to
next
October 15, 2025 22:30
benesjan
force-pushed
the
10-09-feat_more_robust_unconstrained_tagging
branch
from
dfec582 to
f04c9c9
Compare
benesjan
changed the title
benesjan
commented
Oct 16, 2025
benesjan
changed the title
benesjan
commented
Oct 16, 2025
yarn-project/pxe/src/contract_function_simulator/execution_data_provider.ts
Outdated
Show resolved
Hide resolved
benesjan
commented
Oct 16, 2025
yarn-project/pxe/src/contract_function_simulator/pxe_oracle_interface.test.ts
Show resolved
Hide resolved
benesjan
commented
Oct 16, 2025
yarn-project/pxe/src/contract_function_simulator/pxe_oracle_interface.ts
Show resolved
Hide resolved
benesjan
commented
Oct 16, 2025
yarn-project/pxe/src/storage/tagging_data_provider/recipient_tagging_data_provider.ts
Show resolved
Hide resolved
benesjan
force-pushed
the
10-09-feat_more_robust_unconstrained_tagging
branch
2 times, most recently
from
bcf2da9 to
4f1a6ee
Compare
nventuro
reviewed
Dec 15, 2025
Contributor
nventuro
left a comment
nventuro
left a comment
There was a problem hiding this comment.
Left a lot of comments since this is a very large PR. In hindsight we probably should've implemented some of the logic e.g. re. to pending or to processing in separate prs, and then merged that with pxe's work. So unfortutunately there's lots of comments both big and small in the mix.
Sorry it took so long to review this. Thanks for the effort in keeping it up to date and as understandable as possible.
yarn-project/pxe/src/contract_function_simulator/oracle/private_execution_oracle.ts
Outdated
Show resolved
Hide resolved
yarn-project/pxe/src/storage/tagging_data_provider/sender_tagging_data_provider.test.ts
Show resolved
Hide resolved
yarn-project/pxe/src/storage/tagging_data_provider/sender_tagging_data_provider.ts
Outdated
Show resolved
Hide resolved
yarn-project/pxe/src/storage/tagging_data_provider/sender_tagging_data_provider.ts
Outdated
Show resolved
Hide resolved
yarn-project/pxe/src/storage/tagging_data_provider/sender_tagging_data_provider.ts
Outdated
Show resolved
Hide resolved
yarn-project/pxe/src/tagging/sync/sync_sender_tagging_indexes.ts
Outdated
Show resolved
Hide resolved
benesjan
force-pushed
the
10-09-feat_more_robust_unconstrained_tagging
branch
3 times, most recently
from
e2f9768 to
a55fa6b
Compare
benesjan
added
the
ci-no-fail-fast
nventuro
approved these changes
Dec 16, 2025
Contributor
nventuro
left a comment
nventuro
left a comment
There was a problem hiding this comment.
The improvements are excellent, thank you very much for going through my comments and considering each of them.
In this PR I implement the approach that Nico originally described in [this post](https://forum.aztec.network/t/on-note-discovery-and-index-coordination/7165) for sender tagging index sync (for recipient it will be done in a followup PR). To summarize I do the following: 1. Associate the tagging index with a hash of a tx in which the corresponding tax was used, 2. I track the status of the given tx in the tagging data provider, 3. tagging index status can be either PENDING, FINALIZED or DROPPED_OR_REVERTED (this status is only symbolic as when we realize a corresponding tx has been dropped or reverted we just delete the pending index), 4. when choosing the next index to use when sending a private log I choose `HIGHEST_PENDING_INDEX + 1` (or `HIGHEST_FINALIZED_INDEX + 1` if there is no pending index), 5. if the chosen index is further than WINDOW_LENGTH away from the HIGHEST_FINALIZED_INDEX an error is thrown. This should be a good guarantee of us never losing a log because we are always looking for a WINDOW_LENGTH of logs. # Notes for reviewer - I separated `TaggingDataProvider` into `SenderTaggingDataProvider` and `RecipientTaggingDataProvider` because the algorithms are completely disjoint and it makes the code clearer. - Moved the `syncTaggedLogsAsSender` function to `pxe/src/tagging` directory and renamed it as `syncSenderTaggingIndexes`. Now `PrivateExecutionOracle` just calls this function. This is a step in the direction of deprecating `PXEOracleInterface` and it allowed me to write a reasonably clear unit tests. # Issues created in this PR - #17775 - #17776 Co-authored-by: Jan Beneš <janbenes1234@gmail.com>
AztecBot
force-pushed
the
10-09-feat_more_robust_unconstrained_tagging
branch
from
e52830e to
eb5f940
Compare
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
This was referenced Dec 18, 2025
ludamad
pushed a commit
that referenced
this pull request
Dec 19, 2025
In this PR I implement the approach that Nico originally described in [this post](https://forum.aztec.network/t/on-note-discovery-and-index-coordination/7165) for sender tagging index sync (for recipient it will be done in a followup PR). To summarize I do the following: 1. Associate the tagging index with a hash of a tx in which the corresponding tax was used, 2. I track the status of the given tx in the tagging data provider, 3. tagging index status can be either PENDING, FINALIZED or DROPPED_OR_REVERTED (this status is only symbolic as when we realize a corresponding tx has been dropped or reverted we just delete the pending index), 4. when choosing the next index to use when sending a private log I choose `HIGHEST_PENDING_INDEX + 1` (or `HIGHEST_FINALIZED_INDEX + 1` if there is no pending index), 5. if the chosen index is further than WINDOW_LENGTH away from the HIGHEST_FINALIZED_INDEX an error is thrown. This should be a good guarantee of us never losing a log because we are always looking for a WINDOW_LENGTH of logs. # Notes for reviewer - I separated `TaggingDataProvider` into `SenderTaggingDataProvider` and `RecipientTaggingDataProvider` because the algorithms are completely disjoint and it makes the code clearer. - Moved the `syncTaggedLogsAsSender` function to `pxe/src/tagging` directory and renamed it as `syncSenderTaggingIndexes`. Now `PrivateExecutionOracle` just calls this function. This is a step in the direction of deprecating `PXEOracleInterface` and it allowed me to write a reasonably clear unit tests. # Issues created in this PR - #17775 - #17776
ludamad
pushed a commit
that referenced
this pull request
Dec 19, 2025
In this PR I implement the approach that Nico originally described in [this post](https://forum.aztec.network/t/on-note-discovery-and-index-coordination/7165) for sender tagging index sync (for recipient it will be done in a followup PR). To summarize I do the following: 1. Associate the tagging index with a hash of a tx in which the corresponding tax was used, 2. I track the status of the given tx in the tagging data provider, 3. tagging index status can be either PENDING, FINALIZED or DROPPED_OR_REVERTED (this status is only symbolic as when we realize a corresponding tx has been dropped or reverted we just delete the pending index), 4. when choosing the next index to use when sending a private log I choose `HIGHEST_PENDING_INDEX + 1` (or `HIGHEST_FINALIZED_INDEX + 1` if there is no pending index), 5. if the chosen index is further than WINDOW_LENGTH away from the HIGHEST_FINALIZED_INDEX an error is thrown. This should be a good guarantee of us never losing a log because we are always looking for a WINDOW_LENGTH of logs. # Notes for reviewer - I separated `TaggingDataProvider` into `SenderTaggingDataProvider` and `RecipientTaggingDataProvider` because the algorithms are completely disjoint and it makes the code clearer. - Moved the `syncTaggedLogsAsSender` function to `pxe/src/tagging` directory and renamed it as `syncSenderTaggingIndexes`. Now `PrivateExecutionOracle` just calls this function. This is a step in the direction of deprecating `PXEOracleInterface` and it allowed me to write a reasonably clear unit tests. # Issues created in this PR - #17775 - #17776
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In this PR I implement the approach that Nico originally described in this post for sender tagging index sync (for recipient it will be done in a followup PR).
To summarize I do the following:
HIGHEST_PENDING_INDEX + 1(orHIGHEST_FINALIZED_INDEX + 1if there is no pending index),Notes for reviewer
TaggingDataProviderintoSenderTaggingDataProviderandRecipientTaggingDataProviderbecause the algorithms are completely disjoint and it makes the code clearer.syncTaggedLogsAsSenderfunction topxe/src/taggingdirectory and renamed it assyncSenderTaggingIndexes. NowPrivateExecutionOraclejust calls this function. This is a step in the direction of deprecatingPXEOracleInterfaceand it allowed me to write a reasonably clear unit tests.Issues created in this PR
PXEOracleInterface#17776