Skip to content

HDDS-15193. Move the "atomic key creation" logic from output stream to S3 endpoints#10202

Open
peterxcli wants to merge 5 commits intoapache:masterfrom
peterxcli:codex/HDDS-15193-remove-old-atomic-key-creation
Open

HDDS-15193. Move the "atomic key creation" logic from output stream to S3 endpoints#10202
peterxcli wants to merge 5 commits intoapache:masterfrom
peterxcli:codex/HDDS-15193-remove-old-atomic-key-creation

Conversation

@peterxcli
Copy link
Copy Markdown
Member

@peterxcli peterxcli commented May 6, 2026

What changes were proposed in this pull request?

this is a no-op change

the "atomic key creation" logic in Ozone client output stream is actually only for s3g to do the length check.

removes the "atomic key creation" logic from the Ozone client output stream classes and shifts the responsibility for validating S3 object upload content length from the client layer to the S3 gateway layer. The main effect is that S3-specific length checks are now enforced in the S3 gateway, not in the general Ozone client code, leading to a cleaner separation of concerns and more maintainable code.

the name "atomic key creation" is also misleading people to think it's related to atomicity, but it's actually a integrity check for the written block data for s3g. If users want the real atomicity, they must use conditional request:

relate to: #5524

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15193

How was this patch tested?

(Please explain how this patch was tested. Ex: unit tests, manual tests, workflow run on the fork git repo.)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this.)

@peterxcli peterxcli changed the title HDDS-15193. Remove old atomic key creation HDDS-15193. Remove old S3 atomic key creation May 6, 2026
@peterxcli peterxcli marked this pull request as ready for review May 6, 2026 18:28
@peterxcli peterxcli requested a review from ChenSammi May 8, 2026 06:27
@peterxcli
Copy link
Copy Markdown
Member Author

cc @xichen01, @ivandika3 Could you please take a look? Thanks!

@chungen0126
Copy link
Copy Markdown
Contributor

Thanks @peterxcli for the patch. However, I went back to review the PR for these lines, and it appears they are not related to conditional requests. They were actually added to fix a commit overwrite issue that occurs when multiple S3G instances are writing the same key.

I am concerned that removing them, as I don't think that specific problem has been resolved yet. Could you please provide a bit more detail on why these lines need to be deleted? Thanks!

Signed-off-by: peterxcli <peterxcli@gmail.com>
@peterxcli peterxcli changed the title HDDS-15193. Remove old S3 atomic key creation HDDS-15193. refactor old S3 atomic key creation May 8, 2026
@peterxcli peterxcli changed the title HDDS-15193. refactor old S3 atomic key creation HDDS-15193. Keep length validation logic only in s3g module May 8, 2026
@peterxcli peterxcli changed the title HDDS-15193. Keep length validation logic only in s3g module HDDS-15193. Movesthe "atomic key creation" logic from client output to S3 Endpoint May 8, 2026
@peterxcli peterxcli changed the title HDDS-15193. Movesthe "atomic key creation" logic from client output to S3 Endpoint HDDS-15193. Movesthe "atomic key creation" logic from output stream to S3 Endpoint May 8, 2026
@peterxcli peterxcli changed the title HDDS-15193. Movesthe "atomic key creation" logic from output stream to S3 Endpoint HDDS-15193. Move the "atomic key creation" logic from output stream to S3 endpoints May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants