Skip to content

feat(services/s3): add conditional delete with if-match#7607

Open
YuangGao wants to merge 3 commits into
apache:mainfrom
YuangGao:feat/s3-delete-with-if-match
Open

feat(services/s3): add conditional delete with if-match#7607
YuangGao wants to merge 3 commits into
apache:mainfrom
YuangGao:feat/s3-delete-with-if-match

Conversation

@YuangGao
Copy link
Copy Markdown
Contributor

@YuangGao YuangGao commented May 23, 2026

Which issue does this PR close?

Closes #7090.

Rationale for this change

AWS S3 supports If-Match on DeleteObject since 2024 (docs). Expose it through OpenDAL so callers can do CAS-style conditional deletes

What changes are included in this PR?

Core:

  • Add Capability::delete_with_if_match.
  • Add if_match: Option<String> to OpDelete, DeleteOptions, and a FutureDelete::if_match builder.
  • Add the matching check in correctness_check.rs

S3:

  • Set delete_with_if_match: true on the S3 capability.
  • s3_delete_object injects If-Match when args.if_match() is set.
  • s3_delete_objects propagates per-object if_match into the <ETag> field of the DeleteObjects XML body, so the batch path stays a single request even with conditional deletes.

CI:

  • Add delete_with_if_match=false to OPENDAL_TEST_CAPABILITY_OVERRIDES in the four minio_s3* action.yml files

Tests:

  • test_delete_with_if_match_match / _mismatch in core/tests/behavior/async_delete.rs.

Bindings:

  • Node.js + Python DeleteOptions extended with if_match; Python pyi stubs and async/sync delete() signatures updated. .NET / Java / Haskell / Ruby / OCaml use ..Default::default() and need no changes.

Are there any user-facing changes?

Yes, additive only — op.delete_with(path).if_match(etag) is a new opt-in API. Existing op.delete(path) is unchanged.

AI Usage Statement

AI-assisted implementation.

@YuangGao YuangGao force-pushed the feat/s3-delete-with-if-match branch from a6dd63c to 3af96e5 Compare May 23, 2026 21:43
@YuangGao YuangGao marked this pull request as ready for review May 23, 2026 21:50
@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels May 23, 2026
@Xuanwo
Copy link
Copy Markdown
Member

Xuanwo commented May 24, 2026

S3Deleter::delete_batch falls back to per-key delete_once whenever any entry in the batch carries if_match, since DeleteObjects v2 has no per-object If-Match slot. The non-conditional fast path is unchanged.

Seems to be wrong. There is no DeleteObjects v2 and DeleteObjects did support setting etag. For example:

POST /?delete HTTP/1.1
Host: Bucket.s3.amazonaws.com
x-amz-mfa: MFA
x-amz-request-payer: RequestPayer
x-amz-bypass-governance-retention: BypassGovernanceRetention
x-amz-expected-bucket-owner: ExpectedBucketOwner
x-amz-sdk-checksum-algorithm: ChecksumAlgorithm
<?xml version="1.0" encoding="UTF-8"?>
<Delete xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
   <Object>
      <ETag>string</ETag>
      <Key>string</Key>
      <LastModifiedTime>timestamp</LastModifiedTime>
      <Size>long</Size>
      <VersionId>string</VersionId>
   </Object>
   ...
   <Quiet>boolean</Quiet>
</Delete>

Please double check.

@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels May 24, 2026
@YuangGao
Copy link
Copy Markdown
Contributor Author

yes, I confused it with ListObjectsV2, just updated

  • s3_delete_objects now writes op.if_match() into <ETag>
  • Dropped the delete_once fallback in S3Deleter::delete_batch; the conditional batch path is a single request again.

@YuangGao YuangGao changed the title feat(delete): add conditional delete with if-match feat(services/s3): add conditional delete with if-match May 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new feature: s3 conditional copy & delete

2 participants