feat(s3): add conditional headers (If-Match, If-None-Match) for copy/…

[sksingh2005]

Which issue does this PR close?

Closes #7090 .

Rationale for this change

AWS S3 now supports conditional headers (If-Match and If-None-Match) for CopyObject and DeleteObject operations, enabling atomic operations and preventing race conditions in concurrent scenarios. This PR adds support for these headers to OpenDAL's S3 service, bringing it to feature parity with AWS S3's latest capabilities and matching the existing implementation in the Azure Blob service.

What changes are included in this PR?

This PR implements three new conditional operation capabilities for S3:

1. copy_with_if_not_exists

• Copy operations that only succeed if the target doesn't exist (using If-None-Match: *)

2. copy_with_if_match

• Copy operations that only succeed if the source's ETag matches (using If-Match: <etag>)

3. delete_with_if_match

• Delete operations that only succeed if the file's ETag matches (using If-Match: <etag>)

Core Changes:

• Added copy_with_if_match and delete_with_if_match capability flags
• Extended OpCopy and OpDelete with if_match fields
• Added if_match to CopyOptions and DeleteOptions
• Exposed if_match() methods in FutureCopy and FutureDelete for user-facing API
  S3 Service Implementation:
• Updated S3 backend to advertise new capabilities
• Implemented conditional headers in s3_copy_object() and s3_delete_object()
• Added proper header handling for If-Match and If-None-Match
  Testing:
• Added 4 new behavior tests covering success and failure scenarios
• Tests follow existing OpenDAL patterns and are registered in the test suite

Are there any user-facing changes?

Yes, this PR adds new user-facing methods:

// Copy with If-Match header
op.copy_with("source", "target")
    .if_match("\"etag-value\"")
    .await?;
// Delete with If-Match header
op.delete_with("path")
    .if_match("\"etag-value\"")
    .await?;

AI Usage Statement

This PR was implemented with assistance from Claude Sonnet 4.5 through the Antigravity coding assistant. The AI helped with:

• Test case design based on existing behavior tests

All code was reviewed, tested, and validated by the human contributor.

[Xuanwo]

Most changes LGTM, thank you for working on this!

[Xuanwo]

Seems we changed list: true to rename: false? Why this change?

[sksingh2005]

My apologies, it seems list: true was accidentally removed during the rename capability update. I have restored it. We need rename: false to force the Copy + Delete polyfill because S3 doesn't support native rename, but list: true is definitely required i guess sir ?? Review needed @Xuanwo sir

[Xuanwo]

Thank you @sksingh2005 for working on this. I believe most of the changes in this PR are correct. The root cause here is that minio does not support copy with if-match & if-none-match or delete with if-match.

[sksingh2005]

Thank you @Xuanwo sir for the review and for confirming the root cause.
The implementation is correct for AWS S3. The test failures are expected since MinIO doesn't support If-Match/If-None-Match headers for CopyObject and DeleteObject yet.
Please let me know if you'd like me to add a configuration flag to disable these features for S3-compatible services, or if the current implementation is acceptable.

[dentiny]

Seems env vars will be parsed at test start:

opendal/core/testkit/src/utils.rs

Lines 55 to 61 in ab9274c

	let mut cfg = env::vars()
	.filter_map(|(k, v)| {
	k.to_lowercase()
	.strip_prefix(&prefix)
	.map(|k| (k.to_string(), v))
	})
	.collect::<HashMap<String, String>>();

So I guess we could add an optional to disable

diff --git a/.github/services/s3/0_minio_s3/action.yml b/.github/services/s3/0_minio_s3/action.yml
index 2ee8bc8bb..479522ddc 100644
--- a/.github/services/s3/0_minio_s3/action.yml
+++ b/.github/services/s3/0_minio_s3/action.yml
@@ -43,4 +43,5 @@ runs:
         OPENDAL_S3_ACCESS_KEY_ID=minioadmin
         OPENDAL_S3_SECRET_ACCESS_KEY=minioadmin
         OPENDAL_S3_REGION=us-east-1
+        OPENDAL_S3_DISABLE_COPY_WITH_IF_MATCH=true
         EOF
diff --git a/.github/services/s3/minio_s3_with_anonymous/action.yml b/.github/services/s3/minio_s3_with_anonymous/action.yml
index 31c5e0851..114bf7780 100644
--- a/.github/services/s3/minio_s3_with_anonymous/action.yml
+++ b/.github/services/s3/minio_s3_with_anonymous/action.yml
@@ -48,4 +48,5 @@ runs:
         OPENDAL_S3_REGION=us-east-1
         OPENDAL_S3_ALLOW_ANONYMOUS=on
         OPENDAL_S3_DISABLE_EC2_METADATA=on
+        OPENDAL_S3_DISABLE_COPY_WITH_IF_MATCH=true
         EOF
diff --git a/.github/services/s3/minio_s3_with_list_objects_v1/action.yml b/.github/services/s3/minio_s3_with_list_objects_v1/action.yml
index 46904d7aa..ef209c79c 100644
--- a/.github/services/s3/minio_s3_with_list_objects_v1/action.yml
+++ b/.github/services/s3/minio_s3_with_list_objects_v1/action.yml
@@ -44,4 +44,5 @@ runs:
         OPENDAL_S3_SECRET_ACCESS_KEY=minioadmin
         OPENDAL_S3_REGION=us-east-1
         OPENDAL_S3_DISABLE_LIST_OBJECTS_V2=true
+        OPENDAL_S3_DISABLE_COPY_WITH_IF_MATCH=true
         EOF
diff --git a/.github/services/s3/minio_s3_with_versioning/action.yml b/.github/services/s3/minio_s3_with_versioning/action.yml
index e2db993a8..8143df537 100644
--- a/.github/services/s3/minio_s3_with_versioning/action.yml
+++ b/.github/services/s3/minio_s3_with_versioning/action.yml
@@ -45,4 +45,5 @@ runs:
         OPENDAL_S3_SECRET_ACCESS_KEY=minioadmin
         OPENDAL_S3_REGION=us-east-1
         OPENDAL_S3_ENABLE_VERSIONING=true
+        OPENDAL_S3_DISABLE_COPY_WITH_IF_MATCH=true
         EOF
diff --git a/core/services/s3/src/backend.rs b/core/services/s3/src/backend.rs
index 3fd62ee69..36e17dfe2 100644
--- a/core/services/s3/src/backend.rs
+++ b/core/services/s3/src/backend.rs
@@ -555,6 +555,14 @@ impl S3Builder {
         self
     }
 
+    /// Disable copy with if match so that opendal will not send copy request with if match headers.
+    ///
+    /// For example, MinIO doesn't support copy with if match.
+    pub fn disable_copy_with_if_match(mut self) -> Self {
+        self.config.disable_copy_with_if_match = true;
+        self
+    }
+
     /// Enable write with append so that opendal will send write request with append headers.
     pub fn enable_write_with_append(mut self) -> Self {
         self.config.enable_write_with_append = true;
diff --git a/core/services/s3/src/config.rs b/core/services/s3/src/config.rs
index 8c219de6c..cba6491b5 100644
--- a/core/services/s3/src/config.rs
+++ b/core/services/s3/src/config.rs
@@ -210,6 +210,11 @@ pub struct S3Config {
     /// For example, Ceph RADOS S3 doesn't support write with if matched.
     pub disable_write_with_if_match: bool,
 
+    /// Disable copy with if match so that opendal will not send copy request with if match headers.
+    ///
+    /// For example, MinIO doesn't support copy with if match.
+    pub disable_copy_with_if_match: bool,
+
     /// Enable write with append so that opendal will send write request with append headers.
     pub enable_write_with_append: bool,
 
diff --git a/core/services/s3/src/docs.md b/core/services/s3/src/docs.md
index df279f35b..e0f47efd3 100644
--- a/core/services/s3/src/docs.md
+++ b/core/services/s3/src/docs.md
@@ -30,6 +30,7 @@ This service can be used to:
 - `disable_config_load`: Disable aws config load from env.
 - `enable_virtual_host_style`: Enable virtual host style.
 - `disable_write_with_if_match`: Disable write with if match.
+- `disable_copy_with_if_match`: Disable copy with if match / if none match. Some S3-compatible services like MinIO don't support conditional copy.
 - `enable_request_payer`: Enable the request payer for backend.
 - `default_acl`: Define the default access control list (ACL) when creating a new object. Note that some s3 services like minio do not support this option.