Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions modules/learn/pages/clusters-and-availability/xdcr-filtering.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -324,3 +324,48 @@ Note that before and during migration, both the xref:learn:services-and-indexes/
For information on configuring deletion filters with the UI, see xref:manage:manage-xdcr/filter-xdcr-replication.adoc#deletion-filters[Deletion Filters];
with the CLI, see xref:cli:cbcli/couchbase-cli-xdcr-replicate.adoc[xdcr-replicate];
with the REST API, see xref:rest-api:rest-xdcr-create-replication.adoc[Creating XDCR Replications].

== Applying Expressions to Tombstones

When a document is deleted or expires, Couchbase creates a *tombstone*[cite: 1]. A tombstone contains only the **document key (ID)** and system metadata (xattrs); it does not contain the document body[cite: 1].

Because tombstones lack a body, standard filter expressions that reference document fields cannot be evaluated and are bypassed, allowing the deletion to replicate to all targets[cite: 1]. The new option, *Apply deletions (tombstones) to filter expressions using doc keys only*, allows you to filter these deletions if your expression relies exclusively on the document ID[cite: 1].

=== Use Case: Non-Unique Global Keys
In certain architectures, document IDs might not be globally unique across all sources[cite: 1]. Without tombstone filtering, a deletion from Source A could inadvertently delete a document with the same ID on Target B, even if Source A was never intended to replicate to Target B[cite: 1]. Enabling this feature ensures that a deletion only replicates if the document ID matches the specified filter[cite: 1].

=== Requirements and Limitations

To enable tombstone filtering, your replication must meet the following criteria:

* **Active Filter Expression**: A filter expression must be defined for the replication[cite: 1].
* **Key-Only Expressions**: The expression must reference *only* the document ID (e.g., using `META().id`)[cite: 1]. If the expression references document values or extended attributes (xattrs), this option cannot be enabled[cite: 1].
* **Opt-in Basis**: This feature is disabled by default to maintain backward compatibility with earlier Server behavior[cite: 1].
* **Enterprise Edition**: XDCR filtering features are available only in Couchbase Server Enterprise Edition[cite: 1].

.Expression Compatibility Reference
[cols="2,1", options="header"]
|===
| Filter Expression | Can Enable Tombstone Filtering?
| `REGEXP_CONTAINS(META().id, "^ABC-")` | Yes[cite: 1]
| `REGEXP_CONTAINS(country, "France")` | No (Uses doc values)[cite: 1]
| `REGEXP_CONTAINS(META().id, "^abc") AND REGEXP_CONTAINS(country, "France")` | No (Mixed keys/values)[cite: 1]
|===

=== Interaction with Deletion Filters

Existing XDCR deletion filters take precedence over the tombstone filtering option[cite: 1].

* **If "Do not replicate DELETE operations" is enabled**: Application-level deletes are blocked entirely[cite: 1]. The tombstone filtering option will only apply to document expirations (TTL)[cite: 1].
* **If "Do not replicate document expirations" is enabled**: Expirations are blocked entirely[cite: 1]. The tombstone filtering option will only apply to application-level deletes[cite: 1].
* **If both are enabled**: All deletions are blocked; the tombstone filtering option has no effect[cite: 1].

=== Configure Tombstone Filtering

You can enable or update this setting via the Couchbase Server Web Console, REST API, or CLI[cite: 1].

==== Using the Web Console
. Go to **XDCR** and click **Add Replication** (or click **Edit** on an existing replication)[cite: 1].
. Enable **Filter Replication** and enter your expression[cite: 1].
. Under **Deletion Filters**, check the box labeled **Apply deletions (tombstones) to filter expressions using doc keys only**[cite: 1].
. Click **Save Replication**[cite: 1].
34 changes: 34 additions & 0 deletions modules/xdcr-reference/pages/xdcr-filtering-expressions.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -703,3 +703,37 @@ The relationships between available expressions for XDCR Advanced Filtering are
| ExistsClause
| ( "EXISTS" "(" Field ")" )
|===

== Configure Tombstone Filtering

You can enable or update this setting via the Couchbase Server Web Console, REST API, or CLI[cite: 1].

=== Using the REST API
Use the `filterDeletionsWithKey` parameter when creating or updating a replication[cite: 1].

[source,console]
----
curl -X POST http://localhost:8091/controller/createReplication \
-u Administrator:password \
-d fromBucket="travel-sample" \
-d toCluster="remote-cluster" \
-d toBucket="backup" \
-d filterExpression="REGEXP_CONTAINS(META().id, '^route_')" \
-d filterDeletionsWithKey=true
----

=== Using the CLI
Use the `--filter-deletions-with-key` flag with the `xdcr-replicate` command[cite: 1].

[source,console]
----
couchbase-cli xdcr-replicate -c 127.0.0.1:8091 -u Administrator -p password \
--create --xdcr-from-bucket source --xdcr-to-bucket target \
--xdcr-to-cluster remote-cluster \
--filter-expression "REGEXP_CONTAINS(META().id, '^ABC-')" \
--filter-deletions-with-key 1
----

== See Also
* Refer to the file "PRD _ XDCR - option to apply deletions to filter expressions using doc key.docx" for detailed technical specifications and requirements[cite: 1].
* xref:learn/clusters-and-availability/xdcr-filtering.adoc[XDCR Advanced Filtering]