Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 20 additions & 3 deletions dm/dm-compatibility-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,27 @@ DM supports migrating data from different sources to TiDB clusters. Based on the
| MariaDB 10.1.2 ~ 10.5.10 | Experimental | |
| MariaDB > 10.5.10 | Not tested | Expected to work in most cases after bypassing the [precheck](/dm/dm-precheck.md). See [MariaDB notes](#mariadb-notes). |

### Incompatibility with foreign key CASCADE operations
### Foreign key `CASCADE` operations

- DM creates foreign key **constraints** on the target, but they are not enforced while applying transactions because DM sets the session variable [`foreign_key_checks=OFF`](/system-variables.md#foreign_key_checks).
- DM does **not** support `ON DELETE CASCADE` or `ON UPDATE CASCADE` behavior by default, and enabling `foreign_key_checks` via a DM task session variable is not recommended. If your workload relies on cascades, **do not assume** that cascade effects will be replicated.
> **Warning:**
>
> This feature is experimental.

Starting from v8.5.6, DM provides **experimental** support for replicating tables that use foreign key constraints. This support includes the following improvements:

- **Safe mode**: during safe mode execution, DM sets `foreign_key_checks=0` for each batch and skips the redundant `DELETE` step for `UPDATE` statements that do not modify primary key or unique key values. This prevents `REPLACE INTO` (which internally performs `DELETE` + `INSERT`) from triggering unintended `ON DELETE CASCADE` effects on child rows. For details, see [DM Safe Mode](/dm/dm-safe-mode.md#foreign-key-handling-new-in-v856).
- **Multi-worker causality**: when `worker-count > 1`, DM reads foreign key relationships from the downstream schema at task start and injects causality keys. This ensures that DML operations on parent rows complete before operations on dependent child rows, preserving binlog order across workers.

The following limitations apply to foreign key replication:

- In safe mode, DM does not support `UPDATE` statements that modify primary key or unique key values. The task is paused with the error `safe-mode update with foreign_key_checks=1 and PK/UK changes is not supported`. To replicate such statements, set `safe-mode: false`.
- When `foreign_key_checks=1`, DM does not support DDL statements that create, modify, or drop foreign key constraints during replication.
- Table routing is not supported when `worker-count > 1`. If you use table routing with tables that include foreign keys, set `worker-count` to `1`.
- The block-allow list must include all ancestor tables in the foreign key dependency chain. If ancestor tables are filtered out, the task is paused with an error during incremental replication.
- Foreign key metadata must be consistent between the source and downstream. If inconsistencies are detected, run `binlog-schema update --from-target` to resynchronize metadata.
- `ON UPDATE CASCADE` is not correctly replicated in safe mode when an `UPDATE` modifies primary key or unique key values. DM rewrites such statements as `DELETE` + `REPLACE`, which triggers `ON DELETE` actions instead of `ON UPDATE` actions. In this case, DM rejects the statement and pauses the task. `UPDATE` statements that do not modify key values are replicated correctly.

In versions earlier than v8.5.6, DM creates foreign key constraints in the downstream but does not enforce them because it sets the session variable [`foreign_key_checks=OFF`](/system-variables.md#foreign_key_checks). As a result, cascading operations are not replicated to the downstream.

### MariaDB notes

Expand Down
2 changes: 1 addition & 1 deletion dm/dm-precheck.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Regardless of the migration mode you choose, the precheck always includes the fo

- Compatibility of the upstream MySQL table schema

- Check whether the upstream tables have foreign keys, which are not supported by TiDB. A warning is returned if a foreign key is found in the precheck.
- Check whether the upstream tables have foreign keys. TiDB supports foreign keys (GA since v8.5.0), and DM provides experimental support for replicating tables with foreign key constraints starting from v8.5.6. During the precheck, DM returns a warning if foreign keys are detected. For supported scenarios and limitations, see [DM Compatibility Catalog](/dm/dm-compatibility-catalog.md#foreign-key-cascade-operations).
- Check whether the upstream tables use character sets that are incompatible with TiDB. For more information, see [TiDB Supported Character Sets](/character-set-and-collation.md).
- Check whether the upstream tables have primary key constraints or unique key constraints (introduced from v1.0.7).

Expand Down
49 changes: 49 additions & 0 deletions dm/dm-safe-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ In safe mode, DM guarantees the idempotency of binlog events by rewriting SQL st
* `INSERT` statements are rewritten to `REPLACE` statements.
* `UPDATE` statements are analyzed to obtain the value of the primary key or the unique index of the row updated. `UPDATE` statements are then rewritten to `DELETE` + `REPLACE` statements in the following two steps: DM deletes the old record using the primary key or unique index, and inserts the new record using the `REPLACE` statement.

Starting from v8.5.6, when you set `foreign_key_checks=1` in the task session, DM skips the `DELETE` step for `UPDATE` statements that do not modify primary key or unique index values. For more information, see [Foreign key handling](#foreign-key-handling-new-in-v856).

`REPLACE` is a MySQL-specific syntax for inserting data. When you insert data using `REPLACE`, and the new data and existing data have a primary key or unique constraint conflict, MySQL deletes all the conflicting records and executes the insert operation, which is equivalent to "force insert". For details, see [`REPLACE` statement](https://dev.mysql.com/doc/refman/8.0/en/replace.html) in MySQL documentation.

Assume that a `dummydb.dummytbl` table has a primary key `id`. Execute the following SQL statements repeatedly on this table:
Expand Down Expand Up @@ -91,6 +93,53 @@ mysql-instances:
syncer-config-name: "global" # Name of the syncers configuration.
```

## Foreign key handling <span class="version-mark">New in v8.5.6</span>

> **Warning:**
>
> This feature is introduced in v8.5.6 and is experimental.

When you enable safe mode and set `foreign_key_checks=1` in the downstream task session, the default `DELETE` + `REPLACE` rewrite for `UPDATE` statements can trigger unintended `ON DELETE CASCADE` effects on child rows. Starting from v8.5.6, DM introduces the following improvements to address this issue.

### Non-key `UPDATE` optimization

For `UPDATE` statements that do not modify primary key or unique key values, DM skips the `DELETE` step and executes only `REPLACE INTO`. Because the primary key remains unchanged, `REPLACE INTO` overwrites the existing row without triggering cascade deletes. This optimization is applied automatically in safe mode.

For example, consider the following upstream statement, where `id` is the primary key:

```sql
UPDATE dummydb.dummytbl SET int_value = 888999 WHERE id = 123;
```

In versions earlier than v8.5.6, safe mode rewrites this statement as follows:

```sql
DELETE FROM dummydb.dummytbl WHERE id = 123; -- Triggers ON DELETE CASCADE
REPLACE INTO dummydb.dummytbl (id, int_value, ...) VALUES (123, 888999, ...);
```

Starting from v8.5.6, safe mode rewrites the statement as follows:

```sql
REPLACE INTO dummydb.dummytbl (id, int_value, ...) VALUES (123, 888999, ...); -- No cascade
```

> **Warning:**
>
> When `foreign_key_checks=1`, DM does not support `UPDATE` statements that modify primary key or unique key values. In this case, the task is paused with the error `safe-mode update with foreign_key_checks=1 and PK/UK changes is not supported`. To replicate such `UPDATE` statements on tables with foreign keys, set `safe-mode: false`.

### Session-level `foreign_key_checks`

During batch execution in safe mode, DM executes `SET SESSION foreign_key_checks=0` before executing `INSERT` and `UPDATE` batches, and restores the original value afterward. This prevents `REPLACE INTO` (which internally performs `DELETE` + `INSERT`) from triggering cascade operations in the downstream.

This session-level setting introduces a small overhead per batch (two `SET SESSION` round trips). In most workloads, this overhead is negligible.

### Multi-worker foreign key causality

When you set `worker-count` to a value greater than 1 and the task includes tables with foreign keys, DM reads foreign key relationships from the downstream `CREATE TABLE` schema when the task starts. For each DML operation, DM injects causality keys based on these relationships. This ensures that operations on parent rows and their dependent child rows are assigned to the same DML worker queue.

For detailed constraints, see [DM Compatibility Catalog](/dm/dm-compatibility-catalog.md#foreign-key-cascade-operations).

## Notes for safe mode

If you want to enable safe mode during the entire replication process for safety reasons, be aware of the following:
Expand Down
2 changes: 1 addition & 1 deletion foreign-key.md
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,7 @@ Create Table | CREATE TABLE `child` (

<CustomContent platform="tidb">

- [DM](/dm/dm-overview.md) does not support foreign keys. DM disables the [`foreign_key_checks`](/system-variables.md#foreign_key_checks) of the downstream TiDB when replicating data to TiDB. Therefore, the cascading operations caused by foreign keys are not replicated from the upstream to the downstream, which might cause data inconsistency.
- [DM](/dm/dm-overview.md): starting from v8.5.6, DM provides experimental support for replicating tables that use foreign key constraints. For supported scenarios and limitations, see [DM Compatibility Catalog](/dm/dm-compatibility-catalog.md#foreign-key-cascade-operations). In versions earlier than v8.5.6, DM disables the [`foreign_key_checks`](/system-variables.md#foreign_key_checks) system variable when replicating data to TiDB, so cascading operations are not replicated to the downstream cluster.
- [TiCDC](/ticdc/ticdc-overview.md) v6.6.0 is compatible with foreign keys. The previous versions of TiCDC might report an error when replicating tables with foreign keys. It is recommended to disable the `foreign_key_checks` of the downstream TiDB cluster when using a TiCDC version earlier than v6.6.0.
- [BR](/br/backup-and-restore-overview.md) v6.6.0 is compatible with foreign keys. The previous versions of BR might report an error when restoring tables with foreign keys to a v6.6.0 or later cluster. It is recommended to disable the `foreign_key_checks` of the downstream TiDB cluster before restoring the cluster when using a BR earlier than v6.6.0.
- When you use [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md), if the target table uses a foreign key, it is recommended to disable the `foreign_key_checks` of the downstream TiDB cluster before importing data. For versions earlier than v6.6.0, disabling this system variable does not take effect, and you need to grant the `REFERENCES` privilege for the downstream database user, or manually create the target table in the downstream database in advance to ensure smooth data import.
Expand Down
Loading