Skip to content

Add --chunk-concurrent-size for parallel row-copy#1688

Open
dnovitski wants to merge 1 commit into
github:masterfrom
dnovitski:pr-1398-rebased
Open

Add --chunk-concurrent-size for parallel row-copy#1688
dnovitski wants to merge 1 commit into
github:masterfrom
dnovitski:pr-1398-rebased

Conversation

@dnovitski
Copy link
Copy Markdown
Contributor

@dnovitski dnovitski commented May 22, 2026

Summary

Port of #1398 by @shaohk to current master, with correctness improvements.

Adds --chunk-concurrent-size flag that allows multiple row-copy chunks to execute in parallel within each iteration using errgroup. Default is 1 (no behavior change).

Motivation

On large tables with fast storage (NVMe/SSD), the single-threaded row-copy loop can become a bottleneck. This flag enables parallel chunk copying to improve migration throughput.

Performance Results

1M rows, ADD COLUMN extra_col INT DEFAULT 0, Docker MySQL 8.0, chunk-size=1000:

Concurrency Wall-clock Speedup
1 (default) 30.6s baseline
4 23.9s 22% faster
8 20.9s 32% faster

Benefits scale with table size and storage throughput.

Key Design Decisions

  • Two execution paths: concurrency=1 matches master's retry semantics exactly (range calc inside retry loop for hook-based chunk size reduction); concurrency>1 pre-calculates ranges under mutex for safe parallel execution
  • Thread-safe range calculation: CalculateNextIterationRangeEndValues(advanceCursor bool) protected by mutex, returns *IterationRangeValues struct with isolated Min/Max per goroutine
  • No shared mutable state in hot path: SQL warnings returned as function value (eliminates data race on shared MigrationLastInsertSQLWarnings field)
  • errgroup with real migration context: Proper cancellation propagation when any chunk fails
  • DB pool auto-sizing: Connection pool increased when --chunk-concurrent-size exceeds default pool size
  • Backward compatible: Default concurrency=1 preserves existing single-threaded behavior exactly

Changes from original #1398

  • Adapted to current master API (builder pattern, receiver names, retryBatchCopyWithHooks)
  • Fixed thread-safety: ApplyIterationInsertQuery returns SQL warnings instead of writing to shared field
  • Correct retry behavior: single-threaded path recalculates range on retry (matches master); concurrent path retries same range (INSERT IGNORE is idempotent)
  • Proper IncludeMinValues handling for first iteration
  • Uses real migration context (not context.Background())

Testing

  • All CI checks pass (build, lint, CodeQL, migration tests on MySQL 5.7/8.0/8.4/Percona)
  • Performance benchmarked (22-32% improvement with concurrency 4-8)
  • Data integrity verified (1M rows, checksum match)
  • TestRetryBatchCopyWithHooks passes (hook-based chunk size reduction works correctly)

Checklist

  • Tests pass
  • Documentation updated (doc/command-line-flags.md)
  • Backward compatible (default=1)

Based on work by @shaohk in #1398.

@dnovitski dnovitski force-pushed the pr-1398-rebased branch 4 times, most recently from 4e3c02d to 18f91e8 Compare May 22, 2026 11:15
Port of PR github#1398 by @shaohk: allows multiple row-copy chunks to execute
in parallel within each iteration using errgroup.

Key changes:
- Add IterationRangeValues struct for thread-safe range passing
- Serialize range calculation with CalculateNextIterationRangeEndValuesLock
- Rewrite iterateChunks to spawn N goroutines per queue item via errgroup
- Return SQL warnings from ApplyIterationInsertQuery (eliminates race on
  shared MigrationLastInsertSQLWarnings field)
- Increase DB connection pool when concurrency > default pool size
- Add --chunk-concurrent-size CLI flag (default 1, no behavior change)

Co-authored-by: shaohk <shaohk@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant