feat(migrator): [6/7] CLI commands and documentation with rvl migrate subcommand, user guide, and concept docs by nkanu17 · Pull Request #565 · redis/redis-vl-python

nkanu17 · 2026-04-01T22:39:04Z

Summary

CLI interface for migration operations (plan, execute, wizard) via rvl migrate subcommand. Includes comprehensive user guide, concept documentation for index migrations and field attributes, and updated CLI reference.

Also adds AGENTS.md project context and updates CONTRIBUTING.md with conventional commit and branch guidelines.

Files

redisvl/cli/migrate.py, main.py, utils.py
docs/ - concept docs, how-to guide, CLI reference updates
.gitignore, AGENTS.md, CONTRIBUTING.md

Stack

[1/7] Migration foundation > feat(migrator): [1/7] Migration foundation with models, schema-aware planner, validation, and shared utilities #560
[2/7] Sync executor with reliability and quantization > feat(migrator): [2/7] Sync executor with reliability checkpointing, crash-safe resume, and quantization support #561
[3/7] Async migration > feat(migrator): [3/7] Async migration with non-blocking planner, executor, validator, and readiness utilities #562
[4/7] Batch migration > feat(migrator): [4/7] Batch migration with multi-index glob selection, ordered execution, and state tracking #563
[5/7] Interactive wizard
[6/7] CLI and documentation (this PR)
[7/7] Benchmarks

Note

Medium Risk
Introduces a new CLI surface that can drop/recreate indexes and optionally rewrite stored vectors, so incorrect usage or edge cases could cause downtime or data mutations. Core changes are mostly additive, but they touch user-facing CLI parsing and batch status accounting.

Overview
Adds a new rvl migrate command group to the CLI, wiring it into redisvl/cli/main.py and implementing subcommands for migration planning, interactive wizarding, apply (sync/async), disk-space estimation, validation, and batch plan/apply/resume/status in redisvl/cli/migrate.py.

Updates CLI connection handling by centralizing Redis connection flags via add_redis_connection_options() and fixing URL scheme selection (redis:// vs rediss://) in create_redis_url(), plus a small correctness tweak in batch report success counting (BatchMigrationExecutor).

Expands documentation substantially: adds a full CLI reference page (docs/api/cli.rst), new concepts page for migrations (docs/concepts/index-migrations.md), enriches field-attribute docs with vector datatype/quantization and migration support, and adds a detailed how-to guide (docs/user_guide/how_to_guides/migrate-indexes.md) alongside navigation updates and minor CLI notebook updates. Also adds AGENTS.md, updates contributing guidelines for conventional branches/commits, and extends .gitignore for local/migration artifacts.

^{Written by Cursor Bugbot for commit a1d54c9. This will update automatically on new commits. Configure here.}

jit-ci · 2026-04-01T22:40:52Z

🛡️ Jit Security Scan Results

✅ No security findings were detected in this PR

^{Security scan by Jit}

cursor

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

docs/user_guide/13_sql_query_exercises.ipynb

.gitignore

redisvl/cli/migrate.py

nkanu17 · 2026-04-01T22:43:56Z

@codex review

Copilot

Pull request overview

Adds a new rvl migrate CLI command group to expose the index migration planner/executor/validator workflow, alongside substantial documentation covering migration concepts, field attribute updates, and user-facing how-to guidance.

Changes:

Introduces redisvl/cli/migrate.py and wires it into the top-level CLI (rvl migrate ...).
Refactors CLI option helpers to share Redis connection flags, and expands docs to include migration concepts + a migration how-to guide.
Adds/updates contributor guidance (AGENTS.md, CONTRIBUTING.md) and updates ignore patterns for migration artifacts.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
redisvl/cli/utils.py	Refactors CLI arg helpers; centralizes Redis connection options.
redisvl/cli/migrate.py	New migration CLI command group (plan/apply/validate + batch operations).
redisvl/cli/main.py	Wires `migrate` into top-level CLI routing/usage text.
docs/user_guide/index.md	Adds migrations to user guide landing page cards.
docs/user_guide/how_to_guides/migrate-indexes.md	New end-to-end migration how-to guide for the CLI workflow.
docs/user_guide/how_to_guides/index.md	Adds migration guide to how-to index and toctree.
docs/user_guide/cli.ipynb	Updates CLI notebook to include `rvl migrate` commands.
docs/user_guide/13_sql_query_exercises.ipynb	Adds a new SQLQuery exercises notebook.
docs/concepts/search-and-indexing.md	Mentions the migration workflow in indexing lifecycle docs.
docs/concepts/index.md	Adds “Index Migrations” to concepts navigation.
docs/concepts/index-migrations.md	New concept doc explaining migration modes/limits and sync vs async.
docs/concepts/field-attributes.md	Expands vector datatype docs + adds migration/wizard attribute support notes.
docs/api/cli.rst	Adds (new) CLI reference page.
CONTRIBUTING.md	Adds conventional branch/commit guidance.
AGENTS.md	Adds project context, workflows, and repo conventions.
.gitignore	Ignores migration output artifacts and adds additional local dev ignores.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

redisvl/cli/utils.py

redisvl/cli/migrate.py

docs/user_guide/how_to_guides/migrate-indexes.md

docs/concepts/index-migrations.md

docs/api/cli.rst

docs/user_guide/13_sql_query_exercises.ipynb

chatgpt-codex-connector · 2026-04-01T22:48:18Z

Codex Review: Didn't find any major issues. Keep it up!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 8 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

redisvl/cli/migrate.py

docs/user_guide/how_to_guides/migrate-indexes.md

docs/api/cli.rst

AGENTS.md

docs/api/cli.rst

nkanu17 · 2026-04-01T22:58:16Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 751a0e5950

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

redisvl/cli/migrate.py

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 7 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

docs/user_guide/how_to_guides/migrate-indexes.md

redisvl/cli/migrate.py

docs/api/cli.rst

docs/user_guide/13_sql_query_exercises.ipynb

- Fix create_redis_url: use 'rediss://' scheme instead of 'redis://rediss://' - Exit with status 1 on unknown subcommand - Fix SVS_VAMANA -> SVS-VAMANA in help text - Add field_rename and key_rename progress steps (8 total) - Exit with status 1 when batch state file not found

Per AGENTS.md convention, local imports should have a comment explaining why they are not at module level.

- Remove leftover numbered list fragment in migrate-indexes.md (#565 #20) - Extract duplicated progress_callback into shared _make_progress_callback method in migrate.py (#565 #21) - Add rvl migrate section to CLI reference docs (#565 #19) - Add try/except for matplotlib import in visualize_results.py (#566 #14)

nkanu17 · 2026-04-02T00:32:24Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 49ce997cbd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

redisvl/cli/migrate.py

…ate subcommand, user guide, and concept docs CLI interface for migration operations (plan, execute, wizard) via rvl migrate subcommand. Includes comprehensive user guide, concept documentation for index migrations and field attributes, and updated CLI reference. Also adds AGENTS.md project context and updates CONTRIBUTING.md with conventional commit and branch guidelines.

- Fix create_redis_url: use 'rediss://' scheme instead of 'redis://rediss://' - Exit with status 1 on unknown subcommand - Fix SVS_VAMANA -> SVS-VAMANA in help text - Add field_rename and key_rename progress steps (8 total) - Exit with status 1 when batch state file not found

Per AGENTS.md convention, local imports should have a comment explaining why they are not at module level.

…ndexes cleanup

- Align batch-status display to use 'success' matching BatchIndexState model - Clean up batch executor status check to use canonical 'success' only

- Remove leftover numbered list fragment in migrate-indexes.md (#565 #20) - Extract duplicated progress_callback into shared _make_progress_callback method in migrate.py (#565 #21) - Add rvl migrate section to CLI reference docs (#565 #19) - Add try/except for matplotlib import in visualize_results.py (#566 #14)

nkanu17 · 2026-04-02T03:59:42Z

@codex review

Copilot

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

redisvl/cli/utils.py:26

create_redis_url() will always include a username in the constructed URL because --user defaults to "default" and the code appends args.user whenever it is truthy. This produces URLs like redis://default@localhost:6379 even when the user didn’t pass --user, which can be incompatible with some Redis client/auth configurations. Consider defaulting --user to an empty string/None and only injecting username@ (and :password) when the user explicitly provided credentials (or at least when a password is provided).

        if args.ssl:
            url = "rediss://"
        else:
            url = "redis://"
        if args.user:
            url += args.user
            if args.password:
                url += ":" + args.password
            url += "@"
        url += args.host + ":" + str(args.port)

redisvl/migration/batch_executor.py:336

BatchIndexState.status appears to be standardized on success/failed/skipped, but there’s existing usage of status: "succeeded" in persisted checkpoint examples/tests (e.g. tests/integration/test_batch_migration_integration.py simulates a checkpoint with "succeeded"). With this change, any legacy checkpoint/state file containing succeeded will be counted as skipped here (and also won’t contribute to BatchState.success_count). Consider keeping backward compatibility by treating succeeded as an alias of success when loading state / computing summaries.

            if idx_state.status == "success":
                succeeded += 1
            elif idx_state.status == "failed":
                failed += 1
            else:
                skipped += 1

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-02T04:03:23Z

docs/api/cli.rst

+**Syntax**
+
+.. code-block:: bash
+
+    rvl migrate plan --index <name> (--patch <patch.yaml> | --target-schema <schema.yaml>) [OPTIONS]
+
+**Required Options**
+
+.. list-table::
+   :widths: 30 70
+   :header-rows: 1
+
+   * - Option
+     - Description
+   * - ``--index``, ``-i``
+     - Name of the source index to migrate
+   * - ``--patch``
+     - Path to a YAML schema patch file (mutually exclusive with ``--target-schema``)
+   * - ``--target-schema``
+     - Path to a full target schema YAML file (mutually exclusive with ``--patch``)
+
+**Optional Options**
+
+.. list-table::
+   :widths: 30 70
+   :header-rows: 1
+
+   * - Option
+     - Description
+   * - ``--output``, ``-o``
+     - Output path for the migration plan YAML (default: ``migration_plan.yaml``)
+
+**Example**
+
+.. code-block:: bash
+
+    rvl migrate plan -i my_index --patch changes.yaml -o plan.yaml
+


The migrate CLI reference here documents --patch/-o/--output, but the implementation in redisvl/cli/migrate.py uses --schema-patch/--target-schema and --plan-out instead. This mismatch will lead users to run commands that error. Update the documented flags and example invocation to match the actual CLI arguments.

Copilot · 2026-04-02T04:03:23Z

docs/api/cli.rst

+**Optional Options**
+
+.. list-table::
+   :widths: 30 70
+   :header-rows: 1
+
+   * - Option
+     - Description
+   * - ``--async``
+     - Run migration asynchronously (recommended for large quantization jobs)
+   * - ``--query-check``
+     - Path to a YAML file with post-migration query checks
+   * - ``--resume``
+     - Path to a checkpoint file for crash-safe recovery
+


The rvl migrate apply options listed here include --query-check, but the CLI implementation uses --query-check-file (and also supports --report-out / --benchmark-out). Align this table and examples with the actual flags so users can successfully run apply/validate workflows.

Copilot · 2026-04-02T04:03:23Z

docs/user_guide/13_sql_query_exercises.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# SQL-to-Redis Query Translation: Hands-On Exercises\n",
+    "\n",
+    "This notebook provides hands-on exercises for learning the new **SQLQuery** feature in RedisVL, which allows you to write familiar SQL syntax that automatically translates to Redis Search commands.\n",
+    "\n",
+    "## What You'll Learn\n",
+    "\n",
+    "1. How to use the `SQLQuery` class to write SQL-like queries\n",
+    "2. Three equivalent approaches for the same queries:\n",
+    "   - **RedisVL Python API** - Using native query classes (`FilterQuery`, `VectorQuery`, etc.)\n",
+    "   - **RedisVL SQL** - Using the new `SQLQuery` class with SQL syntax\n",
+    "   - **Raw Redis FT.SEARCH** - The equivalent Redis Search command\n",
+    "3. Various query types: filtering, numeric ranges, text search, aggregations, and vector similarity\n",
+    "\n",
+    "## Prerequisites\n",
+    "\n",
+    "- Redis Stack running locally (or Redis Cloud)\n",
+    "- RedisVL with SQL support: `pip install redisvl[sql-redis]`\n",
+    "\n",
+    "## Documentation References\n",
+    "\n",
+    "- [RedisVL Documentation](https://docs.redisvl.com)\n",
+    "- [Redis Search Query Syntax](https://redis.io/docs/latest/develop/ai/search-and-query/query/)\n",
+    "- [Redis Aggregations](https://redis.io/docs/latest/develop/ai/search-and-query/advanced-concepts/aggregations/)\n",
+    "- [sql-redis Package](https://pypi.org/project/sql-redis/)\n",
+    "\n",
+    "---"
+   ]


This PR is scoped to migrator CLI/docs, but this notebook adds a large set of SQLQuery exercises that appear unrelated to index migrations. If this is intentional, it may deserve its own PR to keep the migrator series focused; if not, consider dropping it from this change to reduce review surface and documentation churn.

Copilot · 2026-04-02T04:03:24Z

docs/user_guide/13_sql_query_exercises.ipynb

+    "# YOUR CODE HERE - Method 1: RedisVL Python API\n",
+    "# Hint: Use Tag(\"category\") == \"electronics\" with FilterQuery\n",
+    "q= FilterQuery(\n",
+    "    filter_expression=Tag(\"category\") == \"electronics\",\n",
+    "    return_fields=[\"name\", \"category\", \"price\"],\n",
+    "    num_results=10\n",
+    ")\n",


Minor formatting issue in this exercise cell: q= FilterQuery( is missing a space around = (q = FilterQuery(...)). Since this is user-facing tutorial content, fixing it improves readability and copy/paste quality.

nkanu17 · 2026-04-02T04:04:05Z

@codex review

chatgpt-codex-connector · 2026-04-02T04:08:19Z

Codex Review: Didn't find any major issues. More of your lovely PRs please.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot AI review requested due to automatic review settings April 1, 2026 22:39

nkanu17 mentioned this pull request Apr 1, 2026

feat(migrator): [7/7] Migration performance benchmarks with throughput, latency, memory measurement, and visualization #566

Open

Copilot started reviewing on behalf of nkanu17 April 1, 2026 22:39 View session

cursor bot reviewed Apr 1, 2026

View reviewed changes

docs/user_guide/13_sql_query_exercises.ipynb Show resolved Hide resolved

.gitignore Show resolved Hide resolved

redisvl/cli/migrate.py Outdated Show resolved Hide resolved

Copilot AI reviewed Apr 1, 2026

View reviewed changes

nkanu17 requested a review from Copilot April 1, 2026 22:49

Copilot started reviewing on behalf of nkanu17 April 1, 2026 22:50 View session

Copilot AI reviewed Apr 1, 2026

View reviewed changes

nkanu17 requested a review from Copilot April 1, 2026 22:56

Copilot started reviewing on behalf of nkanu17 April 1, 2026 22:57 View session

chatgpt-codex-connector bot reviewed Apr 1, 2026

View reviewed changes

redisvl/cli/migrate.py Show resolved Hide resolved

redisvl/cli/migrate.py Outdated Show resolved Hide resolved

Copilot AI reviewed Apr 1, 2026

View reviewed changes

nkanu17 force-pushed the feat/migrate-wizard branch from b94e57a to 541db34 Compare April 2, 2026 00:30

nkanu17 added a commit that referenced this pull request Apr 2, 2026

fix: add explanatory comments for local imports in CLI (#565)

a464914

Per AGENTS.md convention, local imports should have a comment explaining why they are not at module level.

nkanu17 force-pushed the feat/migrate-cli-docs branch from 751a0e5 to 49ce997 Compare April 2, 2026 00:30

chatgpt-codex-connector bot reviewed Apr 2, 2026

View reviewed changes

redisvl/cli/migrate.py Outdated Show resolved Hide resolved

nkanu17 added 5 commits April 1, 2026 23:57

fix: add explanatory comments for local imports in CLI (#565)

63e85dc

Per AGENTS.md convention, local imports should have a comment explaining why they are not at module level.

chore: add rvl migrate CLI docs + progress callback dedup + migrate-i…

46a8a04

…ndexes cleanup

fix: address review round 3 for migrate-cli-docs (#565)

a1d54c9

- Align batch-status display to use 'success' matching BatchIndexState model - Clean up batch executor status check to use canonical 'success' only

nkanu17 force-pushed the feat/migrate-wizard branch from 541db34 to 7eea293 Compare April 2, 2026 03:58

Copilot AI review requested due to automatic review settings April 2, 2026 03:58

nkanu17 force-pushed the feat/migrate-cli-docs branch from 49ce997 to a1d54c9 Compare April 2, 2026 03:58

Copilot started reviewing on behalf of nkanu17 April 2, 2026 03:59 View session

Copilot AI reviewed Apr 2, 2026

View reviewed changes

Conversation

nkanu17 commented Apr 1, 2026 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Files

Stack

Uh oh!

jit-ci bot commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛡️ Jit Security Scan Results

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nkanu17 commented Apr 1, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot commented Apr 1, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nkanu17 commented Apr 1, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nkanu17 commented Apr 2, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

nkanu17 commented Apr 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

nkanu17 commented Apr 1, 2026 •

edited by cursor bot

Loading

jit-ci bot commented Apr 1, 2026 •

edited

Loading