Skip to content

Add Range Index data type based on Bf-Tree (single instance)#1613

Merged
badrishc merged 15 commits intodevfrom
badrishc/bf-tree-integration-plan
Apr 22, 2026
Merged

Add Range Index data type based on Bf-Tree (single instance)#1613
badrishc merged 15 commits intodevfrom
badrishc/bf-tree-integration-plan

Conversation

@badrishc
Copy link
Copy Markdown
Collaborator

@badrishc badrishc commented Mar 6, 2026 •
edited
Loading

Description of Change

Adds Range Index as a new Garnet data type, backed by Bf-Tree — a high-performance B-tree for ordered key-value storage with range scan support. Gated behind --enable-range-index-preview.

Commands implemented (9):

  • RI.CREATE — Create index with DISK or MEMORY backend
  • RI.SET / RI.GET / RI.DEL — Point operations on entries
  • RI.SCAN / RI.RANGE — Range queries (DISK backend only)
  • RI.EXISTS / RI.CONFIG / RI.METRICS — Utility commands
  • TYPE returns "rangeindex" for RI keys
  • DEL / UNLINK frees the underlying BfTree

Lifecycle infrastructure (Tsavorite IRecordTriggers):

  • OnFlush — Snapshot BfTree to flush.bftree, set FlagFlushed for lazy promote
  • OnEvict — Free BfTree under exclusive lock
  • OnDiskRead — Zero stale TreeHandle
  • OnCheckpoint — VersionShift (barrier), FlushBegin (snapshot), CheckpointCompleted (cleanup)
  • OnRecovery / OnRecoverySnapshotRead — Set recovered token / FlagRecovered

Checkpoint consistency:

  • Trees with SnapshotPending=1 at barrier time are snapshotted; v+1 trees are skipped
  • Concurrent restores serialized via exclusive lock
  • Old checkpoint snapshots purged at CheckpointCompleted (gated on removeOutdated)
  • RIPROMOTE clears source handle in PostCopyUpdater (after CAS success, not before)
  • Restored trees use data.bftree as working file (not snapshot artifacts)

AOF logging:

  • RI.SET/RI.DEL logged via synthetic no-op RMW (VectorManager pattern)
  • RI.CREATE logged with stub bytes; replay creates fresh BfTree
  • AOF-only recovery (no checkpoint) supported

WRONGTYPE safety:

  • Read, RMW, and Upsert paths block cross-type access
  • GET/SET on RI keys → WRONGTYPE error
  • RI commands on non-RI keys → WRONGTYPE error
  • UpsertAction.WrongType added to Tsavorite

55 tests covering CRUD, scan, range, lifecycle (flush/evict/promote/restore), checkpoint recovery, AOF replay, AOF-only recovery, WRONGTYPE,
concurrent stress (4 threads + blocking SAVE with strict prefix verification), and utility commands.

Documentation:

  • Dedicated Range Index commands page (website/docs/commands/range-index.md)
  • API compatibility listing
  • Design doc updated with implementation status

Not in this PR (deferred)

  • Batch commands (RI.MSET/MGET/MDEL)
  • Cluster replication/migration
  • Transaction (MULTI/EXEC) support
  • Memory-only tree snapshot (blocked on upstream bf-tree)

kevin-montrose
kevin-montrose reviewed Mar 6, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@kevin-montrose kevin-montrose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good to me - couple questions to poke at, but no blockers that I see.

Comment thread website/docs/dev/range-index-resp-api.md Outdated
Comment thread website/docs/dev/range-index-resp-api.md Outdated
Comment thread website/docs/dev/range-index-resp-api.md Outdated
@badrishc badrishc changed the title [Doc] Range index integration plan (first draft) Range index integration: design doc + prototype Mar 18, 2026
@badrishc badrishc force-pushed the badrishc/bf-tree-integration-plan branch from 320af88 to 90fbe55 Compare March 30, 2026 21:47
@badrishc badrishc force-pushed the badrishc/bf-tree-integration-plan branch 9 times, most recently from 8f93b4c to c5afb94 Compare April 15, 2026 02:11
@badrishc badrishc changed the title Range index integration: design doc + prototype Add Range Index data type based on Bf-Tree (single instance) Apr 16, 2026
@badrishc badrishc marked this pull request as ready for review April 16, 2026 01:23
Copilot AI review requested due to automatic review settings April 16, 2026 01:23
Copilot AI reviewed Apr 16, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new RangeIndex data type to Garnet, backed by the native bf-tree library, including command surface (RI.*), storage/session plumbing, lifecycle (flush/evict/checkpoint/recovery) hooks, and supporting metadata/docs/tests/benchmarks. This integrates RangeIndex into the main store with type-safety enforcement and adds a Rust cdylib + C# interop layer for BfTree.

Changes:

  • Introduces RangeIndex runtime support: RI.CREATE/SET/GET/DEL/SCAN/RANGE, RangeIndexManager, and session ops that execute against a BfTree native pointer stored in a Tsavorite value stub.
  • Extends Tsavorite trigger surface for checkpoint/recovery callbacks and wires Garnet record triggers to snapshot/evict/restore BfTrees.
  • Adds command metadata/docs updates plus new interop tests and benchmarks; updates CI/pipelines to install Rust and build/sign/package the native library.

Reviewed changes

Copilot reviewed 61 out of 66 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
test/Garnet.test/TestUtils.cs Adds server option flag for enabling RangeIndex preview in tests.
test/Garnet.test/RespCommandTests.cs Marks internal RI RMW commands as “no metadata required”.
test/Garnet.test/Resp/ACL/RespCommandTests.cs Extends ACL coverage logic and adds RI.* ACL tests.
test/BfTreeInterop.test/BfTreeInteropTests.cs New integration tests for C# ↔ native BfTree interop.
test/BfTreeInterop.test/BfTreeInterop.test.csproj New test project for BfTree interop layer.
playground/CommandInfoUpdater/SupportedCommand.cs Adds RI.* commands to command info updater list.
playground/CommandInfoUpdater/GarnetCommandsInfo.json Adds command-info entries for RI.*.
playground/CommandInfoUpdater/GarnetCommandsDocs.json Adds docs entries for RI.* and updates quoting/encoding.
libs/storage/Tsavorite/cs/src/core/Index/StoreFunctions/StoreFunctions.cs Adds trigger passthrough for recovery/checkpoint callbacks.
libs/storage/Tsavorite/cs/src/core/Index/StoreFunctions/IStoreFunctions.cs Extends store functions interface for new callbacks.
libs/storage/Tsavorite/cs/src/core/Index/StoreFunctions/IRecordTriggers.cs Adds recovery and checkpoint trigger APIs to record triggers.
libs/storage/Tsavorite/cs/src/core/Index/StoreFunctions/CheckpointTrigger.cs New enum for checkpoint trigger points.
libs/storage/Tsavorite/cs/src/core/Index/Recovery/Recovery.cs Calls recovery triggers and distinguishes snapshot recovery path.
libs/storage/Tsavorite/cs/src/core/Index/Checkpointing/HybridLogCheckpointSMTask.cs Invokes checkpoint triggers at version-shift/flush-begin.
libs/storage/Tsavorite/cs/src/core/Allocator/ObjectAllocatorImpl.cs Clarifies flush trigger comment for external-resource snapshotting.
libs/server/StoreWrapper.cs Plumbs shared RangeIndexManager through StoreWrapper lifecycle.
libs/server/Storage/Session/MainStore/RangeIndexOps.cs Implements StorageSession RangeIndex operations and RESP writes.
libs/server/Storage/Functions/UnifiedStore/DeleteMethods.cs Minor formatting change.
libs/server/Storage/Functions/MainStore/VarLenInputMethods.cs Adds RI* sizing support for RMW value allocation.
libs/server/Storage/Functions/MainStore/ReadMethods.cs Enforces type-safety for RI keys on read paths.
libs/server/Storage/Functions/MainStore/RMWMethods.cs Adds RI* RMW handling for create/promote/restore stubs.
libs/server/Storage/Functions/MainStore/PrivateMethods.cs Ensures RI.* commands copy output without RESP header.
libs/server/Storage/Functions/GarnetRecordTriggers.cs Adds BfTree lifecycle handling via record triggers.
libs/server/Storage/Functions/FunctionsState.cs Exposes RangeIndexManager via FunctionsState.
libs/server/Servers/GarnetServerOptions.cs Adds EnableRangeIndexPreview server option.
libs/server/Resp/RespServerSession.cs Routes RI.* commands to dedicated handlers.
libs/server/Resp/RangeIndex/RespServerSessionRangeIndex.cs New RESP handlers for RI.* parsing/execution.
libs/server/Resp/RangeIndex/RangeIndexResult.cs New result enum for RangeIndex operations.
libs/server/Resp/RangeIndex/RangeIndexManager.cs New manager for live BfTrees + flush/checkpoint snapshot orchestration.
libs/server/Resp/RangeIndex/RangeIndexManager.Locking.cs Adds RI shared/exclusive locking + lazy restore/promote paths.
libs/server/Resp/RangeIndex/RangeIndexManager.Index.cs Defines stub layout and stub mutation helpers.
libs/server/Resp/Parser/RespCommand.cs Adds RI commands, parsing fast-paths, and RangeIndex legality checks.
libs/server/Resp/CmdStrings.cs Adds RI.* command string constants.
libs/server/GarnetDatabase.cs Stores RangeIndexManager per DB; plumbs through DB construction.
libs/server/Garnet.server.csproj Adds project reference to BfTreeInterop.
libs/server/API/IGarnetApi.cs Adds RangeIndex API surface methods.
libs/server/API/GarnetApi.cs Implements RangeIndex API calls via StorageSession.
libs/server/ACL/ACLParser.cs Supports dot-commands (e.g., RI.CREATE) parsing for ACLs.
libs/resources/RespCommandsInfo.json Adds RI.* command info entries to shipped resources.
libs/native/bftree-garnet/src/lib.rs New Rust FFI wrapper over bf-tree with point ops, scans, snapshot/recovery.
libs/native/bftree-garnet/rust-toolchain.toml Pins Rust toolchain channel to stable.
libs/native/bftree-garnet/examples/bench.rs Adds standalone Rust benchmark example.
libs/native/bftree-garnet/NativeBfTreeMethods.cs Adds P/Invoke declarations for native bftree_garnet.
libs/native/bftree-garnet/Cargo.toml New Rust crate manifest for cdylib build.
libs/native/bftree-garnet/Cargo.lock Locks Rust dependencies (bf-tree, transitive).
libs/native/bftree-garnet/BfTreeService.cs Managed wrapper for BfTree native API and scan helpers.
libs/native/bftree-garnet/BfTreeInterop.csproj Builds/copies native library and packages runtime assets.
libs/native/bftree-garnet/.gitignore Ignores Rust build artifacts.
libs/host/defaults.conf Adds default config for RangeIndex preview flag.
libs/host/GarnetServer.cs Constructs RangeIndexManager and wires triggers into store creation.
libs/host/Configuration/Options.cs Adds CLI option --enable-range-index-preview.
benchmark/BDN.benchmark/Operations/RangeIndexOperations.cs Adds BDN benchmarks for RI.* operations.
benchmark/BDN.benchmark/BfTree/BfTreeOperations.cs Adds BDN benchmarks for BfTree point + scan operations.
benchmark/BDN.benchmark/BDN.benchmark.csproj References BfTreeInterop from benchmarks.
Garnet.slnx Adds new project/folders to solution layout.
.github/workflows/ci.yml Installs Rust toolchain for GitHub Actions CI.
.azure/pipelines/createbinaries.ps1 Copies bftree native binary into publish output.
.azure/pipelines/azure-pipelines.yml Installs Rust toolchain on Windows/Linux builds.
.azure/pipelines/azure-pipelines-internal-release.yml Builds/downloads bftree native and signs additional artifacts.
.azure/pipelines/azure-pipelines-external-release.yml Builds/downloads bftree native and includes in release signing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread libs/server/Storage/Session/MainStore/RangeIndexOps.cs
Comment thread libs/server/Storage/Session/MainStore/RangeIndexOps.cs Outdated
Comment thread libs/server/Storage/Session/MainStore/RangeIndexOps.cs Outdated
Comment thread libs/native/bftree-garnet/BfTreeService.cs Outdated
Comment thread benchmark/BDN.benchmark/Operations/RangeIndexOperations.cs Outdated
Comment thread libs/server/Resp/RangeIndex/RespServerSessionRangeIndex.cs
Comment thread playground/CommandInfoUpdater/GarnetCommandsDocs.json Outdated
Comment thread libs/native/bftree-garnet/BfTreeInterop.csproj Outdated
@badrishc badrishc force-pushed the badrishc/bf-tree-integration-plan branch 3 times, most recently from c207468 to 9ac1c64 Compare April 16, 2026 02:39
@badrishc badrishc requested a review from Copilot April 16, 2026 02:42
@badrishc badrishc force-pushed the badrishc/bf-tree-integration-plan branch from 9ac1c64 to 3e67835 Compare April 16, 2026 03:02
Copilot AI reviewed Apr 16, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@badrishc badrishc force-pushed the badrishc/bf-tree-integration-plan branch 4 times, most recently from 89856a2 to d602ae0 Compare April 16, 2026 18:24
@badrishc badrishc force-pushed the badrishc/bf-tree-integration-plan branch 3 times, most recently from a0d9ef2 to 4f60a0b Compare April 20, 2026 21:48
@badrishc
Merge dev into bf-tree-integration-plan
516bd88 
Integrates 4 dev commits (#1716, #1712, #1711, #1710).

Key change: #1712 internalized heap-size tracking into Tsavorite and
refactored IRecordDisposer → IRecordTriggers with OnEvict(EvictionSource).

Resolution:
- IRecordTriggers: take dev's version (EvictionSource enum, OnEvict with
  EvictionSource param, OnDisposeDiskRecord, removed OnDisposeValueObject),
  add bf-tree checkpoint hooks (OnRecovery, OnCheckpoint, OnRecoverySnapshotRead)
- GarnetRecordTriggers: update OnEvict to accept EvictionSource; remove
  OnDisposeValueObject and heap-tracking from OnDispose (now internal to
  Tsavorite); keep bf-tree dispatch (OnFlush, OnEvict, OnDiskRead, OnDispose,
  OnCheckpoint, OnRecovery)
- GarnetServer: keep cacheSizeTracker + rangeIndexManager constructor
- RMWMethods PostCopyUpdater: keep ClearTreeHandle for RIPROMOTE
- Fix unnecessary using in ReadMethods.cs

All tests pass: 55 RangeIndex (3 runs), 345 RespTests, 4 CacheSizeTracker.
Format clean (Garnet + Tsavorite).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@badrishc badrishc force-pushed the badrishc/bf-tree-integration-plan branch from 4f60a0b to 516bd88 Compare April 20, 2026 21:53
badrishc and others added 2 commits April 20, 2026 17:07
@badrishc
Block RI.SCAN/RI.RANGE for MEMORY-mode indexes; fix shutdown ordering
485ccf6 
MEMORY-mode BfTree indexes do not support scan operations in the native
library (bf-tree 0.4.9 panics in range_scan.rs:508, crashing the process).

Add StorageBackend check in RangeIndexScan/RangeIndexRange that returns
MemoryModeNotSupported before calling the native scan FFI. Network handlers
return a clear error: 'ERR RI.SCAN/RI.RANGE is not supported for MEMORY-mode
indexes'. Server remains responsive after the rejected command.

Also fix shutdown ordering: dispose RangeIndexManager before DatabaseManager
so BfTree native resources are freed before the Tsavorite store closes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@badrishc
Improve RI.SET error message for invalid key/value sizes
5388195 
Replace generic 'ERR invalid key or value size' with descriptive message
showing the configured MINRECORD, MAXRECORD, MAXKEYLEN constraints and the
actual sizes that were rejected. Example:
  ERR key+value size must be between 16 and 1024 bytes (got 4), max key length 128 (got 2)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@badrishc badrishc force-pushed the badrishc/bf-tree-integration-plan branch from 7056ba7 to 5388195 Compare April 21, 2026 00:21
badrishc and others added 3 commits April 20, 2026 17:40
@badrishc
Merge remote-tracking branch 'origin/dev' into badrishc/bf-tree-integ…
4a24e24 
…ration-plan
@badrishc
Pin Rust toolchain to 1.94 and add --locked to cargo builds
81c9faa 
Pin rust-toolchain.toml to channel 1.94 (was 'stable') for reproducible
builds across developers and CI, following DiskANN's pattern.

Add --locked flag to all cargo build invocations so Cargo.lock is used
exactly and the build fails if it's out of date, rather than silently
updating dependencies.

Files changed:
- libs/native/bftree-garnet/rust-toolchain.toml
- libs/native/bftree-garnet/BfTreeInterop.csproj
- .azure/pipelines/azure-pipelines-external-release.yml
- .azure/pipelines/azure-pipelines-internal-release.yml

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@badrishc
Make BfTreeService.Dispose thread-safe; reject negative RI.CREATE values
cc233da 
BfTreeService.Dispose: use Interlocked.Exchange on _disposed flag to
prevent double-free in concurrent scenarios.

RI.CREATE: change numeric option variables from unsigned to signed long,
check <= 0 (was == 0) in validation block. Negative values previously
wrapped to huge unsigned values, potentially causing massive allocations.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@badrishc badrishc force-pushed the badrishc/bf-tree-integration-plan branch from 01749bc to 0ee0ba3 Compare April 21, 2026 02:31
@badrishc
Fix CI rustup race condition in parallel multi-TFM builds
2707408 
The BfTreeInterop.csproj cargo build target ran for each TFM in parallel,
causing concurrent rustup toolchain installs to race on download file renames.

Fix: gate the cargo build target on IsCrossTargetingBuild so it runs once
in the outer MSBuild orchestration before per-TFM inner builds start.
The 'or TargetFrameworks empty' fallback handles single-TFM builds
(e.g. dotnet build -f net10.0).

Also pin rust-toolchain.toml to exact version 1.94.0 (was 1.94) to match
what setup-rust-toolchain@v1 pre-installs in CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@badrishc badrishc force-pushed the badrishc/bf-tree-integration-plan branch from 0ee0ba3 to 2707408 Compare April 21, 2026 02:38
badrishc and others added 2 commits April 20, 2026 20:06
@badrishc
Add macOS prebuilt native binaries (x64 + arm64)
2fdfbfc 
Cross-compiled via cargo-zigbuild for x86_64-apple-darwin and
aarch64-apple-darwin. Enables building Garnet on macOS without
Rust installed — .NET RID probing finds the prebuilt dylib at
runtimes/osx-{x64,arm64}/native/.

Also add osx-arm64 ContentWithTargetPath to BfTreeInterop.csproj.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@badrishc
Add BfTreeInterop to Docker restore layer in all Dockerfiles
f3bd428 
The Garnet.server.csproj now references BfTreeInterop.csproj, so the
restore layer needs the csproj copied for dotnet restore to resolve
the project graph.

Verified: Docker build succeeds, RI commands (CREATE, SET, GET, SCAN,
TYPE) all work correctly in the production container using prebuilt
native library from runtimes/linux-x64/native/.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@badrishc badrishc force-pushed the badrishc/bf-tree-integration-plan branch from 9c8344e to f3bd428 Compare April 21, 2026 05:01
tiagonapoli
tiagonapoli reviewed Apr 21, 2026
View reviewed changes
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.cs Outdated
tiagonapoli
tiagonapoli reviewed Apr 21, 2026
View reviewed changes
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.cs
tiagonapoli
tiagonapoli reviewed Apr 21, 2026
View reviewed changes
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Index.cs Outdated
tiagonapoli
tiagonapoli reviewed Apr 21, 2026
View reviewed changes
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Index.cs Outdated
tiagonapoli
tiagonapoli reviewed Apr 21, 2026
View reviewed changes
Comment thread libs/server/Resp/RangeIndex/RangeIndexManager.Locking.cs Outdated
badrishc and others added 3 commits April 21, 2026 17:16
@badrishc
Harden stub size validation: use != and throw on corrupt AOF entries
99b6e79 
Change all stubSpan.Length checks from < to != for exact size matching.
A mismatched stub size (too short OR too long) signals corruption.

In HandleRangeIndexCreateReplay, throw GarnetException instead of
silently returning — corrupt AOF entries during recovery should fail
loudly rather than silently losing data.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@badrishc
Address open PR review comments from tiagonapoli
4ddf3fb 
1. Log error on BfTree creation failure during AOF replay instead of
   silently swallowing the exception (RangeIndexManager.cs:536).

2. Remove unused DisposeTreeIfOwned method — all callers use
   DisposeTreeUnderLock which acquires the exclusive lock
   (RangeIndexManager.Index.cs).

3. Change SetFlushedFlag and ClearTreeHandle parameter from
   ReadOnlySpan<byte> to Span<byte> — these methods mutate the span
   via Unsafe.As, so the type should reflect the mutation
   (RangeIndexManager.Index.cs).

4. Throw GarnetException instead of returning NOTFOUND when
   ReadRangeIndex gets an unexpected stub size — this indicates
   store corruption and should fail loudly (RangeIndexManager.Locking.cs).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@badrishc
Merge remote-tracking branch 'origin/dev' into badrishc/bf-tree-integ…
e90dddd 
…ration-plan
@badrishc badrishc merged commit 6e33956 into dev Apr 22, 2026
30 checks passed
@badrishc badrishc deleted the badrishc/bf-tree-integration-plan branch April 22, 2026 01:12
@badrishc badrishc restored the badrishc/bf-tree-integration-plan branch April 22, 2026 01:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kevin-montrose kevin-montrose kevin-montrose left review comments

@tiagonapoli tiagonapoli tiagonapoli left review comments

Copilot code review Copilot Copilot left review comments

@TedHartMS TedHartMS Awaiting requested review from TedHartMS TedHartMS is a code owner

@vazois vazois Awaiting requested review from vazois vazois is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@badrishc @kevin-montrose @tiagonapoli