Skip to content

Add cuvs-bench-elastic: HTTP backend for Elasticsearch GPU vector search#1907

Draft
afourniernv wants to merge 4 commits intorapidsai:mainfrom
afourniernv:fea-1856-cuvs-lucene-backend
Draft

Add cuvs-bench-elastic: HTTP backend for Elasticsearch GPU vector search#1907
afourniernv wants to merge 4 commits intorapidsai:mainfrom
afourniernv:fea-1856-cuvs-lucene-backend

Conversation

@afourniernv
Copy link

Introduce cuvs-bench-elastic as an optional plugin for cuvs-bench that provides an Elasticsearch backend. The backend communicates with Elasticsearch via HTTP and supports HNSW indexing with optional GPU acceleration when using the Elasticsearch GPU image (cuVS-accelerated vector search).

  • Add cuvs_bench_elastic package with backend and config loader entry points
  • Extend cuvs_bench registry and search spaces for pluggable backends
  • Add elastic and integration optional dependencies to cuvs-bench
  • Add modularization tests and integration test scaffolding (disabled until CI has ES GPU image, cuVS libs, and GPU runner)

@copy-pr-bot
Copy link

copy-pr-bot bot commented Mar 10, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@cjnolet cjnolet added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Mar 10, 2026
ep.load()()
except ImportError as e:
if "elasticsearch" in str(e).lower() or "elasticsearch" in str(e):
raise ImportError(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. This is what I discussed with @cjnolet pertaining to lazy import and used Milvus as an example

class MilvusBackend(BenchmarkBackend):
    def __init__(self, config: Dict[str, Any]):
        super().__init__(config)
        try:
            from pymilvus import connections, Collection
        except ImportError:
            raise ImportError(
                "pymilvus is required for MilvusBackend. "
                "Install with: pip install pymilvus"
            )
        connections.connect(host=config["host"], port=config["port"])

Introduce cuvs-bench-elastic as an optional plugin for cuvs-bench that
provides an Elasticsearch backend. The backend communicates with
Elasticsearch via HTTP and supports HNSW indexing with optional GPU
acceleration when using the Elasticsearch GPU image (cuVS-accelerated
vector search).

- Add cuvs_bench_elastic package with backend and config loader entry points
- Extend cuvs_bench registry and search spaces for pluggable backends
- Add elastic and integration optional dependencies to cuvs-bench
- Add modularization tests and integration test scaffolding (disabled until
  CI has ES GPU image, cuVS libs, and GPU runner)

Signed-off-by: Alex Fournier <afournier@nvidia.com>
Use single-doc format (_index, _id, vector_field) instead of two-part
NDJSON (index action + source) so ES accepts the bulk request.

Signed-off-by: Alex Fournier <afournier@nvidia.com>
Expose ELASTIC constant and convenience wrappers for build-only,
search-only, or full benchmark runs.

Signed-off-by: Alex Fournier <afournier@nvidia.com>
@afourniernv afourniernv force-pushed the fea-1856-cuvs-lucene-backend branch from ec0b3e3 to 7b25521 Compare March 20, 2026 16:45
- Document run_build, run_search, run_benchmark convenience API
- Document ELASTIC constant and orchestrator usage
- Add username/password support in config loader (converts to basic_auth)

Signed-off-by: Alex Fournier <afournier@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improves an existing functionality non-breaking Introduces a non-breaking change

Projects

Development

Successfully merging this pull request may close these issues.

3 participants