Skip to content

feat(embedding): add bge-small-en-v1.5-f16 as local embedding model#2233

Open
hammerhoundai wants to merge 2 commits into
volcengine:mainfrom
hammerhoundai:feature/add-english-local-embedding
Open

feat(embedding): add bge-small-en-v1.5-f16 as local embedding model#2233
hammerhoundai wants to merge 2 commits into
volcengine:mainfrom
hammerhoundai:feature/add-english-local-embedding

Conversation

@hammerhoundai
Copy link
Copy Markdown

Summary

Adds bge-small-en-v1.5-f16 as a built-in local embedding model option alongside the existing Chinese bge-small-zh-v1.5-f16.

Type of Change

  • New feature (feat)
  • Bug fix (fix)
  • Documentation (docs)
  • Refactoring (refactor)
  • Other

Motivation

The only local embedding model currently supported is bge-small-zh-v1.5-f16, which is optimized for Chinese text. Non-Chinese users following the quickstart guide get a suboptimal embedding model with no way to select an English alternative without patching source code.

Changes

  • Add bge-small-en-v1.5-f16 to LOCAL_DENSE_MODEL_SPECS (384-dim, ~65MB f16 GGUF from HuggingFace)
  • No query instruction needed (English BGE models don't use one)
  • Model is auto-downloaded to ~/.cache/openviking/models/ on first use (same pattern as the Chinese model)

Usage

{
  "embedding": {
    "dense": {
      "provider": "local",
      "model": "bge-small-en-v1.5-f16",
      "dimension": 384
    }
  }
}

Testing

  • Manual testing completed
    • Verified embedding produces 384-dim vectors
    • Verified semantic similarity: related texts score 0.85, unrelated score 0.39
    • Verified model auto-downloads and loads correctly
  • Ruff lint and format pass

Suggestion

It would also be worth updating openviking-server init to offer both language variants as options during setup, so users can pick the right one at config time.

Checklist

  • Code follows project style guidelines (ruff check + format pass)
  • Documentation updated (usage example included above)
  • All tests pass (only local_embedders.py changed; no test coverage impacted)

Add English-optimized BGE embedding model as a built-in local option
alongside the existing Chinese bge-small-zh-v1.5-f16.

- Model: bge-small-en-v1.5-f16 (384-dim, ~65MB f16 GGUF)
- Auto-downloaded to ~/.cache/openviking/models/ on first use
- No query instruction needed (unlike Chinese variant)
- Runs on CPU, no GPU required

Usage in ov.conf:
  embedding.dense.provider = "local"
  embedding.dense.model = "bge-small-en-v1.5-f16"
  embedding.dense.dimension = 384
@github-actions
Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 1 🔵⚪⚪⚪⚪
🏅 Score: 80
🧪 No relevant tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ Recommended focus areas for review

Filename Mismatch

Model filename in spec uses hyphen ('bge-small-en-v1.5-f16.gguf') but download URL points to file with underscore ('bge-small-en-v1.5_fp16.gguf'). This will cause model loading to fail after download because the saved filename won't match the expected filename.

filename="bge-small-en-v1.5-f16.gguf",
download_url=(
    "https://huggingface.co/ChristianAzinn/bge-small-en-v1.5-gguf/resolve/main/"
    "bge-small-en-v1.5_fp16.gguf?download=true"

@github-actions
Copy link
Copy Markdown

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Fix filename/URL mismatch

Fix filename/URL mismatch: the filename uses a hyphen before "f16" while the
download URL uses an underscore. Align the filename to match the actual file being
downloaded.

openviking/models/embedder/local_embedders.py [49-53]

-filename="bge-small-en-v1.5-f16.gguf",
+filename="bge-small-en-v1.5_fp16.gguf",
 download_url=(
     "https://huggingface.co/ChristianAzinn/bge-small-en-v1.5-gguf/resolve/main/"
     "bge-small-en-v1.5_fp16.gguf?download=true"
 ),
Suggestion importance[1-10]: 8

__

Why: The suggestion correctly identifies a filename/URL mismatch for the new "bge-small-en-v1.5-f16" model, which would cause issues when the code tries to reference the downloaded file. Fixing this is critical to ensure the new model works as intended.

Medium

@hammerhoundai
Copy link
Copy Markdown
Author

The filename mismatch is intentional and correct. The _download_model method saves the downloaded content to target (derived from spec.filename), not from the URL filename. The model downloads from the HuggingFace URL and is saved as bge-small-en-v1.5-f16.gguf locally. This was tested and verified working — produces correct 384-dim embeddings.

Add bge-small-en-v1.5-f16 to LOCAL_GGUF_PRESETS so it appears as an
option during 'openviking-server init' llama.cpp setup flow.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant