🚀 KoboldCPP Vector Database

• 🔥 Supported on Windows systems with an NVIDIA GPU.

REQUIREMENTS

1. 🐍 Python 3.11, 3.12, or 3.13

2. 📁 Git

3. 📁 Git Large File Storage

4. 🌐 Pandoc

5. CUDA does NOT need to be installed system-wide — the installer pulls all the CUDA 12.8 runtime wheels it needs.

6. Build Tools.

   Some dependencies don't ship pre-compiled "wheels" so you must be able to build them with Microsoft Build Tools and/or Visual Studio. Visual Studio is recommended — make sure to select the "Desktop development with C++" workload and check the four boxes on the right that contain "SDK."

   EXAMPLE ERROR ON WINDOWS

   
   EXAMPLE SOLUTION ON WINDOWS

   

INSTALLATION

Step 1

Download the latest "release," extract its contents, and navigate to the src folder to run the commands below.

NOTE: If you clone this repository directly you will get the in-development version of this program, which may or may not be stable.

Step 2

Open a command prompt inside the src folder and create a virtual environment:

python -m venv .

Step 3

Activate the virtual environment:

.\Scripts\activate

Step 4

Run the setup script. It will:

1. Walk you through pre-flight checks (Python version, NVIDIA GPU, Git/Git LFS/Pandoc/Build Tools).
2. Install uv as the package manager.
3. Install torch 2.9 with CUDA 12.8, flash-attention, and the matching NVIDIA wheels.
4. Install the rest of the project's pinned dependencies.
5. Overlay the project's patched versions of pdf.py, instructor.py, and SentenceTransformer.py into the freshly installed packages.

python setup_windows.py

If something goes wrong and you want to reinstall from scratch, the easiest path is to deactivate, delete the venv folder, and start again from Step 2. If you'd rather wipe just the installed packages:

pip freeze > requirements_uninstall.txt && pip uninstall -r requirements_uninstall.txt -y && del requirements_uninstall.txt

Step 5

Launch the GUI:

python gui.py

USAGE

Start Kobold

• Run python download_kobold.py from your activated virtual environment to grab the latest KoboldCpp binary, or download the appropriate executable directly from KoboldCpp's releases page. The installer presents three options:
  • koboldcpp.exe — default, includes CUDA. Pick this if you have an NVIDIA GPU.
  • koboldcpp-nocuda.exe — Vulkan/CPU only. Pick this if you don't have an NVIDIA GPU.
  • koboldcpp-oldpc.exe — for older CPUs without AVX2.
• Configure your launch options as desired, but uncheck "open browser". The "quiet" option is recommended to keep KoboldCpp's terminal output from duplicating.