Skip to content

Commit 634a712

Browse files
committed
Update README.md for OpenVINO/Metal/Vulkan/SYCL
Signed-off-by: JamePeng <jame_peng@sina.com>
1 parent f22b819 commit 634a712

File tree

1 file changed

+94
-4
lines changed

1 file changed

+94
-4
lines changed

README.md

Lines changed: 94 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,8 @@ Installing a CUDA-supported version requires the `CUDA Toolkit` environment to b
132132

133133
See here: https://developer.nvidia.com/cuda-toolkit-archive
134134

135+
More Information see: https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#cuda
136+
135137
Then, set the `GGML_CUDA=on` environment variable before installing:
136138

137139
```bash
@@ -169,13 +171,64 @@ CMAKE_ARGS="-DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS" pip install "llama-cpp-p
169171
```
170172
</details>
171173

174+
<details>
175+
<summary>OpenVINO</summary>
176+
177+
### Install OpenVINO Runtime
178+
179+
Follow the guide to install OpenVINO Runtime from an archive file: [Linux](https://docs.openvino.ai/2026/get-started/install-openvino/install-openvino-archive-linux.html) | [Windows](https://docs.openvino.ai/2026/get-started/install-openvino/install-openvino-archive-windows.html)
180+
181+
- **Linux:**
182+
183+
<details>
184+
<summary>📦 Click to expand OpenVINO installation from an archive file on Ubuntu</summary>
185+
<br>
186+
187+
```bash
188+
wget https://raw.githubusercontent.com/ravi9/misc-scripts/main/openvino/ov-archive-install/install-openvino-from-archive.sh
189+
chmod +x install-openvino-from-archive.sh
190+
./install-openvino-from-archive.sh
191+
```
192+
193+
Verify OpenVINO is initialized properly:
194+
```bash
195+
echo $OpenVINO_DIR
196+
```
197+
</details>
198+
199+
### Supported Devices
200+
201+
OpenVINO backend supports the following hardware:
202+
203+
- Intel CPUs
204+
- Intel GPUs (integrated and discrete)
205+
- Intel NPUs
206+
207+
Although OpenVINO supports a wide range of [Intel hardware](https://docs.openvino.ai/2026/about-openvino/release-notes-openvino/system-requirements.html), the llama.cpp OpenVINO backend has been validated specifically on AI PCs such as the Intel® Core™ Ultra Series 1 and Series 2.
208+
209+
More Information see: https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/OPENVINO.md
210+
211+
To install with OpenVINO, set the `GGML_OPENVINO=ON` environment variable before installing:
212+
213+
```bash
214+
# Linux
215+
source /opt/intel/openvino/setupvars.sh
216+
# Windows
217+
"C:\Program Files (x86)\Intel\openvino_2026.0\setupvars.bat"
218+
# Build
219+
CMAKE_ARGS="-DGGML_OPENVINO=ON" pip install "llama-cpp-python @ git+https://github.com/JamePeng/llama-cpp-python.git"
220+
```
221+
</details>
222+
172223
<details>
173224
<summary>Metal</summary>
174225

175-
To install with Metal (MPS), set the `GGML_METAL=on` environment variable before installing:
226+
On MacOS, Metal is enabled by default(`GGML_METAL=ON`). Using Metal makes the computation run on the GPU.
227+
228+
To disable the Metal build at compile time use the `CMAKE_ARGS="-DGGML_METAL=OFF"` cmake option.
176229

177230
```bash
178-
CMAKE_ARGS="-DGGML_METAL=on -DGGML_METAL_USE_BF16=on" pip install "llama-cpp-python @ git+https://github.com/JamePeng/llama-cpp-python.git"
231+
pip install "llama-cpp-python @ git+https://github.com/JamePeng/llama-cpp-python.git"
179232
```
180233

181234
**Pre-built Wheel (New)**
@@ -213,7 +266,20 @@ More details see here: https://github.com/ggml-org/llama.cpp/blob/master/docs/bu
213266
214267
- For Windows User: Download and install the [`Vulkan SDK`](https://vulkan.lunarg.com/sdk/home#windows) with the default settings.
215268
216-
- For Linux User: Follow the official LunarG instructions for the installation and setup of the Vulkan SDK in the [Getting Started with the Linux Tarball Vulkan SDK](https://vulkan.lunarg.com/doc/sdk/latest/linux/getting_started.html) guide.
269+
- For Linux User:
270+
* First, follow the official LunarG instructions for the installation and setup of the Vulkan SDK in the [Getting Started with the Linux Tarball Vulkan SDK](https://vulkan.lunarg.com/doc/sdk/latest/linux/getting_started.html) guide.
271+
272+
* After completing the first step, ensure that you have used the `source` command on the `setup_env.sh` file inside of the Vulkan SDK in your current terminal session. Otherwise, the build won't work. Additionally, if you close out of your terminal, you must perform this step again if you intend to perform a build. However, there are ways to make this persistent. Refer to the Vulkan SDK guide linked in the first step for more information about any of this.
273+
274+
- For Mac User:
275+
* Generally, follow LunarG's [Getting Started with the MacOS Vulkan SDK](https://vulkan.lunarg.com/doc/sdk/latest/mac/getting_started.html) guide for installation and setup of the Vulkan SDK. There are two options of Vulkan drivers on macOS, both of which implement translation layers to map Vulkan to Metal. They can be hot-swapped by setting the `VK_ICD_FILENAMES` environment variable to point to the respective ICD JSON file. Check the box for "KosmicKrisp" during the LunarG Vulkan SDK installation.
276+
277+
* Set environment variable for the LunarG Vulkan SDK after installation (and optionally add to your shell profile for persistence):
278+
```bash
279+
source /path/to/vulkan-sdk/setup-env.sh
280+
```
281+
282+
More Information see: https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#vulkan
217283
218284
Then install with Vulkan support by set the `GGML_VULKAN=on` environment variable before installing:
219285
@@ -226,11 +292,35 @@ CMAKE_ARGS="-DGGML_VULKAN=on" pip install "llama-cpp-python @ git+https://github
226292
<details>
227293
<summary>SYCL</summary>
228294
295+
### Supported OS
296+
297+
| OS | Status | Verified |
298+
|---------|---------|------------------------------------------------|
299+
| Linux | Support | Ubuntu 22.04, Fedora Silverblue 39, Arch Linux |
300+
| Windows | Support | Windows 11 |
301+
302+
### Intel GPU
303+
304+
SYCL backend supports Intel GPU Family:
305+
306+
- Intel Data Center Max Series
307+
- Intel Flex Series, Arc Series
308+
- Intel Built-in Arc GPU
309+
- Intel iGPU in Core CPU (11th Generation Core CPU and newer, refer to [oneAPI supported GPU](https://www.intel.com/content/www/us/en/developer/articles/system-requirements/intel-oneapi-base-toolkit-system-requirements.html#inpage-nav-1-1)).
310+
311+
On older Intel GPUs, you may try [OpenCL](/docs/backend/OPENCL.md) although the performance is not optimal, and some GPUs may not support OpenCL nor have any GPGPU capabilities.
312+
313+
More Information see here: https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/SYCL.md
314+
229315
To install with SYCL support, set the `GGML_SYCL=on` environment variable before installing:
230316
231317
```bash
232-
source /opt/intel/oneapi/setvars.sh
318+
# Export relevant ENV variables
319+
source /opt/intel/oneapi/setvars.sh
320+
# Option 1: Use FP32 (recommended for better performance in most cases)
233321
CMAKE_ARGS="-DGGML_SYCL=on -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx" pip install "llama-cpp-python @ git+https://github.com/JamePeng/llama-cpp-python.git"
322+
# Option 2: Use FP16
323+
CMAKE_ARGS="-DGGML_SYCL=on -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_SYCL_F16=ON" pip install "llama-cpp-python @ git+https://github.com/JamePeng/llama-cpp-python.git"
234324
```
235325
</details>
236326

0 commit comments

Comments
 (0)