You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The HIP backend enables the WASI-NN GGML plugin to leverage AMD GPUs for accelerated inference using the llama.cpp library. This provides significant performance improvements when running large language models on AMD hardware.
272
+
273
+
**Prerequisites**
274
+
275
+
- AMD GPU with ROCm support
276
+
- Linux operating system (Ubuntu 20.04/22.04, RHEL/CentOS 8+)
277
+
- CMake 3.18 or later
278
+
- ROCm/HIP development environment
279
+
280
+
**Installation**
281
+
282
+
1.**Install ROCm/HIP**
283
+
284
+
Follow the official ROCm installation guide: https://rocm.docs.amd.com/
- `WASMEDGE_PLUGIN_WASI_NN_GGML_LLAMA_HIP`: Enable HIP backend (default: OFF)
322
+
- `WASMEDGE_PLUGIN_WASI_NN_GGML_LLAMA_HIP_ARCH`: Semicolon separated HIP GPU architectures (e.g., `gfx90a;gfx1030`). When set, forwarded to `CMAKE_HIP_ARCHITECTURES`.
323
+
- `CMAKE_HIP_ARCHITECTURES`: (Advanced) Direct CMake variable; normally you can just use the plugin-specific variable above.
324
+
325
+
Once built with HIP support, the WASI-NN plugin will automatically detect and use AMD GPUs when available. No additional configuration is needed in your WebAssembly applications.
326
+
269
327
### Build with llama.cpp Backend on Windows
270
328
271
329
#### Install Dependencies for llama.cpp And Build on Windows
0 commit comments