Skip to content

Commit 426ef7c

Browse files
committed
docs: add WASI-NN GGML HIP backend documentation
1 parent 68bd9ba commit 426ef7c

File tree

1 file changed

+58
-0
lines changed

1 file changed

+58
-0
lines changed

docs/contribute/source/plugin/wasi_nn.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,6 +266,64 @@ cmake -GNinja -Bbuild -DCMAKE_BUILD_TYPE=Release \
266266
cmake --build build
267267
```
268268

269+
#### Ubuntu/Debian with HIP on AMD GPU
270+
271+
The HIP backend enables the WASI-NN GGML plugin to leverage AMD GPUs for accelerated inference using the llama.cpp library. This provides significant performance improvements when running large language models on AMD hardware.
272+
273+
**Prerequisites**
274+
275+
- AMD GPU with ROCm support
276+
- Linux operating system (Ubuntu 20.04/22.04, RHEL/CentOS 8+)
277+
- CMake 3.18 or later
278+
- ROCm/HIP development environment
279+
280+
**Installation**
281+
282+
1. **Install ROCm/HIP**
283+
284+
Follow the official ROCm installation guide: https://rocm.docs.amd.com/
285+
286+
For **Ubuntu 22.04**:
287+
```bash
288+
sudo apt update
289+
wget -q -O - https://repo.radeon.com/rocm/rocm.gpg.key | sudo apt-key add -
290+
echo "deb [arch=amd64] https://repo.radeon.com/rocm/apt/5.7/ ubuntu main" | sudo tee /etc/apt/sources.list.d/rocm.list
291+
sudo apt update
292+
sudo apt install -y rocm-dev rocm-utils hipblas hip-dev
293+
```
294+
295+
2. **Setup Environment**
296+
297+
You can manually set environment variables:
298+
```bash
299+
export ROCM_PATH=/opt/rocm
300+
export PATH=$ROCM_PATH/bin:$PATH
301+
export LD_LIBRARY_PATH=$ROCM_PATH/lib:$LD_LIBRARY_PATH
302+
```
303+
304+
3. **Build WasmEdge with HIP Support**
305+
306+
Build manually:
307+
```bash
308+
cd <path/to/your/wasmedge/source/folder>
309+
mkdir build-hip && cd build-hip
310+
cmake .. \
311+
-DWASMEDGE_PLUGIN_WASI_NN_GGML_LLAMA_HIP=ON \
312+
-DWASMEDGE_PLUGIN_WASI_NN_GGML_LLAMA_HIP_ARCH=gfx90a;gfx1030 \
313+
-DWASMEDGE_PLUGIN_WASI_NN_BACKEND=ggml \
314+
-DWASMEDGE_BUILD_PLUGINS=ON \
315+
-DCMAKE_BUILD_TYPE=Release
316+
cmake --build . --target wasmedgePluginWasiNN -j$(nproc)
317+
```
318+
319+
**Configuration Options**
320+
321+
- `WASMEDGE_PLUGIN_WASI_NN_GGML_LLAMA_HIP`: Enable HIP backend (default: OFF)
322+
- `WASMEDGE_PLUGIN_WASI_NN_GGML_LLAMA_HIP_ARCH`: Semicolon separated HIP GPU architectures (e.g., `gfx90a;gfx1030`). When set, forwarded to `CMAKE_HIP_ARCHITECTURES`.
323+
- `CMAKE_HIP_ARCHITECTURES`: (Advanced) Direct CMake variable; normally you can just use the plugin-specific variable above.
324+
325+
Once built with HIP support, the WASI-NN plugin will automatically detect and use AMD GPUs when available. No additional configuration is needed in your WebAssembly applications.
326+
269327
### Build with llama.cpp Backend on Windows
270328

271329
#### Install Dependencies for llama.cpp And Build on Windows

0 commit comments

Comments
 (0)