Feature Description
node-llama-cpp ships prebuilt binaries for linux x64/arm64/armv7l, macOS, and Windows, but not for linux riscv64. On a RISC-V host, npm install node-llama-cpp falls back to building llama.cpp from source. That works, but it pulls in a full C/C++ toolchain and roughly 30 minutes of native compile time. RISC-V Linux boards and CI runners are getting more common, and that from-source step is a real wait for anyone who just wants to install and run.
The Solution
Add a linux-riscv64 prebuilt binary, built on a native riscv64 runner and shipped as a @node-llama-cpp/linux-riscv64 optional dependency, the same way the other per-platform prebuilts already work. The binding resolver gets a riscv64 branch so installs on RISC-V pick up the prebuilt instead of compiling.
Considered Alternatives
Building from source on install already works today, but it costs a toolchain plus that ~30 minute native compile, which is the thing a prebuilt is meant to skip.
Cross-compiling the binary on x64 was the other option. The dist build toolchain (rolldown) has no riscv64 binding yet, and ggml's RVV paths are easier to trust when they're built and run natively, so a native runner felt like the safer route.
Additional Context
The native riscv64 leg uses the RISE RISC-V GitHub App, which provides the ubuntu-24.04-riscv runner. That app needs to be installed on the repo for the leg to get a runner. I can help coordinate that with the RISE project if you want to go this way.
The runner also needs a few platform tweaks I worked out: Node.js from the unofficial-builds index (setup-node has no riscv64 binary), a retry around npm ci (the unofficial riscv64 node occasionally throws internal errors), ELECTRON_SKIP_BINARY_DOWNLOAD (no riscv64 electron prebuilt), and gcc-14 (recent ggml uses RVV _Float16/zvfh vector intrinsics that only exist in GCC 14+).
I validated the whole flow on a fork: dist built on x64, binary compiled natively on the RISE runner, producing an ELF RISC-V llama-addon.node that passes the smoke check. PR is ready, so I'm happy to open it. Open to a different shape if this isn't how you'd want riscv64 handled.
Are you willing to resolve this issue by submitting a Pull Request?
Yes, I have the time, and I know how to start.
Feature Description
node-llama-cpp ships prebuilt binaries for linux x64/arm64/armv7l, macOS, and Windows, but not for linux riscv64. On a RISC-V host,
npm install node-llama-cppfalls back to building llama.cpp from source. That works, but it pulls in a full C/C++ toolchain and roughly 30 minutes of native compile time. RISC-V Linux boards and CI runners are getting more common, and that from-source step is a real wait for anyone who just wants to install and run.The Solution
Add a
linux-riscv64prebuilt binary, built on a native riscv64 runner and shipped as a@node-llama-cpp/linux-riscv64optional dependency, the same way the other per-platform prebuilts already work. The binding resolver gets ariscv64branch so installs on RISC-V pick up the prebuilt instead of compiling.Considered Alternatives
Building from source on install already works today, but it costs a toolchain plus that ~30 minute native compile, which is the thing a prebuilt is meant to skip.
Cross-compiling the binary on x64 was the other option. The dist build toolchain (rolldown) has no riscv64 binding yet, and ggml's RVV paths are easier to trust when they're built and run natively, so a native runner felt like the safer route.
Additional Context
The native riscv64 leg uses the RISE RISC-V GitHub App, which provides the
ubuntu-24.04-riscvrunner. That app needs to be installed on the repo for the leg to get a runner. I can help coordinate that with the RISE project if you want to go this way.The runner also needs a few platform tweaks I worked out: Node.js from the unofficial-builds index (setup-node has no riscv64 binary), a retry around
npm ci(the unofficial riscv64 node occasionally throws internal errors),ELECTRON_SKIP_BINARY_DOWNLOAD(no riscv64 electron prebuilt), and gcc-14 (recent ggml uses RVV_Float16/zvfh vector intrinsics that only exist in GCC 14+).I validated the whole flow on a fork: dist built on x64, binary compiled natively on the RISE runner, producing an ELF RISC-V
llama-addon.nodethat passes the smoke check. PR is ready, so I'm happy to open it. Open to a different shape if this isn't how you'd want riscv64 handled.Are you willing to resolve this issue by submitting a Pull Request?
Yes, I have the time, and I know how to start.