Extend Mac Support to all backends

[rodrigoalviani]

LocalAI version:
v3.10.1 (dmg)

Environment, CPU architecture, OS, and Version:
Darwin Kernel Version 25.2.0, Apple M1 Pro, Tahoe 26.2

Describe the bug
The error: 'Error installing backend "ANY BACKEND": not a valid backend: run file not found' occurs for every backend I attempt to install through the web interface. The models are being installed normally.

To Reproduce
Try to install any backend

Expected behavior
Successfully instalation

Logs
[10:14:11] STDOUT: Jan 26 10:14:11 INFO core/startup process completed!
[10:14:11] STDOUT: Jan 26 10:14:11 INFO LocalAI is started and running address="127.0.0.1:9001"
[10:14:15] STDOUT: Jan 26 10:14:15 INFO HTTP request method="POST" path="/api/backends/install/localai@metal-stablediffusion-ggml" status=200
[10:14:15] STDOUT: Jan 26 10:14:15 WARN installing backend backend="localai@metal-stablediffusion-ggml"
[10:14:15] STDOUT: Jan 26 10:14:15 INFO Using metal capability (arm64 on mac) env="LOCALAI_FORCE_META_BACKEND_CAPABILITY"
[10:14:15] STDOUT: Jan 26 10:14:15 INFO HTTP request method="GET" path="/api/backends/job/e9ced6f2-fab8-11f0-ad82-8a045046c4bf" status=200
[10:14:15] STDOUT: Jan 26 10:14:15 INFO Using metal capability (arm64 on mac) env="LOCALAI_FORCE_META_BACKEND_CAPABILITY"
[10:14:15] STDOUT: Jan 26 10:14:15 Run file not found runFile="/Users/xpto/.localai/backends/metal-stablediffusion-ggml/run.sh"
[10:14:15] STDOUT: Jan 26 10:14:15 error installing backend error=not a valid backend: run file not found "/Users/xpto/.localai/backends/metal-stablediffusion-ggml/run.sh" backend="localai@metal-stablediffusion-ggml"

Additional context
I also tried changing the server ports, but that didn’t solve the issue either

[leonbollerup]

Sane mere - MacOS Tahoe, M4 mini and Macbook Air

[ericblade]

Strong suspicion it's trying to download binary packages for a platform that doesn't exist.

If you're into building a version with a patch, you might try grabbing this ericblade@fb8e501 change and rebuilding, it fixes so download failures have more useful error messages.

[rodrigoalviani]

@ericblade I don't think so. None of the backends install, not even those that use the CPU. And no, I did not build the application. I used the precompiled packages.

[miglosch]

I encountered the same error (see also #8261): no backend could be installed. Unfortunately, there are no instructions on how to install the backend manually — at least, I couldn't find any. I downloaded llama-cpp, but there is no information on how to add it to LocalAI. (Apple Mac Mini M4Pro, macOS Tahoe, LocalAI v3.10.1).

[mudler]

logs tells that likely an empty image was downloaded. We did have many changes on 3.10 regarding backend capabilities, so we might have introduced regressions. However, since Mac is not part of our automated test suite we don't catch these quite easily. I'll grab my mac Mini and test it here this weekend.

[aeyecandy]

I have included debug logs from both command line and Launcher execution of the server in #8261. Sorry I did not find this issue when I posted mine.

[ddstevens]

Not sure if it is related, but not all the Python dependencies are being installed with the chatterbox backend on macOS. Here is the lead up to a "ModuleNotFoundError: No module named 'torch'" error:

Jan 28 14:05:52 DEBUG GRPC stdout id="chatterbox-127.0.0.1:61587" line="Initializing libbackend for metal-chatterbox" caller={caller.file="/home/runner/work/LocalAI/LocalAI/pkg/model/process.go" caller.L=158 }
Jan 28 14:05:52 DEBUG GRPC stdout id="chatterbox-127.0.0.1:61587" line="Using portable Python" caller={caller.file="/home/runner/work/LocalAI/LocalAI/pkg/model/process.go" caller.L=158 }
Jan 28 14:05:53 DEBUG GRPC stderr id="chatterbox-127.0.0.1:61587" line="Traceback (most recent call last):" caller={caller.file="/home/runner/work/LocalAI/LocalAI/pkg/model/process.go" caller.L=149 }
Jan 28 14:05:53 DEBUG GRPC stderr id="chatterbox-127.0.0.1:61587" line=" File "~/localai/backends/metal-chatterbox/backend.py", line 14, in " caller={caller.file="/home/runner/work/LocalAI/LocalAI/pkg/model/process.go" caller.L=149 }
Jan 28 14:05:53 DEBUG GRPC stderr id="chatterbox-127.0.0.1:61587" line=" import torch" caller={caller.file="/home/runner/work/LocalAI/LocalAI/pkg/model/process.go" caller.L=149 }
Jan 28 14:05:53 DEBUG GRPC stderr id="chatterbox-127.0.0.1:61587" line="ModuleNotFoundError: No module named 'torch'" caller={caller.file="/home/runner/work/LocalAI/LocalAI/pkg/model/process.go" caller.L=149 }

[mudler]

I have verified that llama.cpp backends/models works as expected.

We don't build (yet) all the backend for MacOS, because of lack of testing. I can try to enable these in a separate PR - but so far diffusers, llama-cpp , whisper and chatterbox are built for Mac.