[Request] MP3 support in the packages

[easyjoh]

Hi,

I notice the frontends support audio_decode and mp3_encode.

Would it be possible to provide an docker image with these two frontends enabled? For me it is on the CUDA driver (arm), dgx spark that is.

Thank you for the great work.

[0xShug0]

@easyjoh Thanks for your kind words! And yes, a separate Docker image/release is the plan (#541 (comment)). I just need some time to figure this out.

[easyjoh]

Sounds good. Thank you.

[0xShug0]

@easyjoh Let me know if this works https://github.com/0xShug0/audio.cpp-server-frontends/pkgs/container/audio.cpp-server-frontends

[Tech-no-1]

Hi, could we please get a Vulkan/CPU version as well? Not everyone has got an overpriced Nvidia GPU/system 😅. I can offer testing it again.

[0xShug0]

Sure!

[0xShug0]

Please check the latest release of the frontend repo.

[Tech-no-1]

Okay, so here's an overview of what I tested, including the corresponding result:

• CPU & Vulkan backends/images -> both functional ✅
• Frontend modules: audio_decode, mp3_encode and websocket -> all functional ✅
• Open WebUI STT (transcription) & TTS via http/ws inside an internal docker network -> all functional ✅
• Docker healthcheck via the /health endpoint -> fails because curl is missing, the endpoint is accessible from another container though and returns the expected status
• Not tested: HTTPS module (because isolated/internal only)

Some annotations and thoughts:

• Open WebUI slim images do not include the necessary transcoding components, therefore transcoding to MP3 is not possible before the file is sent to audio.cpp, resulting in a 400 bad request (because Open WebUI is sending webm). Switching to the default (full) image solves this. This is not an issue with audio.cpp, it's just meant for information as I ran into that issue.
• You need to mount //.cache to a volume, if you want the Vulkan (Mesa) shader cache to be persistent. Additionally, if you run the container as non root (unprivileged user) and you do not mount //.cache to a volume, you get a warning that the user does not have the permissions to access //.cache and that it is/gets deactivated. The user running the container (obviously) must have permissions to access the mounted volume. This could be included in the docs for the Vulkan image.
• I don't know if this already exists, but it would be great to have an optional ttl (time-to-live) variable. Globally or even for each model. Currently, the models remain permanently loaded in RAM/VRAM, which is good for fast responses, but not so much for efficient resource management (on a server). They can occupy several GB that could be used by or are needed for other tasks. It would be beneficial to unload them after a certain idle period in order to free up resources.
• A somewhat more sophisticated access management would also be nice 😅, that is a more specific cors_origins and preferably API key authentication.

These are just some thoughts and ideas for the future, I hope it's not overwhelming.

[0xShug0]

@Tech-no-1 Thanks for testing and for the feedback! For TTL, there is a server config idle_unload_ms. This unloads resident models after X secs without model load/run activity across the server. Busy models are skipped, and subsequent requests reload models automatically. The default is 0 (disabled).

Let me look into the other issues you mentioned. I may defer access management for now, as it falls outside the intended scope of a thin inference server and adds maintenance overhead. If you'd like, you're welcome to submit a PR adding it as an optional frontend module tailored to your needs.

[0xShug0]

Doc updated!