llama-server: --system-prompt-file unavailable

[d-a-v]

The option --system-prompt-file is available with llama-cli but not with llama-server.

What would be the way to get such a feature with the current available options in llama-server ?
I tried to get and update the chat-template from model with no luck.

Is it worth asking for a feature request ?

[CISC]

The --system-prompt option is not available for llama-server either.

I don't think it makes much sense to have it as you either use what the model and/or chat template defaults to, or set it in the UI, alternatively pass it through the API.

[d-a-v]

I understand the point about using the web UI or curl for ad-hoc prompts, but that doesn't fully cover certain specialized use cases. Template defaults is indeed a way to achieve my goal but not generic enough - or a bit complex to modify since models carry their own.

For instance, with large-context LLMs, one could configure an easy-to-use for everybody web-llama-server to share local knowledge and turn it into a dedicated chatbot for a local community. In such scenarios, preloading a system prompt—depending on its size—could be highly efficient and practical for this tailored, always-on setup.

Would it be worth reconsidering the --system-prompt-file option for these kinds of deployments?

[d-a-v]

I successfully preloaded a huge system prompt (260kB txt -> ~100kT) using Qwen3 (ctx:256kT).
In this way this instance has been made into a chatbot for anybody connecting to it.
The provided prompt is swallowed only once for everybody.

Here's the changes, I have some questions for polishing it:

• Would there be a more straightforward way to achieve the same goal ?
• Would it be acceptable as it is ?
• Is there a way to get tags <|im_start|>system\n ... <|im_end|>\n extracted from the chat template (I couldn't find out yet)

https://github.com/ggml-org/llama.cpp/compare/master...d-a-v:llama.cpp:serversystemprompt?expand=1

llama-server --system-prompt-file large-prompt.txt --model models/model.gguf ...

[CISC]

* Would there be a more straightforward way to achieve the same goal ?

Most likely.

* Would it be acceptable as it is ?

No, you need to fill in the system prompt part of the chat completion instead so it either uses the jinja chat template or the chosen chat format. Though, for your purposes I'd imagine you would have to append any user provided system prompt to the one provided at startup. I really don't think that would be acceptable.

* Is there a way to get tags `<|im_start|>system\n` ... `<|im_end|>\n` extracted from the chat template  (I couldn't find out yet)

Not really, you could render a partial chat and figure it out, but that would be the wrong way to go anyway.