Short term roadmap

[mudler]

I'm trying to collect most-wanted features by the community, along aside what I think should be there. This is a under-construction list, so feel free to chime in with what you'd like to see.

• feature: whisper api #138
• feature: model gallery #100
• feature: embedding support #70
• feature: binary releases #66
• feature: stopwords #64
• feature: unbuffered token stream #109

This is a community-driven project! Feel free to submit PRs and actively partecipate to development! If you want to have a look at any of the issues, just comment on it.

[kinthaiofficial]

The short-term roadmap looks solid. A few areas I'd be curious about the prioritization on:

Agent-to-agent API surface: Right now LocalAI is great for "one app talks to one model". As multi-agent patterns become more common, the question is whether LocalAI will support agent-to-agent calls that stay local — so agent A can call agent B without either of them needing to go out to an external API. This would be a meaningful differentiator vs cloud-only multi-agent frameworks.

Per-request cost attribution: When you have multiple apps sharing a LocalAI instance, knowing which app consumed which tokens matters for internal chargeback and capacity planning. Even a simple request-level metadata tag that flows through to logging would help.

Context window management for long sessions: Local models tend to have smaller context windows than cloud counterparts. Some kind of built-in compaction strategy (summarize old turns, keep recent turns verbatim) would make LocalAI more viable for stateful applications.

Health endpoint enrichment: The current health check tells you if the service is up, but not if a specific model is loaded and ready. For production deployments, you want /health/models/{model_id} → {status: "ready" | "loading" | "unloaded", loaded_context_size, requests_in_flight}.

LocalAI's positioning as privacy-first is genuinely valuable. The limiting factor has been closing the capability gap with cloud APIs for stateful, multi-turn, multi-agent use cases. Would love to see those on the roadmap.