feat(router): per-workspace model allowlist for managed inference

[zanetworker]

Problem Statement

Managed inference (inference.local) pins every request to a single model. The router unconditionally overwrites the client's model field with the route-configured value (backend.rs:308-313). Every major agent framework and coding tool (12/12 tested) sends different model strings per-request to the same endpoint for different tasks (planning vs execution, coding vs commit messages, chat vs embeddings). The proxy silently discards all of them. No error, no log, no signal.

This is a layer concern. OpenShell is an execution runtime (Layer 2) that should enforce governance, not override harness-level (Layer 3) model decisions. Today:

• Agent Ops cannot curate a set of approved models for a workspace.
• Agent Developers cannot select from approved models per agent role.
• The runtime collapses the entire model selection surface to one knob (openshell inference set --model), making both platform curation and developer selection impossible.

Relationship to existing issues

Seven issues touch this area. Each addresses a valid concern but none propose governed multi-model selection, which is the capability gap.

Issue	Proposal	State	Why it doesn't close the gap
#2039	--model passthrough disables model rewriting entirely	Open, untriaged	Too permissive. Agent can use any model the API key accesses. Operator loses governance. Swings from "one model, operator-chosen" to "any model, agent-chosen" with no middle ground.
#1825	--model model1 model2 multi-model syntax	Open, untriaged	Right need, incomplete design. Does not specify rejection behavior for unlisted models, proxy selection semantics, or interaction with credential injection and audit logging.
#2063	Log or reject model mismatches	Open, maintainer declined incremental changes	Complementary but does not enable multi-model. An agent that needs gpt-4o-mini for commit messages still cannot get it, even if the mismatch is logged.
#776	Per-sandbox inference routes	Open, community discussion	Solves multi-tenant (different sandboxes, different providers). Does not solve multi-model within a single sandbox. Each sandbox still gets one pinned model.
#203	Alias-based multi-model routing (full spike, 15+ files mapped)	Closed by maintainer	Was the comprehensive solution. Closed in favor of the user/system endpoint split, which shipped as inference.local + sandbox-system but does not provide multi-model selection within either route.
#207	User/system endpoint split (the "different direction" from #203)	Closed	sandbox-system is an internal Rust API not reachable from sandbox code. Serves platform functions, not multi-model agent inference.
#1734	Pluggable model routing RFC	Open, stale	Scoped as a metadata contract and plugin interface for cost/residency/sensitivity signals. Valuable long-term architecture but not a concrete solution to the immediate multi-model gap. The allowlist could serve as the first routing policy within a future pluggable framework.

The gap across all seven: today's pinning gives the operator full control but zero flexibility. Passthrough (#2039) gives the agent full flexibility but zero governance. No existing issue proposes the middle ground where the operator defines an approved set and the agent selects within it.

Evidence: affected tools

Agent frameworks (all send per-request model strings to the same endpoint):
OpenAI Agents SDK, LangChain/LangGraph, CrewAI, AutoGen, Semantic Kernel, LlamaIndex.

Coding agents:

Tool	Multi-model pattern	What breaks
Aider	3 roles: architect, editor, weak model. Each sends its own model string.	Architect/editor split defeated. Per-model behavior settings (edit format, temperature) mismatch the actual model, causing parse failures.
Codex	4 roles: main, Guardian review, memory extraction, memory consolidation. Uses /v1/responses exclusively.	All sub-task routing nullified.
Claude Code	Per-subagent model selection (Haiku for Explore, Opus for review).	Subagent tiering nullified. Capability detection breaks (substring matching on model names for feature gating).
OpenCode	4 agent types: coder, task, title, summarizer. Each independently configurable.	Cross-role model differentiation lost.
Cline	Plan/Act modes with separate model configs.	Planning and acting use same model.
Continue	4 roles: chat, autocomplete, embed, rerank.	Embedding model rewritten to chat model — vector dimensions mismatch the index, retrieval fails.
NemoClaw	Per-agent model configs in openclaw.json.	Config is structural fiction — all requests model-pinned by OpenShell. Three closed NemoClaw issues: #994, #1248, #6315.

Proposed Design

Introduce a model allowlist per inference route. The operator approves N models; the proxy passes through the app's model field if it's on the list; rejects (4xx) or falls back to a default if it's not.

User-facing interface:

# Operator approves specific models for this workspace
openshell inference set \
  --provider openai-prod \
  --model gpt-4o \
  --allowed-models gpt-4o,gpt-4o-mini,o3-mini

# Or allow all models the provider serves
openshell inference set \
  --provider openai-prod \
  --model gpt-4o \
  --allowed-models '*'

How --model and --allowed-models interact:

--model serves a dual role depending on whether an allowlist is present:

Allowlist	Role of --model	Behavior
Absent (default)	Pinned model. Every request is rewritten to this value.	Current behavior, fully backward-compatible.
Explicit list	Default/fallback model. Used when the agent omits the model field or requests a model not on the list.	Proxy passes through the agent's model if it's on the list. Falls back to --model or rejects with 4xx otherwise.
'*' (wildcard)	Default model. Used only when the agent omits the model field.	Proxy passes through any model the agent sends. Equivalent to #2039's passthrough, but opt-in per route.

Interaction with curated backends (MaaS, model gateways, multi-model vLLM):

When the upstream is a curated model serving platform (e.g., a MaaS gateway, an enterprise model gateway, or a vLLM instance hosting multiple models), the backend already governs which models are available. The allowlist operates at a different scope:

• The backend curates at the infrastructure level: what models are deployed and serving.
• The allowlist curates at the workspace level: what subset of those models this specific team or agent is permitted to use.

A backend may serve 20 models. The security team's workspace should only reach 3 production models; the research team can reach all 20. This is a workspace-scoped policy decision that the backend does not make.

The operator chooses the right mode for their deployment:

Backend type	Recommended allowlist	Rationale
Raw cloud API (OpenAI, Anthropic)	Explicit list	OpenShell is the only governance layer. Restricts which of the provider's models this workspace may use.
Curated backend (MaaS, model gateway)	'*' passthrough	Backend already curates. Avoids duplicating the catalog in OpenShell and the sync burden when the backend adds or removes models.
Curated backend + workspace restrictions	Explicit subset	Defense in depth. Different teams get different subsets of the same backend's catalog.

This composability is why the allowlist is preferable to #2039's all-or-nothing passthrough: it supports all three patterns through a single mechanism rather than a binary toggle.

Where it changes:

• crates/openshell-router/src/backend.rs: prepare_backend_request skips the obj.insert("model", ...) when the client model is in the allowlist.
• crates/openshell-router/src/config.rs: ResolvedRoute gains an allowed_models: Vec<String> field.
• crates/openshell-server/src/inference.rs: InferenceRouteConfig stores the allowlist. Validation rejects empty strings in the list.
• crates/openshell-cli/src/main.rs: --allowed-models flag on inference set and inference update.
• proto/inference.proto: InferenceRouteConfig and ResolvedRoute gain a repeated string allowed_models field.

What does not change:

• Credential injection: the proxy still strips sandbox credentials and injects the provider's API key regardless of which model is selected.
• Audit logging: OCSF events should record both the requested model and the served model (closes the observability gap in feat(router): surface or optionally reject client models overwritten by managed-inference pinning #2063).
• Provider binding: the route still binds to one provider. Cross-provider routing is out of scope for this issue.
• Default behavior: routes without --allowed-models behave exactly as today.

Alternatives Considered

1. Passthrough (inference set --model passthrough — preserve client-supplied model in managed inference #2039): All-or-nothing. The operator cannot restrict which models agents use. Acceptable for trusted development environments; insufficient for production workspaces where model access must be governed.

2. Alias-based routing (feat: support multiple models with alias-based routing in inference #203): Full multi-model with named aliases and per-alias provider binding. More comprehensive, but was closed. The allowlist is a simpler first step that does not require alias resolution, proto restructuring, or changes to route selection logic. It composes well: alias routing could layer on top of the allowlist in the future.

3. Per-sandbox routes (feat: Support multiple sandboxes with different inference providers #776): Solves multi-tenant isolation (different sandboxes, different providers). Orthogonal to multi-model within a single sandbox.

4. Model discovery validation: Validate the client model against the provider's /v1/models response. Provider-dependent, does not express operator intent (some models may be available but not approved for a given workspace), and adds a runtime dependency on model catalog availability.

The allowlist is the minimal change that introduces governance without requiring a new routing architecture.

Agent Investigation

Investigated at HEAD. Key findings:

• Model rewrite is unconditional for standard routes in prepare_backend_request (backend.rs:308-313). No conditions skip it. No passthrough, allowlist, or feature flag exists.
• route.model is a required String; empty values rejected at CLI (main.rs:1241), server (inference.rs:283-284), and bundle resolution (inference.rs:1120).
• Integration test proxy_overrides_model_in_request_body (backend_integration.rs:256) explicitly verifies the overwrite.
• OCSF audit log (proxy.rs:2442) records the requested model via extract_model_from_request, not the served model. The mismatch is invisible in audit data.
• Routes are workspace-scoped (not gateway-scoped as some docs state). Each store operation takes a workspace parameter.
• The sandbox-system route is internal-only (Rust API, not HTTP). Agent code cannot reach it. It does not help with multi-model workflows.
• Reviewed all seven related issues. None propose governed selection (allowlist). The maintainer's position (from feat(router): surface or optionally reject client models overwritten by managed-inference pinning #2063): "Until we can take a deeper look at the inference.local functionality and UX more holistically I am inclined to not add incremental changes here."

Checklist

• I've reviewed existing issues and the architecture docs
• This is a design proposal, not a "please build this" request