feat(models): 模型路由——用户可配的有序模型 failover 链 - #604
Open
angri450 wants to merge 2 commits into
Open
Conversation
…ls page
- New ModelRouting section: ordered model list (1..8), add/move/remove,
saved via PATCH /api/preferences { model_routing: [...] }
- preferences api types + useProviders expose modelRouting
- i18n zh/en
- preferences: model_routing key (validated refs, dedupe, max 8) stored per-user next to preferred_model; PATCH validates every ref is usable - processor: resolve the routing chain per turn (drop unusable refs, multimodal-aware head) and pass it to harness via build_harness_request configurable; chain[0] becomes the turn model while routing is set - dashboard: Models page routing section (ordered list 1..8 with add/move/remove, instant save via PATCH /api/preferences), zh/en i18n - runtime failover itself lives in harness_agent (ModelFailoverMiddleware reads configurable[model_routing]); harmless no-op until the harness package ships the middleware
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
功能
模型设置页新增「模型路由」：把可用模型手动排成有序链（1..8）。调用时按顺序执行——第 1 个模型失败（限流 / 故障 / 不可用）自动切换第 2 个，依此类推，全链耗尽才报错。解决单一上游模型 RPM 限流 / 故障导致整轮对话失败的问题。
启用路由后优先于偏好模型（星星）生效；未配置路由时行为完全不变。
本仓库职责
运行时 failover 位于 harness_agent 层（ModelFailoverMiddleware，读取
configurable["model_routing"]按序切换request.model；与 PR #600 的媒体 provider 同样属于 harness 包，随包发布）。本仓库负责：users/preferences.py+users/manager.py）：用户级model_routing键，ref 规范化 + 去重 + 上限 8，PATCH 时逐项校验模型可用（不可用直接 400）gateway/process/processor.py+harness_request.py）：每轮解析用户路由链（过滤不可用项 + 多模态头位适配 resolve_model_for_multimodal_turn），链非空时 chain[0] 作为本轮主模型，整链经build_harness_request(configurable)透传给 harness；当前 harness 版本会忽略该键（惰性无害），中间件随包发布后自动激活测试
tests/unit/users/test_preferences.py+4 用例：路由链去重保序 / 超限拒绶4 / merge 往返与清除 / 非法载荷容错，14/14 通过tests/unit/gateway281 项全过（processor 改动回归）