Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ratio #2038

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Conversation

WqyJh
Copy link
Contributor

@WqyJh WqyJh commented Jan 17, 2025

close #2023

我已确认该 PR 已自测通过,相关截图如下:

默认倍率
image

自定义倍率配置
image

自定义倍率前后对比
image

@Laisky
Copy link
Contributor

Laisky commented Jan 17, 2025

还是统一使用 ctxkey 比较好,你可以让 KeyRequestBody=gin.BodyBytesKey

@Laisky
Copy link
Contributor

Laisky commented Jan 17, 2025

请教下这个重构的意义是什么,带来了什么新功能吗或解决了什么问题吗?

@WqyJh
Copy link
Contributor Author

WqyJh commented Jan 17, 2025

请教下这个重构的意义是什么,带来了什么新功能吗或解决了什么问题吗?

See #2023

@Laisky
Copy link
Contributor

Laisky commented Jan 19, 2025

看了下,感觉很不错。不过我觉得没必要在页面上保留那两个【已废弃】,能自动将历史数据转为新格式就更好了。

@WqyJh
Copy link
Contributor Author

WqyJh commented Jan 20, 2025

看了下,感觉很不错。不过我觉得没必要在页面上保留那两个【已废弃】,能自动将历史数据转为新格式就更好了。

Good idea~ 已经实现了

截图中展示旧的倍率

Snipaste_2025-01-20_17-37-54

新代码启动之后,自动迁移到新倍率中

Snipaste_2025-01-20_17-42-49

导入一次之后,新倍率会写到 options 表中

Snipaste_2025-01-20_17-43-11

可以在次基础上修改新倍率

Snipaste_2025-01-20_17-49-13

再次重启 one-api,新倍率已持久化,不会被旧倍率覆盖
Snipaste_2025-01-20_17-49-13

之后的版本中可以删除 ModelRatio, CompletionRatio 相关代码和配置

@WqyJh
Copy link
Contributor Author

WqyJh commented Jan 21, 2025

Hi @songquanpeng, would you please help review and merge the code?

@popdo
Copy link

popdo commented Jan 22, 2025

感觉是多此一举,补全倍率很少去配置,这样修改后反而每个模型都要配置补全,反向优化

@Laisky
Copy link
Contributor

Laisky commented Jan 22, 2025

DELETED, 详见下面的新回复

@Laisky
Copy link
Contributor

Laisky commented Jan 23, 2025

@WqyJh 感觉还是有几个地方可以优化下

加载 Option 的时候,仅当 Ratio 为空时才需要合并旧数据,不用每次启动都合并一次

one-api/model/option.go

Lines 101 to 114 in f2bff58

for _, option := range options {
if option.Key == "Ratio" {
option.Value = billingratio.AddOldRatio(oldModelRatio, oldCompletionRatio)
err := updateOptionMap(option.Key, option.Value)
if err != nil {
logger.SysError("failed to update option map: " + err.Error())
}
err = UpdateOption(option.Key, option.Value)
if err != nil {
logger.SysError("failed to update option map: " + err.Error())
}
logger.SysLog("ratio merged")
}
}

Option 内的 Ratio 支持更细粒度的缺省值

解决 @popdo 提到的问题,用户编辑倍率的时候,可能并不需要为每一个模型都写上 Input 和 Output 倍率,而是仅需要编写和默认值不一样的项即可。

在下面的代码里,让读取 ratio 更细粒度一些。在从 billingratio.GetRatio 中读取到 ratio 后,再进一步检查其中的每一个成员项,Input/Output/LongThreshold/LongInput/LongOutput,如果为空的话用 adaptor.Ratio 补上。这样可以最大程度减轻维护的负担

func GetRatio(meta *meta.Meta, adaptor adaptor.Adaptor) ratio.Ratio {
result := billingratio.GetRatio(meta.OriginModelName, meta.ChannelType)
if result != nil {
return *result
}
ratio := adaptor.GetRatio(meta)
if ratio != nil {
return *ratio
}
logger.SysError("model ratio not found: " + meta.OriginModelName)
return billingratio.FallbackRatio
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ModelRatio and CompletionRatio should be maintained by adapter
3 participants