You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Why not expose entire LiteLLM config as parameter or something when running evals?
I'm actually looking to run open-r1 evals with any api provider. This don't seem possible with current. The OpenAIClient class should also allow using any base_url. Parameter like temperature should be available to modify directly.
The text was updated successfully, but these errors were encountered:
I saw @NathanHB and @gary149 just committed some improvements so that various configuration options can be submitted via a YAML file (gj!), which is workable, but I think that proper flags or a way to submit via command line would still be useful for building eval pipelines (vs having to add YAML file generation into my scripts, which is a big bloat/PITA).
The text was updated successfully, but these errors were encountered: