Skip to content

[litellm] Add model parameters input component #1438

@dlqqq

Description

@dlqqq

Problem

We need a new React component in the Jupyter AI settings page that allows a user to add arbitrary model parameters. Each model parameter will be a keyword argument passed to LiteLLM.

Proposed Solution

Build a <ModelParametersInput /> component, which will:

  1. Show just a "Add a custom model parameter" button initially.
  2. After clicking that button, there should a new input for the parameter name (e.g. temperature or api_url), parameter type (e.g. float or string), and parameter value (e.g. 0.7 or https://localhost:8989). These should appear as 3 text fields on the same line.
  3. After clicking that button, there should also be a new button that says "Save Model Parameters". For now, this button can just log the JSON object of its input state to the browser console.
  4. A validation error should be emitted if the user specifies a parameter value but does not specify its type or its name. If a user specifies an empty parameter value, it should be ignored entirely, regardless of the type or name inputs.
  5. The "Add a model parameter" button should still show after the user clicks it once, i.e. users should be able to add any number of model parameters.

Additional context

  • Stretch goal: There are some fields which are standard to all models. It would be nice to be able to define "static parameters" whose name & type are fixed. This would be useful for basic parameters like temperature: float and api_url: string. Maybe these can be shown after a user clicks a different button like "Specify API URL" above/ under "Add a custom model parameter"?

  • Feel free to change the UI to your liking. I do not know the best UI for this, and what I've suggested is just my best initial guess at what should exist long-term. If you can build a better / more usable UI, please feel free to include it in your PR.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions