Skip to content

JS support for client httpOptions (to unlock support for Priority PayGo) #4873

@Matt-Jennings-GitHub

Description

@Matt-Jennings-GitHub

Is your feature request related to a problem? Please describe.

In order to access Priority PayGo, the header X-Vertex-AI-LLM-Shared-Request-Type: priority must be sent along with the request.

The docs show this is action in the Python SDK, however no such option is exposed from in the GenKitOptions when initialising a genkit client in the JS SDK.

This makes accessing alternative PayGo tiers difficult for anyone building with the JS SDK and wanting to send requests along with the VertexAI Genkit plugin.

Describe the solution you'd like

A way to provide httpOptions to facilitate setting default custom headers sent along with requests when initialising the genkit client from the js sdk, in order to unlock Priority PayGo without workarounds.

Describe alternatives you've considered

Unglamorous middleware and request interception methods, which are not preferable.

Additional context

Currently it's difficult to access Priority PayGo feature for anyone building in js, which is a useful feature for combatting rate limits and limited regional model resources. Resource contention was blocking our production rollout ( Case #68357887 ) and we were forced to move to a global endpoint which was not preferable for us, alongside aggressive retries and model fallbacks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions