-
Notifications
You must be signed in to change notification settings - Fork 680
Description
Is your feature request related to a problem? Please describe.
In order to access Priority PayGo, the header X-Vertex-AI-LLM-Shared-Request-Type: priority must be sent along with the request.
The docs show this is action in the Python SDK, however no such option is exposed from in the GenKitOptions when initialising a genkit client in the JS SDK.
This makes accessing alternative PayGo tiers difficult for anyone building with the JS SDK and wanting to send requests along with the VertexAI Genkit plugin.
Describe the solution you'd like
A way to provide httpOptions to facilitate setting default custom headers sent along with requests when initialising the genkit client from the js sdk, in order to unlock Priority PayGo without workarounds.
Describe alternatives you've considered
Unglamorous middleware and request interception methods, which are not preferable.
Additional context
Currently it's difficult to access Priority PayGo feature for anyone building in js, which is a useful feature for combatting rate limits and limited regional model resources. Resource contention was blocking our production rollout ( Case #68357887 ) and we were forced to move to a global endpoint which was not preferable for us, alongside aggressive retries and model fallbacks.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status