Rate limits on LLM model and task execution

[0xbohu]

Hi, I encounter this error from anthropic Claude 3 Haiku model, when it exceeds 50,000 input tokens per minute. The program just keep trying it and throwing the same error.

429 {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 50,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}

maybe need a configurable option to limit the rates to query LLM model, or put the goals on pending and execute at a later time

[ponderingdemocritus]

yeah good idea - there is no rate limit currently, my API key has higher so i didn't hit this.

You could calculate in the class the number of tokens being used per second an rate limit the call function. Use tiktoken todo this - welcome the PR!

[0xbohu]

thank you @ponderingdemocritus I will look into it!

Also do you know if I can run Eternum locally and serve chain data to a local (or remote) graphQL server so I can configure it in Daydream .env as an external context?

I am trying to run it locally and encounter some deployment issue I have raised in Eternum repo.

I've found Eternum GraphQL url https://api.cartridge.gg/x/sepolia-rc-18/torii/graphql from code commit and configured that in daydream and it runs. but Eternum web app is currently under maintenance so I cannot see the progress visually when AI agent is playing it. like the whitewizard demo.

Any advice on how to run it and validate it with Eternum (locally or on sepolia), really appreciate your help and I am excited to build with daydream and eliza.