Repository navigation
fix(ai): classify per-window token throttles as retryable rate limit - #58796
vitalyisaev2 wants to merge 1 commit into
Conversation
Adds the phrase "too many tokens per minute/second/hour/day" to the context-overflow exclusions and the rate-limit text, so the provider throttle "Token limit reached: too many tokens per minute for this model" is treated as a retryable 429 rather than as a context overflow that compaction shrinks and then fails on. Mirrors the Bedrock per-day wording fix. Fixes anomalyco#58794
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Looks good. I ran the #58794 reproduction script in packages/ai on the base and with this change. On the base, the per-minute and per-second throttle messages classify as InvalidRequest (context overflow) and aren't retryable, with a 429 and with no status. With the change, both come back as a retryable RateLimit and aren't treated as overflow. bun test test/provider-error.test.ts (41 pass) and typecheck:tsc in packages/ai pass.
The new exclusion is what fixes the 429 path, because overflow is checked before the rate-limit branch. The RATE_LIMIT_TEXT addition also covers responses that arrive without a status. The new phrases don't match the 429 quota text, so these errors can't be classified as a quota error instead. A plain "Too many tokens" still counts as overflow.
Public API: unchanged. This is internal error classification in packages/ai.
No blocking concerns.
Adds the phrase "too many tokens per minute/second/hour/day" to the context-overflow exclusions and the rate-limit text, so the provider throttle "Token limit reached: too many tokens per minute for this model" is treated as a retryable 429 rather than as a context overflow that compaction shrinks and then fails on. Mirrors the Bedrock per-day wording fix.
Fixes #58794
Issue for this PR
Closes #58794
Type of change
What does this PR do?
DeepSeek-V4-Flash can throttle with the message
Token limit reached: too many tokens per minute for this model.classifyProviderFailure()matches the plain/too many tokens/ipattern insideisContextOverflow, so this 429 was mistaken for a context overflow. The delivery loop would shrink the transcript via compaction and then fail — but shortening a transcript can't help a tokens-per-minute ceiling, so it should just be retried instead.The change adds
too many tokens per (minute|second|hour|day)to two places: theexclusionslist soisContextOverflowno longer matches it, andRATE_LIMIT_TEXTso it's classified asRateLimit(retryable). Same approach as an earlier Bedrock per-day wording fix, just extended to the per-window phrasings. Bare"Too many tokens"still categorizes as overflow.How did you verify your code works?
Added a regression test mirroring the existing Bedrock one: for
...too many tokens per minute...and...per second...it asserts the failure classifies asRateLimit,isContextOverflow/isContextOverflowFailurearefalse, andisRetryableistrue; bare"Too many tokens"still compacts. Verified withbun test test/provider-error.test.ts(41 pass),bun run typecheck:tscinpackages/ai, and the repo-widebun run check(34/34 tasks pass).Screenshots / recordings
N/A — logic-only change, no UI.
Checklist