Fallback text generation model, for when the primary hits a usage limit

[Onkarj012]

Problem

The "Text generation model" setting drives four things: thread titles, commit messages, PR title/body, and branch names. When that model fails, nothing recovers.

maybeGenerateThreadTitleForFirstTurn retries 3 times with exponential backoff, then swallows the failure into a log warning (apps/server/src/orchestration/Layers/ProviderCommandReactor.ts). The thread keeps the title the client wrote optimistically, which is the raw first message truncated to 50 chars. Hitting a provider usage limit is the common way to land here, and burning 3 attempts in ~6 seconds against a limit that clears in hours is wasted work.

Proposal

An optional textGenerationFallbackModelSelection, tried once when the primary fails. It applies to all four operations, because the setting is named for all of them and splitting the behavior would be arbitrary.

The shape mirrors sourceControlWriterModelSelection throughout, which is already a nullable secondary ModelSelection on the same setting family: same project-scoped key list, same atomic whole-value patch replacement, same environment-local rather than synced treatment.

• Contracts: nullable key, defaults to null, so off unless set.
• resolveTextGenerationFallbackModelSelection returns null when the setting is off, its provider is disabled or unavailable, or it duplicates the primary's instance and model. Retrying the identical model is not a fallback.
• One runWithFallback helper in TextGeneration.ts serves all four operations, so the retry logic exists in exactly one place. Callers that pass no fallback behave exactly as today.
• Settings gets a second row in the existing Text generation section: a switch that toggles null ↔ a selection, with the model picker only when on. No traits picker; a reasoning-effort control on a fallback is noise.

Costs worth naming before you ask

The existing retry now covers both models. ProviderCommandReactor wraps the thread-title call in Effect.retry({ times: 2 }). A title where both models are broken costs 6 CLI spawns instead of 3. Bounded, and only when everything is down. I left the retry alone to keep the change to one concern, but narrowing it is a one-line follow-up if you'd rather.

It fires on any failure, not specifically on usage limits. normalizeCliError carries no error type today and no provider parses rate-limit stderr. Any-failure covers the motivating case plus auth expiry, a missing CLI, and timeouts. Classifying the error properly is a separate piece of work.

Not mirrored to mobile or synced across environments. apps/mobile has no UI for textGenerationModelSelection at all, so there is nothing to mirror. SHARED_SERVER_SETTING_KEYS deliberately excludes sourceControlWriterModelSelection; I followed that precedent, since adding the fallback would mean extending the instance-compatibility guard in filterSharedServerPatch and the drift check, which is a second concern.

Implementation

Working branch: main...Onkarj012:feat/text-generation-fallback-model

12 files, 479 insertions. Targeted tests pass (195 across TextGeneration.test.ts, serverSettings.test.ts, settings.test.ts), including cases for fallback-runs-on-failure, fallback-never-runs-on-success, error-propagates-unchanged-with-no-fallback, and identical-fallback-attempts-once. Typecheck and lint clean on apps/server, apps/web, packages/contracts, packages/shared. No new lint or typecheck findings.

Verified in a real client against a snapshot of real state: the switch starts off because the key is absent, toggling on seeds from the primary and persists, picking a different model persists, and toggling off strips the key back out.

UI

The new row sits directly under the existing one in Settings → General → Text generation, described as "Used when the text generation model fails, including when its usage limit is reached. Off means no fallback." Off, the switch is dark and there is no picker, which is the default state. On, a model picker appears beside it, seeded from the primary. Unset reads as "Off" in the inheritance chain rather than "Not set".

Before/after screenshots and a short toggle video follow in an edit on this post.

Before I do anything with it

I read CONTRIBUTING, so this is a discussion rather than a PR. Happy to open one, shrink it, or drop it entirely. If the direction is wrong, say so and I will not push it further.

Written by Claude Opus 5 in T3 Code.

[kr4chinin]

+1. I hit this not only on usage limits but also when a provider is down entirely. Title generation retries, logs a warning, and the thread never gets a generated title. The existing fallback in resolveTextGenerationProvider only applies when the provider is disabled. It does nothing when the provider is enabled but failing.

Title generation has since moved to orchestration-v2 (ThreadTitleRegenerationService.ts), and ProviderCommandReactor.ts no longer exists. The linked branch is about 3.1k commits behind main, so the server side had to be redone. The proposed shape still fits current main: a nullable textGenerationFallbackModelSelection that mirrors sourceControlWriterModelSelection, and one fallback helper in TextGeneration.ts.

Maintainers, would you accept this direction? I ported it to current main and opened the PR myself: #23946. If the direction or scope is wrong, I'm happy to adjust it or close it.