Skip to content

Bound dashboard delivery batches before shortening the publisher lease #382

Description

@trask

The pull request dashboard publisher holds a repository-scoped Git lease while it delivers side effects and updates the dashboard issue. Most publisher jobs finish in under a minute, but several delivery paths can process all accumulated repository work, so the current 50-minute job timeout and 60-minute lease cannot safely be reduced based on typical runtime alone.

Add explicit, durable per-run bounds for the currently unbounded delivery paths:

  • Slack notifications across open non-draft pull requests
  • prepared author nudges
  • Copilot review requests
  • dashboard command replies
  • dashboard issue rendering or any other work whose cost grows with the repository's open pull request count

Preserve pending work for later publishers, as the status-comment rollout already does with its 50-item batch. Define bounds using API cost and worst-case runtime, not only pull request count, and ensure retries cannot lose or duplicate external effects.

Once all lock-held work has a reliable upper bound, reduce the publisher job timeout and publisher-lock lease together. The job must always be terminated before its lease can expire, while a crashed publisher should stall its repository for substantially less than the current one-hour maximum.

This follows up on the publisher-starvation work in #347. Acceptance criteria:

  • Every delivery path executed while holding the publisher lock has an explicit per-run bound.
  • Deferred work is durable and is drained by subsequent targeted or hourly publishers.
  • Tests cover partial batches, retries, new work arriving during delivery, and completion across multiple runs.
  • Production runtime data supports a shorter job timeout and lease with documented safety margin.
  • The stale-lock recovery window is reduced without allowing a live publisher to outlast its lease.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions