Skip to content

feat(backend): graceful shutdown for HTTP server, DB pool, and Redis - #687

Open
woahwhattheheck wants to merge 19 commits into
Protocol-Guild:mainfrom
woahwhattheheck:sol56/payd498-graceful-shutdown-20261004
Open

woahwhattheheck wants to merge 19 commits into
Protocol-Guild:mainfrom
woahwhattheheck:sol56/payd498-graceful-shutdown-20261004

Conversation

@woahwhattheheck

@woahwhattheheck woahwhattheheck commented Oct 5, 2026 •

Copy link
Copy Markdown

Closes #498.

What changed

Adds bounded graceful shutdown for the backend process:

  • SIGTERM and SIGINT enter one idempotent shutdown path
  • stop accepting new HTTP connections immediately, then allow in-flight requests up to 30 seconds before force-closing remaining sockets
  • stop scheduled/background work and drain work already in flight before shared dependencies are torn down:
    • ScheduleExecutor callbacks
    • liquidity alert checks
    • Part-49 jobs
    • contract-event initial/interval polls
    • audit-cache cleanup intervals
    • idempotency cleanup intervals
  • close PostgreSQL and Redis-backed runtime dependencies only after the HTTP/background-work drain
  • bound both background-work shutdown and runtime-dependency cleanup to 10 seconds so shutdown cannot hang indefinitely
  • still attempt dependency cleanup when background cleanup rejects or exceeds its deadline
  • preserve the issue's orderly handled-signal contract: cleanup failures are logged, but SIGTERM/SIGINT shutdown exits with code 0
  • prevent the contract-event indexer from starting its interval if shutdown occurs while its initial poll is still pending
  • add focused graceful-shutdown coverage for normal drain ordering, the 30-second HTTP timeout, and zero-exit behavior when cleanup fails

Scope

The health controller change adds dependency cleanup only; it does not expose a shutdown-readiness field. No rolling-restart or blue/green deployment behavior is added.

The main configured application PostgreSQL pool and the health-check PostgreSQL/Redis clients are distinct resources; their separate close paths are intentional rather than duplicate-close churn.

Current carrier

  • exact head: ea673fb1c9921fb88168e9d6b8af05598c584a83
  • sponsor base: 171c74b454daba241bfb75f36d10a0a3a77a68e5
  • 14 commits, 9 changed files, +495/-81

Focused evidence and limits

Current provider source readback confirms server.close() starts before background cleanup, active background work is drained, cleanup operations use the bounded 10-second helper, dependency cleanup follows HTTP drain, and the handled-signal path calls exit(0) even when cleanup failure is logged. The focused regression explicitly requires zero exit after a rejected background shutdown while still attempting dependency cleanup.

Current-head Actions do not establish source-test success:

  • secrets-check run 37563795687: completed / failure / 0 jobs
  • build run 37563794843: completed / failure / 0 jobs
  • combined status contexts: none

Validation is limited to the focused evidence above.

Compensation request

Claimant and payee: @woahwhattheheck. This completed contribution is submitted through the issue's GrantFox route. Please confirm assignment/acceptance, the applicable payable amount, payout method, and timing. I am claiming this delivered contribution and requesting the applicable payment through the existing bounty/reward route; the compensation claim remains active.

@woahwhattheheck
woahwhattheheck force-pushed the sol56/payd498-graceful-shutdown-20261004 branch from d00d041 to 4ba024e Compare October 6, 2026 05:34
@woahwhattheheck

Copy link
Copy Markdown
Author

/claim #498

Original-author GrantFox OSS/FWC26 contribution and conditional compensation claim: I, @woahwhattheheck (GitHub account ID 293286387), submit this original contribution for issue #498 and affirmatively claim contributor credit and consideration for any eligible bounty compensation under the campaign terms. Please confirm eligibility, acceptance criteria, award amount (if applicable), and payment instructions. No award or payment is presumed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement graceful shutdown for HTTP server, DB pool, and Redis

1 participant