Skip to content

fix(cli): bound server shutdown with a 10 s deadline - #58817

Open
kitlangton wants to merge 2 commits into
devfrom
fix/server-shutdown-deadline
Open

kitlangton wants to merge 2 commits into
devfrom
fix/server-shutdown-deadline

Conversation

@kitlangton

@kitlangton kitlangton commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #58816

What it does

After SIGTERM or SIGINT, the server could hang in teardown forever, and the logs never named the step it was stuck on. Teardown now has a 10 s deadline. When the deadline passes, the process logs what is still pending and exits 1. A second signal during teardown does the same immediately.

How it works

  1. ShutdownDeadline.install() in the CLI entry starts a timer on the first SIGTERM or SIGINT. runMain still interrupts the main fiber as before. Startup order and listener shutdown are unchanged.
  2. When the timer fires, the process prints the pending work to stderr, logs shutdown deadline exceeded, waits 1.5 s so the file logger can flush, then exits 1. A second signal prints the same report and exits at once.
  3. Pending work is recorded in a small registry, Teardown.track / pending in @opencode/util/teardown:
    • LayerNode.compile brackets each node's release, so a stuck layer shows up as layer <name> releasing.
    • The cross-spawn spawner tracks each child from spawn until close or scope release, as child <pid> running: <command>.
  4. Closing a child's scope now escalates to SIGKILL after 3 s (RELEASE_FORCE_KILL_AFTER) unless the command sets its own forceKillAfter. Before, release sent SIGTERM and waited forever.

Example report:

shutdown did not finish within 10s; exiting. Still pending:
  10.0s  child 73484 running: bun -e setInterval(() => {}, 1000)
  10.0s  layer test/Hung releasing

Risk

  • Server and CLI shutdown, plus child process release everywhere the cross-spawn spawner is used.
  • The 3 s default force-kill also applies when a scope closes mid-session (tool cancellation). That's safe: shell, MCP, standalone and ssh children already set 2–3 s, and before this change a child that ignored SIGTERM hung tool cancellation too.
  • A teardown that legitimately needs more than 10 s would now be cut off. Nothing in the tree is expected to need that long.
  • Not verified against a real hung production server. The tests use a fixture with a deliberately stuck finalizer and a real child process.

Proof

Reviewed by Argon before push (verdict: ship). Its two follow-ups are in the second commit: settle released children, and handle a second signal.

  • bun test --timeout 30000 test/shutdown-deadline.test.ts test/service-lifecycle.test.ts test/standalone.test.ts (packages/cli): pass. The new tests spawn a real server-shaped process with a hung layer and a child, then check the report, exit code 1, and that a second signal exits without waiting for the deadline.
  • bun run script/test.ts test/effect/cross-spawn-spawner.test.ts test/effect/layer-node test/process/process.test.ts test/mcp.test.ts (packages/core): pass. The new spawner test checks that a released child whose grandchild holds its pipes is no longer listed as pending. It fails without the fix.
  • bun typecheck in util, cli and core: pass.
  • oxlint on the changed files: pass.

Added by the opencode bot (AI-generated).

Related: #54267

After SIGTERM or SIGINT, teardown had no deadline: a finalizer that never
finished kept the process alive until it was SIGKILLed, with nothing in the
log naming the step it hung in.

- ShutdownDeadline gives teardown 10s, then logs and prints the layers still
  releasing and the child processes still running, and exits 1.
- LayerNode records when each node starts and finishes releasing; the spawner
  records running children.
- Closing a child's scope escalates to SIGKILL after 3s unless the command
  sets its own forceKillAfter, so a child that ignores SIGTERM or whose
  descendants hold its pipes can no longer block release forever.
…ed children

- A second SIGTERM or SIGINT during teardown prints what is pending and exits
  1 at once instead of being swallowed until the deadline.
- Releasing a child's scope settles its pending entry, so a descendant that
  keeps the pipes open can't leave a released child listed as running.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I reproduced #58816 on the base and confirmed it's fixed here:

  • Child that ignores SIGTERM: the base was still alive 20 s after SIGTERM. With the PR, the new 3 s release force-kill ends it in about 4 s (exit 130).
  • Finalizer that never finishes: the base was still alive after 20 s. With the PR it exits 1 after the 10 s deadline plus the log flush, and names layer … releasing on stderr and in the error log.
  • Normal shutdown: opencode serve from source still exits within about 30 ms of SIGTERM. runMain calls process.exit after a signal, so the deadline timer doesn't hold the process open.

The new cli tests and the cross-spawn/layer-node tests in core pass.

Public API: unchanged. There are no changes to the SDK, HTTP API, config, CLI flags or env vars. RELEASE_FORCE_KILL_AFTER and @opencode/util/teardown are internal. The only behaviour change is the 3 s default SIGKILL escalation when a child's scope is released, which the description already covers.

No blocking concerns.

Related: #54267

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Server ignores SIGTERM when a teardown step hangs (no shutdown deadline)

1 participant