Skip to content

feat(coord): NBD slots are a placement dimension; chart nbdsMax for two devices per guest (ADR 0112 addendum, phase 4) - #1607

Merged
nikhilunni merged 1 commit into
mainfrom
feat/swap-nbd-slots
Oct 7, 2026
Merged

nikhilunni merged 1 commit into
mainfrom
feat/swap-nbd-slots

Conversation

@nikhilunni

Copy link
Copy Markdown
Contributor

Summary

Phase 4 of the chunked-swap design (ADR 0112 addendum): NBD slots become a placement dimension, and the chart defaults size the slot pool for two devices per swap-enabled guest. Stacked on #1605 (phase 3). This is the last code phase; the dev-VM measurements of swap-in latency and capture upload cost are an operator step before the fleet roll.

What changes

  • Heartbeat. HostUtilization gains nbd_slots_total, nbd_slots_in_use (claimed, parked and quarantined slots; warm and validation claims excluded) and the attached device count per sandbox. The allocator keeps an atomic claimed counter, incremented at every slot construction and decremented in its drop.
  • Ledger. Migration 0125 adds the host columns and sessions.nbd_slot_need (one root slot plus one for swap). Existing rows are backfilled from the image config they were created under or from a snapshot that carries a swap manifest; a live guest is counted by its host's sample either way. The reservation SQL uses the same session, capture-job and teleport arms as the memory ledger and subtracts devices the heartbeat already reports, so nothing is counted twice.
  • Placement. Create, queued create, resume, affinity, pinned host, capture placement and teleport admission all pass the slot gate. A host with nbd_slots_total == 0 is exempt (file fallback). No-fit details report nbd_slots; queue fit classes include the need.
  • Fleet view. Both counters on HostView (proto fields 35, 36; TypeScript bindings regenerated).
  • Chart. nbdsMax and warmSlots default to 128, with the node-recreate note in all three values files.

Posture

Resume and teleport pickers read the ledger without a transaction, as the memory ledger does today; the host allocator remains the final arbiter. Recorded in the ADR addendum.

Tests

  • Conformance (sim + live PG): host placement with a partial attach and quarantine release, teleport reservation on both sides, capture placement and reassignment.
  • Pure placement tests: the gate holds in every tier (measured, unmeasured, affinity, soft fallback) and reports nbd_slots; queue classes split on need; allocator counts across acquire, quarantine, reclaim and drop.

Validation

  • just check green.
  • Linux cross clippy for host-agent, protocol, core, coordinator: clean.
  • Live-Postgres host, placement, queue and capture-job scenarios: green.
  • helm lint passes; just gen-proto reproduces the bindings.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UHxL6gj4o8EvxYpgtwEWaM

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow CI / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed⏩ skipped⏩ skipped✅ passedOct 7, 2026, 8:26 PM

@nikhilunni
nikhilunni force-pushed the feat/swap-nbd-slots branch from 5718cc4 to 21e72c6 Compare October 7, 2026 18:00
@nikhilunni
nikhilunni added this pull request to stack #1609 October 7, 2026 18:12
@nikhilunni
nikhilunni force-pushed the feat/swap-nbd-slots branch from 21e72c6 to f329913 Compare October 7, 2026 19:00
@nikhilunni
nikhilunni force-pushed the feat/swap-nbd-slots branch from f329913 to 1ad50cd Compare October 7, 2026 19:23
@nikhilunni
nikhilunni force-pushed the feat/swap-nbd-slots branch from 1ad50cd to 13fe92a Compare October 7, 2026 19:50
@nikhilunni
nikhilunni force-pushed the feat/swap-nbd-slots branch from 13fe92a to 16c12c1 Compare October 7, 2026 20:16
@nikhilunni
nikhilunni force-pushed the feat/swap-nbd-slots branch from 16c12c1 to 590a6a3 Compare October 7, 2026 20:19
Base automatically changed from feat/swap-postcopy to main October 7, 2026 20:19
…wo devices per guest (ADR 0112 addendum, phase 4)

A swap-enabled guest takes two NBD slots. Placement now counts them so
exhaustion is `NoFit`, not a wait inside the host allocator.

- The host heartbeat reports `nbd_slots_total`, `nbd_slots_in_use`
  (claimed, parked and quarantined; never warm or validation claims) and
  the attached device count per sandbox. Migration 0125 adds the host
  columns and `sessions.nbd_slot_need`, backfilled from the image config
  or a swap-bearing snapshot.
- Every placer that lands a chunked disk passes the slot gate: create,
  queued create, resume, affinity, pinned host, capture placement and
  teleport admission. The ledger reserves only the remaining need for a
  sandbox the heartbeat has not yet reported, so a device is never counted
  twice. A host with NBD off is exempt (file fallback). No-fit details
  report `nbd_slots`; queue fit classes include the need.
- Fleet views expose both counters (proto fields 35 and 36).
- Chart defaults: `nbdsMax` and `warmSlots` 128. A change needs a node
  recreate because the kernel reads the module parameter at load.

Conformance scenarios cover host placement with a partial attach, teleport
reservation on both sides, and capture placement and reassignment, on both
stores. Pure placement tests cover every tier and the no-fit reason.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UHxL6gj4o8EvxYpgtwEWaM
@nikhilunni
nikhilunni force-pushed the feat/swap-nbd-slots branch from 590a6a3 to 5a9513b Compare October 7, 2026 20:20
@nikhilunni
nikhilunni merged commit c2139cc into main Oct 7, 2026
3 checks passed
@nikhilunni
nikhilunni deleted the feat/swap-nbd-slots branch October 7, 2026 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant