Compose VM rootfs from shared layer blobs

[chruffins]

summary

Stacked stage of the image-storage project: the production composition path.

• composeRootfs builds one merged rootfs per image by applying the manifest's ordered layer blobs from the shared OCI cache in tar order. Whiteout and opaque-directory markers are interpreted as each layer is applied; entries replace conflicting types safely; hardlink targets resolve against the accumulated tree. The merged tree is exported to the same single read-only disk the guest mounts as its lower filesystem.
• validateManifestModel enforces one config diff id per manifest layer and requires rootfs.type == "layers" (same guard the previous unpack path enforced), so a corrupt manifest fails the build with a clean error instead of indexing past the pairing (preserves the captured-fixture regression behavior). Composition is the only unpack path: the umoci unpackLayers path and its helpers are removed.
• Layers are applied with umoci's layer.UnpackLayer using the DirRootfs on-disk format, so a layer can replace a directory with a file (and vice versa) and non-empty directories are cleared before writing.
• Unlike the previous unpack path, the composed rootfs preserves each layer's tar ownership when running as root (no container-root → current-user uid/gid mapping), matching the layer-artifact flow.
• The staged tree is installed by removePath(dest) + rename into the export directory. This replaces any stale tree from a previous partial build, but is not atomic: a failure between the remove and the rename leaves dest absent, and a crash can strand a .compose-* staging dir (same class as the .unpack-* leftovers the layer-artifact path already tolerates). Current callers always pass fresh temp dirs and never read dest concurrently.
• Each layer's blob digest and diff id are re-verified against the manifest model on every compose, including cache hits. The layer_unpack build-phase metric label is retained for dashboard continuity, though the measured work is now composition.

why this path

The guest init mounts exactly one read-only lower (/dev/vda) and one writable upper (/dev/vdb), and QEMU microvm virtio-mmio slots bound how many devices can be attached. A one-disk-per-layer layout therefore cannot be supported uniformly across cloud-hypervisor, Firecracker, QEMU, and vz, so it was rejected rather than attached blindly. Sharing happens at the content-addressed blob and layer-artifact level; every hypervisor keeps its existing disk contract (vda read-only rootfs, vdb writable overlay), and guest init and hypervisor config are unchanged.

validation

• Synthetic two-layer fixture with whiteout, opaque mask, file/directory replacement, and mode checks composes to the expected tree; composed output exports to erofs and passes fsck.erofs --extract.
• Captured corrupt-manifest fixture still fails cleanly with the original integrity error.
• go test ./lib/images ./lib/paths ./lib/builds ./lib/scopes ./cmd/api/api: green except Docker Hub pull tests (anonymous rate limit in this environment) and VM lifecycle tests, which require network-bridge privileges (verified failing identically on unmodified main).

---

Note

Medium Risk
Switches the primary rootfs build path for normal images and changes tar extraction semantics shared with layer unpacking; guest disk contract is unchanged and umoci fallback limits blast radius.

Overview
Image export now merges manifest-ordered layer blobs from the shared OCI cache into one staging rootfs (composeRootfs) instead of using the old umoci unpackLayers path, which this change deletes. Whiteout and opaque-directory markers are applied as each layer is streamed, so the composed tree is meant for a single read-only guest disk without relying on overlayfs to interpret .wh.* files.

validateModelPairing rejects configs where rootfs.diff_ids length mismatches manifest layer count (same integrity message as the umoci path). If no model or empty layers, umoci unpackLayers remains the fallback.

extractTarEntry now uses clearExisting (including RemoveAll for directories) so later layer entries can replace a path with a different type—e.g. directory → file—fixing composition cases like replacedir in the new tests.

Tests cover whiteout/opaque/replacement ordering, missing blobs, and erofs export + fsck.erofs --extract on the composed tree.

^{Reviewed by Cursor Bugbot for commit 2b465b6. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 2b465b6. Configure here.}

[cursor]

Whiteouts escape compose root

High Severity

Opaque and whiteout handling joins header.Name onto dest with filepath.Join and never runs safeJoin, unlike regular entries. A layer whose whiteout path contains .. can make clearDirContents / clearExisting delete files outside the compose tree during image build, including host paths under the build parent.

 

^{Reviewed by Cursor Bugbot for commit 2b465b6. Configure here.}

[cursor]

Empty whiteout deletes parent

Medium Severity

A whiteout whose name is exactly .wh. yields an empty hidden target. After skipping only . and .., clearExisting runs on the parent directory and RemoveAlls it, so a single marker can wipe the containing directory instead of hiding one entry.

 

^{Reviewed by Cursor Bugbot for commit 2b465b6. Configure here.}