initrd: layered ISO boot with initrd compat check, loopback.cfg, and boot option markers#2112
initrd: layered ISO boot with initrd compat check, loopback.cfg, and boot option markers#2112tlaurion wants to merge 18 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR enhances Heads’ USB ISO kexec-boot flow with layered ISO parameter handling, initramfs filesystem-compatibility safeguards, and clearer user-facing boot option presentation, plus accompanying documentation and a new ISO parser test harness.
Changes:
- Add Layer 1 initramfs filesystem compatibility inspection and Layer 2 loopback.cfg GRUB-variable resolution/fallback ISO parameter injection in
kexec-iso-init.sh. - Improve boot option UX in
kexec-select-boot.sh(compatibility markers, richer menu/confirm display) and tighten logging output (strip ANSI fromdebug.log). - Extend initrd unpacking to better handle concatenated multi-segment cpio archives and add extensive docs + a new ISO parser harness.
Reviewed changes
Copilot reviewed 11 out of 17 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/iso-parser/run.sh | New harness to exercise real parser/unpacker/ISO-init logic against mock trees and optional real ISOs. |
| initrd/etc/functions.sh | Adds _strip_ansi() and routes log-file writes through it; skips EFI configs in scan_boot_options(). |
| initrd/etc/DEBUG_LOG_COPY_INSTRUCTIONS | Adds recovery-shell instructions for signing ISOs. |
| initrd/bin/unpack_initramfs.sh | Improves multi-segment initramfs extraction handling (GNU vs BusyBox behavior, zstd fallback). |
| initrd/bin/kexec-select-boot.sh | Adds compat markers, enhanced menu/confirm display, and new “different ISO” selection path. |
| initrd/bin/kexec-parse-boot.sh | Refactors parsing and adds append-param cleanup for ISO boot entries. |
| initrd/bin/kexec-parse-bls.sh | Minor parsing/refactor cleanup (quoting/modern subshell style). |
| initrd/bin/kexec-iso-init.sh | Implements layered ISO boot design: signature handling, fs compat checks, loopback.cfg fast path, fallback params. |
| initrd/bin/kexec-boot.sh | Improves cmdline handling/logging and warns on oversized kernel command lines. |
| doc/ux-patterns.md | Documents BG_COLOR_MAIN_MENU semantics and compatibility markers UX. |
| doc/modules.md | New doc describing initrd tool/module inclusion and BusyBox vs standalone. |
| doc/logging.md | Documents ANSI stripping and ISO boot logging/message conventions. |
| doc/iso_boot.md | New reference explaining injected ISO boot parameters and distro framework coverage. |
| doc/index.md | New documentation index. |
| doc/busybox_perks.md | New reference documenting BusyBox vs GNU tool differences relevant to initrd scripts. |
| doc/boot-process.md | Adds Stage 2b ISO boot flow documentation and marker semantics. |
| doc/architecture.md | Updates module list description to include zstd reference and link to modules doc. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Takes u-root/u-root#3578 insights on loopback usage + iso underlying fs validation prior of boot, which renders no valid boot options to boot debian dvd unfortunately being the only one permitting to partition properly the disk with unencrypted /boot and encrypted rootfs |
3c81359 to
35ff45f
Compare
35ff45f to
22df7ff
Compare
fixed in 597cd75 |
22df7ff to
3568508
Compare
934250d to
53e0a8c
Compare
|
Fixed the doc/logging.md:397 claim per Copilot review — /dev/kmsg now accurately described as receiving plain text only (ANSI goes to /dev/console). Commit 9a7d74d. |
9a7d74d to
6e92ecf
Compare
|
Fixed two items per Copilot review:
|
6e92ecf to
505758c
Compare
| # only parse these if $option_file is still empty | ||
| if [ ! -s "$option_file" ] && [ -d "$bootdir/loader/entries" ]; then | ||
| for i in $(find "$bootdir" -name "$config"); do | ||
| find "$bootdir" -name "$config" -print | while IFS= read -r i; do |
| - `[OK]` = initrd has the needed module as `.ko`, has it in | ||
| `modules.builtin`, or has no `.ko` files at all (minimal initrd with | ||
| everything built into the kernel — nothing to check against). | ||
| - `[!]` = initrd has loadable kernel modules but none for the USB | ||
| filesystem type. No built-in assumption — we report what we find. |
9c5d3a3 to
9780131
Compare
…mpatibility - Strip xxd -p 60-column padding with tr -d '\n ' (BusyBox xxd pads, GNU does not) - Detect TRAILER!!! offset in plain cpio segments and feed exactly one segment via dd bs=N count=1 so GNU cpio doesn't consume subsequent segments - Add -d flag to cpio invocation and swallow exit code with || true (GNU cpio exits 2 when trailing data follows TRAILER in concatenated archives) - Break loop early when remaining bytes < 110 (minimum cpio header size) - Try zstd-decompress then zstd as fallback for initrds compressed with zstd - Remove dead TODO comments for untested compression formats
…fig parsers
kexec-parse-boot.sh:
- Replace backtick command substitution with $()
- Trim leading whitespace via sed before cut parsing
- Sanitize entry names with tr -d '|' to prevent field boundary injection
- Strip GRUB --- bootloader separator from append
- Soft-check kernel/initrd paths (|| return) instead of skipping entries on miss
- Remove TRACE_FUNC calls
- Improve debug message clarity with parse-boot: prefix
- Simplify fix_path() by removing debug echo
- Replace cut -d= -f2 with ${param#initrd=} syntax
kexec-parse-bls.sh:
- Replace backtick command substitution with $()
- Trim leading whitespace via sed before cut parsing
- Remove TRACE_FUNC calls
…r stripping - Add documentation header describing options (-b, -e, -r, -a, -o, -f, -i) - Strip GRUB --- bootloader separator from cmdline in adjust_cmd_line() - Warn when kernel command line exceeds 2047 bytes (CONFIG_COMMAND_LINE_SIZE) - Include cmdline length in DIE message on kexec load failure - Replace backticks with $() and expr with $((...)) - Remove TRACE_FUNC (no longer tracing this function) - Add DEBUG logging for entry/params/cmdline throughout - Remove stderr redirect (2>/dev/null) from DO_WITH_DEBUG eval so failure messages appear in the log
…SO signing instructions functions.sh: - Skip *EFI*, *efi*, *x86_64-efi* config files in scan_boot_options() to avoid parsing EFI-only bootloader configs - Remove stale 'plain text - no ANSI codes' comments from logging functions (ANSI codes are preserved in debug.log) DEBUG_LOG_COPY_INSTRUCTIONS: - Add ISO signing instructions to recovery shell wall message
…compat check
Design overview
- Layer 1: initramfs compatibility check (check_initrd_compat)
Verify each initrd has the USB filesystem module AND a DRM/KMS
display driver (i915, nouveau, amdgpu, bochs, cirrus, etc.).
Results written to /tmp/kexec_initrd_compat.txt (fs) and
/tmp/kexec_fb_compat.txt (display driver).
Also checks for framebuffer drivers that can work after kexec
without full DRM/KMS reinit.
- Layer 2: loopback.cfg fast path
Parse GRUB ${\iso_path}/${\isofile} variables from the ISO's standard
USB-boot config to extract kernel parameters. Falls through to
universal params if no GRUB vars found.
- Layer 3: kexec-select-boot.sh (interactive boot menu)
Parameter injection changes
- Replace fromiso= with findiso= (covers Debian live-boot + NixOS)
- Add live-media= device filter (for casper/live-boot)
- Retain iso-scan/filename=, img_dev=, img_loop=, iso=
- Inject universal set unconditionally when loopback.cfg has no GRUB vars
Unsigned ISO flow improvements
- Separate signed, failed-signature, and unsigned flows
- Add ISO signing instructions to recovery shell wall message
- Reference u-root's boot/iso implementation (PR 3578)
…y-only ISO-param stripping, dedup and TPM2 guard New functions: - boot_marker(): reads fs and fb compat files, returns combined [OK]/[!]/blank - fmt_boot_target(): formats kernel/initrd as "[path | path]" - fmt_display_params(): strips ISO-finding params from menu display only (full params preserved in entry passed to kexec-boot.sh and saved defaults) parse_option() rewrite: extract kernel, initrd, params from pipe-delimited entries Menu display (get_menu_option): - Show compat markers [OK]/[!] per entry (fs + display driver combined) - Display "[OK] name (params) [kernel | initrd]" in whiptail and CLI - Add "b" option: "Select different ISO" (exit code 2) - Add "a" option: abort (exit code 1) - awk dedup (replaces sort -t\| -k1 -u which BusyBox evaluates by full line) - Restore original entry order for non-unique paths using cp - Context-aware legend: short when all OK, full with [!] explanation Confirmation dialog (confirm_menu_option): - Show full kernel, initrd, params, Board adds/removes, ISO params, final cmdline - Add "Back to menu" (b) option TPM2 primary handle guard: - Skip primhdl hash check when valid_rollback is already "y" (ISO/force boot modes), avoiding spurious warning when paramsdir is tmpfs
The _prod_quiet variants test the normal desktop boot path that preserves quiet/splash. They should not strip those params. Affected: - qemu-coreboot-fbwhiptail-tpm1-hotp-prod_quiet - qemu-coreboot-fbwhiptail-tpm1-prod_quiet - qemu-coreboot-fbwhiptail-tpm2-hotp-prod_quiet - qemu-coreboot-fbwhiptail-tpm2-prod_quiet
Generic test harness covering all ISO boot scripts: - kexec-parse-boot.sh: 11 mock trees testing GRUB, syslinux, BLS, tab-indented, --- separators, deep paths, GRUB variables, loopback.cfg (INLINE/SOURCE) - kexec-parse-bls.sh: BLS format configs - unpack_initramfs.sh: multi-segment cpio, kernel module detection - kexec-select-boot.sh: boot_marker() and fmt_boot_target() for per-initrd [OK]/[!]/blank display testing - --with-isos mode: mounts real ISOs via fuseiso, runs full pipeline
…ce, and UX patterns New files: - doc/iso_boot.md: ISO boot parameter reference with framework coverage table, known limitations, u-root boot/iso credit - doc/busybox_perks.md: BusyBox vs GNU differences - doc/modules.md: module list from Makefile - doc/index.md: documentation index Updated files: - doc/architecture.md: display output after kexec section with ASCII diagram - doc/boot-process.md: Stage 2b USB ISO Boot flow - doc/logging.md: ANSI policy update - doc/ux-patterns.md: compat markers and legend
…ry shell User cancellations (Unsigned ISO warning, USB compat warning, Display Driver Warning) now return to the ISO selection menu instead of falling through to DIE/recovery shell. kexec-iso-init.sh: replace 6 DIE calls for user cancellations with exit 1, so the caller can handle cancellation gracefully. media-scan.sh: wrap ISO selection + boot in a while true loop that returns to the ISO menu when kexec-iso-init.sh exits non-zero (user cancellation). Real errors (DIE) still go to recovery shell.
220628d to
6edae72
Compare
… Tails warning text Extend display driver check to also search for efifb in modules.builtin. Tails (Debian-based) has efifb built into the kernel but no DRM KMS .ko modules in its initrd. Previously triggered a misleading warning about minimal distributions. Remove 'CorePlus/TinyCore' from warning text since the check can also trigger for non-minimal distros that rely on efifb.
…nd distro maintainers The Display Driver Warning now includes: - Clear user message: screen may be blank after boot - Technical hint for distro maintainers: include a KMS driver (i915, nouveau, amdgpu, bochs, cirrus) or ensure CONFIG_FB_EFI=y in the kernel config
…ver checks to fail grep -E treats backslash-pipe (\|) as a literal pipe character, not alternation. The DRM/KMS driver grep pattern i915\|nouveau\|... with -E never matched anything, causing EVERY ISO to get display [!]. Fix: use basic grep (without -E) which correctly interprets \| as alternation, matching any of i915, nouveau, amdgpu, radeon, bochs, virtio-gpu, cirrus, qxl, mgag200, ast.
…dule initrd collection - Fix grep -E bug: grep -E treats backslash-pipe as literal pipe, not alternation. Use basic grep (without -E) so \| works correctly for matching i915, nouveau, amdgpu, etc. - Add Xen multiboot support: collect initrd paths from module entries (field 4/5) in addition to initrd entries. Xen uses module2 for both kernel and initrd, so both fields are scanned. tests/iso-parser/run.sh: - Fix kernel path check: strip params after path (Xen has 'kernel /xen.gz console=none') - Fix ISO search: use find instead of glob so ISOs in subdirectories (like Qubes-R4.3.0-x86_64/) are found
…ation docs Document BusyBox grep alternation pitfalls (ERE vs BRE), strings options, tail -c, and tr octal escape handling. Update summary table with sort and grep entries.
…d tool findings BusyBox grep has no -b (byte-offset) option. Tr octal escapes work. tail -c +N, dd bs=1 skip=N, stat -c %s all confirmed. zstd-decompress is at /bin/zstd-decompress (standalone binary).
Line numbers are not maintainable across code changes. Replace with functional descriptions.
Adapted from Linux kernel's scripts/extract-ikconfig for BusyBox: - Replaced grep -b (not available) with od -A d for byte offsets - Uses strings -t d to find IKCFG_ST marker with decimal offset - Decompressors mapped to Heads paths (zstd-decompress, unxz, etc.) - Correct tail -c+9 indexing for 0-based vs 1-based offsets - Works on both GNU host and BusyBox Heads environment Heads and most distro kernels don't ship with CONFIG_IKCONFIG=y, so the script is included for future use and external kernels.
Summary
This PR implements a layered ISO boot system with safeguards, clear user guidance,
and comprehensive test coverage. Before showing boot options, Heads now verifies
whether the ISO's initramfs contains kernel modules capable of reading the USB
filesystem after kexec — and displays per-option compatibility markers so the
user knows which entries SHOULD work (see limitations below for efifb requirements
in the ISO's initramfs).
The layered approach was inspired by u-root's boot/iso package.
Design: Three-Layer ISO Boot
Layer 1 — initramfs filesystem compatibility check
Each initrd referenced by the ISO's boot config is unpacked and inspected for
the kernel module matching the USB partition's filesystem type. Results are
written to
/tmp/kexec_initrd_compat.txtand displayed as per-option markers:[OK](green) — initrd has the USB fs module (confirmed)[!](yellow) — initrd has loadable modules but NOT the needed one (may fail)If no initrd is confirmed compatible (all entries are
[!]or blank), the useris warned that the ISO is likely a hybrid ISO designed for direct USB writing,
and is told which filesystem module is missing (e.g. ext4). The user can still
proceed, but the warning prevents silent boot failure — the Debian DVD ISO is
a known case that triggers this.
Layer 2 — loopback.cfg fast path
Parses GRUB
${iso_path}and${isofile}variables from the ISO's standardUSB-boot config (
/boot/grub/loopback.cfg) to extract kernel parameters.If no GRUB variables are found, falls through to a universal parameter set
compatible with all major initramfs frameworks.
Layer 3 — interactive boot menu (kexec-select-boot.sh)
Shows per-option
[OK]/[!]markers,fmt_display_params()for display-onlyISO-param stripping (preserves full params in saved defaults), back-to-menu
navigation, and a full kernel command line breakdown in the boot confirmation
screen:
Key Changes from origin/master
ISO boot parameter injection
fromiso=/dev/disk/by-uuid/UUID/ISOfindisoin Debian live-bootfindiso=$ISO_PATH— Debian live-boot + NixOS stage-1live-media=$ISO_DEV— device filter for casper/live-bootiso-scan/filename=,img_dev=,img_loop=,iso=Unsigned ISO flow
Initramfs infrastructure
unpack_initramfs.sh: fixed multi-segment cpio (BusyBox xxd vs GNU),BusyBox cpio exit code handling, zstd fallback
kexec-parse-boot.sh/kexec-parse-bls.sh:$()modernization,ISO parameter stripping removed from parser (moved to display layer),
---separator removal, tab-indented config supportkexec-boot.sh: kernel cmdline length warning,---separator strippingfunctions.sh: skip*EFI*/*efi*/*x86_64-efi*configs inscan_boot_options(); remove_strip_ansi()(ANSI preserved in debug.log)boards/qemu-*-prod_quiet: fix CONFIG_BOOT_KERNEL_REMOVE to preserve quiet/splashUX improvements
[OK]/[!]compat markers with ANSI coloringfmt_display_params()strips ISO-finding params from menu onlysort -t\| -k1 -u)Known Limitations
efifb required in target kernel
The display pipeline relies on the target (ISO/installed OS) kernel having
CONFIG_FB_EFI=y(efifb). simplefb and simpledrm are incompatible with theefifb-framebuffer that Heads hands off via kexec.
Linux no longer exposes the physical framebuffer address (
smem_start) touserspace via
FBIOGET_FSCREENINFO, so kexec-tools writeslfb_base = 0into the new kernel's boot params. The target kernel's efifb sees a zero
address and cannot map the framebuffer — the display stays blank on boards
without a DRM/KMS driver (i915, nouveau, amdgpu, etc.) that reinitialises
the display after kexec.
See doc/architecture.md for the full efifb chain
ASCII diagram and doc/iso_boot.md for the technical
walkthrough.
TPM Disk Unlock Key is the supported workaround for LUKS-encrypted
systems: Heads injects the sealed key before kexec, so the initramfs never
prompts for a passphrase on a blank display.
ISOs without efifb support (TinyCore/CorePlus)
Distributions shipping minimal kernels without
CONFIG_FB_EFI=yproduce noframebuffer driver at all after kexec. The display stays blank even if the
kernel boots successfully. CorePlus-current.iso is a known example.
Debian DVD installer ISOs
The Debian installer initramfs uses
cdrom-detectwhich has noiso-scan/filename=orfindiso=support. The loop device used by Headsto mount the ISO does not survive kexec, so the installer cannot find the
installation media. Write the ISO directly to USB with
ddand use theexternal USB boot option as a workaround.
Note that the Debian DVD installer is still needed to properly partition
the drive with an unencrypted
/boot(required by Heads — seedoc/prerequisites.md) while keeping the rest of
the OS (root, swap, etc.) LUKS-encrypted for use with TPM Disk Unlock Key.
Documentation
doc/boot-process.md: new Stage 2b (USB ISO Boot) sectiondoc/iso_boot.md: kernel parameter reference — which initramfs frameworkuses which parameter, value rules, known limitations
doc/busybox_perks.md: BusyBox vs GNU tool differences (xxd, cpio, sort-k1 -u incompatibility documented)
doc/modules.md: module list from Makefile — which tools are BusyBox appletsvs standalone
doc/logging.md: ANSI policy update — codes preserved in debug.logdoc/architecture.md: efifb display chain ASCII diagramdoc/ux-patterns.md: boot option format and compat marker documentationdoc/index.md: cross-referencesTest Harness
tests/iso-parser/run.sh— a comprehensive test suite covering:---separators, deep paths, GRUB variables, loopback.cfg (INLINE/SOURCE)no-modules initrd (blank marker), ext4/fat/btrfs verification
--with-isos <path>): mounts ISOs via fuseiso,runs the full pipeline, displays per-ISO matrix
--with-isos): user-facing menu and confirmationoutput for every boot option