Tracker issue for automotive use of bootc

[alexlarsson]

This issue is intended to track the status of use of bootc in the automotive sig project. The initial goal is to use automotive-image-builder to produce OCI images in bootc format and then use bootc-image-builder to create physical images from them.

I have done some initial work in other modules:

• Initial automotive work: custom selinux policy, separate build container for bootc, and ext4 verity osbuild/images#1519
• Automotive initial work osbuild/bootc-image-builder#928
• postprocess: Don't hardcode "targeted" policy coreos/rpm-ostree#5382
• Fixes for ESP partition: Make optional, set label osbuild/images#1525
• https://gitlab.com/CentOS/automotive/src/automotive-image-builder/-/merge_requests/310
• image: If using a separate build container, copy bootc customization to it osbuild/images#1571

But, further work needs to happen:

• To use transient /etc, we need to pre-generate the mount files for e.g. /boot, and we need to extend the bootc install customization so that we can pass the uuids to bc-i-b to use for these partitions / filesystems.
• We need a way to tell bc-i-b to not create its own copy of the various mount files. Maybe its enough to just detect the existance of pre-existing mount files to make it skip this.
• We need the ability to tell bc-i-b to add custom partitions to the image. This is used for example for /var and /var/qm in automotive
• We need to support aboot/ukiboot, which involves: not using grub, creating special partitions (boot_a, boot_b, etc) with special uuids, and writing files from the image to these partitions in bc-i-b. This may require integration work with bootupd.
  • Allow creating exact, detailed partition table, not just blueprint disk customization
  • Allow writing file from image to partition at deploy time (i.e. write /boot/aboot.img to the "aboot_a" partition)
• I get (from rpm-ostree or osbuild) a var.mount that references /sysroot/ostree/deploy/$osname/var. If this $osname doesn't match what bc-i-b uses at deploy time this breaks. These need to be synchronized somehow.
• We need to rethink how composefs signatures work because we can't sign the commit in a-i-b anymore, since the final commit will get a different hash.
• bc-i-b seems to default to hybrid mode GPT, we want legacy-free.
• bootc install to-filesystem reads various options from the build container (such as e.g. prepare-root.conf) which may not be the same as the target container in the automotive case.

This issue is for tracking, lets do the work on the individual things in separate issus.

[cgwalters]

To use transient /etc, we need to pre-generate the mount files for e.g. /boot,

Do you actually need a separate /boot? What's the rootfs storage? Is dm-crypt/dm-integrity involved?

We need a way to tell bc-i-b to not create its own copy of the various mount files.

Yes, a lot of prior related discussion in e.g. #926 among others.

We need to rethink how composefs signatures work because we can't sign the commit in a-i-b anymore, since the final commit will get a different hash.

Did you see all the work happening in #1190 in https://github.com/containers/composefs-rs/

That said in the short term, we definitely can support the existing signature scheme that works in ostree with bootc. It would not be a lot of work.

[alexlarsson]

To use transient /etc, we need to pre-generate the mount files for e.g. /boot,

Do you actually need a separate /boot? What's the rootfs storage? Is dm-crypt/dm-integrity involved?

You're right, the rootfs is just a plain ext4 by default, so a separate /boot is not actually needed. We have some (although not very tested) support for dm-verity, which would make it required. However, once we finalize dropping grub for ukiboot, I don't think it will ever be needed.

Did you see all the work happening in #1190 in https://github.com/containers/composefs-rs/

I did not, but will look at it.

[ericcurtin]

In a lot of our Automotive images at present there is no separate /boot . And a lot of the things in /boot directory aren't actually used as such but we still keep various files there as they are expected to be there by tools, rpms, OSTree

[alexlarsson]

@ericcurtin yes, but we can have a /boot directory on the main rootfs, it doesn't have to be a partition.

[ericcurtin]

@ericcurtin yes, but we can have a /boot directory on the main rootfs, it doesn't have to be a partition.

In the aboot builds, I think we do this already, works great.

[alexlarsson]

To use transient /etc, we need to pre-generate the mount files for e.g. /boot,

Do you actually need a separate /boot? What's the rootfs storage? Is dm-crypt/dm-integrity involved?

We need a way to tell bc-i-b to not create its own copy of the various mount files.

Yes, a lot of prior related discussion in e.g. #926 among others.

We need to rethink how composefs signatures work because we can't sign the commit in a-i-b anymore, since the final commit will get a different hash.

Did you see all the work happening in #1190 in https://github.com/containers/composefs-rs/

That said in the short term, we definitely can support the existing signature scheme that works in ostree with bootc. It would not be a lot of work.

@cgwalters How do you expect this to work? "bootc install to-filesystem" always generates a new commit object, so any signatures on the original commit will be lost, even if the end result is the same content tree.

[alexlarsson]

Some add on work on the "booc install reads config from build container" here: osbuild/images#1571

[cgwalters]

@cgwalters How do you expect this to work? "bootc install to-filesystem" always generates a new commit object, so any signatures on the original commit will be lost, even if the end result is the same content tree.

That's basically a design flaw, it can be fixed (by requiring signed images to use pre-computed commits, but the bigger thing is we need to store the image metadata out of band).

I think we should probably chat about this one in #1190 as it's most closely related.

[alexlarsson]

With the merging of osbuild/images#1571 much of this stuff is now working. The remaining issues are mostly around the fact that we can't specify a hyper-detailed lowlevel partition table as required by e.g. aboot or ukiboot support. The goal there is to make bc-i-b be able to read a yaml description of the full partitioning table, of the same format we use elsewhere. @achilleas-k is looking at doing that.

It will also require backporting the "dd file to a partition" stage from osbuild-auto to osbuild. I will look into that.

And, on top of that we need to figure out the story for signatures, but lets wait with that.