Create MicroShift iso using image mode and bootc image builder #999

praveenkumar · 2025-01-10T08:48:44Z

With 4.18 microshift removed the steps of creating the iso using image builder and there is no more build.sh script which is consumed by mircoshift.sh script to create it. This PR use the image mode and bootc image builder (BIB) to create the iso which is now microshift team also pushing forward.

openshift-ci · 2025-01-10T08:48:49Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

praveenkumar · 2025-01-13T05:41:25Z

/test all

praveenkumar · 2025-01-13T09:07:14Z

/test e2e-microshift

praveenkumar · 2025-01-14T06:21:10Z

/test e2e-microshift

praveenkumar · 2025-01-15T05:23:40Z

/cherry-pick release-4.18

openshift-cherrypick-robot · 2025-01-15T05:23:43Z

@praveenkumar: once the present PR merges, I will cherry-pick it on top of release-4.18 in a new PR and assign it to you.

In response to this:

/cherry-pick release-4.18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

praveenkumar · 2025-01-15T14:35:35Z

/hold

praveenkumar · 2025-01-24T09:36:17Z

/unhold

microshift.sh

image-mode/microshift/config/Containerfile.bootc-rhel9

cfergeau · 2025-02-05T11:07:51Z

image-mode/microshift/config/config.toml.template

+firewall-offline-cmd --zone=trusted --add-source=169.254.169.1
+
+
+# Configure systemd journal service to persist logs between boots and limit their size to 1G


1G is quite big.

what do you think would be best 200M ?

Or we could set the storage as persistent but keep the default values?

the comment is now outdated though.

removed this complete section because we don't need to persist logs between the boots since, this is something required for prod kind of setup.

image-mode/microshift/config/config.toml.template

image-mode/microshift/build.sh

cfergeau · 2025-02-05T11:21:58Z

image-mode/microshift/config/Containerfile.bootc-rhel9

+      # This is required to update the gpgcheck for repoID
+      repoID=$(echo "${MIRROR_REPO#*://}" | tr '/:' '_'); \
+      dnf config-manager --add-repo "${MIRROR_REPO}" \
+          --add-repo "https://mirror.openshift.com/pub/openshift-v4/$(uname -m)/dependencies/rpms/${USHIFT_VER}-el9-beta" \


Does this https://mirror.openshift.com/pub/openshift-v4/... repo need to be hardcoded in addition to the user-specified "${MIRROR_REPO}"?

yes this is dependency repo and it is different from mirror repo.

Even with --use-mirror-repo documentation (Use mirror repo to get release candidate and engineering preview rpms) it's not immediatly obvious to me why we would force the use of ${USHIFT_VER}-el9-beta instead of a non-beta repo.
Side-note, if --use-mirror-repo is only meant for release candidate and engineering preview, the name is not great as it's too generic.

I don't find a good name around but adding the comment why this dependencies repo is required.

maybe unreleased-mirror or such? I'm worried the use of -beta is unexpected in some cases, and is really non obvious from the option name.

Used unreleased-mirror as environment variable to avoid confusion.

createdisk.sh

cfergeau · 2025-02-07T09:20:27Z

image-mode/microshift/config/Containerfile.bootc-rhel9

+RUN firewall-offline-cmd --zone=public --add-port=80/tcp && \
+    firewall-offline-cmd --zone=public --add-port=443/tcp && \
+    firewall-offline-cmd --zone=public --add-port=30000-32767/tcp && \
+    firewall-offline-cmd --zone=public --add-port=30000-32767/udp


we disable firewall in createdisk.sh for microshift bundles, this seems redundant?

Yes, removing this from container file.

cfergeau · 2025-02-07T09:21:04Z

image-mode/microshift/config/config.toml.template

+firewall-offline-cmd --zone=trusted --add-source=169.254.169.1
+
+
+# Configure systemd journal service to persist logs between boots and limit their size to 1G


the comment is now outdated though.

As per doc https://github.com/openshift/microshift/blob/main/docs/user/howto_config.md#drop-in-configuration-directory it is better to use the drop in configuration directory then changing the default one.

cfergeau · 2025-02-10T13:01:21Z

ci_microshift.sh

@@ -5,6 +5,10 @@ set -exuo pipefail
 sudo yum install -y make golang

 ./shellcheck.sh
+ARCH=$(uname -m)
+if [[ "$ARCH" == "aarch64" ]]; then
+    export SNC_NON_FATAL_PREFLIGHT_CHECKS=1


it would make more sense to fix the capabilities check on aarch64. I don't think this is directly related to this PR?

It is not directly related to this PR but I wanted to make sure both arm and amd build succeed before merge it.

fix the capabilities check on aarch64.

Yes, I think GCP doesn't allow native nested virtualization for arm64 and current job is using emulation only and it is slow but at least we are getting bundle build for arm64, will check if some different cloud provider can be used.

If we want to allow emulation on a specific platform, it should be the script starting snc on this platform which sets this variable. Ideally we'd have a way to only disable the "kvm" check there.

Ideally we'd have a way to only disable the "kvm" check there.

@cfergeau yes, something like SNC_DISABLE_KVM_CHECK ?

Dropping this commit as of now and going with merge, will create follow up PR for it.

openshift-ci · 2025-02-10T13:07:59Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cfergeau

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [cfergeau]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

praveenkumar · 2025-02-11T05:23:35Z

/retest

praveenkumar · 2025-02-11T09:16:22Z

/retest

openshift-ci · 2025-02-11T09:21:02Z

@praveenkumar: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-microshift-arm	`febf74c`	link	true	`/test e2e-microshift-arm`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

With 4.18 microshift removed the steps of creating the iso using image builder and there is no more `build.sh` script which is consumed by mircoshift.sh script to create it. This PR use the image mode and bootc image builder (BIB) to create the iso which is now microshift team also pushing forward.

openshift-ci · 2025-02-11T15:01:32Z

New changes are detected. LGTM label has been removed.

praveenkumar · 2025-02-11T15:02:09Z

/cherry-pick release-4.18

openshift-cherrypick-robot · 2025-02-11T15:02:33Z

@praveenkumar: #999 failed to apply on top of branch "release-4.18":

Applying: microshift: Use config.d directory for drop-in config directory
Applying: Create MicroShift iso using image mode and bootc image builder
Using index info to reconstruct a base tree...
M	microshift.sh
Falling back to patching base and 3-way merge...
Removing podman_changes.ks
Auto-merging microshift.sh
CONFLICT (content): Merge conflict in microshift.sh
Removing 00-microshift-dns.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0002 Create MicroShift iso using image mode and bootc image builder

In response to this:

/cherry-pick release-4.18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-cherrypick-robot · 2025-02-11T15:03:09Z

@praveenkumar: #999 failed to apply on top of branch "release-4.18":

Applying: microshift: Use config.d directory for drop-in config directory
Applying: Create MicroShift iso using image mode and bootc image builder
Using index info to reconstruct a base tree...
M	microshift.sh
Falling back to patching base and 3-way merge...
Removing podman_changes.ks
Auto-merging microshift.sh
CONFLICT (content): Merge conflict in microshift.sh
Removing 00-microshift-dns.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0002 Create MicroShift iso using image mode and bootc image builder

In response to this:

/cherry-pick release-4.18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci bot added the do-not-merge/work-in-progress label Jan 10, 2025

praveenkumar force-pushed the issue_998 branch 4 times, most recently from 6ab8c1e to c074a21 Compare January 13, 2025 05:40

praveenkumar marked this pull request as ready for review January 13, 2025 05:41

openshift-ci bot requested review from anjannath and gbraad January 13, 2025 05:41

praveenkumar force-pushed the issue_998 branch 2 times, most recently from 125c5ac to 83bd510 Compare January 13, 2025 08:09

praveenkumar force-pushed the issue_998 branch 6 times, most recently from 3610d4d to fd6b49f Compare January 14, 2025 04:09

praveenkumar changed the title ~~[WIP] Create MicroShift iso using image mode and bootc image builder~~ Create MicroShift iso using image mode and bootc image builder Jan 15, 2025

openshift-ci bot removed the do-not-merge/work-in-progress label Jan 15, 2025

praveenkumar force-pushed the issue_998 branch from fd6b49f to 8e204a4 Compare January 15, 2025 05:29

openshift-ci bot added the do-not-merge/hold label Jan 15, 2025

praveenkumar force-pushed the issue_998 branch from 8e204a4 to 2c867a0 Compare January 17, 2025 05:14

openshift-ci bot removed the do-not-merge/hold label Jan 24, 2025

praveenkumar self-assigned this Feb 5, 2025

cfergeau reviewed Feb 5, 2025

View reviewed changes

praveenkumar force-pushed the issue_998 branch from 2c867a0 to f051c02 Compare February 6, 2025 07:14

cfergeau reviewed Feb 7, 2025

View reviewed changes

microshift: Use config.d directory for drop-in config directory

8cac863

As per doc https://github.com/openshift/microshift/blob/main/docs/user/howto_config.md#drop-in-configuration-directory it is better to use the drop in configuration directory then changing the default one.

praveenkumar force-pushed the issue_998 branch 4 times, most recently from dcac478 to febf74c Compare February 10, 2025 11:47

cfergeau approved these changes Feb 10, 2025

View reviewed changes

openshift-ci bot assigned cfergeau Feb 10, 2025

openshift-ci bot added the lgtm label Feb 10, 2025

openshift-ci bot added the approved label Feb 10, 2025

praveenkumar force-pushed the issue_998 branch from febf74c to 1df1ccb Compare February 11, 2025 15:01

openshift-ci bot removed the lgtm label Feb 11, 2025

praveenkumar merged commit 1fb32f8 into crc-org:master Feb 11, 2025
0 of 4 checks passed

		firewall-offline-cmd --zone=trusted --add-source=169.254.169.1


		# Configure systemd journal service to persist logs between boots and limit their size to 1G

Create MicroShift iso using image mode and bootc image builder #999

Create MicroShift iso using image mode and bootc image builder #999

Conversation

praveenkumar commented Jan 10, 2025

openshift-ci bot commented Jan 10, 2025

praveenkumar commented Jan 13, 2025

praveenkumar commented Jan 13, 2025

praveenkumar commented Jan 14, 2025

praveenkumar commented Jan 15, 2025

openshift-cherrypick-robot commented Jan 15, 2025

praveenkumar commented Jan 15, 2025

praveenkumar commented Jan 24, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openshift-ci bot commented Feb 10, 2025

praveenkumar commented Feb 11, 2025

praveenkumar commented Feb 11, 2025

openshift-ci bot commented Feb 11, 2025

openshift-ci bot commented Feb 11, 2025

praveenkumar commented Feb 11, 2025

openshift-cherrypick-robot commented Feb 11, 2025

openshift-cherrypick-robot commented Feb 11, 2025