Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the Helm procedure to use oc-mirror for air-gapped environments #868

Open
wants to merge 9 commits into
base: release-1.4
Choose a base branch
from

Conversation

Fortune-Ndlovu
Copy link
Contributor

@Fortune-Ndlovu Fortune-Ndlovu commented Jan 21, 2025

IMPORTANT: Do Not Merge - To be merged by Docs Team Only

Version(s):

This PR updates the air-gapped installation documentation for Helm using oc-mirror. The key updates include:

  • Adding detailed steps for:
  • Creating the ImageSetConfiguration file.
  • Running the mirroring process.
  • Transferring and extracting the archive in an air-gapped environment.
  • Installing and verifying the Helm chart.

Curious about the visual output take a look at this gist of the update: https://gist.github.com/Fortune-Ndlovu/8b93a529ae6e79e7d79bf256d078bf38

Issue:
Related Issue: https://issues.redhat.com/browse/RHIDP-5351

Issue where the testing was verified: https://issues.redhat.com/browse/RHIDP-1723

@rhdh-bot
Copy link
Collaborator

rhdh-bot commented Jan 21, 2025

@Fortune-Ndlovu
Copy link
Contributor Author

Closing PR because docs member is already working on this PR update

----

. Go to *YAML view* and update the `image` section for `backstage` and `postgresql` using the following values:
. Deploy the Helm chart in the air-gapped environment using the `helm`
Copy link
Member

@linfraze linfraze Jan 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like we show a code sample instead of a command.

Where is the command that the user can copy and paste?

What is the code sample showing and what does the user do with it?

Copy link
Contributor Author

@Fortune-Ndlovu Fortune-Ndlovu Jan 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the command that the user can copy and paste?

[source,terminal]
----
scp ./mirror-archive/mirror_seq1_000000.tar /tmp
----

What is the code sample showing and what does the user do with it?
The code sample demonstrates the process of transferring the generated archive (mirror_seq1_000000.tar) to the air-gapped environment using scp. Once transferred, the user extracts the contents of the archive using the tar command. This step is necessary to make the mirrored resources (such as container images, Helm charts, or Operator catalogs) accessible in the air-gapped environment.

Finally, the user uses these extracted resources to deploy and configure their applications or clusters in the disconnected environment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, for a fully disconnected env, the next steps once the image set has been mirrored to disk should be something like "Transfer the archive to the disconnected environment".
They might use scp, rsync or even physically move from a flash drive to their disconnected environment. This is up to them, and we cannot know that.

Copy link
Member

@linfraze linfraze Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See this thread for suggested resolution

…s that you want to mirror. For example:

Signed-off-by: Fortune Ndlovu <[email protected]>
Signed-off-by: Fortune Ndlovu <[email protected]>
kind: ImageSetConfiguration
storageConfig:
local:
path: ./mirror-output
Copy link
Member

@rm3l rm3l Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, we should distinguish between mirroring to a mirror registry in a partially disconnected environment or in a fully disconnected environment.
In a partially disconnected disconnected environment, users can mirror the image set directly to the target mirror registry. So they won't need to mirror everything to disk first (using this storageConfig.local.path field). Because it might result in downloading several hundreds of gigabytes of data to disk, which is useless in that case IMO.
What you are describing here makes sense in a fully disconnected environment, where users would first mirror the image set to disk, then transfer the archive to the disconnected env, and finally mirror from the disk to the target registry.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should create two separate procedure modules - one for fully disconnected users and one for partially disconnected users

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How the procedures are different:

Fully disconnected:

  1. mirror everything to disk using this storageConfig.local.path field
  2. transfer the archive to the disconnected environment
  3. mirror from the disk to the target registry

Partially disconnected:

  1. mirror the image set directly to the target mirror registry.
    (Skip steps 1 & 2 because it might result in unnecessarily downloading several hundreds of gigabytes of data to disk)

Copy link
Member

@linfraze linfraze Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proper CCS modularization standards would have us create two separate modules, but I'm open including both procedures in the same module (but written as if they are in separate modules) for now and modularizing under a separate PR in the future.

So we would have the following structure:

= Installing {product} in an air-gapped environment with the Helm Chart
[conceptual info]

== Installing {product} in a partially disconnected environment with the Helm Chart
[conceptual and procedural info for this use case]

== Installing {product} in a fully disconnected environment with the Helm Chart
[conceptual and procedural info for this use case]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the prerequisites different? Should we mention the need for some connectivity to complete the partially connected procedure?

- name: redhat-developer-hub
version: "1.4.1"
----
. Initiate the mirroring process using the `ImageSetConfiguration` file. The `oc-mirror` command mirrors the specified resources as defined in the configuration file and generates a `.tar` archive containing all the mirrored resources.
Copy link
Member

@linfraze linfraze Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
. Initiate the mirroring process using the `ImageSetConfiguration` file. The `oc-mirror` command mirrors the specified resources as defined in the configuration file and generates a `.tar` archive containing all the mirrored resources.
. Mirror the resources specified in the`ImageSetConfiguration` file by running the following command:

Make the following a Note or move to the conceptual portion of the module before the procedure (maybe both):

The oc-mirrorcommand mirrors the specified resources as defined in the configuration file and generates a.tar archive containing all the mirrored resources.

Copy link
Member

@linfraze linfraze Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Fortune-Ndlovu Is the changed step still technically accurate?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conceptual information in the module is very generic and replicates the information in the higher-level module. Replace it with more specific information - if we split into two modules (one for fully disconnected, one for partially disconnected), it would be a good idea to give specific conceptual information for both of those scenarios and explain how / why the process is unique to each use case.

Note: we've received general feedback that customers want more conceptual info (to better understand what and why) in Red Hat docs.

Signed-off-by: Fortune-Ndlovu <[email protected]>
…e between fully and partially disconnected env
----
podman pull registry.redhat.io/rhdh/rhdh-hub-rhel9:{product-version}
oc-mirror --config=mirror-config.yaml file://mirror-archive
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Fortune-Ndlovu Do you confirm this works as expected by mirroring the right images? I just tested it, and it seems it is not able to detect the images:

$ oc mirror -config=.asoro/airgap/my-image-set-configuration.yaml file://mirror-archive
Checking push permissions for registry.localhost:5000
Found: oc-mirror-workspace/src/publish
Found: oc-mirror-workspace/src/v2
Found: oc-mirror-workspace/src/charts
Found: oc-mirror-workspace/src/release-signatures
backend is not configured in .asoro/airgap/my-image-set-configuration.yaml, using stateless mode
backend is not configured in .asoro/airgap/my-image-set-configuration.yaml, using stateless mode
No metadata detected, creating new workspace
Pulling chart redhat-developer-hub
error image list []
No new images detected, process stopping

url: https://charts.openshift.io
charts:
- name: redhat-developer-hub
version: "{product-version}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is the right variable or an issue with the preview, but it seems this is not getting replaced in the final preview:
image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing that out. We need to make sure we include subs="attributes+" in the [source] tags

+
[source,terminal]
----
oc set image-lookup rhdh-hub-rhel9
scp ./mirror-archive/mirror_seq1_000000.tar /tmp
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example is a bit misleading IMO. It should emphasize transferring to a remote host IMO.
I think we should remove this example. The bullet point could simply be "Transfer the generated .tar archive file (for example, mirror++_++seq1++_++000000.tar) to the air-gapped environment".
It is hard to know exactly how people transfer bits to their fully disconnected envs.
WDYT?

----

. Go to *YAML view* and update the `image` section for `backstage` and `postgresql` using the following values:
. Extract the contents of the archive into a directory (for example, `/tmp`) by running the `tar` command. For example:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should clarify that this extraction needs to be happen in the disconnected network. So the host to which the archive was transferred. Otherwise, it looks like this can be run locally.

.Verification
. To verify that all resources are successfully deployed and running, check the status of your deployments and pods in the cluster.
. To validate the route, ensure that the route matches the expected URL based on the router base of the cluster.
.. If necessary, update the route to align with your expected configuration.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we expect users to validate and update this?
How about using a similar procedure as what we have for "Deploying Developer Hub on OCP with the Helm CLI", by providing the command for the user to get the right router base URL? Like so:

CLUSTER_ROUTER_BASE=$(oc get route console -n openshift-console -o=jsonpath='{.spec.host}' | sed 's/^[^.]*\.//')
helm install redhat-developer-hub./mirror-archive/oc-mirror-workspace/src/charts/redhat-developer-hub-1.4.1.tgz \
    --set global.clusterRouterBase="$CLUSTER_ROUTER_BASE"

+
[source,terminal,source,subs="attributes+"]
[source,terminal]
Copy link
Member

@linfraze linfraze Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[source,terminal]
[source,terminal,subs="attributes+"]

Oops, I think I previously added an extra source by mistake, but subs="attributes+" ensures that our attributes are rendered in the code blocks

. Pull {product-short} and PostgreSQL images from https://catalog.redhat.com/software/containers/search[{company-name} Image registry] to your workstation, by running the following commands:
. Create an `ImageSetConfiguration` file to specify the resources that you want to mirror. For example:
+
[source,terminal]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[source,terminal]
[source,terminal,subs="attributes+"]

+
[source,terminal,source,subs="attributes+"]
[source,terminal]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[source,terminal]
[source,terminal,subs="attributes+"]

+
. Create an `ImageSetConfiguration` file to specify the resources that you want to mirror. For example:
+
[source,terminal]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[source,terminal]
[source,terminal,subs="attributes+"]

----
. Transfer the generated `.tar` archive file (for example, `mirror++_++seq1++_++000000.tar`) to the air-gapped environment using `scp` or another file transfer tool. For example:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
. Transfer the generated `.tar` archive file (for example, `mirror++_++seq1++_++000000.tar`) to the air-gapped environment using `scp` or another file transfer tool. For example:
. Transfer the generated `.tar` archive file (for example, `mirror++_++seq1++_++000000.tar`) to the air-gapped environment.

----

. Go to *YAML view* and update the `image` section for `backstage` and `postgresql` using the following values:
. Extract the contents of the archive into a directory (for example, `/tmp`) by running the `tar` command. For example:
Copy link
Member

@linfraze linfraze Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
. Extract the contents of the archive into a directory (for example, `/tmp`) by running the `tar` command. For example:
. In your air-gapped environment, extract the contents of the archive into a directory (for example, `/tmp`) by running the `tar` command. For example:

@rm3l is this better?

So the host to which the archive was transferred

^should we explain this better, too?

kind: ImageSetConfiguration
storageConfig:
local:
path: ./mirror-output
Copy link
Member

@linfraze linfraze Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace path: ./mirror-output with the actual path on your system

use variables and/or definitions to communicate this to the user (like this)

^consider doing this for all of the values in this example

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants