gcp-crio: retry instance creation across all zones in the region

[bitoku]

The provisioning step always picked the first zone (zones[0]) from the region. When that zone was out of capacity (ZONE_RESOURCE_POOL_EXHAUSTED), the job failed with no fallback. Iterate over all zones in randomized order and try each until instance creation succeeds.

Assisted-by: Claude Code https://claude.com/claude-code

Summary by CodeRabbit

Updates GCP CRI-O build host provisioning for OpenShift CI infrastructure. The script randomizes all zones in the selected region and retries instance creation in each zone. It records the successful zone for subsequent operations and fails only after all zones fail, including failures such as ZONE_RESOURCE_POOL_EXHAUSTED.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: d000e9cc-3eab-445b-add7-ebd1a1b0ab31

📥 Commits

Reviewing files that changed from the base of the PR and between 312455e and fe05514.

📒 Files selected for processing (1)

• ci-operator/step-registry/gcp/crio/provision/buildhost/gcp-crio-provision-buildhost-commands.sh

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

Build host provisioning now shuffles regional zones, retries instance creation across zones, records the successful zone, and fails when no zone succeeds.

Changes

Build host provisioning

Layer / File(s)	Summary
Zone selection and instance creation fallback ci-operator/step-registry/gcp/crio/provision/buildhost/gcp-crio-provision-buildhost-commands.sh	The script shuffles available zones, retries instance creation for each zone, records the successful zone in ZONE_0, and exits with an error when all zones fail.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to fe055

The change adds zone fallback during instance creation, and no actionable merge-blocking risk remains beyond normal checks and review.

Suggested reviewers: dgoodwin, stbenjam

🚥 Pre-merge checks | ✅ 15

✅ Passed checks (15 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: retrying GCP CRI-O instance creation across all zones in the region.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	PASS: The pull request changes only one .sh provisioning script. The exact HEAD^..HEAD diff contains no It(), Describe(), Context(), or When() declarations and introduces no test title. Th…
Test Structure And Quality	✅ Passed	PASS: The pull request changes only one shell script, gcp-crio-provision-buildhost-commands.sh. It adds randomized GCP zone selection and instance-creation retries. It does not add or modify Ginkgo …
Microshift Test Compatibility	✅ Passed	PASS: The pull request changes only the GCP CRI-O build-host shell script. The exact diff adds no Ginkgo e2e tests or test declarations such as It, Describe, Context, or When. Therefore, the MicroShif…
Single Node Openshift (Sno) Test Compatibility	✅ Passed	PASS — The pull request changes only the GCP CRI-O build-host provisioning shell script. The committed diff adds zone selection and instance-creation retries; it adds no Ginkgo tests or other e2e test…
Topology-Aware Scheduling Compatibility	✅ Passed	PASS: The pull request changes only a GCP CRI-O build-host provisioning shell script. The diff adds randomized GCP zone selection and instance-creation retries. It does not add or modify deployment ma…
Ote Binary Stdout Contract	✅ Passed	PASS: The pull request changes only the Bash GCP provisioning step. The step metadata runs it as a pre-provisioning CI workflow step, not as an OTE binary. The added echo statements are provisioning…
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	PASS: The pull request changes one provisioning shell script only. The exact diff adds no Ginkgo e2e tests and no It, Describe, Context, or When declarations. Therefore this custom check is no…
No-Weak-Crypto	✅ Passed	The pull request changes only zone enumeration and GCP instance-creation retry logic. The exact diff adds shuf, a zone loop, and instance state handling. It adds no MD5, SHA1, DES, 3DES, RC4, Blowfi…
Container-Privileges	✅ Passed	PASS: The commit changes only the GCP provisioning shell script. The diff adds zone shuffling and instance-creation retries. It adds no privileged, hostPID, hostNetwork, hostIPC, SYS_ADMIN, …
No-Sensitive-Data-In-Logs	✅ Passed	The change adds logs for GCP zone names and the existing region value. These values are not passwords, tokens, API keys, PII, session IDs, internal hostnames, or customer data. The instance creation c…

Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1 files.

Full details: Stable And Deterministic Test Names

Explanation

PASS: The pull request changes only one .sh provisioning script. The exact HEAD^..HEAD diff contains no It(), Describe(), Context(), or When() declarations and introduces no test title. The dynamic zone values remain operational shell variables, not Ginkgo test names.

Full details: Test Structure And Quality

Explanation

PASS: The pull request changes only one shell script, gcp-crio-provision-buildhost-commands.sh. It adds randomized GCP zone selection and instance-creation retries. It does not add or modify Ginkgo test code, It blocks, fixtures, waits, or assertions, so the listed test-quality checks are not applicable.

Full details: Microshift Test Compatibility

Explanation

PASS: The pull request changes only the GCP CRI-O build-host shell script. The exact diff adds no Ginkgo e2e tests or test declarations such as It, Describe, Context, or When. Therefore, the MicroShift test compatibility check is not applicable.

Full details: Single Node Openshift (Sno) Test Compatibility

Explanation

PASS — The pull request changes only the GCP CRI-O build-host provisioning shell script. The committed diff adds zone selection and instance-creation retries; it adds no Ginkgo tests or other e2e test constructs. The SNO compatibility check is therefore not applicable.

Full details: Topology-Aware Scheduling Compatibility

Explanation

PASS: The pull request changes only a GCP CRI-O build-host provisioning shell script. The diff adds randomized GCP zone selection and instance-creation retries. It does not add or modify deployment manifests, operators, controllers, pod scheduling constraints, replicas, affinity, tolerations, or PDBs. The topology-aware scheduling check is therefore not applicable.

Full details: Ote Binary Stdout Contract

Explanation

PASS: The pull request changes only the Bash GCP provisioning step. The step metadata runs it as a pre-provisioning CI workflow step, not as an OTE binary. The added echo statements are provisioning diagnostics and do not affect an openshift-tests JSON test-listing process. No main(), TestMain(), Ginkgo suite setup, klog, or Go stdout code changed.

Full details: Ipv6 And Disconnected Network Test Compatibility

Explanation

PASS: The pull request changes one provisioning shell script only. The exact diff adds no Ginkgo e2e tests and no It, Describe, Context, or When declarations. Therefore this custom check is not applicable. The script's existing external download is not a new Ginkgo test.

Full details: No-Weak-Crypto

Explanation

The pull request changes only zone enumeration and GCP instance-creation retry logic. The exact diff adds shuf, a zone loop, and instance state handling. It adds no MD5, SHA1, DES, 3DES, RC4, Blowfish, ECB, custom cryptography, or secret/token comparisons. The existing ed25519 SSH key generation is unchanged and is not a prohibited algorithm.

Full details: Container-Privileges

Explanation

PASS: The commit changes only the GCP provisioning shell script. The diff adds zone shuffling and instance-creation retries. It adds no privileged, hostPID, hostNetwork, hostIPC, SYS_ADMIN, allowPrivilegeEscalation, or root-user setting. The related step YAML and metadata are unchanged.

Full details: No-Sensitive-Data-In-Logs

Explanation

The change adds logs for GCP zone names and the existing region value. These values are not passwords, tokens, API keys, PII, session IDs, internal hostnames, or customer data. The instance creation command still uses the existing server, network, subnet, and public SSH key values; its xtrace exposure is pre-existing. No new sensitive-data logging condition is introduced.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-merge-bot]

@bitoku: pj-rehearse could not automatically process this event because the request waited in queue for longer than 5 minutes. Use /pj-rehearse to trigger rehearsals manually.

[bitoku]

/pj-rehearse

[openshift-merge-bot]

@bitoku: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[bitoku]

/pj-rehearse auto-ack

[openshift-merge-bot]

@bitoku: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[bitoku]

/retest

[bitoku]

/test pull-ci-openshift-release-main-step-registry-metadata

[bitoku]

/pj-rehearse auto-ack

[openshift-merge-bot]

@bitoku: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[bitoku]

/test step-registry-metadata

[bitoku]

/pj-rehearse pull-ci-cri-o-cri-o-main-ci-crun-e2e pull-ci-cri-o-cri-o-main-ci-cgroupv2-e2e-crun pull-ci-cri-o-cri-o-main-ci-cgroupv2-e2e-features

[openshift-merge-bot]

@bitoku: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[openshift-merge-bot]

@bitoku: pj-rehearse could not automatically process this event because the request waited in queue for longer than 5 minutes. Use /pj-rehearse to trigger rehearsals manually.

[bitoku]

/pj-rehearse auto-ack

[openshift-merge-bot]

@bitoku: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[bitoku]

/pj-rehearse ack

[openshift-merge-bot]

@bitoku: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[haircommander]

/lgtm
/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bitoku, haircommander

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• ci-operator/step-registry/gcp/crio/provision/buildhost/OWNERS [haircommander]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@bitoku: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/rehearse/cri-o/cri-o/main/ci-cgroupv2-integration	fe05514	link	unknown	/pj-rehearse pull-ci-cri-o-cri-o-main-ci-cgroupv2-integration
ci/rehearse/cri-o/cri-o/main/ci-cgroupv2-e2e	fe05514	link	unknown	/pj-rehearse pull-ci-cri-o-cri-o-main-ci-cgroupv2-e2e
ci/rehearse/cri-o/cri-o/main/ci-cgroupv2-e2e-crun	fe05514	link	unknown	/pj-rehearse pull-ci-cri-o-cri-o-main-ci-cgroupv2-e2e-crun
ci/rehearse/cri-o/cri-o/main/ci-cgroupv2-e2e-features	fe05514	link	unknown	/pj-rehearse pull-ci-cri-o-cri-o-main-ci-cgroupv2-e2e-features
ci/rehearse/cri-o/cri-o/main/ci-crun-e2e	fe05514	link	unknown	/pj-rehearse pull-ci-cri-o-cri-o-main-ci-crun-e2e

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.