Skip to content

OCPBUGS-94109: cherry-pick reconciler optimizations#503

Open
dtantsur wants to merge 8 commits into
openshift:release-4.22from
dtantsur:OCPBUGS-94109
Open

OCPBUGS-94109: cherry-pick reconciler optimizations#503
dtantsur wants to merge 8 commits into
openshift:release-4.22from
dtantsur:OCPBUGS-94109

Conversation

@dtantsur

Copy link
Copy Markdown
Member

Adds the following commits from 5.0:

  • Avoid calling getNode in GetFirmwareSettings
  • Avoid calling getNode in GetFirmwareComponents
  • Reduce and simplify requeue delays in the Ironic provisioner
  • Remove unused RequeAfter delay
  • Provisioner: drop the separate TryInit call
  • Don't silently ignore errors from /v1/drivers
  • Cache Ironic client and its status across reconciliations
  • Fix lint error

dtantsur and others added 8 commits June 30, 2026 15:46
It's pointless since we only use node UUID, which is already known.

Fix the unit test to check what it actually claims to check.

Signed-off-by: Dmitry Tantsur <dtantsur@protonmail.com>
(cherry picked from commit b6462d8)
The only actual purpose of the call there is to see if firmware
components are supported by the Node. Which, in turn, is needed
to determine if the HFC object needs to be created by the BMH
controller.

This commit changes the behavior slightly: now HFC is not created when
the list of components is empty. This happens both when firmware
settings are not supported and during early stages of enrollment.

Most users will only see that the HFC resource may get created slightly
later than before.

Signed-off-by: Dmitry Tantsur <dtantsur@protonmail.com>
(cherry picked from commit 6835bdb)
Replace five old delay variables with two:

- shortRetryDelay (3s): used when we asked Ironic to do something and
  are waiting for it to finish (state changes, power operations, abort
  requests, lock retries)
- longRetryDelay (10s): used when hardware is doing something slow like
  booting a ramdisk (inspection, cleaning, servicing, deployment)

Also add random jitter (0-25%) to all delays so that many hosts don't
hit Ironic at the exact same time.

Signed-off-by: Sunnatillo <sunnat.samadov@est.tech>
Signed-off-by: Sunnatillo <sunnat.samadov@est.tech>
All of our production code calls TryInit right after NewProvisioner,
so there is little point for them to be separate. A new ErrNotReady
error indicates what returning ready=false used to indicate.

This simplification is needed for my follow-up changes to improve
caching of Ironic parameters between reconciliations.

Signed-off-by: Dmitry Tantsur <dtantsur@protonmail.com>
Signed-off-by: Dmitry Tantsur <dtantsur@protonmail.com>
Currently, BMO checks the available features and the number of drivers
on each reconciliation. These two calls add up when a lot of hosts are
going through provisioning. It's not reasonable to expect Ironic to
constantly change, and even if it suddenly crashes, the worst outcome is
less optimal error messages.

This change caches the result of the endpoint validation for 5 seconds
(subject to further tuning). When Ironic resource is used for
configuration, the resulting client is also cached. Only successful
checks are cached, the behavior on errors is the same as before.

Cache TTL can be tuned via environment variable IRONIC_CLIENT_CACHE_TTL,
including an ability to opt out of caching by setting it to 0.

The change has some implications for unit tests since bypassing API
calls now requires a different approach.

Signed-off-by: Dmitry Tantsur <dtantsur@protonmail.com>
The current code in BMO fails to build because of a missing import

Error: hostfirmwarecomponents_controller.go:132:6: undefined: errors (typecheck)

This commit fixes this issue.

Signed-off-by: Iury Gregory Melo Ferreira <imelofer@redhat.com>
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jun 30, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@dtantsur: This pull request references Jira Issue OCPBUGS-94109, which is invalid:

  • expected Jira Issue OCPBUGS-94109 to depend on a bug targeting a version in 5.0.0 and in one of the following states: MODIFIED, ON_QA, VERIFIED, but no dependents were found

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Adds the following commits from 5.0:

  • Avoid calling getNode in GetFirmwareSettings
  • Avoid calling getNode in GetFirmwareComponents
  • Reduce and simplify requeue delays in the Ironic provisioner
  • Remove unused RequeAfter delay
  • Provisioner: drop the separate TryInit call
  • Don't silently ignore errors from /v1/drivers
  • Cache Ironic client and its status across reconciliations
  • Fix lint error

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c4c918f5-21da-4838-acb7-e744a9b04518

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@dtantsur

Copy link
Copy Markdown
Member Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jun 30, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@dtantsur: This pull request references Jira Issue OCPBUGS-94109, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note text is set and does not match the template
  • dependent bug Jira Issue OCPBUGS-85527 is in the state ON_QA, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
  • dependent Jira Issue OCPBUGS-85527 targets the "5.0.0" version, which is one of the valid target versions: 5.0.0
  • bug has dependents
Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot requested review from hroyrh and zaneb June 30, 2026 14:07
@openshift-ci

openshift-ci Bot commented Jun 30, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dtantsur

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 30, 2026
@openshift-ci

openshift-ci Bot commented Jun 30, 2026

Copy link
Copy Markdown

@dtantsur: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@dtantsur

dtantsur commented Jul 1, 2026

Copy link
Copy Markdown
Member Author

/label backport-risk-assessed

I realize that these changes carry certain risks, but I don't believe they're very high, and I'd also rather discover them in 4.22 than when preparing the next release.

@openshift-ci openshift-ci Bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Jul 1, 2026
@elfosardo

Copy link
Copy Markdown

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants