Skip to content

feat: deny releases using experimental Hermeto backends#1736

Open
jsmid1 wants to merge 1 commit into
conforma:mainfrom
jsmid1:EC-1799
Open

feat: deny releases using experimental Hermeto backends#1736
jsmid1 wants to merge 1 commit into
conforma:mainfrom
jsmid1:EC-1799

Conversation

@jsmid1

@jsmid1 jsmid1 commented Jun 2, 2026

Copy link
Copy Markdown

Builds whose SBOM contains components fetched by an experimental Hermeto backend (hermeto:backend:experimental:*) now violate the release policy. Covers both SPDX and CycloneDX SBOM formats.

Issue: EC-1799

Co-Authored-By: Claude

@coderabbitai

coderabbitai Bot commented Jun 2, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@jsmid1, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 52 minutes. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 1df350ef-3246-4fa9-a04f-a7ddabd8d036

📥 Commits

Reviewing files that changed from the base of the PR and between ca05deb and 086c1d7.

📒 Files selected for processing (8)
  • antora/docs/modules/ROOT/pages/packages/release_sbom_cyclonedx.adoc
  • antora/docs/modules/ROOT/pages/packages/release_sbom_spdx.adoc
  • antora/docs/modules/ROOT/pages/release_policy.adoc
  • antora/docs/modules/ROOT/partials/release_policy_nav.adoc
  • policy/release/sbom_cyclonedx/sbom_cyclonedx.rego
  • policy/release/sbom_cyclonedx/sbom_cyclonedx_test.rego
  • policy/release/sbom_spdx/sbom_spdx.rego
  • policy/release/sbom_spdx/sbom_spdx_test.rego
📝 Walkthrough

Walkthrough

This PR adds a new experimental_hermeto_backend validation rule for both CycloneDX and SPDX SBOM formats. The rules detect when SBOM components or packages were fetched via experimental Hermeto backends by scanning for specific annotation markers. Both formats include comprehensive Rego policy rules, test coverage, and integrated documentation across three standard policy rulesets.

Changes

Experimental Hermeto backend detection

Layer / File(s) Summary
CycloneDX rule implementation and validation
policy/release/sbom_cyclonedx/sbom_cyclonedx.rego, policy/release/sbom_cyclonedx/sbom_cyclonedx_test.rego
Implements experimental_hermeto_backend deny rule by matching annotations with hermeto:backend:experimental: prefix, mapping to components via bom-ref, and emitting denials with component identifiers. Tests assert experimental backends are denied while stable hermeto:backend:gomod annotations pass.
SPDX rule implementation and validation
policy/release/sbom_spdx/sbom_spdx.rego, policy/release/sbom_spdx/sbom_spdx_test.rego
Implements experimental_hermeto_backend deny rule by scanning package annotations for Tool: hermeto:backend annotator with comments starting with hermeto:backend:experimental:, emitting denials keyed by package PURL. Tests verify experimental backends deny and stable backends pass.
Documentation and policy ruleset integration
antora/docs/modules/ROOT/pages/packages/release_sbom_cyclonedx.adoc, antora/docs/modules/ROOT/pages/packages/release_sbom_spdx.adoc, antora/docs/modules/ROOT/pages/release_policy.adoc, antora/docs/modules/ROOT/partials/release_policy_nav.adoc
Adds rule documentation sections describing failure conditions and messages for both CycloneDX and SPDX rules. Updates navigation entries and registers both rules in minimal, redhat, and redhat_rpms policy rulesets.

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: denying releases using experimental Hermeto backends, which aligns with the comprehensive policy updates across SPDX and CycloneDX SBOM formats.
Description check ✅ Passed The description clearly explains the policy change, specifying that builds with experimental Hermeto backend components are now denied, and covering both SBOM formats with a reference to the issue.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@policy/release/sbom_spdx/sbom_spdx.rego`:
- Around line 370-386: The rule deny contains result currently requires finding
an externalRef with referenceType "purl" (variable externalref and id :=
externalref.referenceLocator), which causes annotated packages without a purl to
skip this deny; modify the logic in the deny rule so that if no purl externalRef
exists you still produce a result by deriving id from a fallback identifier
(e.g., pkg.name or pkg.SPDXID) instead of requiring externalref; update the
variables used by metadata.result_helper_with_term (the id passed to
metadata.result_helper_with_term and any uses of pkg.externalRefs) to use the
purl when present and the fallback when absent so annotated SPDX packages are
denied consistently.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: b580bfad-756c-4f64-9a1b-317e1dbf7c92

📥 Commits

Reviewing files that changed from the base of the PR and between 9da3421 and ca05deb.

📒 Files selected for processing (8)
  • antora/docs/modules/ROOT/pages/packages/release_sbom_cyclonedx.adoc
  • antora/docs/modules/ROOT/pages/packages/release_sbom_spdx.adoc
  • antora/docs/modules/ROOT/pages/release_policy.adoc
  • antora/docs/modules/ROOT/partials/release_policy_nav.adoc
  • policy/release/sbom_cyclonedx/sbom_cyclonedx.rego
  • policy/release/sbom_cyclonedx/sbom_cyclonedx_test.rego
  • policy/release/sbom_spdx/sbom_spdx.rego
  • policy/release/sbom_spdx/sbom_spdx_test.rego

Comment thread policy/release/sbom_spdx/sbom_spdx.rego
@codecov

codecov Bot commented Jun 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag Coverage Δ
unit-tests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
policy/release/sbom_cyclonedx/sbom_cyclonedx.rego 100.00% <100.00%> (ø)
...cy/release/sbom_cyclonedx/sbom_cyclonedx_test.rego 100.00% <100.00%> (ø)
policy/release/sbom_spdx/sbom_spdx.rego 100.00% <100.00%> (ø)
policy/release/sbom_spdx/sbom_spdx_test.rego 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Builds whose SBOM contains components fetched by an experimental
Hermeto backend (hermeto:backend:experimental:*) now violate
the release policy. Covers both SPDX and CycloneDX SBOM formats.

Ref: EC-1799

Signed-off-by: Jan Smid <jsmid@redhat.com>
Co-Authored-By: Claude
# - redhat_rpms
deny contains result if {
some s in sbom.cyclonedx_sboms
some annotation in s.annotations

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This indicates that the annotation key is at the top level of the sbom data. Is that right?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I look at the requirements in the Jira and it seems like that's right.

@simonbaird simonbaird left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's good. Can you think of a way to test it on a real image with a real sbom that has an experimental Hermeto? The reporter might be able to share an image like that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants