Write an SPDX 2.2 package manifest into the image and each sysext

[Vince Perri (vinceaperri)]

Summary

Every ACL image and sysext now carries an SPDX 2.2 document listing the installed RPM packages:

• /usr/share/os-manifests/package-manifest.spdx.json for the image
• /usr/share/os-manifests/package-manifest.<sysext>.spdx.json for each sysext

systemd-sysext merges the sysext copies alongside the image's, so a consumer reading that directory on a booted machine sees the image's package list plus one list per merged sysext.

Each sysext list carries only what the sysext added. A sysext whose package diff against the base is empty gets no manifest.

Consumer

Vulnerability scanning. A node-side scanner needs a package inventory it can report, so that CVE matching can happen off-node against it. On a conventional RPM distro a scanner collects that inventory by querying the package database, typically by shelling out to rpm -qa. ACL ships no rpm binary, so that route is not available here.

These documents are that inventory, in a form a scanner can read off the filesystem with no package manager present. The intended consumer is a collection rule that reads referenceLocator (a package URL) and supplier out of /usr/share/os-manifests/package-manifest*.spdx.json, which between them carry the name, version, release, epoch, architecture and supplier of every package. Bottlerocket is covered the same way today, by a rule that parses its application-inventory.json.

Reading the package URL is deliberate. PURL's rpm type defines arch and epoch as named qualifiers, so whoever writes the collection rule has a specification to work from.

That collection rule is owned by the scanning service and is not in place yet.

Type of Change

• Image build change (base image, sysexts, OEM images)
• Package/SPEC update
• CI/automation change
• SDK/toolchain update
• Configuration change
• Documentation update
• Bug fix

Does this affect the image build?

• Yes
• No

Merge order

1. Merge azure-container-linux-mantle#35.
2. Wait for a mantle LKG container carrying it to be published to ACR.
3. Merge this PR. It is what adds generate_package_manifest.py to acl-scripts.
4. Merge acl-pipelines !28911 last.

Test Methodology

• validate_golden_manifest.sh passes: SPDX 2.2 clean, NTIA conformant on all seven requirements.
• Full build for acldevel 1192360.

Merge Checklist

• Image builds successfully with this change (or image build is not affected)
• Any updated packages/SPECs build successfully
• Relevant kola tests pass
• All package sources are available
• Source files have up-to-date hashes/manifests
• Documentation has been updated to match any changes
• Ready to merge

[Copilot]

Pull request overview

This PR adds deterministic SPDX 2.2 “package manifest” documents to Azure Container Linux images and sysexts (RPM mode), so consumers can read installed RPM components from /usr/share/os-manifests/ without double-counting between base and sysext layers.

Changes:

• Introduces a stdlib-only Python generator to convert RPM NEVRA lists into SPDX 2.2 JSON manifests.
• Writes package manifest documents into the image rootfs and sysext rootfs during builds (RPM mode), using a reproducible timestamp source.
• Adds a conformance test (SPDX 2.2 + NTIA minimum elements) and enforces a kola test for the new manifest.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
build_sysext	Makes SOURCE_DATE_EPOCH handling stricter under set -e and writes sysext package manifests when RPM packages are present.
build_library/rpm/tests/testdata/packages.txt	Adds a NEVRA fixture used for SPDX/NTIA conformance validation.
build_library/rpm/tests/requirements.txt	Pins Python validator dependencies used by the conformance test.
build_library/rpm/tests/check_generate_package_manifest_conformance.sh	Adds an automated conformance test for manifest determinism and validation.
build_library/rpm/generate_package_manifest.py	Implements SPDX 2.2 JSON package manifest generation from NEVRA input.
build_library/reports_util.sh	Adds write_package_manifest helper used by image/sysext build flows.
build_library/prod_image_util.sh	Writes the image package manifest in RPM mode with reproducible created timestamp.
acl/tests/kola_enforcing.yaml	Enforces the packages.package-manifest kola test.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

build_sysext:358

• The conditional skip for an empty ${SYSEXTNAME}_packages.txt is based on an incorrect premise: the generator already produces a valid SPDX document even when the package list is empty (it still emits the root package + DESCRIBES relationship). Skipping here means some sysexts won’t carry a manifest, which contradicts the PR description and complicates consumers that expect one per sysext.

  # A sysext whose packages are all already in the base image diffs to nothing.
  # That is a valid sysext, and SPDX has no way to describe an empty package set.
  if [[ -s "${BUILD_DIR}/${SYSEXTNAME}_packages.txt" ]]; then

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

build_library/reports_util.sh:155

• generate_package_manifest.py refuses to overwrite an existing manifest unless --force is provided. Since build directories/rootfs trees can be reused across runs, this can cause non-obvious rebuild failures when the output file already exists. Consider always passing --force here (the generator is intended to be deterministic).

        --manifest-version="${manifest_version}" \
        --created-epoch="${created_epoch}"

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

build_library/rpm/generate_package_manifest.py:354

• The argparse description/help says the input is an "rpm -qa NEVRA list", but this script expects one NEVRA per line (including epoch when present) and will error on gpg-pubkey-* entries that rpm -qa commonly emits. To avoid misleading users, consider updating the description/help to point at the filtered NEVRA output (e.g. produced with a queryformat / this repo’s rpm_query_packages).

def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Generate an SPDX 2.2 package manifest from an rpm -qa NEVRA list."
    )

    parser.add_argument(
        "--packages-file",
        required=True,
        type=validate_input_file,
        help="The path to the package list to convert, one NEVRA per line. (required)",
    )

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/workflows/pr-workflows.yaml:41

• package_manifest sets permissions: {} but still uses actions/checkout@v4. With an empty GITHUB_TOKEN permission set, checkout may fail to read the repository. Consider granting the minimal required permission (e.g. contents: read) or removing the job-level override so it inherits the workflow defaults.

    # Deliberately outside the SDK container since the validators come from pip.
    # The job still gates on pre_check: it runs PR-authored shell and installs
    # from PyPI, so it belongs behind the same approval as every other job here.
    permissions: {}
    steps:
      - uses: actions/checkout@v4

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (2)

build_library/reports_util.sh:150

• write_package_manifest invokes generate_package_manifest.py directly via sudo. This relies on the script being executable (+x) and having a working shebang; elsewhere in build_library/rpm the repo runs Python helpers via python3 (e.g., package_catalog.sh). To make builds more robust and consistent, invoke it as sudo python3 "$BUILD_LIBRARY_DIR/rpm/generate_package_manifest.py" ... instead of executing the file directly.

    # build_image runs as the sdk user, so writing into an image rootfs needs sudo.
    sudo install -d -m 0755 "${output%/*}"
    sudo "${BUILD_LIBRARY_DIR}/rpm/generate_package_manifest.py" \
        --packages-file="${packages_file}" \
        --manifest-file="${output}" \
        --manifest-name="${manifest_name}" \
        --manifest-version="${manifest_version}" \
        --created-epoch="${created_epoch}"

build_library/rpm/tests/check_generate_package_manifest_conformance.sh:52

• The conformance test executes generate_package_manifest.py as a program ("${GENERATOR}"). This will fail if the file mode isn't executable in the checkout. Since the script is pure Python and the test already depends on python3, invoke it as python3 "${GENERATOR}" ... to avoid relying on executable bits.

echo "=== Generating manifest from ${PACKAGES_FILE##*/} ==="
"${GENERATOR}" \
    --packages-file="${PACKAGES_FILE}" \
    --manifest-file="${MANIFEST}" \
    --manifest-name="${MANIFEST_NAME}" \
    --manifest-version="${MANIFEST_VERSION}" \
    --created-epoch="${CREATED_EPOCH}" \
    --force

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (3)

build_library/build_image_util.sh:838

• Typo/grammar in the new comment: "image'S" should be "image's", and "pacakges" should be "packages". This is in a user-facing code comment that may get copied around, so it’s worth fixing for clarity.

      # Writes the image'S package list and SPDX package manifest from the final rpmdb.
      #
      # Must be called after finish_image_backup_etc_rpm uninstalls the azurelinux-repos* pacakges above,
      # and before the rootfs state loop deletes /var (and the rpmdb) below.

build_library/rpm/tests/check_generate_package_manifest_conformance.sh:29

• The comment describing how to regenerate requirements.txt says to run pip-compile with --no-header, but requirements.txt in this directory includes the autogenerated header. Either remove --no-header from the instructions or regenerate requirements.txt with --no-header so the docs match the actual file.

# Updating this file by editing ./requirements.in and running:
#   pip-compile --generate-hashes --no-header --strip-extras --output-file=requirements.txt requirements.in
REQUIREMENTS="${TESTS_DIR}/requirements.txt"

.github/workflows/pr-workflows.yaml:43

• The new package_manifest workflow job relies on creating a venv with whatever python3 happens to be on ubuntu-latest, but the pinned, hashed requirements.txt was generated with Python 3.12 (per its header). To keep CI deterministic (and avoid --require-hashes failures when ubuntu-latest bumps Python), add an actions/setup-python step to pin Python (e.g., 3.12) before running the conformance script.

  package_manifest:
    name: "Check the SPDX package manifest generator"
    # Behind pre_check's approval gate despite not needing the SDK.
    needs: [ pre_check ]
    runs-on: ubuntu-latest
    # Drop the unnecessary workflow-level write token.
    permissions: {}
    steps:
      - uses: actions/checkout@v4
      - name: Check SPDX 2.2 and NTIA conformance
        shell: bash
        run: build_library/rpm/tests/check_generate_package_manifest_conformance.sh

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.