ACL: add fasttrack support - #26
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds “FastTrack preview” repository support to the ACL image build flow by propagating a FASTTRACK_REPO_FILE environment variable into the SDK container and using it during RPM installation to prefer CVE-patched RPMs, with additional logging to help verify behavior.
Changes:
- Forward
FASTTRACK_REPO_FILEthroughacl/build_rpm_image.sh→run_sdk_container→sdk_lib/sdk_entry.sh. - In RPM-mode installs, copy the fasttrack
.repofile into the installroot and log fasttrack-available / fasttrack-installed package information. - Add repoquery-based diagnostics around fasttrack availability and installed versions.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| sdk_lib/sdk_entry.sh | Persists FASTTRACK_REPO_FILE into the SDK user environment via .bashrc. |
| run_sdk_container | Passes FASTTRACK_REPO_FILE into the SDK container environment for builds. |
| build_library/rpm/rpm_install.sh | Installs fasttrack repo config into installroot and adds fasttrack-related repoquery/verification logging. |
| acl/build_rpm_image.sh | Exports FASTTRACK_REPO_FILE so it is available to the SDK container build pipeline. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
f119d70 to
10449ac
Compare
10449ac to
2a5f10d
Compare
2a5f10d to
4e2aa2d
Compare
4e2aa2d to
eeb0851
Compare
eeb0851 to
f1292b7
Compare
f1292b7 to
9ad5069
Compare
9ad5069 to
95c8599
Compare
95c8599 to
0d693b1
Compare
| # Compare versions first, then releases | ||
| if [[ "${pmc_ver}" != "${base_ver}" ]]; then | ||
| if is_older "${base_ver}" "${pmc_ver}"; then | ||
| warn " ${pkg}: STALE - based on ${base_ver}-${base_rel}, upstream is at ${pmc}" | ||
| STALE=$((STALE + 1)) | ||
| else | ||
| info " ${pkg}: OK (ACL version newer than upstream)" | ||
| fi | ||
| elif [[ "${pmc_rel}" -gt "${base_rel}" ]] 2>/dev/null; then | ||
| warn " ${pkg}: STALE - based on ${base_ver}-${base_rel}, upstream is at ${pmc}" | ||
| STALE=$((STALE + 1)) | ||
| else | ||
| info " ${pkg}: OK" | ||
| fi |
| if [[ -x "${BUILD_LIBRARY_DIR}/rpm/check_pmc_staleness.sh" ]]; then | ||
| if ! "${BUILD_LIBRARY_DIR}/rpm/check_pmc_staleness.sh" "${root_fs_dir}"; then | ||
| error "PMC staleness check failed - rebase stale ACL SPECs before building." | ||
| return 1 | ||
| fi | ||
| fi |
| -e IMAGE_VERSION_ID="${IMAGE_VERSION_ID:-}" | ||
| -e IMAGE_BUILD_ID="${IMAGE_BUILD_ID:-}" | ||
| -e EXTRA_KERNEL_CMDLINE="${EXTRA_KERNEL_CMDLINE:-}" | ||
| -e FASTTRACK_REPO_FILE="${FASTTRACK_REPO_FILE:+/mnt/host/source/src/scripts/__build__/fasttrack-repo/fasttrack-preview.repo}" |
| local ft_name ft_evr | ||
| while IFS=' ' read -r ft_name ft_evr; do | ||
| [[ -z "${ft_name}" || -z "${ft_evr}" ]] && continue | ||
| local installed_ver | ||
| installed_ver=$(sudo rpm --dbpath="${root_fs_dir}/var/lib/rpm" -q "${ft_name}" 2>/dev/null || true) |
| dnf_output="$(timeout 120 /usr/bin/dnf5 repoquery \ | ||
| --repofrompath=fasttrack-check,${FASTTRACK_URL} \ | ||
| --repo=fasttrack-check \ | ||
| --setopt=fasttrack-check.gpgcheck=0 \ | ||
| --available --latest-limit=1 \ | ||
| --queryformat="%{name} %{version}-%{release}\n" \ | ||
| ${pkg_list} 2>/dev/null)" || { warn "Failed to query fasttrack repo - skipping staleness check"; exit 0; } | ||
| else | ||
| # Prod/dev build: check against PMC directly | ||
| PMC_BASE="https://packages.microsoft.com/azurelinux/3.0/prod/base/${ARCH}" | ||
| PMC_EXTENDED="https://packages.microsoft.com/azurelinux/3.0/prod/extended/${ARCH}" | ||
| info "Checking against PMC (no fasttrack repo configured)" | ||
| info " base: ${PMC_BASE}" | ||
| info " extended: ${PMC_EXTENDED}" | ||
| info "Querying for ${#BASE_VER[@]} packages..." | ||
|
|
||
| dnf_output="$(timeout 120 /usr/bin/dnf5 repoquery \ | ||
| --repofrompath=pmc-base,${PMC_BASE} \ | ||
| --repofrompath=pmc-extended,${PMC_EXTENDED} \ | ||
| --repo=pmc-base --repo=pmc-extended \ | ||
| --setopt=pmc-base.gpgcheck=0 \ | ||
| --setopt=pmc-extended.gpgcheck=0 \ | ||
| --available --latest-limit=1 \ | ||
| --queryformat="%{name} %{version}-%{release}\n" \ | ||
| ${pkg_list} 2>/dev/null)" || { warn "Failed to query PMC repos - skipping staleness check"; exit 0; } |
0d693b1 to
34e733d
Compare
| -e IMAGE_VERSION_ID="${IMAGE_VERSION_ID:-}" | ||
| -e IMAGE_BUILD_ID="${IMAGE_BUILD_ID:-}" | ||
| -e EXTRA_KERNEL_CMDLINE="${EXTRA_KERNEL_CMDLINE:-}" | ||
| -e FASTTRACK_REPO_FILE="${FASTTRACK_REPO_FILE:+/mnt/host/source/src/scripts/__build__/fasttrack-repo/fasttrack-preview.repo}" |
| call_docker exec "${name}" sh -c 'cp /etc/hosts /etc/hosts2; umount /etc/hosts ; mv /etc/hosts2 /etc/hosts' | ||
|
|
||
| call_docker exec "${tty[@]}" -i -e INJECT_DOCKER_SYSEXT="${INJECT_DOCKER_SYSEXT:-false}" "${name}" /mnt/host/source/src/scripts/sdk_lib/sdk_entry.sh "$@" | ||
| call_docker exec "${tty[@]}" -i -e INJECT_DOCKER_SYSEXT="${INJECT_DOCKER_SYSEXT:-false}" -e FASTTRACK_REPO_FILE="${FASTTRACK_REPO_FILE:+/mnt/host/source/src/scripts/__build__/fasttrack-repo/fasttrack-preview.repo}" "${name}" /mnt/host/source/src/scripts/sdk_lib/sdk_entry.sh "$@" |
| # Check ACL SPECs for PMC staleness - fails the build if any package is stale. | ||
| # The check compares pmc_base_version/pmc_base_release metadata in each | ||
| # ACL SPEC against the current PMC repo to detect packages that need rebase. | ||
| # Packages in the exception list are allowed to be stale (e.g. waiting for | ||
| # ACL changes to land upstream). |
| # Usage: check_pmc_staleness.sh <installroot> | ||
| # |
| dnf_output="$(timeout 120 /usr/bin/dnf5 repoquery \ | ||
| --repofrompath=fasttrack-check,${FASTTRACK_URL} \ | ||
| --repofrompath=pmc-base,${PMC_BASE} \ | ||
| --repofrompath=pmc-extended,${PMC_EXTENDED} \ | ||
| --repo=fasttrack-check --repo=pmc-base --repo=pmc-extended \ | ||
| --setopt=fasttrack-check.gpgcheck=0 \ | ||
| --setopt=pmc-base.gpgcheck=0 \ | ||
| --setopt=pmc-extended.gpgcheck=0 \ | ||
| --available --latest-limit=1 \ | ||
| --queryformat="%{name} %{version}-%{release}\n" \ | ||
| ${pkg_list} 2>/dev/null)" || { warn "Failed to query repos - skipping staleness check"; exit 0; } |
| dnf_output="$(timeout 120 /usr/bin/dnf5 repoquery \ | ||
| --repofrompath=pmc-base,${PMC_BASE} \ | ||
| --repofrompath=pmc-extended,${PMC_EXTENDED} \ | ||
| --repo=pmc-base --repo=pmc-extended \ | ||
| --setopt=pmc-base.gpgcheck=0 \ | ||
| --setopt=pmc-extended.gpgcheck=0 \ | ||
| --available --latest-limit=1 \ | ||
| --queryformat="%{name} %{version}-%{release}\n" \ | ||
| ${pkg_list} 2>/dev/null)" || { warn "Failed to query PMC repos - skipping staleness check"; exit 0; } |
34e733d to
9e08940
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.
Suppressed comments (7)
run_sdk_container:163
- FASTTRACK_REPO_FILE is treated as a boolean and replaced with a hard-coded in-container path via
${FASTTRACK_REPO_FILE:+...}. This prevents callers from specifying an explicit repo file path (the value they set is discarded), which makes the environment variable semantics surprising and can break non-CI usage.
-e IMAGE_VERSION_ID="${IMAGE_VERSION_ID:-}"
-e IMAGE_BUILD_ID="${IMAGE_BUILD_ID:-}"
-e EXTRA_KERNEL_CMDLINE="${EXTRA_KERNEL_CMDLINE:-}"
-e FASTTRACK_REPO_FILE="${FASTTRACK_REPO_FILE:+/mnt/host/source/src/scripts/__build__/fasttrack-repo/fasttrack-preview.repo}"
run_sdk_container:188
- This exec invocation also replaces the caller-provided FASTTRACK_REPO_FILE value with a hard-coded path. If the intent is to support a default pipeline-generated repo file, consider setting FASTTRACK_REPO_FILE to that default earlier (when empty) rather than overwriting non-empty values here.
call_docker exec "${tty[@]}" -i -e INJECT_DOCKER_SYSEXT="${INJECT_DOCKER_SYSEXT:-false}" -e FASTTRACK_REPO_FILE="${FASTTRACK_REPO_FILE:+/mnt/host/source/src/scripts/__build__/fasttrack-repo/fasttrack-preview.repo}" "${name}" /mnt/host/source/src/scripts/sdk_lib/sdk_entry.sh "$@"
build_library/rpm/rpm_install.sh:394
dnf5 repoquery --repo=fasttrack-preview --availablecan return (and sort) a very large package list, and this block runs on everyrpm_install_packageinvocation in fasttrack builds. This can significantly slow image builds and flood logs. Consider restricting the query to the packages being installed in this call.
if [[ -n "${FASTTRACK_REPO_FILE:-}" ]] && [[ -f "${root_fs_dir}/etc/yum.repos.d/$(basename "${FASTTRACK_REPO_FILE}")" ]]; then
info "=== FastTrack preview repo packages available ==="
sudo /usr/bin/dnf5 repoquery --installroot="${root_fs_dir}" --releasever=3.0 \
${forcearch_arg} --repo=fasttrack-preview --available 2>/dev/null | sort || true
info "=== End of fasttrack packages ==="
build_library/rpm/rpm_install.sh:438
- This verification loop iterates over all packages available in fasttrack-preview and runs
rpm -qfor each, which can be extremely expensive and noisy. A lower-cost approach is to scope the repoquery to the packages being installed in this call (or to the subset detected in the transaction log).
done < <(sudo /usr/bin/dnf5 repoquery --installroot="${root_fs_dir}" --releasever=3.0 \
${forcearch_arg} --repo=fasttrack-preview --available --queryformat="%{name} %{evr}\n" 2>/dev/null || true)
build_library/rpm/check_pmc_staleness.sh:12
- The usage header says the script only takes
<installroot>, but it also supports a second argument for the exception list. Keeping the usage text accurate helps avoid confusion when it fails withUsage: ....
# package and fails the build when PMC has moved ahead.
#
# Usage: check_pmc_staleness.sh <installroot>
#
build_library/rpm/build_image_util.sh:105
- This PR is described as adding FastTrack repo support, but it also introduces a new build-blocking PMC staleness gate (including network repo queries) in the core RPM image build path. That’s a significant behavioral/CI policy change and should either be called out explicitly in the PR description (and rationale) or split into a separate PR so reviewers can evaluate it independently.
# Check ACL SPECs for PMC staleness - fails the build if any package is stale.
# The check compares pmc_base_version/pmc_base_release metadata in each
# ACL SPEC against the current PMC repo to detect packages that need rebase.
# Packages in the exception list are allowed to be stale (e.g. waiting for
# ACL changes to land upstream).
local staleness_exceptions="selinux-policy"
if [[ -x "${BUILD_LIBRARY_DIR}/rpm/check_pmc_staleness.sh" ]]; then
if ! "${BUILD_LIBRARY_DIR}/rpm/check_pmc_staleness.sh" "${root_fs_dir}" "${staleness_exceptions}"; then
build_library/rpm/check_pmc_staleness.sh:103
- FASTTRACK_URL extraction only matches lines that start exactly with
baseurl=. Many .repo files use whitespace (e.g.baseurl = ...) or may indent entries, which would make FASTTRACK_URL empty and cause the staleness check to silently skip (due to the repoquery failure handler).
if [[ -n "${FASTTRACK_REPO_FILE:-}" ]] && [[ -f "${FASTTRACK_REPO_FILE}" ]]; then
# Fasttrack build: query fasttrack + PMC (fasttrack has latest, PMC covers the rest)
FASTTRACK_URL="$(grep -m1 '^baseurl=' "${FASTTRACK_REPO_FILE}" | cut -d= -f2-)"
info "Checking against fasttrack repo: ${FASTTRACK_URL}"
Signed-off-by: Mayank Singh <mayansingh@microsoft.com>
Signed-off-by: Mayank Singh <mayansingh@microsoft.com>
…path Signed-off-by: Mayank Singh <mayansingh@microsoft.com>
9e08940 to
3d099c0
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
Suppressed comments (1)
build_library/rpm/check_pmc_staleness.sh:25
- INSTALLROOT is required by the script interface but is never used/validated, which makes the usage misleading and can hide caller errors (e.g., passing a non-existent rootfs dir). At minimum, validate the directory exists before proceeding.
INSTALLROOT="${1:?Usage: check_pmc_staleness.sh <installroot>}"
| if [[ -n "${FASTTRACK_REPO_FILE:-}" ]] && [[ -f "${FASTTRACK_REPO_FILE}" ]]; then | ||
| # Fasttrack build: query fasttrack + PMC (fasttrack has latest, PMC covers the rest) | ||
| FASTTRACK_URL="$(grep -m1 '^baseurl=' "${FASTTRACK_REPO_FILE}" | cut -d= -f2-)" | ||
| info "Checking against fasttrack repo: ${FASTTRACK_URL}" | ||
| info " + PMC base: ${PMC_BASE}" | ||
| info " + PMC extended: ${PMC_EXTENDED}" | ||
| info "Querying for ${#BASE_VER[@]} packages..." |
| info "=== Packages installed from fasttrack-preview repo ===" | ||
| grep -i "fasttrack-preview" /tmp/rpm-install.log || info " (none or repo name not shown in transaction log)" | ||
| info "=== Installed versions of fasttrack packages ===" |
Summary
This PR adds new feature support for Fasttrack repository integration to enable faster CVE resolution through priority installation of updated RPMs from the fasttrack-preview repository.
Change Log
Implements Fasttrack preview repository support by enabling the build system to accept and utilize a fasttrack .repo file for preferential installation of CVE-patched RPMs during image builds. The changes propagate the FASTTRACK_REPO_FILE environment variable through the SDK container workflow and add logging to track which packages are installed from the fasttrack repository.
Type of Change
Does this affect the image build?
Associated Issues
https://dev.azure.com/mariner-org/ACL/_build/results?buildId=1156891&view=results
https://dev.azure.com/mariner-org/ACL/_build/results?buildId=1156313&view=results
Merge Checklist
All applicable boxes should be checked before merging