Skip to content

Commit eb47f56

Browse files
authored
feat(infra): Add base_os_version to support parallel builds (#14128)
## What this PR does This PR resolves a critical GLIBC mismatch error in the parallel `trial_build` CI pipeline by enabling dynamic selection of the `base-runner` image. It introduces a hybrid approach to ensure a project's runtime environment always matches its build environment. ## Why this PR is important Currently, fuzzers built on a newer OS (e.g., Ubuntu 24.04) during parallel CI runs fail during the `check_build` step because they are always tested against an older, hardcoded `base-runner:latest` image (Ubuntu 20.04). This change is crucial to unblock parallel builds and allow projects to use newer base images without breaking the CI. ## How the changes were implemented * **CI-Level Override:** The `trial_build` function now passes a `--base-image-tag` to `helper.py`. This tag corresponds to the OS version of the build job (e.g., `ubuntu-24-04`), ensuring the `check_build` step uses the correct runner image. * **Project-Level Configuration:** A new `base_os_version` field is introduced in `project.yaml`. This allows developers to specify a base OS for local runs of `run_fuzzer`, `reproduce`, etc., making local testing consistent with CI. * **Centralized Image Logic:** All commands in `helper.py` that use a runner image now call a refactored `_get_base_runner_image` function. This function implements the priority system: `--base-image-tag` > `base_os_version` > `legacy` default. * **New Consistency Check:** A new GitHub Actions workflow (`check_base_os.yml`) is added. It triggers on pull requests modifying `projects/` and verifies that the `base_os_version` in `project.yaml` matches the `FROM` tag in the project's `Dockerfile`, preventing configuration errors. * **Bug Fix:** The logic for selecting debug images (`reproduce --valgrind`) was also corrected as part of the refactoring. ## How to test 1. The primary validation is through the CI itself. A `trial_build` for a project using a newer `Dockerfile` (e.g., based on Ubuntu 24.04) should now pass the `check_build` step. 2. Locally, a developer can add `base_os_version: ubuntu-24-04` to a `project.yaml` (assuming a matching `Dockerfile`) and confirm that `python3 infra/helper.py run_fuzzer <project> <fuzzer>` uses the `gcr.io/oss-fuzz-base/base-runner:ubuntu-24-04` image. ## Is it a breaking change? No. The default behavior is unchanged. All existing projects without the new `base_os_version` field will continue to use the `legacy` (`:latest`) runner image, making the change fully backward-compatible. ## Related Task * **Taskflow:** [b/441792502](b/441792502)
1 parent f7bbff1 commit eb47f56

File tree

7 files changed

+263
-27
lines changed

7 files changed

+263
-27
lines changed
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
#
15+
################################################################################
16+
17+
name: 'Check Base OS Consistency'
18+
19+
on:
20+
pull_request:
21+
paths:
22+
- 'projects/**'
23+
24+
jobs:
25+
check-consistency:
26+
runs-on: ubuntu-latest
27+
steps:
28+
- name: Checkout repository
29+
uses: actions/checkout@v4
30+
with:
31+
fetch-depth: 0 # Fetch all history to compare with main
32+
33+
- name: Get changed project directories
34+
id: changed-projects
35+
run: |
36+
# Get the list of changed files compared to the target branch
37+
# and filter for unique directories under 'projects/'.
38+
CHANGED_DIRS=$(git diff --name-only ${{ github.base_ref }} ${{ github.head_ref }} | \
39+
grep '^projects/' | \
40+
xargs -n 1 dirname | \
41+
sort -u)
42+
echo "changed_dirs=${CHANGED_DIRS}" >> $GITHUB_OUTPUT
43+
44+
- name: Set up Python
45+
uses: actions/setup-python@v5
46+
with:
47+
python-version: '3.x'
48+
49+
- name: Install dependencies
50+
run: pip install PyYAML
51+
52+
- name: Check each modified project
53+
if: steps.changed-projects.outputs.changed_dirs != ''
54+
run: |
55+
EXIT_CODE=0
56+
for project_dir in ${{ steps.changed-projects.outputs.changed_dirs }};
57+
do
58+
echo "--- Checking ${project_dir} ---"
59+
python3 infra/ci/check_base_os.py "${project_dir}" || EXIT_CODE=$?
60+
done
61+
exit $EXIT_CODE

infra/build/functions/build_lib.py

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -685,12 +685,24 @@ def get_gcb_url(build_id, cloud_project='oss-fuzz'):
685685
f'{build_id}?project={cloud_project}')
686686

687687

688-
def get_runner_image_name(test_image_suffix):
689-
"""Returns the runner image that should be used. Returns the testing image if
690-
|test_image_suffix|."""
688+
def get_runner_image_name(test_image_suffix, base_image_tag=None):
689+
"""Returns the runner image that should be used.
690+
691+
Returns the testing image if |test_image_suffix|.
692+
"""
691693
image = f'gcr.io/{BASE_IMAGES_PROJECT}/base-runner'
694+
695+
# For trial builds, the version is embedded in the suffix.
692696
if test_image_suffix:
693697
image += '-' + test_image_suffix
698+
return image
699+
700+
# For local/manual runs, the version is passed as a tag.
701+
# Only add a tag if it's specified and not 'legacy', as 'legacy' implies
702+
# 'latest', which is the default behavior.
703+
if base_image_tag and base_image_tag != 'legacy':
704+
image += ':' + base_image_tag
705+
694706
return image
695707

696708

infra/build/functions/build_project.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@
7373
class Config:
7474
testing: bool = False
7575
test_image_suffix: Optional[str] = None
76+
base_image_tag: Optional[str] = None
7677
repo: Optional[str] = DEFAULT_OSS_FUZZ_REPO
7778
branch: Optional[str] = None
7879
parallel: bool = False
@@ -453,8 +454,10 @@ def get_build_steps_for_project(project,
453454
f'--architecture {build.architecture} {project.name}\\n' +
454455
'*' * 80)
455456
# Test fuzz targets.
457+
runner_image_name = build_lib.get_runner_image_name(
458+
config.test_image_suffix, config.base_image_tag)
456459
test_step = {
457-
'name': build_lib.get_runner_image_name(config.test_image_suffix),
460+
'name': runner_image_name,
458461
'env': env,
459462
'args': [
460463
'bash', '-c',

infra/build/functions/trial_build.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -321,6 +321,7 @@ def _do_test_builds(args, test_image_suffix, end_time, version_tag):
321321

322322
config = build_project.Config(testing=True,
323323
test_image_suffix=test_image_suffix,
324+
base_image_tag=version_tag,
324325
repo=args.repo,
325326
branch=args.branch,
326327
parallel=False,
@@ -432,7 +433,7 @@ def _do_build_type_builds(args, config, credentials, build_type, projects):
432433
credentials,
433434
build_type.type_name,
434435
extra_tags=tags,
435-
timeout=PROJECT_BUILD_TIMEOUT))
436+
timeout=PROJECT_BUILD_TIMEOUT))['id']
436437
time.sleep(1) # Avoid going over 75 requests per second limit.
437438
except Exception as error: # pylint: disable=broad-except
438439
# Handle flake.

infra/build/functions/trial_build_test.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ def test_build_steps_correct(self, mock_gcb_build_and_push_images,
7373
del mock_wait_on_builds
7474
self.maxDiff = None # pylint: disable=invalid-name
7575
build_id = 1
76-
mock_run_build.return_value = build_id
76+
mock_run_build.return_value = {'id': build_id}
7777
branch_name = 'mybranch'
7878
project = 'skcms'
7979
args = [

infra/ci/check_base_os.py

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
#
15+
################################################################################
16+
"""
17+
A CI script to ensure that the base OS version specified in a project's
18+
project.yaml file matches the FROM line in its Dockerfile.
19+
"""
20+
21+
import os
22+
import sys
23+
import yaml
24+
25+
# Defines the base OS versions that are currently supported for use in project.yaml.
26+
# For now, only 'legacy' is permitted. This list will be expanded as new
27+
# base images are rolled out.
28+
SUPPORTED_VERSIONS = [
29+
'legacy',
30+
# 'ubuntu-20-04',
31+
# 'ubuntu-24-04',
32+
]
33+
34+
# A map from the base_os_version in project.yaml to the expected Dockerfile
35+
# FROM tag.
36+
BASE_OS_TO_DOCKER_TAG = {
37+
'legacy': 'latest',
38+
'ubuntu-20-04': 'ubuntu-20-04',
39+
'ubuntu-24-04': 'ubuntu-24-04',
40+
}
41+
42+
43+
def main():
44+
"""Checks the Dockerfile FROM tag against the project's base_os_version."""
45+
if len(sys.argv) < 2:
46+
print(f'Usage: {sys.argv[0]} <project_path>', file=sys.stderr)
47+
return 1
48+
49+
project_path = sys.argv[1]
50+
project_yaml_path = os.path.join(project_path, 'project.yaml')
51+
dockerfile_path = os.path.join(project_path, 'Dockerfile')
52+
53+
# 1. Get the base_os_version from project.yaml, defaulting to 'legacy'.
54+
base_os_version = 'legacy'
55+
if os.path.exists(project_yaml_path):
56+
with open(project_yaml_path) as f:
57+
config = yaml.safe_load(f)
58+
if config and 'base_os_version' in config:
59+
base_os_version = config['base_os_version']
60+
61+
# 2. Validate that the version is currently supported.
62+
if base_os_version not in SUPPORTED_VERSIONS:
63+
print(
64+
f'Error: base_os_version "{base_os_version}" is not yet supported. '
65+
f'The currently supported versions are: "{", ".join(SUPPORTED_VERSIONS)}"',
66+
file=sys.stderr)
67+
return 1
68+
69+
# 3. Get the expected Dockerfile tag from our mapping.
70+
expected_tag = BASE_OS_TO_DOCKER_TAG[base_os_version]
71+
72+
# 4. Read the Dockerfile and find the tag in the FROM line.
73+
if not os.path.exists(dockerfile_path):
74+
print(f'Error: Dockerfile not found at {dockerfile_path}', file=sys.stderr)
75+
return 1
76+
77+
dockerfile_tag = ''
78+
with open(dockerfile_path) as f:
79+
for line in f:
80+
if line.strip().startswith('FROM'):
81+
try:
82+
if ':' not in line:
83+
print(
84+
f'Error: Malformed FROM line in Dockerfile (missing tag): {line.strip()}',
85+
file=sys.stderr)
86+
return 1
87+
dockerfile_tag = line.split(':')[1].strip()
88+
except IndexError:
89+
print(f'Error: Could not parse tag from Dockerfile FROM line: {line}',
90+
file=sys.stderr)
91+
return 1
92+
break
93+
94+
# 5. Compare and report.
95+
if dockerfile_tag != expected_tag:
96+
print(
97+
f'Error: Mismatch found in {project_path}.\n'
98+
f' - project.yaml (base_os_version): "{base_os_version}" (expects Dockerfile tag "{expected_tag}")\n'
99+
f' - Dockerfile FROM tag: "{dockerfile_tag}"\n'
100+
f'Please align the Dockerfile\'s FROM line to use the tag "{expected_tag}".',
101+
file=sys.stderr)
102+
return 1
103+
104+
print(
105+
f'Success: {project_path} is consistent (base_os_version: "{base_os_version}", Dockerfile tag: "{dockerfile_tag}").'
106+
)
107+
return 0
108+
109+
110+
if __name__ == '__main__':
111+
sys.exit(main())

0 commit comments

Comments
 (0)