ci: workflows: pin python dependencies #87609

nashif · 2025-03-25T08:31:13Z

Pin python dependencies to hashes and cleanup/unify python setup steps in
various workflows.

We now have one dependency file containing all requirements for github
actions that is managed centrally with hashes. No direct pip installs
are needed in workflow files and everything shall go via the
requirements file.

Pinning to specific version and hashes helps with preventing supply
chain attacks.

Signed-off-by: Anas Nashif [email protected]

kernel/init.c

.github/workflows/backport_issue_check.yml

scripts/requirements-actions.in

nashif · 2025-03-25T09:53:09Z

hmm, not sure about this one:

 Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.12.9/x64/bin/gitlint", line 5, in <module>
    from gitlint.cli import cli
  File "/opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/site-packages/gitlint/cli.py", line 3, in <module>
    from gitlint.config import LintConfig, LintConfigError
  File "/opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/site-packages/gitlint/config.py", line 2, in <module>
    import ConfigParser
ModuleNotFoundError: No module named 'ConfigParser'
Compliance error, check for error messages in the "Run Compliance Tests" st

I have not seen this during testing :(

pdgendt · 2025-03-25T11:05:24Z

hmm, not sure about this one:

 Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.12.9/x64/bin/gitlint", line 5, in <module>
    from gitlint.cli import cli
  File "/opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/site-packages/gitlint/cli.py", line 3, in <module>
    from gitlint.config import LintConfig, LintConfigError
  File "/opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/site-packages/gitlint/config.py", line 2, in <module>
    import ConfigParser
ModuleNotFoundError: No module named 'ConfigParser'
Compliance error, check for error messages in the "Run Compliance Tests" st

I have not seen this during testing :(

Can we update the gitlint package? Also for the requirements.in file, we could specify gitlint-core instead.

scripts/requirements-actions.txt

cfriedt · 2025-03-27T23:47:47Z

Should help with caching / false positives as well.

pdgendt · 2025-03-28T07:57:52Z

Why was the "Checkout source code" step skipped in the failed "Publish Unit Tests Results"? 🤔

nashif · 2025-03-28T16:27:52Z

Why was the "Checkout source code" step skipped in the failed "Publish Unit Tests Results"? 🤔

this is fixed now

pdgendt · 2025-03-28T17:53:14Z

The commit to temporarily disable the failing test also touches the .github/workflows/twister.yaml file to checkout the pull request head. Is this intentional?

nashif · 2025-03-28T18:49:29Z

The commit to temporarily disable the failing test also touches the .github/workflows/twister.yaml file to checkout the pull request head. Is this intentional?

no, will fix, thanks.

nashif · 2025-03-28T19:06:31Z

The commit to temporarily disable the failing test also touches the .github/workflows/twister.yaml file to checkout the pull request head. Is this intentional?

no, will fix, thanks.

ok, fixed

kartben · 2025-03-28T20:34:04Z

.github/workflows/bsim-tests.yaml

@@ -178,7 +178,6 @@ jobs:

      - name: Merge Test Results
        run: |
-          pip install junitparser junit2html


Looks like you just dropped them as I can't see any step where you install the requirements?

those are already in the docker image, no need to install them

kartben · 2025-03-28T20:36:25Z

.github/workflows/clang.yaml

+          cache: pip
+          cache-dependency-path: scripts/requirements-actions.txt
+
+      - name: install-packages


If you have a chance to do a s/install-packages/Install Python packages/g so that steps have consistent naming that would be great (non blocking obviously)

kartben · 2025-03-28T20:41:03Z

.github/workflows/scripts_tests.yml

-      - name: Rebase
-        continue-on-error: true
-        env:
-          BASE_REF: ${{ github.base_ref }}
-          PR_HEAD: ${{ github.event.pull_request.head.sha }}
-        run: |
-          git config --global user.email "[email protected]"
-          git config --global user.name "Github Actions"
-          rm -fr ".git/rebase-apply"
-          rm -fr ".git/rebase-merge"
-          git rebase origin/${BASE_REF}
-          git clean -f -d
-          git log --graph --oneline HEAD...${PR_HEAD}
-


Unrelated change?

Pin python dependencies to hashes and cleanup/unify python setup steps in various workflows. We now have one dependency file containing all requirements for github actions that is managed centrally with hashes. No direct pip installs are needed in workflow files and everything shall go via the requirements file. Pinning to specific version and hashes helps with preventing supply chain attacks. Signed-off-by: Anas Nashif <[email protected]>

Pin dependencies on the workflow and move it from using docker to the zephyr setup action. Signed-off-by: Anas Nashif <[email protected]>

This test fails on older python versions (3.10) and only on CI. Disabling it while we investigate. The test itself verifies inline logs options, so the functionality test is not impacted. Tracked in zephyrproject-rtos#87769 Signed-off-by: Anas Nashif <[email protected]>

teburd · 2025-03-29T16:12:58Z

Lets do it with the entire environment using nix :-D Hash all the things!

pdgendt · 2025-03-31T06:57:29Z

Missed it last time I reviewed, but commit messages shouldn't contain Github #number references 🫣

zephyrbot added platform: nRF BSIM Nordic Semiconductors, nRF BabbleSim area: Kernel area: Continuous Integration labels Mar 25, 2025

zephyrbot requested review from aescolar, andyross, ceolin, cfriedt, dcpleung, fabiobaltieri, kartben, npitre, peter-mitsis, stephanosio and TaiJuWu March 25, 2025 08:31

zephyrbot assigned aescolar Mar 25, 2025

aescolar reviewed Mar 25, 2025

View reviewed changes

kernel/init.c Outdated Show resolved Hide resolved

nashif force-pushed the topic/ci/pin_deps branch 2 times, most recently from e13a855 to 68c2a9b Compare March 25, 2025 08:41

aescolar reviewed Mar 25, 2025

View reviewed changes

.github/workflows/backport_issue_check.yml Outdated Show resolved Hide resolved

nashif force-pushed the topic/ci/pin_deps branch 2 times, most recently from 18b79c3 to 374367f Compare March 25, 2025 08:59

pdgendt reviewed Mar 25, 2025

View reviewed changes

scripts/requirements-actions.in Outdated Show resolved Hide resolved

nashif removed the area: Kernel label Mar 25, 2025

nashif removed request for andyross and peter-mitsis March 25, 2025 09:53

pdgendt reviewed Mar 25, 2025

View reviewed changes

scripts/requirements-actions.txt Outdated Show resolved Hide resolved

nashif force-pushed the topic/ci/pin_deps branch 2 times, most recently from f7b1d2e to 4ebcffb Compare March 26, 2025 02:24

github-actions bot requested review from gchwier, golowanow, hakehuang, LukaszMrugala and PerMac March 27, 2025 15:55

nashif force-pushed the topic/ci/pin_deps branch from ba2ad47 to 30216ea Compare March 27, 2025 21:01

nashif force-pushed the topic/ci/pin_deps branch from 30216ea to 81f8987 Compare March 28, 2025 10:19

nashif force-pushed the topic/ci/pin_deps branch from 81f8987 to fb12ef5 Compare March 28, 2025 19:05

nashif assigned nashif and stephanosio and unassigned aescolar Mar 28, 2025

kartben reviewed Mar 28, 2025

View reviewed changes

nashif added 3 commits March 28, 2025 18:25

ci: overhaul and pin dependencies on twister_tests_blackbox

3b3a374

Pin dependencies on the workflow and move it from using docker to the zephyr setup action. Signed-off-by: Anas Nashif <[email protected]>

nashif force-pushed the topic/ci/pin_deps branch from 9b43181 to fc7e570 Compare March 28, 2025 22:26

kartben approved these changes Mar 28, 2025

View reviewed changes

pdgendt approved these changes Mar 29, 2025

View reviewed changes

cfriedt approved these changes Mar 29, 2025

View reviewed changes

nashif merged commit 19c6240 into zephyrproject-rtos:main Mar 29, 2025
72 of 73 checks passed

gbarkadiusz mentioned this pull request Mar 31, 2025

Coverage: Native_sim does not produce the coverage data. #87899

Closed

LukaszMrugala mentioned this pull request Apr 3, 2025

CI Blackbox tests mistakenly disabled #88087

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: workflows: pin python dependencies #87609

ci: workflows: pin python dependencies #87609

nashif commented Mar 25, 2025

nashif commented Mar 25, 2025

pdgendt commented Mar 25, 2025

cfriedt commented Mar 27, 2025

pdgendt commented Mar 28, 2025

nashif commented Mar 28, 2025

pdgendt commented Mar 28, 2025

nashif commented Mar 28, 2025

nashif commented Mar 28, 2025 •

edited

Loading

kartben Mar 28, 2025

nashif Mar 28, 2025

kartben Mar 28, 2025

kartben Mar 28, 2025

teburd commented Mar 29, 2025

pdgendt commented Mar 31, 2025

ci: workflows: pin python dependencies #87609

ci: workflows: pin python dependencies #87609

Conversation

nashif commented Mar 25, 2025

nashif commented Mar 25, 2025

pdgendt commented Mar 25, 2025

cfriedt commented Mar 27, 2025

pdgendt commented Mar 28, 2025

nashif commented Mar 28, 2025

pdgendt commented Mar 28, 2025

nashif commented Mar 28, 2025

nashif commented Mar 28, 2025 • edited Loading

kartben Mar 28, 2025

Choose a reason for hiding this comment

nashif Mar 28, 2025

Choose a reason for hiding this comment

kartben Mar 28, 2025

Choose a reason for hiding this comment

kartben Mar 28, 2025

Choose a reason for hiding this comment

teburd commented Mar 29, 2025

pdgendt commented Mar 31, 2025

nashif commented Mar 28, 2025 •

edited

Loading