Enable native builds on Linux aarch64 #670

zanieb · 2025-06-28T03:36:08Z

Rebase of #607

Uses #672 and #673 to abstract some of the changes to the CI matrix.

I think this will be a useful precursor to #670

As in #672 this seems helpful for #670

zanieb · 2025-06-28T19:09:22Z

As I mentioned at #607 (comment) — I need to review this myself as I did not review the original implementation and there may be some clean-up to be done after hacking things to work.

ci-matrix.py

cpython-unix/build-main.py

zanieb · 2025-06-29T17:41:21Z

cpython-unix/xcb.debian9.Dockerfile

@@ -0,0 +1,3 @@
+{% include 'build.debian9.Dockerfile' %}


See https://github.com/astral-sh/python-build-standalone/pull/670/files#r2173842155

cpython-unix/build.py

geofft

LGTM, I noted a couple of things that could be changed, but they don't cause incorrect behavior, so I think the builds that result from this code are fine.

cpython-unix/build.py

geofft · 2025-07-01T13:36:25Z

cpython-unix/build.py

-            build_binutils(client, get_image(client, ROOT, BUILD, "gcc"), host_platform)
+            build_binutils(
+                client,
+                get_image(client, ROOT, BUILD, docker_image, host_platform),
+                host_platform,
+            )


note to self: This change pairs with a change to the Makefile to pass --docker-image in the binutils call. In turn the value passed there is rewritten from gcc.cross* to gcc (which is why it was previously hard-coded here), but leaves other suffixes like gcc.debian9 alone.

cpython-unix/build.py

docs/running.rst

ci-matrix.py

geofft · 2025-07-01T13:44:17Z

ci-matrix.py

@@ -25,7 +25,10 @@
    {"name": "build", "arch": "x86_64"},
    {"name": "build.cross", "arch": "x86_64"},
    {"name": "build.cross-riscv64", "arch": "x86_64"},
+    {"name": "build.debian9", "arch": "aarch64"},


In general - is "debian9" here because we're mostly using Debian 8 and that's too old for aarch64?

If we upgrade our base builds to Debian 9, can we use the normal build Dockerfile for aarch64?

In general - is "debian9" here because we're mostly using Debian 8 and that's too old for aarch64?

I presume so, that's where I got stuck at #484 (comment) but this was Greg's choice.

cpython-unix/build.py

geofft · 2025-07-01T13:47:47Z

cpython-unix/build.py

@@ -579,7 +591,10 @@ def python_build_info(
        if lib.startswith("-l"):
            lib = lib[2:]

-            if platform == "linux_x86_64" and lib not in linux_allowed_system_libraries:
+            if (


Again, if you do another rev of this PR, I think this one could also use an else: raise.

Not trivial here because the else case would also be lib in <expected>, will omit for now.

geofft · 2025-07-01T14:07:46Z

cpython-unix/targets.yml

+  target_cc: clang
+  target_cxx: clang++
+  target_cflags:
+    - '-fvisibility=hidden'


So this is not a behavior change in practice (my current aarch64 build doesn't export symbols from dependencies, and there's a validation check for it), but I can't figure out why the cross builds seem not to need it. Opened #680.

geofft · 2025-07-01T14:13:53Z

cpython-unix/build.py

+        machine = platform.machine()

-        env["TARGET_TRIPLE"] = (
-            target_triple.replace("x86_64_v2-", "x86_64-")
-            .replace("x86_64_v3-", "x86_64-")
-            .replace("x86_64_v4-", "x86_64-")
-        )
+        if machine == "aarch64":
+            env["BUILD_TRIPLE"] = "aarch64-unknown-linux-gnu"
+        else:
+            env["BUILD_TRIPLE"] = "x86_64-unknown-linux-gnu"
+            env["TARGET_TRIPLE"] = (
+                target_triple.replace("x86_64_v2-", "x86_64-")
+                .replace("x86_64_v3-", "x86_64-")
+                .replace("x86_64_v4-", "x86_64-")
+            )


This doesn't seem right - isn't platform.machine() guaranteed to be x86_64 here? I think this code is fine as is, the new branch won't be taken, and the existing code is currently in use for x86_64 -> aarch64 cross builds (the target_triple.replace stuff doesn't replace anything and leaves the target triple unmodified).

geofft · 2025-07-01T15:03:08Z

cpython-unix/targets.yml

+  # Blocked on:
+  #   BOLT-ERROR: Cannot relax adr in non-simple function
+  #   trampoline_code_table/1. Use --strict option to override
+  # bolt_capable: true


This is from BOLT getting confused at hand-written assembly in libffi. Also there seems to be no way to opt out of this optimization pass, or optimizing this one function? Reported as llvm/llvm-project#146541

This seems to "just work." Let's stay modern.

I think deletion of the legacy proto packages a few commits ago fixed up the build failures the deleted comment alluded to.

I just published an LLVM 20 toolchain for aarch64. The toolchain has support for PGO and BOLT. This commit switches the Linux aarch64 builds to be performed natively on aarch64 machines. PGO and BOLT are enabled on the builds, hopefully making them a bit faster.

Following #670

zanieb added the platform:linux Specific to the Linux platform label Jun 28, 2025

zanieb force-pushed the zb/aarch64-native branch 3 times, most recently from 2aff89e to 1b9b6ce Compare June 28, 2025 03:46

This was referenced Jun 28, 2025

Generate the Docker CI matrix #672

Merged

Generate the matrix for building the pythonbuild crate #673

Merged

zanieb added a commit that referenced this pull request Jun 28, 2025

Generate the Docker CI matrix (#672)

526189d

I think this will be a useful precursor to #670

zanieb added a commit that referenced this pull request Jun 28, 2025

Generate the matrix for building the pythonbuild crate (#673)

449d3e7

As in #672 this seems helpful for #670

zanieb added the ci:dry-run label Jun 28, 2025

zanieb force-pushed the zb/aarch64-native branch from 1b9b6ce to fa119e9 Compare June 28, 2025 17:00

zanieb removed the ci:dry-run label Jun 28, 2025

zanieb force-pushed the zb/aarch64-native branch 5 times, most recently from f8cd0b1 to 801d42a Compare June 28, 2025 18:33

zanieb marked this pull request as ready for review June 28, 2025 19:08

zanieb force-pushed the zb/aarch64-native branch from 801d42a to e06fb79 Compare June 28, 2025 19:08

zanieb mentioned this pull request Jun 28, 2025

Enable native builds on Linux aarch64 #607

Closed

zanieb commented Jun 29, 2025

View reviewed changes

ci-matrix.py Outdated Show resolved Hide resolved

zanieb commented Jun 29, 2025

View reviewed changes

cpython-unix/build-main.py Outdated Show resolved Hide resolved

zanieb commented Jun 29, 2025

View reviewed changes

cpython-unix/build.py Show resolved Hide resolved

zanieb commented Jun 29, 2025

View reviewed changes

cpython-unix/build.py Outdated Show resolved Hide resolved

geofft approved these changes Jul 1, 2025

View reviewed changes

geofft reviewed Jul 1, 2025

View reviewed changes

zanieb mentioned this pull request Jul 1, 2025

Enable PGO and native builds for aarch64-unknown-linux-gnu #484

Closed

geofft approved these changes Jul 1, 2025

View reviewed changes

xtrans 1.5.0 -> 1.6.0

695c72c

This seems to "just work." Let's stay modern.

indygreg and others added 3 commits July 1, 2025 12:31

x11-util-macros 1.20.1 -> 1.20.2

b40be46

xorgproto 2019.1 -> 2024.1

0211a8c

I think deletion of the legacy proto packages a few commits ago fixed up the build failures the deleted comment alluded to.

zanieb force-pushed the zb/aarch64-native branch from 99ab2c7 to 285e307 Compare July 1, 2025 17:31

zanieb merged commit 21cf744 into main Jul 1, 2025
382 checks passed

zanieb deleted the zb/aarch64-native branch July 1, 2025 18:10

zanieb mentioned this pull request Jul 2, 2025

Update wanted release targets for aarch64-unknown-linux-gnu #681

Merged

zanieb added a commit that referenced this pull request Jul 2, 2025

Update wanted release targets for aarch64-unknown-linux-gnu (#681)

9bb8bcb

Following #670

zanieb mentioned this pull request Jul 9, 2025

Build issue, file or directory not found #692

Closed

Uh oh!

Enable native builds on Linux aarch64 #670

Enable native builds on Linux aarch64 #670

Uh oh!

Conversation

zanieb commented Jun 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zanieb commented Jun 28, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

geofft left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

zanieb commented Jun 28, 2025 •

edited

Loading