Skip to content

Commit 146b4c7

Browse files
committed
ci: add ARM64 support with native builds and refactored workflows
Add comprehensive ARM64 (linux/arm64) support for Devito's Docker images and CI/CD pipeline, enabling deployment on AWS Graviton, Apple Silicon, and other ARM64 platforms. Architecture & Build Strategy: - Native ARM64 builds on GitHub's free ubuntu-24.04-arm runners - Separate runners for each architecture (no cross-compilation) - x86_64 builds on ubuntu-22.04 (pinned to avoid ubuntu-latest migration) - Multi-platform Docker image support with proper manifest lists Workflow Improvements: - Refactor docker-bases.yml to use matrix strategy for GCC builds - Consolidate x86_64 and ARM64 base image builds into single job - Remove QEMU setup (not needed for native builds) - Add arm64 workflow dispatch input parameter - Conditional execution based on input flags to run only requested builds - ARM64 GCC limited to default version (building from source too slow) CI Test Coverage: - Add ARM64 test configurations to pytest-core-nompi.yml: - pytest-ubuntu-py312-gcc14-omp-arm64 - pytest-ubuntu-py311-gcc13-omp-arm64 - Add ARM64 MPI tests to pytest-core-mpi.yml (gcc-arm64) - Add ARM64 smoke test to docker-devito.yml - Pin x86_64 tests to ubuntu-22.04 for stability Bug Fixes: - Fix OpenMPI parallel build: change ${nproc} to $(nproc) in Dockerfile.cpu (reduces build time from ~15-20 mins to ~5-8 mins) - Add actionlint.yaml config for self-hosted runner label validation Documentation: - Add ARM64 section to docker/README.md - Document supported ARM64 platforms (AWS Graviton, Apple Silicon, etc.) - Add usage examples for ARM64 images
1 parent f2e9cb9 commit 146b4c7

File tree

7 files changed

+127
-21
lines changed

7 files changed

+127
-21
lines changed

.github/actionlint.yaml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# actionlint configuration file
2+
# https://github.com/rhysd/actionlint/blob/main/docs/config.md
3+
4+
self-hosted-runner:
5+
# Custom labels for self-hosted runners
6+
labels:
7+
- nvidiagpu # Self-hosted runner for NVIDIA GPU builds
8+
- amdgpu # Self-hosted runner for AMD GPU builds

.github/workflows/docker-bases.yml

Lines changed: 47 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,9 @@ on:
1616
cpu:
1717
type: boolean
1818
default: false
19+
arm64:
20+
type: boolean
21+
default: false
1922
nvidia:
2023
type: boolean
2124
default: false
@@ -34,46 +37,79 @@ on:
3437

3538
jobs:
3639
#######################################################
37-
############## Basic gcc CPU ##########################
40+
############## GCC Multi-Architecture Base ############
3841
#######################################################
39-
deploy-cpu-bases:
40-
if: inputs.cpu
41-
name: "cpu-base"
42-
runs-on: ubuntu-latest
42+
deploy-gcc-bases:
43+
if: inputs.cpu || inputs.arm64
44+
name: "gcc-${{ matrix.arch }}-${{ matrix.gcc || 'default' }}"
45+
runs-on: ${{ matrix.runner }}
4346
env:
4447
DOCKER_BUILDKIT: "1"
4548

4649
strategy:
4750
matrix:
48-
gcc: ["", "14"]
51+
include:
52+
# x86_64 builds - both default and GCC 14
53+
- arch: amd64
54+
runner: ubuntu-22.04
55+
platform: linux/amd64
56+
tag_prefix: cpu
57+
gcc: ""
58+
input_flag: cpu
59+
- arch: amd64
60+
runner: ubuntu-22.04
61+
platform: linux/amd64
62+
tag_prefix: cpu
63+
gcc: "14"
64+
input_flag: cpu
65+
# ARM64 build - only default GCC (building from source too slow)
66+
- arch: arm64
67+
runner: ubuntu-24.04-arm
68+
platform: linux/arm64
69+
tag_prefix: arm64
70+
gcc: ""
71+
input_flag: arm64
4972

5073
steps:
74+
- name: Check if should build
75+
id: check
76+
run: |
77+
if [[ "${{ matrix.input_flag }}" == "cpu" && "${{ inputs.cpu }}" == "true" ]] || \
78+
[[ "${{ matrix.input_flag }}" == "arm64" && "${{ inputs.arm64 }}" == "true" ]]; then
79+
echo "skip=false" >> $GITHUB_OUTPUT
80+
else
81+
echo "skip=true" >> $GITHUB_OUTPUT
82+
fi
83+
5184
- name: Checkout devito
85+
if: steps.check.outputs.skip != 'true'
5286
uses: actions/checkout@v5
5387

5488
- name: Check event name
89+
if: steps.check.outputs.skip != 'true'
5590
run: echo ${{ github.event_name }}
5691

57-
- name: Set up QEMU
58-
uses: docker/setup-qemu-action@v3
59-
6092
- name: Set up Docker Buildx
93+
if: steps.check.outputs.skip != 'true'
6194
uses: docker/setup-buildx-action@v3
6295

6396
- name: Login to DockerHub
97+
if: steps.check.outputs.skip != 'true'
6498
uses: docker/login-action@v3
6599
with:
66100
username: ${{ secrets.DOCKER_USERNAME }}
67101
password: ${{ secrets.DOCKER_PASSWORD }}
68102

69-
- name: GCC image
103+
- name: Build and push GCC image
104+
if: steps.check.outputs.skip != 'true'
70105
uses: docker/build-push-action@v6
71106
with:
72107
context: .
73108
file: "./docker/Dockerfile.cpu"
74109
push: true
110+
platforms: ${{ matrix.platform }}
75111
build-args: "gcc=${{ matrix.gcc }}"
76-
tags: "devitocodes/bases:cpu-gcc${{ matrix.gcc }}"
112+
tags: "devitocodes/bases:${{ matrix.tag_prefix }}-gcc${{ matrix.gcc }}"
77113

78114
#######################################################
79115
############## Intel OneApi CPU #######################

.github/workflows/docker-devito.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,12 @@ jobs:
4646
test: 'tests/test_operator.py'
4747
runner: ubuntu-latest
4848

49+
- base: 'bases:arm64-gcc'
50+
tag: 'arm64-gcc'
51+
flag: '--init -t'
52+
test: 'tests/test_operator.py'
53+
runner: ubuntu-24.04-arm # GitHub's free ARM64 runner
54+
4955
steps:
5056
- name: Checkout devito
5157
uses: actions/checkout@v5

.github/workflows/pytest-core-mpi.yml

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -66,25 +66,32 @@ jobs:
6666
runs-on: ${{ matrix.os }}
6767
strategy:
6868
matrix:
69-
name: [gcc, icx]
69+
name: [gcc, icx, gcc-arm64]
7070
include:
7171
- name: gcc
7272
arch: gcc
73-
os: ubuntu-latest
73+
base_prefix: cpu
74+
os: ubuntu-22.04
7475
mpiflag: ""
7576
- name: icx
7677
arch: icx
77-
os: ubuntu-latest
78+
base_prefix: cpu
79+
os: ubuntu-22.04
7880
# Need safe math for icx due to inaccuracy with mpi+sinc interpolation
7981
mpiflag: "-e DEVITO_SAFE_MATH=1"
82+
- name: gcc-arm64
83+
arch: gcc
84+
base_prefix: arm64
85+
os: ubuntu-24.04-arm
86+
mpiflag: ""
8087

8188
steps:
8289
- name: Checkout devito
8390
uses: actions/checkout@v5
8491

8592
- name: Build docker image
8693
run: |
87-
docker build . --file docker/Dockerfile.devito --tag devito_img --build-arg base=devitocodes/bases:cpu-${{ matrix.arch }}
94+
docker build . --file docker/Dockerfile.devito --tag devito_img --build-arg base=devitocodes/bases:${{ matrix.base_prefix }}-${{ matrix.arch }}
8895
8996
- name: Test with pytest
9097
run: |

.github/workflows/pytest-core-nompi.yml

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,9 @@ jobs:
3939
pytest-osx-py312-clang-omp,
4040
pytest-docker-py310-gcc-omp,
4141
pytest-docker-py310-icx-omp,
42-
pytest-ubuntu-py313-gcc14-omp
42+
pytest-ubuntu-py313-gcc14-omp,
43+
pytest-ubuntu-py312-gcc14-omp-arm64,
44+
pytest-ubuntu-py311-gcc13-omp-arm64
4345
]
4446
set: [base, adjoint]
4547
include:
@@ -113,6 +115,20 @@ jobs:
113115
language: "openmp"
114116
sympy: "1.14"
115117

118+
- name: pytest-ubuntu-py312-gcc14-omp-arm64
119+
python-version: '3.12'
120+
os: ubuntu-24.04-arm
121+
arch: "gcc-14"
122+
language: "openmp"
123+
sympy: "1.14"
124+
125+
- name: pytest-ubuntu-py311-gcc13-omp-arm64
126+
python-version: '3.11'
127+
os: ubuntu-24.04-arm
128+
arch: "gcc-13"
129+
language: "openmp"
130+
sympy: "1.13"
131+
116132
- set: base
117133
test-set: 'not adjoint'
118134

docker/Dockerfile.cpu

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ RUN cd /tmp && mkdir openmpi && \
4545
../configure --prefix=/opt/openmpi/ \
4646
--disable-mpi-fortran \
4747
--enable-mca-no-build=btl-uct --enable-mpi1-compatibility && \
48-
make -j ${nproc} && \
48+
make -j $(nproc) && \
4949
make install && \
5050
cd /tmp && rm -rf /tmp/openmpi
5151

docker/README.md

Lines changed: 37 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,12 @@ Devito provides several images that target different architectures and compilers
88

99
### [Devito] on CPU
1010

11-
We provide two CPU images:
12-
- `devito:gcc-*` with the standard GNU gcc compiler.
13-
- `devito:icx-*` with the Intel C compiler for Intel architectures.
11+
We provide CPU images for different architectures and compilers:
12+
- `devito:gcc-*` with the standard GNU gcc compiler (x86_64).
13+
- `devito:icx-*` with the Intel C compiler for Intel architectures (x86_64).
14+
- `devito:arm64-gcc-*` with the standard GNU gcc compiler for ARM64 architectures (AWS Graviton, Apple Silicon, etc.).
1415

15-
These images provide a working environment for any CPU architecture and come with [Devito], `gcc/icx` and `mpi` preinstalled, and utilities such as `jupyter` for usability and exploration of the package.
16+
These images provide a working environment for their respective CPU architectures and come with [Devito], `gcc/icx` and `mpi` preinstalled, and utilities such as `jupyter` for usability and exploration of the package.
1617

1718
To run this image locally, you will first need to install `docker`. Then, the following commands will get you started:
1819

@@ -38,6 +39,38 @@ In addition, the following legacy tags are available:
3839
- `devito:cpu-*` that corresponds to `devito:gcc-*`
3940

4041

42+
### [Devito] on ARM64
43+
44+
We provide ARM64-specific images optimized for ARM64 processors:
45+
- `devito:arm64-gcc-*` with the standard GNU gcc compiler for ARM64 architectures.
46+
47+
These images support various ARM64 platforms including:
48+
- AWS Graviton2/3/4 instances
49+
- Apple Silicon (M1/M2/M3) via Docker Desktop
50+
- ARM-based cloud instances
51+
52+
Devito automatically detects the specific ARM64 variant at runtime and applies appropriate optimizations through its JIT compiler.
53+
54+
To run on ARM64 systems:
55+
56+
```bash
57+
# Pull image and start a bash shell
58+
docker run --rm -it -p 8888:8888 devitocodes/devito:arm64-gcc-latest /bin/bash
59+
60+
# or start a Jupyter notebook server on port 8888
61+
docker run --rm -it -p 8888:8888 devitocodes/devito:arm64-gcc-latest
62+
63+
# Run an example
64+
docker run --rm -it devitocodes/devito:arm64-gcc-latest python examples/seismic/acoustic/acoustic_example.py
65+
```
66+
67+
On AWS Graviton instances with user context:
68+
69+
```bash
70+
docker run --rm -it -v `pwd`:`pwd` -w `pwd` -u $(id -u):$(id -g) devitocodes/devito:arm64-gcc-latest python examples/seismic/acoustic/acoustic_example.py
71+
```
72+
73+
4174
### [Devito] on GPU
4275

4376
Second, we provide three images to run [Devito] on GPUs, tagged `devito:nvidia-nvc-*`, and `devito:amd-*`.

0 commit comments

Comments
 (0)