Skip to content

Conversation

ggorman
Copy link
Contributor

@ggorman ggorman commented Oct 1, 2025

Adds ARM64 (Graviton/Apple Silicon) support to Docker infrastructure:

  • New Dockerfile.arm64 mirroring Dockerfile.cpu architecture
  • Builds OpenMPI 5.0.x from source for GCC 14 compatibility
  • Uses GitHub's free ubuntu-24.04-arm runners (no AWS costs)
  • Adds ARM64 to docker-bases.yml and docker-devito.yml workflows
  • Documents ARM64 usage in README.md
  • Configures actionlint.yaml for custom runner labels

Single ARM64 base image supports all Graviton variants (2/3/4) and Apple Silicon via runtime detection through Devito's JIT compiler.

@ggorman ggorman requested a review from mloubout October 1, 2025 13:14
Copy link
Contributor

@mloubout mloubout left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get why the changes are so big and why it needs a copy paste of the CPU docker file. Only change should be to add the runner and tag to the deploy action

Copy link

codecov bot commented Oct 1, 2025

Codecov Report

❌ Patch coverage is 0% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.05%. Comparing base (f2e9cb9) to head (0a26204).
⚠️ Report is 55 commits behind head on main.

Files with missing lines Patch % Lines
tests/test_interpolation.py 0.00% 10 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2751      +/-   ##
==========================================
- Coverage   83.05%   83.05%   -0.01%     
==========================================
  Files         248      248              
  Lines       50294    50284      -10     
  Branches     4426     4423       -3     
==========================================
- Hits        41773    41762      -11     
+ Misses       7763     7762       -1     
- Partials      758      760       +2     
Flag Coverage Δ
pytest-gpu-aomp-amdgpuX 68.75% <ø> (ø)
pytest-gpu-nvc-nvidiaX 69.27% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ggorman
Copy link
Contributor Author

ggorman commented Oct 1, 2025

I don't get why the changes are so big and why it needs a copy paste of the CPU docker file. Only change should be to add the runner and tag to the deploy action

You are right. Will fix and push again.

tags: "devitocodes/bases:cpu-gcc${{ matrix.gcc }}"

#######################################################
############## GCC ARM64 Base #########################
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can me berged with the GCC above with a matrix

  Add comprehensive ARM64 (linux/arm64) support for Devito's Docker images
  and CI/CD pipeline, enabling deployment on AWS Graviton, Apple Silicon,
  and other ARM64 platforms.

  Architecture & Build Strategy:
  - Native ARM64 builds on GitHub's free ubuntu-24.04-arm runners
  - Separate runners for each architecture (no cross-compilation)
  - x86_64 builds on ubuntu-22.04 (pinned to avoid ubuntu-latest migration)
  - Multi-platform Docker image support with proper manifest lists

  Workflow Improvements:
  - Refactor docker-bases.yml to use matrix strategy for GCC builds
  - Consolidate x86_64 and ARM64 base image builds into single job
  - Remove QEMU setup (not needed for native builds)
  - Add arm64 workflow dispatch input parameter
  - Conditional execution based on input flags to run only requested builds
  - ARM64 GCC limited to default version (building from source too slow)

  CI Test Coverage:
  - Add ARM64 test configurations to pytest-core-nompi.yml:
    - pytest-ubuntu-py312-gcc14-omp-arm64
    - pytest-ubuntu-py311-gcc13-omp-arm64
  - Add ARM64 MPI tests to pytest-core-mpi.yml (gcc-arm64)
  - Add ARM64 smoke test to docker-devito.yml
  - Pin x86_64 tests to ubuntu-22.04 for stability

  Bug Fixes:
  - Fix OpenMPI parallel build: change ${nproc} to $(nproc) in Dockerfile.cpu
    (reduces build time from ~15-20 mins to ~5-8 mins)
  - Add actionlint.yaml config for self-hosted runner label validation

  Documentation:
  - Add ARM64 section to docker/README.md
  - Document supported ARM64 platforms (AWS Graviton, Apple Silicon, etc.)
  - Add usage examples for ARM64 images
  Replace rank-specific hardcoded assertions with ownership-based verification
  using _dist_datamap. The test now validates sparse point interpolation values
  based on actual point ownership rather than assuming a specific MPI topology,
  ensuring consistent results across different decompositions.
@mloubout
Copy link
Contributor

mloubout commented Oct 6, 2025

Having a different image for ARM doesn't make sense, it should be a single cpu image with multiple platform support. You can have a look at the arm-base-mloubout branch, it does it properly for cpu and push it the right way

image

@mloubout
Copy link
Contributor

Done in #2773

@mloubout mloubout closed this Oct 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants