Skip to content

Commit 7eba9e8

Browse files
burgholzerburgholzer
authored
♻️ Refactor Packaging and CI (#283)
## Description This PR brings a major refactoring for the overall Python packaging as well as the CI employed in the project. ### Packaging We are (finally) moving away from `setuptools` and towards modern `scikit-build-core` (https://scikit-build-core.readthedocs.io/) This brings many advantages such as - truly editable installs that also include the C++ part; this implies incredibly fast rebuilds that can be further improved by installing ccache. - completely eliminates the `setup.py` (and the `MANIFEST.in`) file and all custom build code that was required previously. As part of this change, we adopt the established best practice of a src-layout for the Python package. This implies that the Python package no longer lives in the main package directory, but is moved to `src/mqt/...`. This avoids several pitfalls when working with Python packages and makes the overall handling way easier. This also means there is no direct build-time dependency on cmake and ninja anymore as this is automatically handled by scikit-build-core on demand. As part of this rewrite, several `nox` sessions are also updated ### CI MQT Core has introduced reusable workflows in - cda-tum/mqt-core#396 These have been further refined in - cda-tum/mqt-core#407 Overall, this allows to - remove almost all workflow files of this repository - eliminate a lot of maintenance required in this repository to keep up with the latest best practices - rely on dependabot for workflow updates once MQT Core has settled enough so that it follows proper versioning and we can use tags for the workflows here instead of `@main` while at the same time: - having automatic change detection and only triggering jobs that are really necessary; with a single pass check that can be used for branch protection - setting up ccache for all supported workflow jobs which greatly improves compilation times - setting up mold as a better linker under linux - splitting jobs into as many subjobs as possible and feasible to distribute work - automatically creating wheels for all supported platforms (including exotic platforms that need emulation) and Python versions using PyPI trusted publishing - enabling automatic retries for flaky jobs such as MSVC builds or coverage uploads - enabling automatic build parallelization via using Ninja wherever suitable - running an automatic workflow using the minimal versiosn for all Python dependencies to guarantee compatibility As a nice side effect, the new CI has unveiled several issues in the HSF simulator and it's task logic that have subsequently been resolved. Upon writing regression tests, an endianness bug in the Amplitude Mode HSF simulation surfaced. With this PR, the expected results are returned. ## Checklist: <!--- This checklist serves as a reminder of a couple of things that ensure your pull request will be merged swiftly. --> - [x] The pull request only contains commits that are related to it. - [x] I have added appropriate tests and documentation. - [x] I have made sure that all CI jobs on GitHub pass. - [x] The pull request introduces no new warnings and follows the project's style guidelines.
2 parents c6f95af + 3121f99 commit 7eba9e8

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

65 files changed

+1270
-1014
lines changed

‎.cirrus.yml

+113
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
.clone_script: &clone |
2+
if [ -z "$CIRRUS_PR" ]; then
3+
git clone --recursive --branch=$CIRRUS_BRANCH https://x-access-token:${CIRRUS_REPO_CLONE_TOKEN}@github.com/${CIRRUS_REPO_FULL_NAME}.git $CIRRUS_WORKING_DIR
4+
git reset --hard $CIRRUS_CHANGE_IN_REPO
5+
else
6+
git clone https://x-access-token:${CIRRUS_REPO_CLONE_TOKEN}@github.com/${CIRRUS_REPO_FULL_NAME}.git $CIRRUS_WORKING_DIR
7+
git fetch origin pull/$CIRRUS_PR/head:pull/$CIRRUS_PR
8+
git checkout $CIRRUS_BASE_BRANCH
9+
git -c user.email="[email protected]" merge --no-commit pull/$CIRRUS_PR
10+
git submodule update --init --recursive
11+
fi
12+
13+
.statistics_script: &statistics |
14+
ccache -s -v
15+
echo $(python -m pip cache dir)
16+
echo $(python -m pip cache list)
17+
18+
macos_arm64_test_task:
19+
name: 🐍 Test / 🍎 arm64
20+
alias: macos_arm64_test
21+
clone_script: *clone
22+
macos_instance:
23+
image: ghcr.io/cirruslabs/macos-monterey-xcode:latest
24+
ccache_cache:
25+
folder: .ccache
26+
populate_script:
27+
- mkdir -p .ccache
28+
fingerprint_key: ccache-macosx_arm64
29+
pip_cache:
30+
folder: /Users/admin/Library/Caches/pip
31+
prepare_env_script: |
32+
brew install [email protected] ccache pipx ninja nox
33+
ln -s $(which python3.10) python
34+
export PATH=/opt/homebrew/opt/[email protected]/libexec/bin:$PATH
35+
export PATH=/opt/homebrew/opt/ccache/libexec:$PATH
36+
export PATH=/opt/homebrew/opt/pipx/libexec:$PATH
37+
export PATH=/opt/homebrew/opt/ninja/libexec:$PATH
38+
export PATH=/opt/homebrew/opt/nox/libexec:$PATH
39+
echo "PATH=$PATH" >> $CIRRUS_ENV
40+
echo "CCACHE_DIR=$PWD/.ccache" >> $CIRRUS_ENV
41+
run_nox_script:
42+
- nox -s tests-3.10 --verbose
43+
statistics_script: *statistics
44+
45+
build_and_store_wheels: &BUILD_AND_STORE_WHEELS
46+
install_cibuildwheel_script:
47+
- python -m pip install cibuildwheel~=2.15.0
48+
run_cibuildwheel_script:
49+
- cibuildwheel
50+
wheels_artifacts:
51+
path: "wheelhouse/*"
52+
53+
linux_aarch64_wheels_task:
54+
only_if: "$CIRRUS_RELEASE != ''"
55+
name: 🐍 Packaging / 🎡 🐧 arm64
56+
alias: linux_aarch64_wheels
57+
clone_script: *clone
58+
compute_engine_instance:
59+
image_project: cirrus-images
60+
image: family/docker-builder-arm64
61+
architecture: arm64
62+
platform: linux
63+
cpu: 8
64+
setup_pyhton_script:
65+
- apt-get install -y python3-venv python-is-python3
66+
<<: *BUILD_AND_STORE_WHEELS
67+
68+
macos_arm64_wheels_task:
69+
only_if: "$CIRRUS_RELEASE != ''"
70+
name: 🐍 Packaging / 🎡 🍎 arm64
71+
alias: macos_arm64_wheels
72+
clone_script: *clone
73+
macos_instance:
74+
image: ghcr.io/cirruslabs/macos-monterey-xcode:latest
75+
ccache_cache:
76+
folder: .ccache
77+
populate_script:
78+
- mkdir -p .ccache
79+
fingerprint_key: ccache-macosx_arm64-wheels
80+
pip_cache:
81+
folder: /Users/admin/Library/Caches/pip
82+
prepare_env_script: |
83+
brew install [email protected] ccache pipx ninja nox
84+
ln -s $(which python3.10) python
85+
export PATH=/opt/homebrew/opt/[email protected]/libexec/bin:$PATH
86+
export PATH=/opt/homebrew/opt/ccache/libexec:$PATH
87+
export PATH=/opt/homebrew/opt/pipx/libexec:$PATH
88+
export PATH=/opt/homebrew/opt/ninja/libexec:$PATH
89+
export PATH=/opt/homebrew/opt/nox/libexec:$PATH
90+
echo "PATH=$PATH" >> $CIRRUS_ENV
91+
echo "CCACHE_DIR=$PWD/.ccache" >> $CIRRUS_ENV
92+
<<: *BUILD_AND_STORE_WHEELS
93+
statistics_script: *statistics
94+
95+
publish_task:
96+
name: 🚀 Deploy to PyPI
97+
container: { image: "python:3.10-bullseye" }
98+
depends_on:
99+
- linux_aarch64_wheels
100+
- macos_arm64_wheels
101+
only_if: "$CIRRUS_RELEASE != ''"
102+
env:
103+
TWINE_REPOSITORY: pypi
104+
TWINE_USERNAME: __token__
105+
TWINE_PASSWORD: "ENCRYPTED\
106+
[87601b93dead1db95c3ccb4b2ea2e0da333a0da3e52d5eb1\
107+
e5eb83dc24ae3c704779789acb68a9d53719a6d823c0692e]"
108+
install_script: pip install twine
109+
publish_script:
110+
- curl -L https://api.cirrus-ci.com/v1/artifact/build/$CIRRUS_BUILD_ID/wheels.zip -o wheels.zip
111+
- unzip wheels.zip
112+
- python -m twine check wheelhouse/*
113+
- python -m twine upload wheelhouse/*

‎.github/ISSUE_TEMPLATE/bug-report.yml

+12-31
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,6 @@
1-
name: Bug report
1+
name: 🐛 Bug report
22
description: Something is not working correctly.
3-
title: "<title>"
4-
labels: ["bug"]
5-
assignees:
6-
- hillmich
3+
title: "🐛 <title>"
74
body:
85
- type: markdown
96
attributes:
@@ -13,35 +10,19 @@ body:
1310
⚠
1411
Verify first that your issue is not [already reported on GitHub](https://github.com/cda-tum/mqt-ddsim/search?q=is%3Aissue&type=issues).
1512
16-
If you are having general questions, please consider [starting a discussion](https://github.com/cda-tum/mqt-ddsim/discussions).
17-
- type: markdown
18-
attributes:
19-
value: >-
20-
**Environment**
21-
- type: input
22-
attributes:
23-
label: mqt.ddsim version
24-
placeholder: For example, mqt.ddsim==1.0.0
25-
validations:
26-
required: true
27-
- type: input
13+
If you have general questions, please consider [starting a discussion](https://github.com/cda-tum/mqt-ddsim/discussions).
14+
- type: textarea
2815
attributes:
29-
label: OS
30-
placeholder: For example, Ubuntu 22.04, macOS Big Sur, Windows etc.
16+
label: Environment information
17+
description: >-
18+
Please provide information about your environment. For example, OS, C++ compiler, mqt.core version etc.
19+
placeholder: |
20+
- OS:
21+
- C++ compiler:
22+
- mqt.ddsim version:
23+
- Additional environment information:
3124
validations:
3225
required: true
33-
- type: input
34-
attributes:
35-
label: Python version
36-
placeholder: For example, Python 3.10
37-
- type: input
38-
attributes:
39-
label: C++ compiler
40-
placeholder: For example, gcc-10
41-
- type: textarea
42-
attributes:
43-
label: Additional environment information
44-
description: Feel free to add more information about your environment here.
4526
- type: textarea
4627
attributes:
4728
label: Description

‎.github/ISSUE_TEMPLATE/feature-request.yml

+2-5
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,6 @@
1-
name: Feature request
1+
name: ✨ Feature request
22
description: Suggest an idea
3-
title: "<title>"
4-
labels: ["enhancement"]
5-
assignees:
6-
- hillmich
3+
title: "✨ <title>"
74
body:
85
- type: markdown
96
attributes:

‎.github/codecov.yml

+39-7
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,54 @@
11
ignore:
2-
- "extern/**/*"
32
- "apps/*"
3+
- "extern/**/*"
4+
- "include/python/**/*"
5+
- "src/python/*.cpp"
46
- "test/*.cpp"
5-
- "mqt/**/*"
67

78
coverage:
8-
range: 60..95
9+
range: 60..90
910
precision: 1
1011
status:
11-
project:
12-
default:
12+
project: off
13+
patch: off
14+
15+
flag_management:
16+
default_rules:
17+
carryforward: true
18+
statuses:
19+
- type: project
20+
target: auto
1321
threshold: 0.5%
14-
patch:
15-
default:
22+
removed_code_behavior: adjust_base
23+
- type: patch
24+
target: 90%
1625
threshold: 1%
26+
individual_flags:
27+
- name: cpp
28+
paths:
29+
- "include"
30+
- "src"
31+
- "!src/python/*.cpp"
32+
after_n_builds: 1
33+
- name: python
34+
paths:
35+
- "src/mqt/**/*.py"
36+
after_n_builds: 3
37+
statuses:
38+
- type: project
39+
threshold: 0.5%
40+
removed_code_behavior: adjust_base
41+
- type: patch
42+
target: 95%
43+
threshold: 1%
1744

1845
parsers:
1946
gcov:
2047
branch_detection:
2148
conditional: no
2249
loop: no
50+
51+
comment:
52+
layout: "reach, diff, flags, files"
53+
require_changes: true
54+
show_carryforward_flags: true

‎.github/codeql-config.yml

-4
This file was deleted.

‎.github/dependabot.yml

+14-9
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,36 @@
11
version: 2
22
updates:
3-
# Enable version updates for git submodules
43
- package-ecosystem: "gitsubmodule"
5-
# Look for `.gitmodules` in the `root` directory
64
directory: "/"
7-
# Check the submodules for updates every week
5+
groups:
6+
submodules:
7+
patterns:
8+
- "*"
89
schedule:
910
interval: "monthly"
10-
day: "friday"
1111
time: "05:00"
1212
timezone: "Europe/Vienna"
1313

14-
# Enable version updates for GitHub Actions
1514
- package-ecosystem: "github-actions"
16-
# Look for `*.yml` files in the `.github/workflows` directory
1715
directory: "/"
18-
# Check for updates to GitHub Actions every week
16+
groups:
17+
github-actions:
18+
patterns:
19+
- "*"
1920
schedule:
20-
interval: "monthly"
21+
interval: "weekly"
2122
day: "friday"
2223
time: "05:00"
2324
timezone: "Europe/Vienna"
2425

2526
- package-ecosystem: "pip"
2627
directory: "/"
28+
groups:
29+
python-dependencies:
30+
patterns:
31+
- "*"
2732
schedule:
28-
interval: "monthly"
33+
interval: "weekly"
2934
day: "friday"
3035
time: "05:00"
3136
timezone: "Europe/Vienna"

‎.github/matchers/pylint.json

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
{
2+
"problemMatcher": [
3+
{
4+
"severity": "warning",
5+
"pattern": [
6+
{
7+
"regexp": "^([^:]+):(\\d+):(\\d+): ([A-DF-Z]\\d+): \\033\\[[\\d;]+m([^\\033]+).*$",
8+
"file": 1,
9+
"line": 2,
10+
"column": 3,
11+
"code": 4,
12+
"message": 5
13+
}
14+
],
15+
"owner": "pylint-warning"
16+
},
17+
{
18+
"severity": "error",
19+
"pattern": [
20+
{
21+
"regexp": "^([^:]+):(\\d+):(\\d+): (E\\d+): \\033\\[[\\d;]+m([^\\033]+).*$",
22+
"file": 1,
23+
"line": 2,
24+
"column": 3,
25+
"code": 4,
26+
"message": 5
27+
}
28+
],
29+
"owner": "pylint-error"
30+
}
31+
]
32+
}

‎.github/pull_request_template.md

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
## Description
2+
3+
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
4+
5+
Fixes #(issue) <!--- Replace (issue) with the issue number that is fixed by this pull request. -->
6+
7+
## Checklist:
8+
9+
<!---
10+
This checklist serves as a reminder of a couple of things that ensure your pull request will be merged swiftly.
11+
-->
12+
13+
- [ ] The pull request only contains commits that are related to it.
14+
- [ ] I have added appropriate tests and documentation.
15+
- [ ] I have made sure that all CI jobs on GitHub pass.
16+
- [ ] The pull request introduces no new warnings and follows the project's style guidelines.

‎.github/release-drafter.yml

+4
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ categories:
99
- title: "🐛 Bug Fixes"
1010
labels:
1111
- "bug"
12+
- "fix"
1213
- title: "📄 Documentation"
1314
labels:
1415
- "documentation"
@@ -18,6 +19,9 @@ categories:
1819
- title: "📦 Packaging"
1920
labels:
2021
- "packaging"
22+
- title: "🧹 Code Quality"
23+
labels:
24+
- "code quality"
2125
- title: "⬆️ Dependencies"
2226
collapse-after: 5
2327
labels:

0 commit comments

Comments
 (0)