Skip to content

Commit

Permalink
Add action to build package (#1)
Browse files Browse the repository at this point in the history
* Add action to build package

* Fine, I'll bring my own constants

* Continue trying to get things to build

* Flail some more

* Keep on tweaking

* Pass artifacts around

* Wait for artifacts to be built

* Test publish

* Try to get Test PyPI working

* Let's try that again

* Hello, GitHub, is this thing on?

* Fail fast

* One more time for the world

* Only publish on push
  • Loading branch information
Jordan Webb authored Apr 19, 2021
1 parent 7b62696 commit 8ee9f63
Show file tree
Hide file tree
Showing 9 changed files with 136 additions and 35 deletions.
105 changes: 105 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
on:
pull_request:
branches:
- main

push:
branches:
- main
tags:
- 'v*'

name: Build renameat2

jobs:
manylinux:
runs-on: ubuntu-20.04

strategy:
fail-fast: true
matrix:
python:
- cp36-cp36m
- cp37-cp37m
- cp38-cp38
- cp39-cp39
arch:
- x86_64
- i686
- aarch64

steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@v1

- uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Build manylinux wheel
run: |
set -euo pipefail
if [ "$ARCH" = "aarch64" ]; then
export BASEPLAT=manylinux2014
fi
docker-compose -f builder/docker-compose.yml build --pull
docker-compose -f builder/docker-compose.yml run manylinux build-wheel.sh
env:
WHICH_PYTHON: ${{ matrix.python }}
ARCH: ${{ matrix.arch }}

- uses: actions/upload-artifact@v2
if: github.event_name == 'push'
with:
name: ${{ matrix.python }}-${{ matrix.arch }}
path: dist/*.whl


sdist:
runs-on: ubuntu-20.04

steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Build sdist
run: |
set -euo pipefail
docker-compose -f builder/docker-compose.yml build --pull
docker-compose -f builder/docker-compose.yml run manylinux /opt/python/cp36-cp36m/bin/python3 setup.py sdist
- uses: actions/upload-artifact@v2
if: github.event_name == 'push'
with:
name: sdist
path: dist/*.tar.gz


testpypi:
runs-on: ubuntu-20.04
if: github.event_name == 'push'

needs:
- manylinux
- sdist

steps:
- uses: actions/download-artifact@v2
with:
path: ./artifacts

- name: Rearrange artifacts
run: |
mkdir -p dist
mv ./artifacts/*/* dist/
- name: Publish package to TestPyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
user: __token__
password: ${{ secrets.TEST_PYPI_TOKEN }}
repository_url: https://test.pypi.org/legacy/
2 changes: 1 addition & 1 deletion builder/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ services:
args:
ARCH: ${ARCH:-x86_64}
environment:
PLAT: manylinux1_${ARCH:-x86_64}
PLAT: ${BASEPLAT:-manylinux1}_${ARCH:-x86_64}
WHICH_PYTHON: ${WHICH_PYTHON:-cp36-cp36m}
volumes:
- ..:/mnt
9 changes: 0 additions & 9 deletions builder/manylinux/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,6 @@ ARG BASE_IMAGE=quay.io/pypa/manylinux2014_${ARCH}:latest

FROM ${BASE_IMAGE}

# We need newer kernel headers, manylinux comes with 3.10 and we need 3.15
RUN rpm --import https://repo.ius.io/RPM-GPG-KEY-IUS-7 && \
yum install -y https://repo.ius.io/ius-release-el7.rpm && \
yum install -y yum-plugin-replace && \
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org && \
yum install -y https://www.elrepo.org/elrepo-release-7.el7.elrepo.noarch.rpm && \
yum --enablerepo=elrepo-kernel replace -y kernel-headers --replace-with=kernel-lt-headers && \
yum clean all

COPY build-wheel.sh /usr/bin/build-wheel.sh

VOLUME [ "/mnt", "/tmp" ]
Expand Down
6 changes: 3 additions & 3 deletions builder/manylinux/build-wheel.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,10 @@ fi
tar -C /build/source -xvf "$sdist"

# Install development requirements
"/opt/python/${WHICH_PYTHON}/bin/pip" install -r /mnt/requirements-dev.txt
"/opt/python/${WHICH_PYTHON}/bin/pip3" install -r /mnt/requirements-dev.txt

# Compile wheel
"/opt/python/${WHICH_PYTHON}/bin/pip" wheel /build/source/"$(basename "$sdist" .tar.gz)" --no-deps -w /build/wheel
"/opt/python/${WHICH_PYTHON}/bin/pip3" wheel /build/source/"$(basename "$sdist" .tar.gz)" --no-deps -w /build/wheel

# Bundle external shared libraries into the wheels
for whl in /build/wheel/*.whl; do
Expand All @@ -43,7 +43,7 @@ done

# Install packages and test
cp /mnt/renameat2_test.py /bin/renameat2_test.py
"/opt/python/${WHICH_PYTHON}/bin/pip" install renameat2 --no-index -f /build/fixedwheel
"/opt/python/${WHICH_PYTHON}/bin/pip3" install renameat2 --no-index -f /build/fixedwheel
cd / && "/opt/python/${WHICH_PYTHON}/bin/python3" -m pytest /bin/renameat2_test.py

mnt_uid=$(stat -c '%u' /mnt)
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ requires = ["setuptools >= 40.6.0", "cffi", "wheel", "setuptools_scm[toml]>=3.4"
build-backend = "setuptools.build_meta"

[tool.setuptools_scm]
local_scheme = "no-local-version"
19 changes: 19 additions & 0 deletions renameat2.c
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,25 @@
#include <sys/syscall.h>
#include <linux/fs.h>

#ifndef SYS_renameat2
# ifdef __x86_64__
# define SYS_renameat2 316
# endif
# ifdef __i386__
# define SYS_renameat2 353
# endif
# ifdef __aarch64__
# define SYS_renameat2 276
# endif
# ifdef __arm__
# define SYS_renameat2 382
# endif
#endif

#ifndef SYS_renameat2
# error SYS_renameat2 is not defined
#endif

/* Newer versions of glibc include this, but none new enough for manylinux */
int renameat2(int olddirfd, const char *oldpath,
int newdirfd, const char *newpath,
Expand Down
6 changes: 3 additions & 3 deletions renameat2/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ def _check_kernel_version():


class Flags(IntFlag):
RENAME_EXCHANGE: int = _lib.RENAME_EXCHANGE # type: ignore
RENAME_EXCHANGE = 2
"""
Atomically exchange oldpath and newpath. Both pathnames must exist but may be of
different types (e.g., one could be a non-empty directory and the other a symbolic
Expand All @@ -49,15 +49,15 @@ class Flags(IntFlag):
RENAME_WHITEOUT.
"""

RENAME_NOREPLACE: int = _lib.RENAME_NOREPLACE # type: ignore
RENAME_NOREPLACE = 1
"""
Don't overwrite newpath of the rename. Return an error if newpath already exists.
RENAME_NOREPLACE requires support from the underlying filesystem. See the
renameat(2) manpage for more information.
"""

RENAME_WHITEOUT: int = _lib.RENAME_WHITEOUT # type: ignore
RENAME_WHITEOUT = 4
"""
Specifying RENAME_WHITEOUT creates a "whiteout" object at the source of the rename
at the same time as performing the rename. The whole operation is atomic, so that
Expand Down
4 changes: 0 additions & 4 deletions renameat2_build.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,6 @@

ffibuilder.cdef( # type: ignore
"""
#define RENAME_EXCHANGE ...
#define RENAME_NOREPLACE ...
#define RENAME_WHITEOUT ...
int renameat2(int olddirfd, const char *oldpath,
int newdirfd, const char *newpath,
unsigned int flags);
Expand Down
19 changes: 4 additions & 15 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,6 @@
from setuptools import setup

# setup.cfg seems to be working but I'm old and still don't trust it
# setup(
# name="renameat2",
# version="1.0.0",
# author="Jordan Webb",
# author_email="[email protected]",
# description="A wrapper around Linux's renameat2 system call",
# url="https://github.com/jordemort/python-renameat2",
# packages=["renameat2"],
# setup_requires=["cffi>=1.0.0"],
# cffi_modules=["renameat2_build.py:ffibuilder"],
# python_requires=">=3.6",
# )

setup(cffi_modules=["renameat2_build.py:ffibuilder"], use_scm_version=True)
setup(
cffi_modules=["renameat2_build.py:ffibuilder"],
use_scm_version={"local_scheme": "no-local-version"},
)

0 comments on commit 8ee9f63

Please sign in to comment.