Skip to content

Update CI to build wheels for linux + osx with cibuildwheel #288

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

jonathangreen
Copy link
Contributor

@jonathangreen jonathangreen commented Mar 15, 2024

This PR overhauls whats being done in CI, so that wheel builds are done in a similar manor to lxml using cibuildwheel. This should let us push wheels for all platforms out to PyPi. Right now this PR just attaches the built wheels to the build as an artifact, since I don't have any access to PyPi for this repo. cibuildwheel has some guidance in their docs as to how to upload the built wheels to PyPi.

Since I've changed around the actions, I don't think they will run until a admin enables them on this PR, or the PR is merged. But you can see in my fork what the build process looks like.

I bumped the minimum supported version of lxml up to 5.0.0, so that the version of libxml2 that is statically linked by the build process matches the version that is shipped with lxml. I also added a CI workflow that tests a matrix of:

  • python:
    • "3.7"
    • "3.12"
  • lxml:
    • "5.0.0"
    • "5.1.0"
  • libxmlsec1:
    • "1.3.3"
    • "1.2.39"

setup.py was updated to:

  • Look up and install the latest version of openssl 1.1.1. Static builds were not linking against the latest before.
    • In the future it would be nice to update this to install openssl 3.x versions, but the build requirements are different, so I didn't take that on as part of this PR.
  • Allow static builds for osx, so we can push static wheels to pypi
    • Allow cross compiling so github actions OSX x86_64 workers can build arm64 osx packages
  • Remove --enable-rebuild-docs arg to configure for libxml2, since it no longer exists
  • Build libxmlsec1 with --enable-md5 build arg, required for the tests to pass, otherwise the built wheel has a symbol missing
  • Bump the supported python versions to 3.6+ to match the wheels that are being built.

Note: This PR does not do anything to build wheels for Windows. I don't have a lot of windows build experience, and it looks like the existing builds depend on pre-built binaries in some other repo. Someone with more experience on Windows will have to do that work. Once the windows wheel builds are working, we should be able to package them up the same way as the osx and linux builds. I left a comment in the wheel.yml job pointing to how to make this change.

@jonathangreen
Copy link
Contributor Author

Happy to break this one up into a series of smaller PRs, if this one is too large / too wide reaching. Happy to incorporate any feedback.

I'd love to see at least some of the work in this PR get merged into this repo. I think being able to publish staticly linked wheels for linux / osx would be a big win, and would ease the pain for end users of this package of making sure libxml2 stays in sync between this package and lxml.

@jonathangreen
Copy link
Contributor Author

@jimjag thanks for merging #289! 🎉

Is there anything you would like me to do to make it easier to merge this PR? Happy to make any changes necessary here.

@jonathangreen
Copy link
Contributor Author

Going to close this one, and open some more targeted PR instead that might be easier to review.

@jonathangreen jonathangreen deleted the feature/update-ci branch November 14, 2024 00:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant