Wrapping C Libraries is Hard

[dstufft]

Wrapping and packaging a C library is hard. Cython needs a seperate compile step with no clear guide as to who should be doing that step, CFFI has an implicit compile and is really hard to package correctly.

How can we make packaging C libraries easier?

[ncoghlan]

cffi supports pre-compilation, but the API for it is thoroughly confusing.

Also for reference:
https://bitbucket.org/cffi/cffi/issue/109/enable-sane-packaging-for-cffi (filed by Donald)

Proposed rename of the verify APIs (from the cffi-in-the-stdlib? discussions after PyCon 2013):
https://bitbucket.org/cffi/cffi/issue/60/proposed-rename-for-verify-and-verifier

Docs improvement request for verification:
https://bitbucket.org/cffi/cffi/issue/59/missing-content-in-cffiverify

[dstufft]

Packaging cffi is actually really hard. It's taken me several tries till I got my cffi packages to a place where I feel like maybe they are packaged correctly :/ But i agree that pacakging C stuff in general is kind of arcane.

[alex]

In terms of cffi specific stuff, a branch was started on fixing it at the PyCon.ZA sprints.

[eevee]

On the Cython side, we (Yelp) just ended up blindly pasting this into a Cython-powered library: http://stackoverflow.com/a/4515279/17875

[brainwane]

I saw a tweet today from @zoobab https://twitter.com/zoobab/status/968486518552760320 asking about whether the new PyPI still has trouble with compiled C libraries. @zoobab could you talk a little more about the trouble you've had?

[ncoghlan]

PyPI doesn't have anything to do with building C extensions, so the backend change won't have any impact on that regardless - the source build UX will still be down to whether or not publishers offer pre-built binary archives, and how good their instructions are for setting up build environments when they don't.

(Over time, pip 10 and PEPs 517/518 may improve matters, by making it easier for projects to automatically install their build dependencies, but that isn't related to the PyPI migration)