Upgrade pip?

[vidstige]

Why does pip need to be downgraded to 7.x, when the latest is 10.0.1? 🤔

[remia]

I think it's related to this paylogic/pip-accel#73

[vidstige]

What's the best course of action here? I've done some research on debian packaging for python myself, and found that a wheel -> deb conversion tool would probably be easiest/most stable. I've seen for example that popular packages such as numpy seems to be tricky to package using setup.py based methods. One additional benefit is that wheels can be downloaded with public interface of pip, e.g. `pip download´. Just my 50c.

[remia]

For now I managed to use py2deb effectively but had to manually specify some dependencies that were specified with environment markers (not handled by old pip 7 I think). I really don't know if there is a more elegant solution :/

See https://github.com/Ymagis/ClairMeta/blob/develop/.travis/build_deb.sh for example.

[xolox]

Sorry for the long silence here. First I'll answer the question in the opening post by @vidstige:

Why does pip need to be downgraded to 7.x, when the latest is 10.0.1? thinking

py2deb uses pip-accel as an API for:

1. Using pip to download Python packages from PyPI, unpack them and collect package metadata.
2. Local building and caching of python setup.py bdist_dumb based binary packages.

Because pip-accel was developed against pip 7.x and is incompatible with newer versions, pip is downgraded when you install py2deb / pip-accel.

---

I've held back on replying in the py2deb and pip-accel issue trackers (regarding the old pip version) for a while now, because I wanted to provide a proper analysis and "best course of action". I actually started writing that up, but didn't manage to finish it, and so haven't shared it. The end result is that issues like these see nothing but silence from my side, for a long time. Sorry about that.

I'm pressed for time right now as well, so will provide a short summary of the state of affairs:

• The pip-accel project is slowly dying due to lack of maintenance (pip upgrades!). This is caused primarily by the deep integration between pip-accel and pip. I kind of worked myself into a very tight corner there 😇.

• Regardless of the fate of pip-accel I 'm deeply invested in the future of the py2deb project. My use-cases mean that I can't switch fully to wheels. So I'd need to support the old python setup.py bdisb_dumb based approach as well as adding support for conversion of (binary) wheels to *.deb packages. In any case py2deb needs to be upgraded to a newer / the newest version of pip to gain extended support for environment markers (as pointed out by @remia above).

In about a month (the first half of August) I'm planning to have a significant chunk of free time (time off from work, at the very least) and I've been planning to dive deep into pip-accel and py2deb then. I would first investigate if it's possible to reinvigorate the pip-accel project (and whether there is even a point in doing so, given that pip has seen great improvements in the past years):

• If it is, then I'll do that first, followed by an updated pip-accel <-> py2deb integration.
• If not, then I would (try to) port the essential bits from pip-accel directly into the py2deb code base and change py2deb to call pip directly instead of using pip-accel to invoke pip.

That's all for now, hopefully I'll have something more useful to share soon!

[vidstige]

Thanks for the update! Makes a lot of sense. I think it would be easier to get started with py2deb if it allowed a fresher version of pip. For now, I'm just plopping all our wheels inside a debian package, which works ok. But ideally I would like to auto-convert all these wheels to deb packages at some point in time. Perhaps I'll write a small script for this, inspired by py2deb. Thanks

[RafalSkolasinski]

Any update on this?

[aTanW]

First of all, thanks for great tool!
Is it possible to make use of pip-accel optional? It will give possibility to develop pip-accel in its own schedule.
py2deb will be functional with latest pips...

[remia]

I think it is now all broken on python3.8, unless I'm mistaken pip7.2 can't install package at all.
AttributeError: module 'platform' has no attribute 'linux_distribution'

[xolox]

Hi all,

Sorry for the lack of updates from my end ^[1].

I've started working on a new, minimal implementation of pip-accel that strips it down to the bare essentials (the reason being that the deep integration that was previously created is what made it so hard to keep up-to-date with newer pip releases, so I don't want to repeat that mistake).

The best plan forward for py2deb seems to be to run this new pip-accel (or plain pip) using a command line like pip install --ignore-installed --prefix ... and decompose the resulting installation directory into separate packages based on information available in the lib/python*/*-packages/*.dist-info files (I have a proof of concept that seems to show it's possible).

Because these are large changes to both projects I will need to sacrifice some features, but at least it will get both projects (most importantly py2deb) back on supported pip releases.

I'm hoping to pick up speed on this in the coming weeks, if so I'll certainly share my results here.

---

^[1] I kind of feel obliged to explain a two year gap of silence. For what it's worth, I didn't just forget about pip-accel and py2deb 😇. Shortly after my most recent message here my wife was involved in a serious car accident (while she was on holiday on the other side of the world). The past two years have been focused on her recovery and as a result I'm only just now starting to get back into programming my open source projects.

[vidstige]

Sorry to hear that. You have no obligation to anyone, just do things if you enjoy it. Hoping the best for your wife. 🤞

[remia]

Hey @xolox, thanks for the update, don't feel any obligation or pressure on our part as @vidstige pointed !

Hoping the best for your wife. 🤞

🤞