Further constrain dependency versions in requirements.txt #940
+98
−122
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The python_version actually means the constraint only applies to that Python version. We need the packages installed no matter the Python version; we just need the constraint on Python < 3.11. It's simpler and still works to not add the python_version, so let's just remove them. In addition, this removes constraints that should belong elsewhere, to keep the requirements.txt file leaner.
A shortcoming of generating requirements.txt with version pins is that the versions of some transitive dependencies may need to be updated to address security releases. Simply re-running pip-compile to produce a new requirements.txt may not update the versions because the dependencies that bring in the transitive dependencies have not themselves changed. The recommended practice for this situation is to put the version constraints into a separate constraint file and pass it as an argument to pip-compile. (pip-compile has explicit support for this.) Some additions to the script scripts/generate_requirements.sh encapsulate and document this process. In addition, I removed some no-longer-needed post-processing of the requirements.txt file. It was a hack in the first place, and with the latest requirements.in, it turns out to be unnecessary. Finally, I simplified this some more and made it follow more style guidelines for Bash scripts.
This is the output of running pip-compile on requirements.in.
This file is used to constrain the versions of transitive dependencies during the `pip-compile` process. This is the recommended way to apply security patches without polluting `requirements.in`.
mhucka
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a few more constraints in
requirements.inon development dependencies that have come to light from further testing on different environments, and removes some entries that were for requirements that are not essential and whose handling needs further thought.In addition, this updates
scripts/generate_requirements.shto deal with a shortcoming of generatingrequirements.txtusingpip-compile. The versions of some transitive dependencies may need to be updated to address security releases, but simply re-runningpip-compileto produce a newrequirements.txtmay not update the versions because the dependencies that bring in the transitive dependencies have not themselves changed. The recommended practice for this situation is to put such version constraints into a separate constraint file and pass it as an argument topip-compile. (pip-compilehas explicit support for this.) Some additions to the script scripts/generate_requirements.sh encapsulate and document this process.In addition, I removed some no-longer-needed post-processing of the
requirements.txtfile fromscripts/generate_requirements.sh. It was a hack in the first place, and with the latestrequirements.in, it turns out to be unnecessary. While at it, I simplified the script some more and made it follow more style guidelines for Bash scripts.