Skip to content

Commit c9a70c2

Browse files
committed
improve text
1 parent 52bfc8a commit c9a70c2

File tree

1 file changed

+15
-36
lines changed

1 file changed

+15
-36
lines changed

docs/blog/posts/pinning_dependecies.md

+15-36
Original file line numberDiff line numberDiff line change
@@ -11,61 +11,40 @@ tags:
1111
draft: true
1212
---
1313

14-
# Pinning test dependencies
14+
# Pinning Test Dependencies
1515

1616
## Motivation
1717

18-
## Test stability
18+
### Guaranteeing test stability
1919

20-
I'm one of the core-developers of [napari](https://github.com/napari/napari) project.
21-
At the moment of writing, this project has 36 direct dependencies and 116 total dependencies for minimum working dependencies.
22-
The test uses 165 dependencies in total. Building docs requires 191 dependencies.
20+
As one of the principal developers of the [napari](https://github.com/napari/napari) project, it is noteworthy to mention that at the time of writing, this project encompasses 36 direct dependencies and 116 total dependencies for minimum operational capability. In terms of testing, it utilizes 165 dependencies in total, while the assembly of documents requires up to 191 dependencies.
2321

24-
This is a lot of dependencies, and their maintainers do not always notice that they will break some use cases.
25-
Sometimes, breaking change is required, and providing deprecated API for a long time is impossible.
22+
Given the substantial volume of these dependencies, there are instances where the maintainers of other libraries might unintentionally disrupt certain API in their packages during updates. These occasional or unintentional disruptions can cause unforeseen complications. On the other hand, there are times when such breakages are deliberate. Providing a backward compatibility layer can be tricky in these situations. Balancing the maintenance of deprecated APIs with both reasonable time constraints and performance requirements often proves unfeasible in the long run. These factors illuminated the need for us to commence pinning our test dependencies.
2623

27-
Before we started pinning test dependencies on average once a month, we had problems with broken tests.
28-
Many of our contributors are learning programming and open-source contributions.
29-
So, it is hard for them to understand why tests are failing and whether it is the fault of their changes.
30-
So next to fixing the test, there is also an explanation to perform additional maintenance work to explain to people what happens.
24+
Prior to implementing the pinning of test dependencies, we frequently encountered issues with broken tests, roughly about once a month. Many of our contributors are novices in programming and contributing to open-source projects. They often struggle with understanding why tests fail and whether it's an outcome of their modifications. Thus, alongside fixing the test, we have to embark on additional maintenance work to clearly explain the circumstances leading to the failure.
3125

26+
### Enhancing reproducibility in science-based projects
3227

33-
### Reproducibility in science
28+
Napari, being in its alpha phase, is substantially a science-based project. As such, we occasionally have to make API modifications. Consequently, newer library updates may not be compatible with older code. However, in scientific research, there's often a need to re-run older codes for comparison purposes.
3429

35-
The napari is a scientific project in the alpha phase. So, we regularly break API.
36-
An old code may not work with the new library version.
37-
However, scientific work sometimes requires running old code to compare results.
38-
There are projects like [`pypi-timemachine`](https://pypi.org/project/pypi-timemachine/)
39-
that create a proxy server that returns PyPi packages until a specific date.
30+
Luckily, there are project tools like [`pypi-timemachine`](https://pypi.org/project/pypi-timemachine/) that design proxy servers to retrieve PyPi packages up till a specified date. Despite this, providing a definitive list of constraints enables the environment to be easily replicated without resorting to additional tools and avoiding discrepancies that could impact results. Also, this list of constraints simplifies the process of instructing others on how to recreate the environment.
4031

41-
However, providing an explicit list of constraints allows the environment
42-
to be reproduced without additional tools, finding differences that may lead to different results.
43-
Also, constraints make it much easier to explain to people how to reproduce the environment.
32+
## Distinguishing between pip's Requirements and Constraints
4433

34+
In pip, requirements are a list of packages that need to be installed and could contain version constraints and [extras](https://pip.pypa.io/en/stable/reference/requirement-specifiers/#requirement-specifiers) definitions. On the other hand, constraints are version boundaries for packages that should be considered during the resolution process.
4535

46-
## Difference between requirements and constraints for pip
47-
48-
The requirements are a list of packages that need to be installed.
49-
It could contain version constraints and [extras](https://pip.pypa.io/en/stable/reference/requirement-specifiers/#requirement-specifiers) definition.
50-
The constraints are a list of version bounds for packages that should be handled during the resolution process.
51-
52-
Requirements could be passed using the package name `pip install numpy` or using a file `pip install -r requirements.txt`.
53-
Constraints could be passed using the file `pip install napari -c constraints.txt` or using environment variables `PIP_CONSTRAINT=constraints.txt pip install napari`.
54-
It will not be installed if a package is mentioned in the constraints but not in the requirements.
36+
Requirements can be passed by using the package name like `pip install numpy`, or via a file like `pip install -r requirements.txt`. Constraints, however, can be passed using the file `pip install napari -c constraints.txt` or via environment variables `PIP_CONSTRAINT=constraints.txt pip install napari`. It's important to note that putting packages in constraints.txt does not install them. It only limits the versions that could be installed.
5537

5638
## What we need
57-
39+
5840
Here is a list of our needs:
59-
41+
6042
1. We need to pin all dependencies for tests and docs.
6143
2. We need to pin per Python version, as some dependencies do not provide single-version support for all Python versions that we support.
6244
3. Constraints need to be automatically updated (creating Pull Requests).
6345
4. As we use upper constraints for some dependencies, we need to be able to upgrade dependencies in any PR, that comes from a forked repository.
6446
5. We need to minimize the infrastructure cost for this process.
65-
47+
6648
## Our solution
67-
49+
6850
Here, I will describe our solution to the problem. Today's date is 2023.11.7. Workflow may change over time. The most recent version of the workflow is available [here](https://github.com/napari/napari/blob/main/.github/workflows/upgrade_test_constraints.yml).
69-
70-
71-

0 commit comments

Comments
 (0)