pytest plugin: Improve performance via caching #472
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tony
force-pushed
the
pytest-plugin-fixture-caching
branch
4 times, most recently
from
a1cea48 to
4e66283
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #472 +/- ##
==========================================
+ Coverage 63.85% 64.17% +0.32%
==========================================
Files 40 40
Lines 3591 3713 +122
Branches 774 788 +14
==========================================
+ Hits 2293 2383 +90
- Misses 772 793 +21
- Partials 526 537 +11 ☔ View full report in Codecov by Sentry. |
tony
force-pushed
the
pytest-plugin-fixture-caching
branch
3 times, most recently
from
e6bcfba to
ae8e219
Compare
tony
force-pushed
the
pytest-plugin-fixture-caching
branch
4 times, most recently
from
68a8bd8 to
3b4d3c0
Compare
tony
force-pushed
the
pytest-plugin-fixture-caching
branch
from
3b4d3c0 to
b7c4149
Compare
tony
force-pushed
the
pytest-plugin-fixture-caching
branch
from
b7c4149 to
faaba05
Compare
tony
force-pushed
the
pytest-plugin-fixture-caching
branch
from
7af8ea6 to
a658c43
Compare
This was referenced Oct 12, 2024
tony
added a commit
that referenced
this pull request
Oct 12, 2024
_Resolves #474_ # Problem Issue #472 requires session-scoped fixtures. Currently, we use `set_home`, `gitconfig`, and `hgconfig` to `monkeypatch.setenv` `$HOME`, pointing it to a temporary directory with user-specific configuration files. This ensures all subsequent `git` and `hg` commands automatically load these configurations. However, [`monkeypatch.setenv`](https://docs.pytest.org/en/8.3.x/reference/reference.html#pytest.MonkeyPatch.setenv) doesn't work with function-scoped fixtures. # Improvement ``` ❯ hyperfine \ --warmup 3 \ --runs 10 \ --prepare 'git checkout master' \ --command-name 'libvcs 0.31.0' \ 'py.test' \ --prepare 'git checkout pytest-config' \ --command-name 'with improved hg/git config fixtures' \ 'py.test' Benchmark 1: libvcs 0.31.0 Time (mean ± σ): 15.150 s ± 0.751 s [User: 16.650 s, System: 4.720 s] Range (min … max): 14.235 s … 16.741 s 10 runs Benchmark 2: with improved hg/git config fixtures Time (mean ± σ): 15.014 s ± 0.307 s [User: 17.246 s, System: 4.865 s] Range (min … max): 14.458 s … 15.642 s 10 runs Summary with improved hg/git config fixtures ran 1.01 ± 0.05 times faster than libvcs 0.31.0 ``` # Changes ## pytest fixtures: Session-scoped `hgconfig` and `gitconfig` These are now set by `set_hgconfig` and `set_gitconfig`, which set `HGRCPATH` and `GIT_CONFIG`, instead of overriding `HOME`.
tony
added a commit
to vcs-python/vcspull
that referenced
this pull request
Oct 13, 2024
# Problem Git, Subversion, and Mercurial repositories are unnecessarily reinitialized for each test via libvcs' pytest plugin. - We're not utilizing session-based scoping. - A single initial repo could be created, then copied to [`tmp_path`](https://docs.pytest.org/en/8.3.x/how-to/tmp_path.html#the-tmp-path-fixture) using [`shutil.copytree`](https://docs.python.org/3/library/shutil.html#shutil.copytree) ([source](https://github.com/python/cpython/blob/v3.13.0/Lib/shutil.py#L550-L605)). # Improvement ``` ❯ hyperfine \ --warmup 3 \ --runs 10 \ --prepare 'pip uninstall -y libvcs && pip install "libvcs==0.30.1"' \ --command-name 'libvcs 0.30.1' \ 'py.test' \ --prepare 'pip uninstall -y libvcs && pip install "libvcs==0.31.0"' \ --command-name 'libvcs 0.31.0' \ 'py.test' \ --prepare 'pip uninstall -y libvcs && pip install "libvcs==0.32.1"' \ --command-name 'libvcs 0.32.1' \ 'py.test' Benchmark 1: libvcs 0.30.1 Time (mean ± σ): 4.784 s ± 0.116 s [User: 4.641 s, System: 0.846 s] Range (min … max): 4.599 s … 4.971 s 10 runs Benchmark 2: libvcs 0.31.0 Time (mean ± σ): 3.436 s ± 0.115 s [User: 2.821 s, System: 0.470 s] Range (min … max): 3.347 s … 3.678 s 10 runs Benchmark 3: libvcs 0.32.1 Time (mean ± σ): 3.173 s ± 0.054 s [User: 2.726 s, System: 0.388 s] Range (min … max): 3.104 s … 3.295 s 10 runs Summary libvcs 0.32.1 ran 1.08 ± 0.04 times faster than libvcs 0.31.0 1.51 ± 0.04 times faster than libvcs 0.30.1 ``` # Changes ## libvcs 0.30.1 -> 0.32.1 See also: - https://github.com/vcs-python/libvcs/blob/v0.32.1/CHANGES#libvcs-0310-2024-10-12 - https://libvcs.git-pull.com/history.html#libvcs-0-32-1-2024-10-12 # Appendix - vcs-python/libvcs#472
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Git, Mercurial, and Subversion repositories are unnecessarily reinitialized for each test.
tmp_pathusingshutil.copytree(source).Issue #471 highlighted this inefficiency, where benchmarks showed tens of thousands of redundant function calls.
Improvement
Changes
Pytest fixtures overhaul