Add FastBarIterator: native iterator accelerator for python-progressbar#15
Add FastBarIterator: native iterator accelerator for python-progressbar#15wolph wants to merge 1 commit into
Conversation
A small Cython cdef iterator (Approach B) that wraps an iterable and drives a progressbar2 ProgressBar, counting items in a C field so the per-item loop does no Python attribute writes. `bar.value`/`previous_value` are synced only at redraw crossings (~20x/sec, like tqdm.n) and once more at finish. The bar implements a small protocol consumed here: `_fast_begin()`, `_fast_tick(value)`, `_fast_end()`, `_fast_end_dirty()` plus the plain attributes `value`, `_next_update`, `_finished`. `__dealloc__` calls `_fast_end_dirty()` when iteration is abandoned (break/exception) so redirected streams are restored, matching the generator's GeneratorExit cleanup (progressbar issue #212). progressbar2 imports this automatically as the `progressbar2[fast]` extra and falls back to its pure-Python gate when absent. Measured ~5 ns/iter (~4x faster than rich) vs ~30 ns pure Python. Includes a .pyi stub and indirect integration tests (fake bar) since Cython modules can't be measured by Python coverage.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a native iterator accelerator (FastBarIterator) implemented in Cython to optimize the performance of progressbar2. It includes the Cython implementation, type stubs, setup configuration, and comprehensive integration tests using a mock progress bar. The feedback suggests catching BaseException instead of Exception in the __dealloc__ method of FastBarIterator to ensure no exceptions escape during teardown or interpreter shutdown.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| if not bar._finished: | ||
| bar.value = self._value | ||
| bar._fast_end_dirty() | ||
| except Exception: |
There was a problem hiding this comment.
In __dealloc__, using except Exception: may not catch all exceptions. Subclasses of BaseException (such as KeyboardInterrupt, SystemExit, or other interpreter shutdown-related exceptions) can still propagate. Since exceptions escaping __dealloc__ cannot be properly handled and will print a warning to stderr, it is safer to use except BaseException: to guarantee that absolutely no exception escapes during teardown.
except BaseException:
There was a problem hiding this comment.
Pull request overview
This PR introduces a new Cython-backed iterator accelerator (speedups.progressbar.FastBarIterator) intended to drive progressbar2’s ProgressBar via a small “fast protocol”, reducing per-iteration Python overhead by keeping the counter in a C field and only syncing state at redraw boundaries and at completion.
Changes:
- Added
FastBarIteratoras a new Cython extension module (speedups/progressbar.pyx). - Added a matching type stub (
speedups/progressbar.pyi) and new integration-style tests using a protocol-compatible fake bar (tests/test_progressbar.py). - Registered the new extension for build via
setup.py.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
speedups/progressbar.pyx |
Adds the Cython FastBarIterator implementation (fast protocol driver + __dealloc__ dirty-finish teardown). |
speedups/progressbar.pyi |
Provides typing for the new native iterator module. |
tests/test_progressbar.py |
Adds integration tests validating iteration order, tick crossings, and dealloc dirty-finish behavior. |
setup.py |
Includes speedups.progressbar in the cythonized extension list. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| The item count lives in a C field; the per-item loop does no Python attribute | ||
| writes. ``bar.value``/``previous_value`` are synced to the bar only at redraw | ||
| crossings (~20x/sec, like ``tqdm.n``) and once more at finish, so they stay | ||
| plain attributes and no other code path pays any overhead. |
| """Iterates ``iterable``, driving ``bar`` and counting in a C field. | ||
|
|
||
| ``bar.value``/``previous_value`` are synced only at redraw crossings | ||
| (~20x/sec, like ``tqdm.n``) and once more at finish, so the per-item loop | ||
| does no Python attribute writes. | ||
| """ |
| def test_break_finishes_dirty_on_dealloc(): | ||
| bar = FakeBar(step=5) | ||
| it = FastBarIterator(bar, range(1000)) | ||
| collected = [] | ||
| for x in it: | ||
| collected.append(x) | ||
| if x == 7: | ||
| break | ||
| assert collected == list(range(8)) | ||
| del it | ||
| gc.collect() | ||
| assert bar.ended == 'dirty' | ||
| assert bar._finished | ||
|
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e4a228f691
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| try: | ||
| item = next(self._it) | ||
| except StopIteration: | ||
| self._bar.value = self._value # sync final count |
There was a problem hiding this comment.
When the iterable ends without the last item crossing _next_update (the common gated case), this clean-exhaustion path only writes bar.value; bar.previous_value remains whatever the last _fast_tick() left behind. The module contract says both value and previous_value are synced once more at finish, so widgets or callers that inspect previous_value after native iteration completes can observe stale state compared with the final count.
Useful? React with 👍 / 👎.
Adds a small Cython
cdefiterator (speedups.progressbar.FastBarIterator, "Approach B") that wraps an iterable and drives aprogressbar2ProgressBar, counting items in a C field so the per-item loop does no Python attribute writes.bar.value/previous_valueare synced only at redraw crossings (~20x/sec, liketqdm.n) and once more at finish.progressbar2 imports this automatically as the
progressbar2[fast]extra and falls back to its pure-Python gate when absent. Measured ~5 ns/iter (~4x faster than rich, ~11x faster than tqdm) vs ~30 ns pure Python.Protocol consumed from the bar:
_fast_begin(),_fast_tick(value),_fast_end(),_fast_end_dirty()+ plain attrsvalue/_next_update/_finished.__dealloc__calls_fast_end_dirty()on early break/exception so redirected streams are restored (progressbar issue #212).Includes a
.pyistub and indirect integration tests (Cython modules can't be measured by Python coverage). Companion progressbar2 PR: wolph/python-progressbar#316.