perf: vectorise the over-sampled grid builder; skip the mask-edge overlay under fast plots - #534
Merged
Merged
Conversation
…rlay under fast plots grid_2d_slim_over_sampled_via_mask_from built every unmasked pixel's sub-pixel block in a Python loop (a linspace, a meshgrid and a stack per pixel). On a 2000x100 CTI frame that is ~180k iterations and ~12s per call, and FitDataset.__init__ forces the dataset's border relocator, which calls it once per fit — four calls, ~50s of the 117s (local) / 89.5s (CI, slowest leg) autocti_workspace imaging_ci/modeling/start_here.py smoke script, for a relocator a CTI fit never uses. The offsets for a given sub-size are the same for every pixel, so they are now built once per distinct sub-size and broadcast onto that sub-size's pixel centres; non-uniform sub-sizes keep the pixel-ordered layout through a per-pixel block start (cumsum of block sizes). Output is bit-identical to the loop — the old construction is kept verbatim as the oracle in a new test over random masks, uniform and mixed sub-sizes (including zeros), and a non-zero origin. plot_array derived the mask's edge grid on every call for the overlay; with PYAUTO_FAST_PLOTS=1 the figure is closed before it is drawn or saved, so the overlay is never seen. It is now skipped under that flag (~0.4s per figure on the CTI frame, ~100 figures per bypassed fit). Measured on the autocti script, cold (dataset simulated inside the run): 117.3s → 58.5s with this change alone → 25.9s with the smoke profile's PYAUTO_FAST_PLOTS=1 as well. test_autoarray: 1460 passed. Found and measured by PyAutoBrain `hygiene ci` + /ci_speedup. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015J8ZLmc55ZT1nSRntZMz9K
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
grid_2d_slim_over_sampled_via_mask_frombuilt every unmasked pixel's sub-pixel block in a Python loop (alinspace, ameshgridand astackper pixel). On a 2000x100 CTI frame that is ~180k iterations and ~12s per call, andFitDataset.__init__forces the dataset's border relocator, which calls it once per fit — four calls, ~50s of the slowest smoke script on the Heart board (autocti_workspaceimaging_ci/modeling/start_here.py, 89.5s on its slowest leg), for a relocator a CTI fit never uses.The offsets for a given sub-size are the same for every pixel, so they are now built once per distinct sub-size and broadcast onto that sub-size's pixel centres; non-uniform sub-sizes keep the pixel-ordered layout through a per-pixel block start (cumsum of block sizes). The output is bit-identical to the loop: the old construction is kept verbatim as the oracle in a new test over random masks, uniform and mixed sub-sizes (including zeros) and a non-zero origin.
plot_arrayalso derived the mask's edge grid on every call for the overlay. WithPYAUTO_FAST_PLOTS=1the figure is closed before it is drawn or saved, so the overlay is never seen; it is now skipped under that flag (~0.4s per figure on the CTI frame, ~100 figures per bypassed fit).Measured on the autocti script, cold (dataset simulated inside the run, libraries from source): 117.3s → 58.5s with this change alone → 25.9s with the smoke profile's
PYAUTO_FAST_PLOTS=1as well (PyAutoLabs/autocti_workspace, same branch name).Found and measured by PyAutoBrain
hygiene ci+/ci_speedup.API Changes
No public API change.
grid_2d_slim_over_sampled_via_mask_fromreturns the same values in the same order; a private helper_sub_pixel_offsets_fromis added beside it.plot_arraybehaves identically unlessPYAUTO_FAST_PLOTS=1, where it no longer computes the (never-drawn) mask-edge overlay.See full details below.
Test Plan
test_autoarray— 1460 passed locally (libraries from source)autocti_workspacesmoke gate, once its profile PR is up, showsimaging_ci/modeling/start_here.pywell under its 89.5s baselineFull API Changes (for automation & release notes)
Removed
Added
autoarray.operators.over_sampling.over_sample_util._sub_pixel_offsets_from(sy, sx, sub_size)— private; the(sub_size², 2)offset block one pixel's sub-pixels take, inmeshgrid(..., indexing="ij")orderMigration
Generated by the PyAutoLabs agent workflow.
🤖 Generated with Claude Code
https://claude.ai/code/session_015J8ZLmc55ZT1nSRntZMz9K
Generated by Claude Code