perf: build nufft_precision_operator_from as a type-1 NUFFT - #541
Merged
Merged
Conversation
The sparse-operator preload is the real part of a type-1 (adjoint) NUFFT of the inverse-variance weights 1/sigma^2 evaluated on the doubled-extent grid: `nufft2d1(-x, y, w, (2Nx, 2Ny), eps, +1)`, followed by an `ifftshift` and zeroing of the Nyquist row `Ny` and column `Nx`. That replaces the brute-force O(N_pix * K) cosine accumulation with an O(K + N_pix log N_pix) transform. - New `nufft_precision_operator_via_nufft_from`, and `method="nufft"` (the new default) on the `nufft_precision_operator_from` dispatcher, with `eps=1e-12` and mandatory visibility chunking taken from the transformer. - The numpy and JAX brute-force builders are kept untouched as the reference implementations (`method="numpy"` / `"jax"`; `use_jax=True` still routes to the JAX one). - Loud, logged fallback to `method="numpy"` when `PYAUTO_DISABLE_JAX=1` is set or nufftax is not importable; never silent, anything else raises. - `dataset.py`: `apply_sparse_operator` and `psf_precision_operator_from` gain `method`, `eps` and `nufft_chunk_size`, defaulting to the transformer's own `eps` / `chunk_size` when it is a `TransformerNUFFT`. Pins: mixed-tolerance parity against the numpy builder at 4.4e-14 of peak at K=300; chunked == one-shot at 5.3e-14; the wrong-sign control fails at 0.187 of peak. Measured in autolens_profiling#229: alma 2101 s -> 7.3 s wall (289x; 111x in CPU-seconds), alma_high 22 s. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hLF3ZAcz5MmaSJBEcLkvF
Base automatically changed from
feature/interferometer-apply-operator-rfft2
to
main
September 8, 2026 17:22
…ce (#539) Every existing workspace `apply_sparse_operator(use_jax=True)` call was mapped onto `method="jax"`, silently putting the production path on the brute force; `use_jax` is now only honoured when a brute-force method is already selected. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hLF3ZAcz5MmaSJBEcLkvF
3 tasks
Collaborator
Author
|
Workspace PR: PyAutoLabs/autogalaxy_workspace#238 — corrects the interferometer preload prose in |
Merged
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #540; merge #540 first, GitHub retargets this PR to
main.Summary
nufft_precision_operator_frombuilds the interferometer sparse-operator preload — the compact(2Ny, 2Nx)array that depends only on the(dy, dx)offset between image pixels. Until now it was built by brute force: anO(N_pix · K)cosine accumulation over every image pixel × visibility pair, which is minutes to hours at real dataset sizes and is why the workspace ships a separate "prepare the preload overnight and cache it to.npy" example.The same array is the real part of a type-1 (adjoint) NUFFT of the inverse-variance weights
w = 1/σ²evaluated on the doubled-extent grid:which costs
O(N_vis · nspread² + M log M)forM = 4·Ny·Nx. The sign convention (-x, +y, isign+1) is the one of eight candidate mappings that reproduces the brute force; the derivation and the search are recorded in the function's docstring.method="nufft"is the new default. The numpy and JAX brute-force builders are untouched and stay as the reference implementations the NUFFT is pinned against.The array is bit-for-bit the same object (to the pins below), so existing workspace
.npypreload caches remain valid — nothing needs regenerating.Measured (autolens_profiling#229, phase 3)
almaalma_highCPU-seconds caveat: the 289× is wall-clock. The brute-force builder is single-threaded, the NUFFT spreader is multi-threaded, so in CPU-seconds the honest figure is 111×; the wall-clock number is what a user experiences on a workstation.
Pins
_via_np_from, seeded 16×16 mask at K=300: max deviation 4.4e-14 of peak (rtol=1e-10, atol=1e-10·P[0,0]— mixed because the off-peak entries are many orders of magnitude below the peak, so a purertolwould pin round-off).chunk_size=64, K > 64) == one-shot: 5.3e-14.uv[:, 0]negated) fails the pin at 0.187 of peak — i.e. the pin has real discriminating power.Ny/ columnNxexactly zero, neighbours non-zero;P[i, j] == P[-i, -j].test_interferometer.pysparse-vs-mapping comparisons run through the new default unchanged.epsand chunkingeps = 1e-12saturates fp64 at every instrument profiled — the measuredmax|Δ|is already at the round-off floor, and it is the only value that holds the array pin.chunk_sizeis not an optimisation, it is a memory ceiling: the spreader's gather buffer is one-shot ~15 GB at K = 5e6, so chunking is mandatory at real scale. When not given it is taken from the transformer's ownchunk_sizeif it is aTransformerNUFFT.Fallback semantics
Two loud, logged fallbacks to
method="numpy"— never silent:PYAUTO_DISABLE_JAX=1(the test-mode kill switch already honoured indataset.py), orBoth emit a
logger.warningnaming theO(N_pix · K)cost being paid. Any other unusablemethodraises.API Changes
Additive only — every existing keyword is kept with its existing meaning, so no caller breaks.
nufft_precision_operator_fromgainsmethod="nufft"(new default),eps=1e-12andchunk_size=None.use_jaxis now only honoured when a brute-force method is selected (method="numpy"+use_jax=True= the JAX brute force); it is ignored under the defaultmethod="nufft", which already runs on JAX.method="jax"is the explicit way to ask for the JAX brute force.chunk_kstill chunks the brute-force builders.Interferometer.apply_sparse_operatorandInterferometer.psf_precision_operator_fromgainmethod="nufft",eps=Noneandnufft_chunk_size=None(None= take the transformer's).nufft_precision_operator_via_nufft_from(...).use_jax=True. Existing workspaceapply_sparse_operator(use_jax=True)calls therefore land on the fast path with no edit; opting back into a brute force now takes an explicitmethod="numpy"ormethod="jax".See full details below.
Test Plan
pytest test_autoarray/inversion test_autoarray/dataset test_autoarray/operators -q→ 685 passedpython -m black --checkon the four changed files → clean/prm: merge perf: apply the interferometer sparse operator with rfft2/irfft2 #540 first, then confirm this PR retargeted tomainand CI is green per legFull API Changes (for automation & release notes)
Added
autoarray.inversion.inversion.interferometer.inversion_interferometer_util.nufft_precision_operator_via_nufft_from(noise_map_real, uv_wavelengths, shape_masked_pixels_2d, grid_radians_2d, *, eps=1e-12, chunk_size=None) -> np.ndarray— builds the preload as a type-1 NUFFT.Changed Signature
nufft_precision_operator_from(..., *, method="nufft", eps=1e-12, chunk_size=None, chunk_k=2048, show_progress=False, show_memory=False, use_jax=False)—method,eps,chunk_sizeadded; all pre-existing keywords unchanged.Interferometer.apply_sparse_operator(nufft_precision_operator=None, batch_size=128, method="nufft", eps=None, nufft_chunk_size=None, chunk_k=2048, show_progress=False, show_memory=False, use_jax=False)Interferometer.psf_precision_operator_from(..., method="nufft", eps=None, nufft_chunk_size=None, ...)Changed Behaviour
.npypreloads stay valid.dataset.pywas rewritten: the preload is now seconds, so the warning is about the DFT transformer, not the preload.Migration
method="numpy"(ormethod="jax"for the JAX one).use_jax=Trueget the NUFFT path unchanged — the kwarg is now a no-op under the default method, so no workspace edit is needed to buy the speed-up.Closes #539
Generated by the PyAutoLabs agent workflow.
🤖 Generated with Claude Code
https://claude.ai/code/session_018hLF3ZAcz5MmaSJBEcLkvF