prompt: close numerical-inversion-failures (refuted); ship + scope the autoarray noise-map cluster - #264
Merged
Conversation
…sqrt-NaN The prompt's premise does not hold on current `main`. Its own 2026-08-21 reproduction gate ran both scripts from a cleared `output/` under each workspace's profile_release.yaml: 2/2 PASS, no LinAlgError, no non-positive-definite failure. The prompt's specific claim that the autolens leg "reproduces on current `main`" is false. Closed rather than parked, on the sibling autofit_sampler_database's ground rather than the green run alone: neither script is parked. Verified against main -- autogalaxy_workspace/config/build/no_run.yaml has no interferometer/features/pixelization/galaxy_reconstruction entry, and autolens_workspace_test's has no interferometer/model_fit entry (it appears only as a BOOTSTRAP-TARGET consumer). Both re-execute in every mode=release pass, so re-validation is automatic. This makes four independent refutations of the non-positive-definite hypothesis, not three -- the fourth predates the release run: complete/2026/07/pix-inversion-not-positive-definite.md found all six of its markers stale and changed no code, the LinAlgError having been cured 2026-04-10 by PyAutoArray's GaussianKernel PD-guarantee f1817af0. Tracking issue PyAutoArray#467 opened and closed with the evidence. No code changed in any repo. One real defect came out of the gate, filed as its own prompt: abstract.py:859 applies np.sqrt elementwise to a whole covariance matrix, so negative off-diagonals are NaN unconditionally -- for any matrix, however well-conditioned. Confirmed on PyAutoArray main @ a6b07cd. It looks exactly like evidence for the refuted hypothesis and is not; the prompt says so prominently, and records the trap that reconstruction_noise_map is derived from this property's diagonal, so a naive fix silently turns a standard deviation into a variance. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
Researching why source-reconstruction noise maps have been unreliable found
the elementwise-sqrt bug is NOT the cause. np.sqrt is elementwise, so it
commutes with taking the diagonal -- diagonal(sqrt(C))[i] == sqrt(C[i,i]) --
and the off-diagonal NaNs provably cannot reach the 1D noise map.
Three deeper defects were found in the same one-line property, split into a
new prompt (reconstruction_noise_map_solver_mismatch.md) because the primary
one is a statistical decision, not a code repair:
1. Estimator mismatch. inv(curvature_reg_matrix) is the posterior covariance
of the unconstrained Warren & Dye solve, but use_positive_only_solver is
true by default, so the reconstruction is an NNLS active-set solve.
Constraining s >= 0 truncates the posterior: noise is overstated near the
boundary and meaningless for pinned pixels. A compact lensed source pins a
large fraction of the mesh at zero, so the formula is worst exactly where
it is most used.
2. Edge-zeroed pixels ignored. use_edge_zeroed_pixels is also default true;
the reconstruction solves on zeroed_ids_to_keep and scatters back zeros,
while the noise map inverts the full matrix -- re-admitting the poorly
constrained boundary vertices the zeroing exists to remove. A pixel whose
reconstruction reads 0 ("not solved") gets a noise value as though it had
been.
3. use_edge_zeroed_pixels is nested inside the use_positive_only_solver
branch, so turning the positive-only solver off silently disables
edge-zeroing. Two orthogonal settings, silently coupled, no warning.
The original prompt keeps the two defects needing no science decision: the
elementwise sqrt, and np.linalg.inv where Cholesky belongs. abstract.py:805
already documents ~1e-6 evidence round-off from "factorizing the explicitly
formed inverse" at cond(C) ~ 1e9 on clustered traced mesh vertices -- applied
to the log-det, never to the noise map. inversion_plots.py:395 already wraps
the noise map in except LinAlgError and writes NaN to the CSV, a guard that
exists because this fails in practice.
Both prompts mark the load-bearing quantitative claims as reasoned but not
measured, and require the A/B before the fix is written.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
…AutoArray#468) Phase 1 implemented and pushed to PyAutoArray on claude/numerical-inversion-failures-7xsp1k. PR not opened. The prompt's A/B section is rewritten with measured results, because running it refuted two of the claims the prompt itself made: np.linalg.inv does NOT give negative diagonals on well-formed SPD matrices (0 across cond 1e3-1e15), is NOT materially less accurate on the diagonal, and near-coincident mesh vertices do NOT degrade the inverse once regularization is present. Recorded as a table so the wrong reasoning is not re-derived. What survived: inv returns asymmetric output (5.2e-7 at cond 1e12 vs 2.6e-16), and inv silently succeeds on indefinite matrices where a covariance is undefined. The argument for Cholesky is detection, not accuracy. Still not established, and flagged as such: that a real curvature_reg_matrix is indefinite in a converged fit. Routed to the sibling prompt to instrument. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
…nce task An adversarial review of the first PyAutoArray commit found a MAJOR regression: scipy's cho_factor raises ValueError, not LinAlgError, on a non-finite matrix. Both callers catch only LinAlgError, and the CSV writer's docstring explicitly promises a failure there may not abort the model-fit -- so a NaN-contaminated curvature matrix would have killed a fit that previously wrote a nan column and continued. Fixed in PyAutoArray f0aefa8. Downstream grep resolves the deprecated alias's API risk empirically: zero consumers of reconstruction_noise_map_with_covariance in PyAutoGalaxy 3ca31bf, PyAutoLens 87e5827 or autolens_workspace. The same grep supplies concrete evidence for the sibling estimator-mismatch prompt, recorded there: autolens_workspace source_science.py scripts compute signal_to_noise_map = reconstruction / reconstruction_noise_map, so the quantity that prompt argues is computed for the wrong estimator feeds the S/N maps on source reconstructions that end up in papers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
…AutoArray#469 PyAutoArray#469 merged as 2784056; issue #468 closed. CI green on all three jobs (3.12, 3.13, nojax). The 3 test_transformer.py pynufft failures seen locally were a sandbox dependency artefact and were green in CI, as predicted. The record keeps the reasoning that did NOT survive, because this cluster has now misled two separate passes: - The elementwise sqrt does not affect the 1D reconstruction_noise_map. sqrt is elementwise, so it commutes with taking the diagonal. It looks exactly like evidence for a non-positive-definite matrix and is neither that nor the cause of unreliable source noise maps. - The A/B refuted three arguments for the fix: inv does not give negative diagonals on well-formed SPD, is not less accurate on the diagonal, and near-coincident mesh vertices do not degrade it once regularization is present. What survived is detection, not accuracy: cho_factor raises on an indefinite matrix where inv silently returns plausible wrong numbers. - The review caught a regression the first commit introduced: scipy's cho_factor raises ValueError, not LinAlgError, on non-finite input, escaping both callers' guards and aborting a fit the CSV writer promises not to abort. Downstream grep resolved the deprecated alias's API risk (zero consumers in PyAutoGalaxy, PyAutoLens, autolens_workspace) and supplied the sibling prompt's strongest evidence: source_science.py computes signal_to_noise_map = reconstruction / reconstruction_noise_map, so the noise map feeds published S/N maps on source reconstructions. Still open: draft/bug/autoarray/reconstruction_noise_map_solver_mismatch.md, the estimator half. Its first job is to instrument a real fit — neither the indefiniteness of a real curvature_reg_matrix nor the size of the NNLS pinned fraction was ever measured, and the argument rests on the latter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
… fraction Human decision: the reconstruction noise map is a posterior (a calibrated uncertainty used for error bars and S/N), not a diagnostic. That closes the "rewrite the docstring rather than the maths" escape hatch -- defects 1 and 2 are real defects. Recorded in the prompt. Then measured the claim the prompt was resting on, using PyAutoArray's real fnnls_cholesky solver rather than reasoning: CONFIRMED - NNLS pins 25-50% of the mesh for a compact source (2-5% of mesh area), across 400/900/1600-pixel meshes; ~1% for an extended source. The premise holds and scales with compactness as argued. BUT the consequence is much smaller than this prompt implied. Shipped full-matrix noise vs active-set-conditional, on the free pixels: median ratio 1.025, max 1.123. A systematic, always-one-direction ~2.5% (up to 12%) OVERSTATEMENT of the error bars -- not the order-of-magnitude error the high grading assumed. Re-graded Priority: high -> medium. Pinned pixels are also less alarming than feared: reconstruction is exactly 0.0 there, so S/N = 0/noise = 0, which is defensible. Caveat recorded prominently: this is a structural proxy, not a lens fit. The mapping matrix is random rather than ray-traced, so neighbouring image pixels do not map to neighbouring source pixels. The pinned fraction is robust to that; the 2.5%/12% figures are indicative only and must be re-measured on a real Delaunay fit before being quoted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
… back to high
Ran the measurement on a REAL ray-traced fit, reproducing the workspace's
source_science.py setup (Isothermal e_r=1.6 + shear,
RectangularBilinearAdaptDensity 28x28, Constant reg, r=3.0" mask, compact
Sersic, PSF + Poisson noise). Defaults confirmed live:
use_positive_only_solver=True, use_edge_zeroed_pixels=True, 108 zeroed pixels.
The synthetic proxy understated the effect by an order of magnitude, for
exactly the reason its own caveat gave: a random mapping matrix spreads data
support over the whole mesh, while real ray tracing concentrates it in the arc.
pinned fraction: 21% (extended) to 97% (compact), vs 25-50% in the proxy
noise ratio : 1.03x (extended) to 2.84x (compact), vs 1.025x in the proxy
individual pixels up to 10x
flux above the workspace's S/N>=5 cut: 0% to -49%
Two clean trends: the gap grows with source compactness and shrinks with
regularization strength. At coeff=100 it essentially vanishes (1.009).
Reframed the finding, which matters more than the numbers: the two covariances
BRACKET the truth rather than one being right. Full-matrix ignores the s>=0
constraint so overstates; active-set-conditional treats the active set as known
so understates. The truncated-Gaussian posterior is between them. So "switch to
the conditional covariance" is NOT the fix -- it swaps one bound for the other.
The finding is that the source noise map is ambiguous at the factor-of-2 level
for compact sources, with the shipped value at the pessimistic end.
Downstream this is a threshold effect, which amplifies it: source_science.py
cuts at S/N < 5 before computing flux and magnification, so a smooth noise bias
becomes a discrete one as pixels cross the line.
Priority medium -> high. The earlier downgrade was made on the proxy and was
wrong. Still not established, and flagged: one config and one noise realization
per row; reg coefficient was fixed by hand when in a real fit it is a free
parameter the sampler optimises, and the effect nearly vanishes at large lambda
-- where fitted lambda lands decides whether this bites in practice. Delaunay
meshes untested.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
… re-graded low The open question was that the sweep used a hand-set regularization coefficient, while in a real fit Constant.coefficient is free under LogUniform(1e-6, 1e6) and the figure of merit is the Bayesian log evidence. Computed argmax_lambda log_evidence deterministically over a 17-point grid instead of running Nautilus. r_eff lambda* pinned% noise x med flux above S/N>=5 0.05 10 96.6% 1.263 0.0% 0.10 10 87.1% 1.055 -1.3% 0.30 10 42.3% 1.007 0.0% lambda* = 10 in every case, well inside the scanned grid. The Bayesian evidence self-selects AWAY from the problematic regime: the factor-of-2.8 gap occurs at lambda ~ 0.1-1, i.e. under-regularized solutions the evidence penalises. At the lambda a real fit chooses, the median gap is 1.007-1.263 and the science outputs move by 0.0% / -1.3% / 0.0%. Pixel counts either side of the S/N cut are all but identical (12/12, 29/30, 138/138). The pinned fraction stays large (42-97%), so Defect 1's mechanism is real and confirmed. It just has little numerical consequence at the operating point. Priority high -> low. Honest accounting: this prompt has now been graded high -> medium -> high -> low. The swings came from measuring progressively less wrong things (synthetic proxy, then real fit at hand-set lambda, then real fit at fitted lambda). Only the last is the operating point. Still worth doing, and recorded: a very compact source (r_eff=0.05) still sees ~26% median overstatement of per-pixel noise, which matters for a quoted error bar even though it moves no flux. Recommended immediate action is a docstring caveat, not code -- the bracket framing means swapping to the conditional covariance would be wrong at any lambda. Biggest caveat on this result, flagged in the prompt: the lens mass was fixed at truth. A real fit has it free, and a poor mass model may need lower lambda to absorb residuals -- exactly where the gap opens. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
…what remains PyAutoArray#472 documents on reconstruction_noise_map that the covariance is that of the unconstrained solve while the default solver is NNLS, quantifies the overstatement at and below the evidence-optimal coefficient, and records that restricting to the free set is not the correction. Documentation only. Scoped what this prompt still owns, so a later session does not re-derive it: 1. Defect 1's actual maths (truncated-Gaussian posterior) -- low, only if someone needs calibrated per-pixel error bars on a very compact source. 2. Defect 2 was never measured in isolation. The real fits had 108 zeroed pixels of 784 folded into the "pinned" counts throughout, so its separate contribution is unknown. 3. Defect 3 -- use_edge_zeroed_pixels nested inside the use_positive_only_solver branch -- is untouched by any of this measurement and unambiguous at any priority. It sits on the reconstruction path so it changes fit results and needs its own sign-off. Flagged as the most likely next piece of real work. 4. The open measurement: re-run the evidence-optimal lambda with the lens mass FREE rather than fixed at truth. Single result most likely to overturn the low grading. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
…rsion-failures-7xsp1k # Conflicts: # dashboard.html # dashboard.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Mind-side record for a
/start_devthat turned out to be a refutation, a shipped fix, and a measurement campaign.1.
numerical_inversion_failuresclosed as refutedIts premise does not hold on
main: the 2026-08-21 gate ran both scripts from a clearedoutput/under each workspace's release profile — 2/2 PASS. The prompt's claim that the autolens leg "reproduces on currentmain" is false.Closed rather than parked on the sibling
autofit_sampler_database's ground, not the green run alone: neither script is parked.autogalaxy_workspace/config/build/no_run.yamlhas no pixelization entry, andautolens_workspace_test's has nointerferometer/model_fitentry — it appears only as aBOOTSTRAP-TARGETconsumer. Both re-execute in everymode=releasepass, so re-validation is automatic.That makes four independent refutations of this non-positive-definite hypothesis. The fourth predates the release run:
complete/2026/07/pix-inversion-not-positive-definite.mdfound all six of its markers stale and changed no code, theLinAlgErrorhaving been cured on 2026-04-10 by theGaussianKernelPD-guaranteef1817af0.Record:
complete/2026/08/numerical-inversion-failures.md. Issue PyAutoArray#467, opened and closed with the evidence. No code changed in any repo.2. The one real defect, shipped
The gate turned up an unconditional bug:
abstract.py:859appliednp.sqrtelementwise to a whole covariance matrix, so negative off-diagonals wereNaNfor any matrix however well-conditioned.Shipped as PyAutoArray#469 (merged) — Cholesky-formed covariance, decoupled noise map, deprecated alias, 7 new tests. Record:
complete/2026/08/reconstruction-noise-map-covariance-sqrt.md.Record deliberately keeps the reasoning that did not survive, because this cluster has now misled two separate passes:
sqrtcommutes with taking the diagonal. It looks exactly like evidence for a non-positive-definite matrix and is neither that nor the cause of unreliable source noise maps.invdoes not give negative diagonals on well-formed SPD, is not less accurate, and near-coincident mesh vertices do not degrade it once regularization is present. What survived is detection, not accuracy.cho_factorraisesValueError, notLinAlgError, on non-finite input — escaping both callers' guards.3. The estimator half — measured, graded four times, landed at
lowdraft/bug/autoarray/reconstruction_noise_map_solver_mismatch.md. The covariance describes the unconstrained Warren & Dye solve whileuse_positive_only_solver: trueis the default. Human decision recorded: the noise map is a posterior, not a diagnostic — so this is a real defect, not docs drift.Then measured, and the grading moved with the evidence:
high → mediummedium → highhigh → lowOnly the last is the operating point. The Bayesian evidence self-selects away from the problematic regime — the factor-of-2.8 gap lives at λ ≈ 0.1–1, which the evidence penalises. At λ* = 10 the science outputs move 0.0% / −1.3% / 0.0%.
Also recorded: the two covariances bracket the truth, so switching to the free-set-restricted one would be wrong at any λ. That killed the obvious fix.
Immediate action taken instead: PyAutoArray#472, a docstring caveat. Documentation only.
Biggest caveat, flagged rather than buried: the lens mass was fixed at truth throughout. A real fit with a free, imperfect mass model may need lower λ — the regime where the gap opens. Named in the prompt as the measurement most likely to overturn the
lowgrading.4. Left open, deliberately
use_edge_zeroed_pixelsis nested inside theuse_positive_only_solverbranch, so turning the positive-only solver off silently disables edge-zeroing. Untouched by any measurement here and unambiguous at any priority — but it sits on the reconstruction path, so it changes fit results and needs its own sign-off.Housekeeping
Registries pass
lifecycle.py checkandindex --check; dashboard regenerated.🤖 Generated with Claude Code
https://claude.ai/code/session_0133X4XhMV91SFjzV2mK4Ejh
Generated by Claude Code