Skip to content

test: evidence-sampled subhalo recovery acceptance test (phase 3) - #243

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/potential-correction-evidence
Jul 31, 2026
Merged

Jammy2211 merged 1 commit into
mainfrom
feature/potential-correction-evidence

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Summary

Phase 3 of PyAutoLabs/PyAutoLens#672: the quantitative acceptance test of the potential-correction implementation proposed by the upstream author — regularization hyper-parameters determined by maximizing the Bayesian evidence, for BOTH the one-shot (single-step) method and the iterative method, with subhalo recovery asserted at each method's evidence maximum.

scripts/imaging/subhalo_recovery_evidence.py (new): on the standard 120×120 mock of subhalo_recovery.py, (1) a 13×12 grid over Matern (coefficient, scale) of the one-shot joint inversion's evidence via dense_util.log_evidence_from_fixed_curvature (per-sample cost = two reg-matrix builds + three Choleskys — the curvature matrix, data vector and mapping are regularization-independent and cached), and (2) a 5×5 grid where the LM engine runs to convergence from a cold start (identity damping + stall guards, PyAutoLens#676) and returns the converged Laplace evidence. Evidence surfaces are saved to npz before any assertion.

Findings (full analysis + review report on the issue): both methods localize the input subhalo at their evidence maxima (peaks 0.36"/0.16" from truth), but within the Matern family evidence-max ≠ map-fidelity-max — the +190-nat preference for shorter scales decomposes as +401 prior-misfit relief (a compact cuspy NFW dkappa misfits a long-scale smooth prior) +102 chi² −313 Occam, and the shorter-scale prior dilutes map correlation (0.13 at the max vs 0.82 on the hand-calibrated ridge). The acceptance criterion therefore asserts localization at the evidence max (distance thresholds + corr noise floor) plus a ridge reference proving the family contains high-fidelity solutions (corr > 0.5 at c=2000, s=4).

Runtime is one to a few hours by design (validation, not smoke): excluded from the automated runners via config/build/no_run.yaml.

Scripts Changed

  • scripts/imaging/subhalo_recovery_evidence.py — new evidence-sampled acceptance test (both methods, grids + acceptance assertions; results npz saved pre-assert; gc.collect() per iterative point).
  • config/build/no_run.yaml — SLOW exclusion for the new script (manual validation, ~1-2h by design).

Upstream PR

PyAutoLabs/PyAutoLens#676 — the iterative leg requires the identity-damping default and stall guards (a Marquardt-damped cold start under-converges at every grid point; without the guards each point pays a ~10× rejection storm).

Test Plan

  • Full end-to-end solo run passes all acceptance assertions (log + npz archived on the issue/task worktree).
  • Script excluded from smoke/release runners (no_run.yaml); smoke count unchanged 21/21.

🤖 Generated with Claude Code

…72 phase 3)

Grids the Matern (coefficient, scale) evidence of BOTH potential-correction
methods — one-shot via the fixed-curvature fast path, iterative to
convergence under identity damping — and asserts the input subhalo is
localized at each method's evidence maximum, with a ridge reference proving
the family contains high-fidelity recoveries. no_run-excluded (manual
validation, ~1-2h by design).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Jul 31, 2026
@Jammy2211
Jammy2211 merged commit 86d0c82 into main Jul 31, 2026
3 checks passed
@Jammy2211
Jammy2211 deleted the feature/potential-correction-evidence branch July 31, 2026 22:11
@Jammy2211 Jammy2211 removed the pending-release PR queued for the next release build label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant